Establishing Consensus Mean and Consensus Uncertainty for Phase Abundance

Measurements from different sources, different laboratories, different instruments, and different methods can exhibit significant between-method variability, as well as distinct within-method variances. Often a reference material is certified based on data from more than one measurement method. This situation occurs when no single method can provide the necessary level of accuracy and/or when there is no single method whose sources of uncertainty are well- understood and quantified. A common goal in the analysis of such data is to compute a best- consensus value and to attach a meaningful uncertainty to that value. The results of the combined XRD and optical microscopy data sets are presented in Table 4. Three methods of combining these data sets are presented and will be discussed below.

The Naive Method

A naive approach [16, 17] is to regard the different method results for the same analyte as being repeated estimates of a single true mean, and to compute the consensus mean as the unweighted mean of the different group means, and the consensus uncertainty as:

where n is the number of groups, the ordinary sample standard deviation s is computed on the mean values themselves, and "t" is the appropriate tabulated value of the Student t percent point function. A very high t value for the case n=2 (t » 13) typically precludes using this approach when only two groups are available. This method is useful in that it is simple, probably something that many practitioners would intuitively attempt to use, and will often give a consensus value identical or very close to other "naturally" weighted methods. The naive method also yields a less conservative uncertainty than those provided by other methods which take other sources of error into account.

The Levenson et al. Method

The point of departure for a new method of consensus mean/uncertainty estimation, proposed by Levenson et al. at NIST [18], is to note that the intent of using multiple methods is to realize systematic effects (biases) of individual methods as variation across the multiple methods results. However, if the number of methods is small--two to four--then the sample standard deviation of the method means will be a poor estimator of the uncertainty of the systematic effects. To overcome this, this method uses a Type B model [19] for the uncertainty of the systematic effects. In practice, a uniform distribution, bounded by the range of method results, has been found to be as effective as a Type B model. Other methods make explicit use of intermethod bias (or bias squared), computed as the difference between the largest and smallest group means or between the largest and grand average of the group means [20], as a proxy for between-method variance.

The Mandel-Paule-Vangel-Rukhin (MPVR) Method

Another approach to computing consensus estimates is to try to get an explicit estimate of the intermethod (intergroup) variance, and sum that with a pooled estimate of the within-method (within-group) variance, use the combination to weight the contributions from the different methods to form a consensus mean, and multiply by an appropriate expansion factor, e.g., 2, to get a nominal 95 % uncertainty interval. An estimation equation approach for the determination of the between-group variance developed by Mandel and Paule [21] has often been used at NIST, particularly in the certification of standard reference materials. Vangel-Rukhin [22] showed that the Mandel-Paule solution can be interpreted as a simplified version of maximum likelihood. While most useful when the number of contributing methods is large, we compute the MPVR estimates of the inter-method variances here and add them to a standard pooled estimates of the within method variances, take the square root to get an estimate of the overall standard errors, and multiply by an expansion factor of 2. This method has the virtue of explicitly quantifying within-group and between-group variation, and being rooted in a broadly applicable important general method of mathematical statistics, namely maximum likelihood. The drawback is that the estimation equation solution and formulations of its variance are asymptotically correct, so that MPVR estimates are better for data sets where many (e.g. >10) methods/groups are present.

Table 4. Combined QXRD / Optical Analyses Mean and 95 % Uncertainty Interval.