Appropriate Statistical Techniques when Using Consensus Statistics to Evaluate Performance

A large variety of statistical methods can be used for proficiency testing (PT) programs in various areas of laboratory testing. Statistical methods described in ISO 13528 and other international standards address PT in a wide variety of applications. The most significant difference in statistical techniques is whether performance evaluations are determined from the participant results using consensus statistics from the current round, or whether the performance criteria are determined independently. For schemes evaluated by consensus, the next most significant factor is the experience of both the scheme and its participants. This is evidenced in the proportion of results reported by participants who lack competence, are newly enrolled, or do not understand the instructions provided. For example, statistical techniques that are necessary for novel schemes (e.g. run for the first time) may not be appropriate for a similar scheme after several rounds with the same participants. Similarly, different techniques may apply for closed schemes that have regular technical review of a limited group of experienced laboratories. Techniques that make allowances for high levels of “contamination” from incompetent or inexperienced participants, such as the z’ score with consensus assigned values, should not be used in experienced schemes using consensus statistics. Other techniques that are more sensitive to small systematic errors should be employed for closer monitoring of experienced laboratories, including statistical techniques that consider the measurement uncertainty of the results. Mature PT schemes and closed schemes for special purposes should evaluate the measurement uncertainty of participant results in any PT scheme used by laboratories that make decisions on conformity assessment, or where improvement of participant agreement is an objective for the scheme. Oversight bodies that require compliance with ISO/IEC 17043 should consider these recommendations, to better ensure global compatibility of measurements.