Laboratory of Pathology - Staff

Test Method / Instrument Validations

Method Validation and Approval:

For new methods implemented after June 15, 2009, there must be an evaluation of the test method analytic validation or verification study (accuracy, precision, etc.) signed by the laboratory director, or designee meeting CAP director qualifications, prior to use in patient testing to confirm the acceptability of the data and approve each nonwaived test for clinical use. LP standard practice is to have the instruments' or analytes' manufacturer provide guidance on method validations, statistical analyses and ensure consistency with industry best practices.

The evaluation must include: 1) a written assessment of each component of the validation or verification study, including the acceptability of the data; 2) a signed approval statement, such as, "I have reviewed the verification (or validation) data for accuracy, precision, reportable range, and reference interval studies (insert other components, as required) for the (insert instrument/test name), and the performance of the method is considered acceptable for patient testing." The Medical Director or designee must also sign a summary statement documenting review of validation studies and approval of each test prior to clinical use. An example statement is, “validations reviewed and performance of method is approved for patient testing.” If data include discordant results, there must be a record of the discordance and investigation of any impact on the approval of the test for clinical use.

FDA-cleared or FDA-approved tests

For introduction of FDA-cleared or approved test system, the section must do the following before reporting patient test results:

• Demonstrate comparable performance specifications established by the manufacturer for accuracy, precision, interferences and the reportable range of test results for the test system. 
• Verify reference intervals (normal values) are appropriate for the laboratory's patient population.
• Validate Accuracy, Precision, Sensitivity, Reportable Range, Interfering Substances, and Other Performance Characteristics relevant to test validation per the manufacturer's guidelines.

The laboratory must follow manufacturer's instructions or provide validation records if the test has been modified. Following manufacturer's instructions includes performing quality control, calibration, calibration verification, and related functions as applicable to the scope of testing. Reagents, fluids, and disposable materials supplied by the laboratory must meet the specifications in the instructions.

Laboratory-Developed Tests (LDTs)

For the purposes of interpreting this requirements, a laboratory-developed test (LDT) is defined as follows: A test used in patient management that has both of the following features.

• The test is performed by the clinical laboratory in which the test was developed wholly or in part; and
• The test is neither FDA-cleared nor FDA-approved.

Each laboratory section must maintain a list of laboratory-developed tests (LDTs) and modified FDA cleared/approved tests implemented by the laboratory. The list must include tests developed in-house, and for laboratories subject to US regulations, tests using analyte-specific reagents (ASRs), and FDA-cleared/approved tests that have been modified by the laboratory.

Elements of Validation

For introduction of all other test systems (e.g., modified FDA-cleared/approved test, in-house laboratory-developed test (LDT), standardized test book procedures, or when test performance specifications are not provided by manufacturer), the following performance characteristics, as applicable, must be demonstrated before reporting patient test results:

• Accuracy - Where current technology permits, accuracy is established by comparing results to a definitive or reference method, or may be verified by comparing results to an established comparative method. Use of reference materials or other materials with known concentrations or activities is suggested in establishing or verifying accuracy. Accuracy is verified or established by comparing results to a definitive or reference method, or an established comparative method. Use of matrix-appropriate reference materials, patient specimens (altered or unaltered), or other commutable materials with known concentrations or activities may be used to verify or establish accuracy. The use of routine quality control materials or calibrators that were used to calibrate the method is not appropriate. 
  
  
• Accuracy - Modified FDA-cleared/approved and LDTs - For modified FDA-cleared/approved and laboratory-developed tests (LDTs), validation of analytical accuracy includes testing with an appropriate number of samples. An appropriate number of samples is defined as the following:
  
  • For quantitative tests, a minimum of 20 samples with analyte concentrations distributed across the analytical measurement range should be used. Proportionate mixtures of samples may be used to supplement the study population. 
  • For qualitative tests, a minimum of 20 samples, including positive, negative, and low positive samples with concentrations near the lower level of detection should be used; equivocal samples should not be used.
    • For certain methods that test multiple analytes (e.g. next-generation sequencing, FISH, HPLC, GC-MS, MALDI-TOF, etc.), analytic accuracy may be established for each method (not necessarily each analyte), as appropriate.

    • If the laboratory uses fewer samples, the laboratory director must record the criteria used to determine the appropriateness of the sample size. In many cases, a validation study with more samples is desirable.

  • For modified FDA-cleared/approved tests and LDTs in use prior to July 31, 2016, for which limited validation studies are recorded, ongoing data supporting acceptable test performance may be used to meet the above minimum sample requirement, unless the laboratory must record the criteria used to determine the acceptability of a smaller sample size. 

    • Examples of such ongoing data include records of proficiency testing, alternative performance assessment, and quality control. 

    • This requirement does not apply to LDTs that employ: Manual microscopy (histology, cytology interpretation) or conventional microbiological cultures and disc susceptibility studies.
      
      
• Precision - Analytical precision for each test is verified using a sufficient number of characterized samples with repeated analysis. Precision is established by repeat measurement of samples at varying concentrations or activities within-run and between-run over a period of time. 
  
  
• Analytical Sensitivity - The laboratory verifies or establishes the analytic sensitivity (lower detection limit) of each assay, as applicable.
  
  
• Analytical specificity to include interfering substances - Implementation of new analytes must include validation of the analytical specificity, which refers to the ability of a test or procedure to correctly identify or quantify an entity in the presence of interfering or cross-reactive substances that might be expected to be present. Published literature can be used for guidance and determine confidence intervals to estimate analytical sensitivity. Interfering substances may pose a significant problem to the clinical laboratory and healthcare providers who may be misled by laboratory results that do not reflect patient clinical status. The laboratory must be aware of common interferences by performing studies or having available studies performed elsewhere (such as by the instrument-reagent manufacturer).
  
  
• Reportable range of test results for the test system. The reportable range of an assay is the range of values that the laboratory reports for that assay. The analytical measurement range (AMR) is the range of analyte values that a method can directly measure on the specimen without any dilution or concentration. In some cases, clinically relevant limits may be narrower than the analytical measurement range, and the clinically relevant limit is used as the limit for the reportable range.
  
  
• Any other performance characteristic required for test performance.
  
  
• Note: If the laboratory modifies manufacturer instructions, the test is categorized as a non-FDA approved/cleared test, and the modification must be validated by the laboratory. A change in the specimen type or collection device is considered a modification.

Each section must determine the test system's calibration procedures and control procedures based upon the performance specifications verified or established as indicated above.

The section must document all activities for the verification or establishment of performance specifications.

Reference Interval (Normal Range) Validation and Reporting

Each clinical section evaluates the appropriateness of reference intervals specific to each analyte. Reference intervals are evaluated: at the introduction of a new analyte / test; if there is a change to analytic methodology; and if the patient population changes.

Laboratory reports that have reference intervals, normal ranges, or interpretations must have the reference values reported with the patient's report to allow proper interpretation of the data.

• Normal values are initially established or verified for each specimen source when appropriate. 
• Clinical and Laboratory Standards Institute (CLSI) guideline EP28-A3C, Defining, Establishing, and Verifying Reference Intervals in the Clinical Laboratory, 3rd ed., is available to staff performing reference interval validations. This CLSI guideline provides guidance on how to conduct reference range validation by recommending: methods, types and numbers of cases to evaluate, and statistical analysis of data to establish or verify reference intervals.

• A reference interval can be verified by testing samples from 20 healthy representative individuals; if no more than two results fall outside the proposed reference interval, that interval can be considered verified for the population studied. 

• Literature references or manufacturer package insert may be appropriate, but should be verified based on patient population. If a formal reference interval study is not possible or practical, then the laboratory should carefully evaluate the use of published data for its own reference ntervals, and retain records of this evaluation.
• Reference intervals must be re-evaluated for new analytes, change in analytic methodology, or change in patient population.

Validation Claims

Clinical claims  about a test's diagnostic sensitivity and specificity, ability to predict the risk of a disease or condition, clinical usefulness, or cost-effectiveness can be published in the laboratory sections test menu or on the pathology repot. LP sections are not required to make clinical claims about a test, but any claims made by the laboratory must be validated.

• If a claim about diagnostic sensitivity and specificity and/or ability to predict risk of a disease or condition, the laboratory must perform a clinical validation study,unless the clinical validity of the test is documented in peer-reviewed literature or textbooks.
• The clinical validation study must include at least 20 samples and must include both positive and negative samples.
• If the laboratory uses fewer samples, the laboratory director must record the criteria used to determine the appropriateness of the sample size.

A list of current test methods is available to clieants upon request, and also available on the Laboratory of Pathology's website and/or order section requisition forms. 

The Section Head of each section determines appropriate method performance specifications. For current test methods, the following performance specifications are available upon request to medical and clinical care staff:

• Summary of the analytical performance specifications for each method, validated or verified by the laboratory to include analytical accuracy, precision, analytical sensitivity, analytical specificity (test method interferences), reference interval, and reportable range, as applicable; and
• Supporting data for clinical performance claims, if applicable, validated or verified by the laboratory or obtained from peer-reviewed literature.

Analytic Methodology Changes

If the laboratory changes its analytic methodology so that test results or their interpretations may be SIGNIFICANTLY different, the change is explained to NIH medical and clinical care staff. This will be accomplished by NIH electronic mail to all clinical clients utilizing the affected sections' service or though Laboratory of Pathology handouts to relevant clinical staff and the Medical Executive Committee.

Intermittent Testing – Test Reactivation

When a test is put back into production, the following requirements must be met:

• Proficiency testing (PT) or alternative assessment performed within 30 days prior to restarting patient testing.

• Method performance specifications verified, as applicable, within 30 days prior to restarting patient testing.

• Competency assessed for analysts within 12 months prior to restarting patient testing.

This requirement applies to tests that are taken out of production for a time (for example, seasonal testing for influenza). A test is considered to be taken out of production when (1) patient testing is not offered AND (2) PT or alternative assessment, as applicable, is suspended. It does not apply to situations where a proficiency testing challenge is not performed due to a temporary, short-term situation, such as a reagent back order or an instrument breakdown. In those situations, the laboratory must perform alternative assessment for that testing event.

For tests in which proficiency testing (PT) is required, if a PT challenge is not offered during the 30-day period prior to restarting patient testing, the laboratory may perform an alternative assessment of the test. In such a case, the laboratory must participate in the next scheduled PT event.

Antibody Validations

Clinical sections that use antibodies for clinical analysis must validate new antibodies, including introduction of a new clone, prior to patient testing. The performance characteristics of each assay must be appropriately validated before being placed into clinical use. The initial goal is to establish the optimal antibody titration, detection system, and antigen retrieval protocol. Once optimized, a panel of tissues must be tested to determine the assay's sensitivity and specificity. The scope of the validation is at the discretion of the section's technical supervisor.

Means of validation may include, but are not limited to: 1) correlating the results using the new antibody with the morphology and expected results; 2) comparing the results using the new antibody with the results of prior testing of the same tissues with a validated assay in the same laboratory; 3) comparing the results using the new antibody with the results of testing the same tissue in another laboratory with a validated assay; or 4) comparing the results using the new antibody with previously validated non-IHC tests or testing previously graded tissue challenges from a formal  proficiency testing program. 

For an initial validation, there should be at least 90% overall concordance between the new test and the comparator test or expected results. 

For validation of a nonpredictive assay, the validation should test a minimum of 10 positive and 10 negative tissues. For validation of predictive markers (with the exception of HER2, ER and PgR), the laboratory should test a minimum of 20 positive and 20 negative tissues. In either situation, when the technical supervisor or medical director determines that fewer validation cases are sufficient for a specific marker (e.g. a rare antigen or tissue), the rationale for that decision needs to be recorded. Positive cases in the validation set should span the expected range of clinical results (expression level), especially for those markers that are reported quantitatively.

When possible, validation tissues that have been processed using the same fixative and processing methods as cases that will be tested clinically should be used. If IHC is regularly done on specimens that are not fixed or processed in the same manner as the tissues used for validation (e.g. alcohol fixed cell blocks, cytologic smears, formalin postfixed tissue, or decalcified tissue), the laboratory should test a sufficient number of such tissues to ensure that assays consistently achieve expected results. The laboratory director is responsible for determining the number of positive and negative cases and the number of predictive and nonpredictive markers to test.