Genetic variation underlies human diversity, causing differences in characteristics such as height, eye color or blood type. Some sequence variants also cause genetic diseases, including sickle cell anemia, cystic fibrosis, and mucopolysaccharidosis type III. However, it is often difficult for scientists to determine which variants cause pathological conditions.
In this Innovation Spotlight, Yuya Kobayashi, clinical genomics scientist at Invitae, discusses how clinical geneticists classify and reclassify variants and how artificial intelligence (AI) can help improve genetic testing.
Kobayashi YuyaPhD
Senior Project Manager
variant classification system
invite
What is the general architecture for classifying genetic variants?
In 2015, the American College of Medical Genetics and Genomics (ACMG) and the Association for Molecular Pathology (AMP) released joint consensus guidelines for the classification of germline genetic variants.1 These recommendations, known as the ACMG Guidelines, provide clinical geneticists with a standardized approach to determining whether there is sufficient evidence to classify a variant as pathogenic or benign.
The ACMG Guidelines establish three parameters. First, it defines the types of evidence to be considered. Second, it determines the value or weight of each piece of evidence and how it is combined to achieve one of five classification levels: pathogenic, likely pathogenic, variant of uncertain significance (VUS), probably benign, or benign. Finally, the guidance specifies target confidence thresholds for each tier, with 90% confidence serving as the threshold for classifying a variant as likely pathogenic or likely benign.
in our latest JAMA Internet Open In this study, we used historical variant classification data for more than 2 million genetic variants over an eight-year period to determine how well current variant classification systems align with these confidence threshold targets.2 By observing how a variant’s classification evolves over time, we can estimate the accuracy of the original classification.
What is Sherloc?
Sherloc (Semi-quantitative, evidence-based stratification rules for site interpretation) is an ACMG guideline-compliant, peer-reviewed, clinically validated variant classification system that defines how to apply the ACMG guideline in a more specific and granular way .3 For example, the ACMG guidance states that variants that are more common in the general population than expected for disease should be classified as benign, but it does not define what should be expected. Systems like Sherlock fill these gaps through analytical tools and rules defined by geneticists. Importantly, Sherlock is a system that can evolve as our knowledge of genetics and available technology advances.
All 2 million variants in our study were classified using Sherloc, so examining how these classifications changed over time gave us a way to estimate their initial classification accuracy.2 Our results show that when Sherloc classifies a variant as likely pathogenic or likely benign, new data confirms this 99.9% of the time. This shows that ACMG-compliant systems, such as Sherloc, achieve accuracy that far exceeds the 90% confidence target set by ACMG/AMP.
Why is there a need to reclassify genetic variants in clinical genomics?
The human genome is about three billion base pairs in length, which means there are many possible genetic variants and the likelihood that any given variant will be well studied or widely observed is low. Our data on genetic variants are generally limited, and therefore, approximately half of the genetic variants encountered are initially classified as VUS. However, as more patients are tested and experimental research methods improve, new data allow us to reassess previously classified variants.
Our study found that nearly all reclassifications either confirmed the likely pathogenic and likely benign variants as pathogenic and benign, respectively, or converted the VUS to a more definitive classification.2 Consistent with other studies, approximately 80% of reclassified VUS were ultimately diagnosed as probably benign or benign. Only in very rare cases, approximately 0.06% of reclassifications, do we see situations where new evidence reverses the original classification (e.g., from benign to pathogenic, or vice versa).
VUS results can be frustrating because it does not provide actionable answers to the patient or clinician. These reclassifications may mean the opportunity to receive an appropriate monitoring regimen or treatment. In some cases, reclassification can provide patient peace of mind by confirming a benign outcome and reducing unnecessary medical intervention. Ultimately, the ability to deliver clearer results paves the way for precision medicine, allowing for more appropriate targeted care.
What methods help reclassify VUS into clear categories?
In their new study, Kobayashi and colleagues determined that most VUS reclassifications are the result of scientists using machine learning tools to reanalyze existing data sets.
Our study identified three main strategies that facilitate VUS reclassification.2 The first strategy was to rely on new data collected from additional patient testing or publicly available datasets, which contributed to 30% of VUS reclassifications. The second strategy involves generating data designed to address VUS, such as testing other family members for segregation analysis or testing the patient’s RNA to better understand the molecular impact of the variant. This strategy accounts for 10% of reclassifications.
Surprisingly, the biggest reason for VUS reclassification is not the result of new material, but the application of machine learning (ML) to reanalyze existing material. These machine learning tools allow us to more accurately weigh the importance of each piece of evidence, helping us draw clearer conclusions. Importantly, the machine learning methods that have had a significant impact on VUS reclassification were developed by clinical geneticists and artificial intelligence scientists with a deep understanding of the complexity of the data.
What impact does your discovery have on advancing the practice of genetic testing?
The main finding of our study is that the current accuracy of variant classification is generally very high and exceeds the target definition set by the ACMG guidelines.2 However, this means that a large number of variants are classified as VUS, despite exceeding the 90% confidence target of probably benign and probably pathogenic. This gap highlights the need for improved communication about the trustworthiness of genetic test results and a better understanding of how they should be handled in clinical care.
Another noteworthy finding is that even with these strict non-VUS classification criteria, we have still made substantial progress in reducing VUS, particularly among historically underrepresented racial, ethnic, and ancestry groups, and machine learning Tools are the key driver. The findings suggest machine learning tools could provide a path toward improving the fairness of genetic testing. However, despite all the progress we have made, nine out of 10 variants classified as VUS today remain unchanged. Continued innovation in data analytics, including the use of machine learning and other artificial intelligence methods, is critical to accelerating progress and improving fairness in genetic testing.
What are the next steps to improve processes and guidelines for variant classification in germline genetic testing?
The ideal goal for our community is to eventually transition to a quantitative classification framework that can output the probability of variant pathogenicity, rather than relying on the qualitative five-tier classification we use today. This shift can circumvent the challenge of reconciling observed classification accuracy with target accuracy.
Artificial intelligence and machine learning technologies will play an important role in this transformation, as evidenced by the positive impacts observed in our research. However, it is critical that clinical geneticists guide the development and implementation of AI-driven systems to ensure they are used thoughtfully and appropriately. Developing guidance on how to validate and incorporate artificial intelligence tools into clinical settings will be a critical next step in advancing genetic testing practices, making them more accurate and easier to use for all patients and clinicians.
