A literature search was performed in PubMed. Studies identified for potential inclusion were obtained from searches in PubMed with no date limits. Search terms included "OVA1", "multivariate index assay", "adnexal mass", “Overa”, “OVA1 next generation”, “second generation multivariate index”, “Risk of Ovarian Malignancy Algorithm”, “ovarian mass assessment” or “adnexal mass assessment”. Studies had to have been published in peer reviewed journals and include a patient population that met the FDA clearance criteria which includes over age 18, ovarian adnexal mass present for which surgery is planned, and not yet referred to an oncologist.
OVA1®
Fifteen studies were identified for OVA1®. Nine were excluded due to a variety of reasons including: OVA1® was not the index test, OVA1® performance assessed as an individual test; test not performed with the intended FDA-cleared intended usage as an adjunct test, the primary endpoint of the study was to examine how pelvic imaging influenced the OVA1® score, not the performance of the OVA1® test, cost-evaluation studies, and use of OVA1® for indications other than adnexal mass assessment.17-25
Five of the six studies had limitations such as including patients who had been evaluated by a gynecologic oncologist prior to being tested and the studies contained overlapping patient populations.26-30 Therefore, the results may not be generalizable or broadly applicable. All the studies were funded by the test manufacturer.
There were 5 studies that evaluated the clinical validity of Ova1®.
The initial study was a prospective multi institutional cohort study of 590 women from 27 sites across the United States which sought to assess the effectiveness of OVA1® in identifying high-risk ovarian tumors.26 Assessment by a non-gynecologic-oncologist physician plus Ova1® had a sensitivity of 92%, a Negative Predictive Value (NPV) of 93%, and a Positive Predictive Value (PPV) of 36% for predicting an ovarian malignancy. Over half the patients were enrolled by gynecologic oncologists, which may have biased these results. Given that this test is meant to determine the need for involvement of a gynecologic oncologist when it has already been predetermined that an adnexal mass requires surgery, including gynecologic oncologists in the trial was a significant limitation in determining the clinical validity of Ova1®. Additionally, the low PPV may be associated with increased false positives, creating potential patient anxiety/stress and unnecessary procedures. Lastly, not all results were reported separately for patients enrolled by non-gynecologic oncologists. This study was designated as “Level III Evidence” by the journal Obstetrics and Gynecology.
|
Non-Gynecologic Oncologist - Physician Assessment plus Multivariate Index Assay
|
Gynecologic Oncologist -Physician Assessment plus Multivariate Index Assay
|
Sensitivity (%)
|
92
|
99
|
Specificity (%)
|
42
|
26
|
Positive Predictive Value (%)
|
36
|
43
|
Negative Predictive Value (%)
|
93
|
98
|
Ueland, 2011
In a subsequent study of 494 patients, OVA1® demonstrated an overall sensitivity and NPV of 95.7% and 98.1% when combined with clinical impression, respectively.28 Similar to the initial trial, the specificity and PPV were low (50.7% and 30.8%, respectively) and clinical assessment was by non-gynecologic oncologists in half the patients. OVA1® correctly identified ovarian malignancy in 91.4%, compared to 65.7% for CA 125. The study also found that both clinical impression and CA 125 were more accurate than OVA1® in correctly identifying benign disease. Menopausal status did not appear to have an impact on the OVA1® test performance; receiver operating characteristic (ROC) curves stratified by menopausal status showed similar discriminative ability at premenopausal area under the curve (AUC) of 0.906 (95% CI, 0.847-0.975) and a postmenopausal AUC of 0.898 (95% CI, 0.847-0.948).
The objective of the third study was to determine the projected impact on referral patterns of patients undergoing surgery for an adnexal mass after initial evaluation by a nongynecologic oncologist using OVA1®, CA 125, modified-ACOG referral guidelines, and clinical assessment using the combined datasets of the two previously reported studies.30 The Dearking modified ACOG guidelines for consultation with a gynecologic oncologist were used for postmenopausal patients meeting one or more of the following criteria: an elevated CA 125 (>35 units/mL), nodular or fixed pelvic mass, ascites, or evidence of abdominal or distant metastasis. OVA1® demonstrated higher sensitivity (90.2%) compared with clinical assessment (73.2%), CA 125 (68.3%) and modified ACOG guidelines (79.3%). The authors also found that use of OVA1® as a risk stratification test was associated with a gynecologic oncology referral rate of 55.7% comparable to actual clinical practice (60.0%). The authors noted that “the factors affecting utilization of gynecologic oncology resources at the time of suspected ovarian cancer diagnosis appear to be multifactorial”.
Stratification of test performance for detection of malignancy by menopausal status did not appear to have an impact on the test performance.
|
Premenopausal |
Postmenopausal |
Sensitivity %
|
87.7
|
91.6
|
Specificity %
|
60.5
|
43.1
|
PPV %
|
25.6
|
41.9
|
NPV %
|
96.9
|
92.0
|
Bristow, 2013
Utilizing the same patient pool as the previously noted studies, Ware-Miller et al aimed to estimate the performance of OVA1® with ACOG guidelines and the effect of replacing CA 125 with the multivariate index assay.27 This study demonstrated that replacing the CA125 with OVA1® in the ACOG guidelines increased the sensitivity from 77% to 94% and NPV from 87% to 93%. However, there was a concomitant decrease in the specificity from 68% to 35%, and in the PPV from 52% to 40%. The ACOG guidelines with OVA1® identified 79% (15/19) of missed malignancies in premenopausal, and 67% (12/18) of malignancies missed in postmenopausal women compared with the ACOG criteria alone. This study was limited due to the fact that some patient included in the study were already under a gynecologic oncologist's care, results were not presented separately by physician specialty and there was no long-term follow-up on outcomes.
Longoria et al analyzed of the effectiveness of OVA1® in identifying early-stage ovarian malignancy compared to clinical assessment, CA 125, and modified ACOG guidelines among women undergoing surgery for an adnexal mass.29 The performance of OVA1® demonstrated a high sensitivity in identifying early-stage ovarian malignancy when compared with clinical assessment, CA 125, and modified ACOG guidelines. The sensitivity was found to be 91.9% for OVA1® testing alone and 95.3% when OVA1® testing was combined with clinical assessment. The specificity with and without clinical assessment was 44.2% and 49.4%, respectively. Twenty-five percent of the patient population was enrolled by a gynecologic oncologist which may have inflated the prevalence of malignancy in the study population versus the OVA1® intended use population. This study was designated as “Level III Evidence” by the journal Obstetrics and Gynecology.
The sixth study by Eskander et al was a chart review assessing the ability of OVA1® to drive referrals of ovarian cancer patients to gynecologic oncologists prior to their first surgical intervention.31 Data for the study were retrospectively collected through an online medical chart review from 22 obstetrician/gynecologists. The study included 136 patients who had received an elevated-risk OVA1® score, with 122 of 136 patients undergoing surgery for the removal of an adnexal mass. Data were collected through chart extraction forms. Completed forms were aggregated for analysis and assessed for the rates at which obstetrician/gynecologists referred patients with an elevated OVA1® score to a gynecologic oncologist prior to surgery. Results showed that 80% of patients with an elevated OVA1® score were referred to a gynecologic oncologist prior to surgery, with postmenopausal and premenopausal patients showing similar referral rates of 82% and 78%, respectively. Additionally, an appropriate referral to a gynecologic oncologist was seen in 95% of patients with primary ovarian malignancies with an elevated OVA1® score. The study was limited by a small patient population and biases inherent to a retrospective chart review study, which could impact the rates of malignancy and referrals. Lastly, patient outcomes such as morbidity and mortality were not assessed, nor were other long-term outcomes.
Overa®
Only one study met inclusion criteria and it addressed clinical validity. There were no peer reviewed studies addressing analytical or clinical utility that met inclusion criteria.19
Clinical validity was evaluated in a non-concurrent prospective study of 493 preoperatively collected serum specimens from premenopausal and postmenopausal women presenting with an adnexal mass requiring surgical intervention from the same population from the aforementioned study by Bristow.28 Overa® test scores were determined based on the analysis of archived serum specimens, and the patients were stratified into low or high risk groups for finding malignancy on surgery. The analysis examined whether patient referral to a gynecologic oncologist was supported when dual assessment was determined to be positive (either Overa® or clinical assessment was positive, or both were positive). A dual assessment was considered negative when both Overa® and clinical assessment were negative.
Among the 493 study participants, 92 (19%) had a final pathology diagnosis of malignancy. The clinical performance of the Overa® assay, when combined with presurgical physician assessment demonstrated:
- Sensitivity of 93.5%
- Specificity of 64.8%
- PPV of 37.9%
- NPV of 97.7%
Overall, the addition of Overa® testing to presurgical physician assessment correctly identified 75% (P<0.001) of the malignancies missed by physician assessment alone.
This study had several limitations, including a small patient population and a retrospective study design. Additionally, preoperative physician assessment was not uniform across the patient population and this may have introduced bias into the study. Also the method used for combining clinical assessment and Overa® test results was to consider the test positive if either clinical assessment or Overa® test was positive. Thus, in practice, Overa® testing would not be necessary if clinical assessment alone indicated cancer. Using Overa® testing in this manner guarantees that Overa® testing will be more sensitive and less specific than clinical assessment alone, even if it has no better than chance capability of detecting ovarian cancer. Sensitivity improved from 74% to 94% and specificity decreased from 93% to 65%. Finally, this study was funded by the manufacturer. This single study provides very-low-quality evidence for the clinical validity of the Overa® test.
ROMATM
Wang et al 2014 published a meta-analysis of studies evaluating the diagnostic accuracy of the ROMATM algorithm and comparing it to the performance of single markers HE4 and CA 125.32 To be included in the meta-analysis, studies had to investigate both HE4 and CA 125 or calculate ROMATM, include women with ovarian cancer and benign gynecologic disease, use pathology diagnosis as the reference standard, and collect blood samples before treatment was initiated. A total of 32 studies met these inclusion criteria; six of these were conducted in the United States. Findings of the overall pooled analysis of diagnostic accuracy are presented in Table 1. Findings were similar when diagnostic performance in premenopausal women and postmenopausal women were evaluated separately. ROMATM had similar or higher sensitivity than HE4 and CA 125, and HE4 had the highest specificity.
Table 1. Diagnostic Performance of ROMATM compared with HE4 and CA 125 from Wang et al: Meta-analysis findings
|
No. Studies
|
Sensitivity
|
Specificity
|
ROMATM
|
14
|
85.3 (81.2-88.6)
|
82.4 (77.4-86.5)
|
HE4
|
28
|
76.3 (72.0-80.1)
|
93.6 (90.0-95.9)
|
CA 125
|
28
|
79.2 (74.0-83.6)
|
82.1 (76.6- 86.5)
|
Dayyani et al conducted a meta-analysis comparing ROMATM with HE4 and CA 125 in patients with suspected ovarian cancer.33 Six studies met the inclusion criteria, four of which were included in the Wang et al meta-analysis. Two studies were published in 2014 or later. ROMATM had statistically higher area under the curve (AUC) values than either HE4 or CA 125 alone (0.921, vs 0.899, and 0.883 for HE4 and CA 125, respectively). Findings of the pooled analysis of diagnostic accuracy are shown in Table 2.
Table 2. Diagnostic Performance of ROMATM compared with HE4 and CA 125 from Dayyani et al: Meta-analysis findings ROMATM
|
No. Studies
|
Sensitivity % (95% CI)
|
Specificity %, 95% CI
|
ROMATM
|
6
|
87.3 (75.2-94.0)
|
85.5 (71.9-93.2)
|
HE4
|
6
|
68.2 (69.3-90.1)
|
85.1 (71.6-92.8)
|
CA 125
|
6
|
79.6 (66.3-88.5)
|
82.5 (66.2- 91.9)
|
Since the publication of the Wang et al and Dayyani et al meta-analyses, multiple studies have described the use of ROMATM in populations of women in whom decisions to pursue surgery had been made, including Al Musalhi et al34 (n=213 cases), Cho et al35 (n=90 cases), Terlikowska et al36 (n=224 cases), and Minar et al37 (n=267).
Diagnostic performance of the ROMATM test was evaluated for FDA approval in a prospective, blinded clinical trial using thirteen demographically mixed subject enrollment sites with company sponsorship (K103358).16 Patients all presented with an adnexal mass and were scheduled to undergo surgery. An Initial Cancer Risk Assessment (ICRA) was performed to determine the detection of benign versus malignant lesions before testing. The prevalence of cancer was 15%.
Using pathologic diagnosis as the gold standard, test performance, when combined with presurgical assessment for benign disease, was as follows in the hands of a mixed population of generalist and specialist physicians:
|
ICRA alone
|
ICRA with ROMA testing
|
Sensitivity |
77% |
91%
|
Specificity |
84% |
67%
|
PPV |
46% |
33%
|
NPV
|
96% |
98% |
Both tests, when added to pre-testing clinical assessment, produced a fall in the positive predictive value of diagnosis with a small increase in the negative predictive value. The changes observed in the negative predictive value were of uncertain statistical and clinical significance.
A study by Moore et al evaluated ROMATM in conjunction with clinical assessment, using either positive clinical assessment or positive ROMATM as a positive test (similar to the recommended usage for OVA1®).38 Using this method of combining tests guarantees a higher sensitivity and lower specificity for the combined test than for either test alone. Used in this way, ROMATM would only need to be evaluated in patients with a negative clinical assessment. In this study, 461 women were enrolled, of whom a total of 86 (19%) had a malignancy. Combined assessment improved sensitivity from 77.9% to 89.7%, but worsened specificity from 84.3% to 67.2%.
It is important to note that all of the above literature assessed ROMATM as a stand-alone test and did not evaluate diagnostic performance in conjunction with clinical assessment, as the test was intended to be used. Therefore, the ability to draw conclusions regarding the test’s diagnostic performance is limited.
Evidence-Based Practice Guidelines, Position Statements, and Third Party Review
The American Congress of Obstetricians and Gynecologists (ACOG)
ACOG recommend at level C scientific evidence (based primarily on consensus and expert opinion) that physicians can use serum biomarker panels as an alternative to CA125 level alone when evaluating women with adnexal masses to determine the need for referral to or consultation with a gynecologic oncologist when an adnexal mass requires surgery. Consultation with or referral to a gynecologic oncologist is recommended at a level B scientific evidence (based on limited or inconsistent scientific evidence) for patients with an elevated score on a formal risk assessment test.12
The Society for Gynecologic Oncology (SGO)
In May 2013, the Society for Gynecologic Oncology (SGO) issued the following position statement on multiplex serum testing for women with pelvic masses39:
“Blood levels of five proteins in women with a known ovarian mass have been reported to change when ovarian cancer is present. Tests measuring these proteins may be useful in identifying women who should be referred to a gynecologic oncologist. Recent data have suggested that such tests, along with physician clinical assessment, may improve detection rates of malignancies among women with pelvic masses planning surgery. Results from such tests should not be interpreted independently, nor be used in place of a physician’s clinical assessment. Physicians are strongly encouraged to reference the American Congress of Obstetricians and Gynecologists’ 2011 Committee Opinion “The Role of the Obstetrician-Gynecologist in the Early Detection of Epithelial Ovarian Cancer” to determine an appropriate care plan for their patients.” SGO does not formally endorse or promote any specific products or brands.
A review by Hayes regarding the use of the OVA1® test to assess malignancy risk in adnexal masses in women with planned surgery was rated as “D2” (there is insufficient published evidence to assess the safety and/or impact on health outcomes or patient management).40 The review concluded that “there is no peer-reviewed evidence for analytical validity, very-low-quality evidence for clinical utility, and low-quality evidence for clinical validity for the use of the OVA1® test. Individual study quality for publications supporting the OVA1® test ranged from poor to very poor. The lack of studies supporting analytical validity and clinical utility downgrades the overall body of evidence for OVA1®. Additionally, clinical validity studies evaluated had several limitations and consisted of low-quality and very-low-quality evidence”.
The National Cancer Institute (NCI)41
Level of evidence: 5
Proteomics
Initially, mass spectroscopy of serum proteins was combined with complex analytic algorithms to identify protein patterns that might distinguish between ovarian cancer cases and controls.42 This approach assumed that pattern recognition alone would be sufficient to permit such discrimination, and that identification of the specific proteins responsible for the patterns identified was not required. This strategy was modified, using similar laboratory tools, to identify finite numbers of specific known serum markers that may be used in place of, or in conjunction with, CA 125 measurements for the early detection of cancer.43 These studies 44,45 have generally been small case-control studies that are limited by sample size and the number of early-stage cancer cases included. Further evaluation is needed to determine whether any additional markers identified in this fashion have clinical utility for the early detection of ovarian cancer in the unselected clinical population of interest.
National Comprehensive Cancer Network (NCCN)
The NCCN ovarian cancer guidelines (v1.2020) includes the following statement46:
“It has been suggested that specific biomarkers (serum HE4 and CA 125) along with an algorithm (Risk of Ovarian Malignancy Algorithm [ROMATM]) may be useful for determining whether a pelvic mass is malignant or benign. The FDA has approved the use of HE4 and CA 125 for estimating the risk of ovarian cancer in women with a pelvic mass. Currently, the NCCN Panel does not recommend the use of these biomarkers for determining the status of an undiagnosed pelvic mass.”
Of note is the fact that the NCCN Ovarian Cancer panel has twice (2016, 2017) unanimously voted down the request by manufacturer to modify the wording in the guidelines to replace CA 125 by OVA1® and /or MIA or other tumor markers in the work up section of the guidelines, to revise the primary treatment section to include establishing a baseline CA 125 as clinically indicated either from OVA1® and or MIA result or separately and to change the wording in section M-5 to indicate that the guidelines recommend that anyone with an elevated OVA1®result undergo surgery by an experienced gynecologic oncologist.
National Institute for Health and Care Excellence
- There is currently not enough evidence to recommend the routine adoption of the IOTA ADNEX model, Overa®, RMI I (at thresholds other than 200 or 250), ROMATM or IOTA Simple Rules in secondary care in the NHS to help decide whether to refer people with suspected ovarian cancer to a specialist multidisciplinary team (MDT).
- The NICE guideline on ovarian cancer recommends that people with an RMI I of 250 or more are referred to a specialist MDT. Evidence suggests that there is no substantial change in accuracy if the threshold for RMI I is lowered to 200.
- The IOTA ADNEX model, Overa®, RMI I (at thresholds other than 250), ROMATM and IOTA Simple Rules show promise. Further research is recommended on test accuracy and the impact of the test results on clinical decision-making.47