To: Administrative File: CAG #00290R
Tumor Antigen by Immunoassay CA 125 (Addition of Primary Peritoneal Adenocarcinoma as a Covered Indication)
From: Steve Phurrough, MD, MPA
Director, Coverage and Analysis Group
Louis Jacques, MD
Division Director
Jackie Sheridan-Moore
Lead Analyst
Maria Ciccanti
Analyst
Jim Rollins, MD, MSHA, PhD
Lead Medical Officer
Subject: Coverage Decision Memorandum for Tumor Antigen by Immunoassay CA 125
(Addition of Primary Peritoneal Adenocarcinoma as a Covered Indication)
Date: November 1, 2005
I. Decision
The Centers for Medicare and Medicaid Services (CMS) has determined that there is sufficient evidence to conclude that CA-125 testing is reasonable and necessary for the surveillance of Primary Peritoneal Carcinoma (PPC) in Medicare beneficiaries following treatment.
We will add PPC to the indications described in the narrative NCD for tumor antigen by immunoassay CA-125. We will also add ICD-9-CM codes 158.8 and 158.9 to the list of covered codes associated with the NCD for tumor antigen by immunoassay CA-125.
II. Background
Epidemiology
Primary Peritoneal Carcinoma (PPC) is also referred to as Primary Peritoneal Adenocarcinoma. Terms which are synonymous with PPC include the following:
Extraovarian peritoneal serous papillary carcinoma
Serous surface papillary carcinoma
Multiple focal extraovarian serous carcinoma
Primary peritoneal papillary serous adenocarcinoma
Serous surface carcinoma of the peritoneum
Papillary serous carcinoma of the peritoneum
Peritoneal papillary carcinoma
The peritoneum is the membrane that lines the abdominal and pelvic cavities as well as the surfaces of the abdominal and pelvic organs. PPC arises in the peritoneum and is very similar to epithelial ovarian cancer in terms of its microscopic appearance, symptoms, pattern of spread, treatment, and prognosis. This is because the lining of the abdomen and the surface of the ovary come from the same tissue when the embryonic female develops in the uterus.
PPC is a rare cancer that occurs exclusively in women. Eltabbakh and colleagues were able to show that the epidemiologic features of women with PPC when compared with women with primary epithelial ovarian cancer show little differences though the former group was older (Eltabbakh, Piver, Natarajan, Mettlin, 1998). Survival rates are poor for patients with PPC, with 100% mortality. The median survival reported is from 12-25 months even with extensive surgery and chemotherapy.
Disease Process
Symptoms of PPC are similar to those of ovarian cancer and the disease is difficult to detect, especially in its early stages. Patients usually present with complaints of diffuse nonspecific abdominal pain or distention secondary to ascites, bloating, nausea, vomiting, indigestion and a change in bowel habits.
Some researchers have hypothesized that a hereditary predisposition may play a role in PPC, because patients with the BRCA1 mutation have an increased risk for this disease. Levine and colleagues have demonstrated that a substantial proportion of Ashkenazi Jewish patients with PPC are BRCA carriers. BRCA-associated PPC patients are younger at the time of diagnosis, and have improved survival compared to patients without the BRCA mutation (Levine Argenta, Yee, Marshal, Narciso, Bogomolniy et al. 2003).
Piura and associates were able to show that PPC was related to the BRCA1 185delAG mutation (Piura, Rabinovich, Yanai-Inbar, 2001). There is an ongoing registry study evaluating whether or not patients who undergo a prophylactic oophorectomy secondary to a family history of ovarian cancer develop PPC (Piver, Jishi, Tsukada, Nava 1993).
Staging of PPC is similar to the staging of ovarian cancer. At the time of diagnosis, most patients suffering with PPC are either found in stage III (contained within the abdomen) or stage IV (disease has spread to distant organs i.e. lungs or liver). Histological specimens are also able to provide information about tumor grading, which can be translated into prognosis. Tumors are graded on a scale of 1 (best prognosis) to 3 (worst prognosis):
- Grade 1 - cells appear to look most like normal tissue (well differentiated). Grade 1 cancers tend to grow and spread more slowly.
- Grade 2 - cells look somewhat like normal tissue, but some atypia noted (moderately well differentiated). Grade 2 cancers tend to grow and spread quickly.
- Grade 3 - cells appear very abnormal with marked atypia (called poorly differentiated or undifferentiated). Grade 3 cancers are the most aggressive and grow and spread more quickly than other grades.
Treatment Options
PPC is treated similarly to ovarian cancer. Currently, cytoreduction and adjuvant therapy with platinum-based chemotherapeutic regimens are the treatments of choice. Treatment consists of total abdominal hysterectomy and bilateral salpingo-oophorectomy as needed, along with debulking of the tumor and follow-up chemotherapy. Carboplatin or cisplatin therapy, in combination with paclitaxel, is associated with a high response rate as well as improvement of median survival.
Interaction of the Intervention with the Disease Process
As noted by Piver, “...although prophylactic oophorectomy will provide 100% protection against the development of ovarian cancer, it is clear that a very small fraction of such women will subsequently develop PPC. Whether the inherited BRCA1 and BRCA2 mutations that result in familial ovarian cancer also predisposes someone to PPC is yet unknown. All such women are followed closely post prophylactic oophorectomy by physician examination and CA-125 blood testing” (Piver, 1996). Because of histologic similarity, as well as similarities in symptoms, pattern of spread, treatment, and prognosis with ovarian cancer, monitoring of CA-125 levels is also performed in the surveillance of PPC, just as it is for ovarian cancer.
CA-125, also known as Carcinoma Antigen 125, is a glycoprotein that is produced by certain tumors. It was first described by Bast and colleagues in the 1980s, and it is expressed by approximately 80% of ovarian epithelial carcinomas. In this setting, its clinical application is for the prognosis and the monitoring of therapeutic response. Wick was one of the first to note the association between CA-125 and PPC (Wick, Mills, Dehner, Bollinger, Fechner, 1989), but it was Altaras and associates who noted the usefulness of CA-125 in the management and follow-up for primary peritoneal papillary serous carcinoma (Altaras, Aviram, Cohen, Cordoba, Weiss, Bayth, 1991).
In a case study done by Skates and associates, rising longitudinal levels of CA-125 were significant in the detection of peritoneal papillary serous carcinoma, and led to the indication for surgery (Skates, Troiano, Knapp 2003). Other authors have found that CA-125 levels correlate with the clinical status of the disease and response to therapy (Killackey, Davis, 1993; Mills Anderson, Fechner, 1988). For the purposes of this NCD request, the focus is on the use of CA-125 is primarily for monitoring therapy in patients with PPC, just as it is used in patients with ovarian cancer. That is because a number of studies have demonstrated similarities between the two types of diseases, including elevated CA-125 levels (Piura, Meirovitz, Bartfeld, 1998; Barda, Menczer, Chetrit et al. 2004).
III. History of Medicare Coverage
Medicare is a defined benefit program. An item or service must fall within a benefit category as a prerequisite to Medicare coverage and not otherwise be excluded from coverage. § 1812 (Scope of Part A); § 1832 (Scope of Part B) § 1861(s) (Definition of Medical and Other Health Services).
The current NCD (portions of which are currently found in the National Coverage Determination Manual, CMS Pub. 100-03, section 190.28), effective November 15, 2002, indicates that the benefit category is "diagnostic laboratory tests." The information provided supports continuing the current benefit category of Social Security Act section 1861(s)(3), "other diagnostic tests," for the new indication.
The current NCD lists the following indications and limitations.
Indications
CA 125 is a high molecular weight serum tumor marker elevated in 80 percent of patients who present with epithelial ovarian carcinoma. It is also elevated in carcinomas of the fallopian tube, endometrium, and endocervix. An elevated level may also be associated with the presence of a malignant mesothelioma.
A CA 125 level may be obtained as part of the initial pre-operative work-up for women presenting with a suspicious pelvic mass to be used as a baseline for purposes of post-operative monitoring. Initial declines in CA 125 after initial surgery and/or chemotherapy for ovarian carcinoma are also measured by obtaining three serum levels during the first month post treatment to determine the patient’s CA 125 half-life, which has significant prognostic implications.
CA 125 levels are again obtained at the completion of chemotherapy as an index of residual disease. Surveillance CA 125 measurements are generally obtained every 3 months for 2 years, every 6 months for the next 3 years, and yearly thereafter. CA 125 levels are also an important indicator of a patient’s response to therapy in the presence of advance or recurrent disease. In this setting, CA 125 levels may be obtained prior to each treatment cycle.
Limitations
These services are not covered for the evaluation of patients with sign of symptoms suggestive of malignancy. The service may be ordered at times necessary to assess either the presence of recurrent disease or the patient’s response to treatment with subsequent treatment cycles.
A CA 125 is specifically not covered for aiding in the differential diagnosis of patients with a pelvic mass as the sensitivity and specificity of the test is not sufficient. In general, a single “tumor marker” will suffice in following a patient with one of these malignancies.
IV. Timeline of Recent Activities
| Date |
Action |
| 6/20/2005 |
CMS opened a national coverage determination (NCD) review to determine if CA-125 testing is reasonable and necessary for the surveillance of PPC in Medicare beneficiaries following treatment in response to receipt of a formal request from Carly Elliott in Rapid City, South Dakota. The initial 30-day public comment period began. |
| 7/20/2005 |
End of initial public comment period. |
| 9/16/2005 |
CMS posted the proposed decision memorandum and opened a second 30 day public comment period. |
| 10/16/2005 |
End of second public comment period. |
V. FDA Status
The FDA has approved many CA-125 assays, (Test, Epithelial Ovarian Tumor-Associated Antigen (CA125), Tumor-associated antigen immunological test system, Product Code LTK) by multiple manufacturers. These can be found on the FDA website at URL http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfivd/index.cfm.
VI. General Methodological Principles
Generally, when making national coverage determinations, CMS evaluates relevant clinical evidence to determine whether or not the evidence is of sufficient quality to support a finding that an item or service falling within a benefit category is reasonable and necessary for the diagnosis or treatment of illness or injury or to improve the functioning of a malformed body member. The critical appraisal of the evidence enables us to determine to what degree we are confident that: 1) the specific assessment questions can be answered conclusively; and 2) the intervention will improve net health outcomes for patients. Improved net health outcomes is one of several considerations in determining whether an item or service is reasonable and necessary. In general, features of clinical studies that improve quality and decrease bias include the selection of a clinically relevant cohort, the consistent use of a single good reference standard, and the blinding of readers of the index test, and reference test results.
A detailed account of the methodological principles of study design that agency staff utilizes to assess the relevant literature on a therapeutic or diagnostic item or service for specific conditions can be found in Appendix B.
VII. Evidence
We are providing a summary of the evidence we considered during our review. The Evidence Table is provided as Appendix A.
The principal outcome of interest in assessing the utility of a diagnostic test is its ability to improve health outcomes of persons who are tested. It can be difficult to conclusively prove that changes in ultimate outcomes such as mortality are the result of a diagnostic test. Some disease courses are not significantly altered by the timing of treatment. If therapeutic options are few, the patient may receive the same treatment regimen upon diagnosis regardless of subsequent test results.
Intermediate outcomes may in some cases form an adequate base from which to draw conclusions of benefit. This is predicated on the characteristics of the specific disease in question, and the characteristics of potential treatments for that disease. Improved intermediate outcomes for tests might include earlier treatment when early treatment improves survival or quality of life, the avoidance of unnecessary and possibly harmful treatment, the choice of a more appropriate patient-targeted treatment, and for life-threatening diseases the prognostic information needed for timely referral to support systems such as hospice.
B. Discussion of evidence reviewed
1. Question
Is the quality of evidence adequate to conclude that CA-125 testing improves net health outcomes in persons with primary peritoneal carcinoma?
2. External technology assessments
We did not request an external technology assessment on this issue and are unaware of any assessments that were conducted independently.
3. Internal technology assessments
Literature search methods
Medical literature was identified using MEDLINE, Cochrane Review as well as multiple oncology, surgery, and gynecology textbooks. Peer-reviewed articles written in English were reviewed. Search terms included Primary Papillary Carcinoma, Extraovarian Peritoneal Serous Papillary Carcinoma, Serous Surface Papillary Carcinoma, Multiple Focal Extraovarian Serous Carcinoma, Primary Peritoneal Papillary Serous Adenocarcinoma, Serous Surface Carcinoma of the Peritoneum, Papillary Serous Carcinoma of the Peritoneum, CA-125, CA-125 immunoassay, and CA-125 tumor antigen.
Evidence Review
In reviewing the evidence, a limited number of studies were found. These findings include a number of case studies as well as case reports that have shown an association between the presence of PPC and elevated levels of CA-125 levels, but suffered from low participation rates (Chiou, Sheu, Wang, Chang, 2003; Skates, Troiano, Knapp, 2003; Wright, Horowitz, Rader, 2002, Zissin, Hertz, Altaras et al. 2001; Furukawa Ueda, Takahashi, Higashino Shimura et al. 1999; Tsujimura, Takeda, Terada, Urmoto 1991). The latter two studies had 3 patients and 2 patients respectively. Only a few of the studies have documented the accuracy of this test in this setting (see Table 1). In evaluating the diagnostic accuracy of this test, most studies only reported that “all patients were positive for CA-125”. No reports of sensitivity (Sn), specificity (Sp), positive predictive value (PPV) or negative predictive value (NPV) were given.
Chiou and associates evaluated imaging features of patients with PPC. Preoperative CT images of 11 women with PPC were retrospectively reviewed along with recorded CA-125 levels (Chiou, Sheu, Wang, Chang, 2003). Elevated CA-125 levels were noted in 91% of cases. CT imaging revealed ascites in 82% of cases, peritoneal nodules or masses in 73% of cases, and omental nodules or omental caking (ascites along with a fixed upper abdominal and pelvic mass)in 64% of the cases. The absence of an overt ovarian mass was noted in 64% of the cases. The authors concluded that the presence of diffuse peritoneal disease and the absence of an ovarian mass on CT and an elevated level of serum CA-125 levels are suggestive of PPC.
Skates and collogues published a case study involving a patient with primary papillary along with rising longitudinal levels of CA-125 (Skates, Troiano, Knapp, 2003). They had noted that prior to this finding, there were little data available which showed the association between the two entities. Finding and monitoring of the elevated CA-125 levels was instrumental in the management of this patient, and later led to surgical intervention.
Wright et al. also published a case study of a postmenopausal woman with gynecological symptoms. Colposcopy, endocervical and endometrial curettage pelvic ultrasound were all normal. CT imaging revealed a large omental cake and ascites. An elevated CA-125 level (2,907 IU/mL) was also noted. Exploratory laparotomy confirmed the presence of PPC.
There are few studies which have evaluated whether or not the use of CA-125 has improved health outcomes in patients with PPC. That is because PPC is a rare condition, and few studies have been done on the topic. Kennedy and associates used CA-125 in monitoring progress of treatment for PPC (Kennedy, Markman, Webster et al. 1998). This study involved 38 patients (36 stage IIIC and 2 stage IV), who were treated with platinum-paclitaxel combination, and monitored for overall survival. CA-125 levels were also monitored. The results of the study revealed that 92% of patients experienced at least a 50% reduction in their CA-125 levels and 55% experienced a greater than 90% reduction. The majority of patients received either a complete (68%) or partial (18%) clinical response. Median progression-free survival was 15 months and median overall survival was 40 months.
Rose and Reale also used CA-125 levels in the management of patients with advanced papillary serous carcinoma of the peritoneum (Rose, Reale 1991). The study involved 2 patients who had previous hysterectomies and bilateral salpingo-oophorectomies secondary to endometrial carcinoma. In both cases CA-125 levels rose before the diagnosis of peritoneal carcinoma occurred, and levels corresponded closely to patient response to therapy.
As noted previously, a number of studies have shown the association between the presence of Breast Cancer Gene (BRCA) mutation and the presence of breast and ovarian cancer. Also previously reviewed are studies that have demonstrated a higher frequency of BRCA mutations among patients with PPC. These findings are generalizable to the Medicare population, since most of the case studies as well as case series contain patients that were postmenopausal. Though PPC is rare, studies do indicate this condition tends to occur more frequently in older females.
4. MCAC
A Medicare Coverage Advisory Committee (MCAC) meeting was not convened on this issue.
5. Evidence-based guidelines
We did not find evidence-based guidelines on the use of CA-125 for monitoring therapy related to the presence of primary peritoneal carcinoma.
6. Professional Society Position Statements
We did not find nor did we receive any professional society positions on the use of CA-125 for monitoring therapy related to PPC.
7. Expert Opinion
We have not currently received any expert opinions on the use of CA-125 for monitoring therapy related to PPC.
8. Public Comments
Initial public comments
We received five public comments, all from physicians, during the initial public comment period. They related personal experience in the use of CA-125 for patients with PPC, and supported the addition of PPC to the list of covered diagnoses. No published scientific evidence was submitted.
Second public comments
We received two public comments on the proposed decision memorandum. One commenter discussed a personal history of kidney transplantation and did not mention the proposed decision or ovarian cancer or tumor markers. We believe this comment was submitted in error and will not respond to it here. The second comment was from a member of an oncology group physician practice. He noted that, in their experience, the test can be quite helpful when positive.
VIII. CMS Analysis
National coverage determinations (NCDs) are determinations by the Secretary with respect to whether or not a particular item or service is covered nationally under title XVIII of the Social Security Act § 1869(f)(1)(B). In order to be covered by Medicare, an item or service must fall within one or more benefit categories contained within Part A or Part B, and must not be otherwise excluded from coverage. Moreover, with limited exceptions the expenses incurred for items or services must be “reasonable and necessary for the diagnosis or treatment of illness or injury or to improve the functioning of a malformed body member.” §1862(a)(1)(A). This section presents the agency’s evaluation of the evidence considered and conclusions reached for the assessment question.
Is the quality of evidence adequate to conclude that CA-125 testing improves net health outcomes in persons with primary peritoneal carcinoma?
The evidence reviewed in this decision memorandum demonstrates that there is an association between PPC and elevated levels of CA-125, and this test is being used in the oncology community to monitor response to therapy for this cancer. A number of studies have demonstrated the role of CA-125 in the management of this condition. Primary papillary carcinoma is apparently incurable, and treatment is largely palliative. This condition has a higher prevalence within certain populations, such as patients who test positive for the BRCA mutation. Patients who have had a previous salpingo-oophorectomy are not totally protected from this condition.
The definitive diagnosis of PPC is made by surgically obtaining tissue for pathologic examination. Given its attendant risks, surgical exploration is only undertaken when there is a high suspicion of disease. One can, with caveats, compute high values for sensitivity and positive predictive from the data in the published case reports. Analysis of the data in most of the reports showed that all of the patients with PPC had elevated levels of CA-125 (i.e., sensitivity = 100%). However, these are retrospective analyses of women with known disease, and these values are not generalizable to other populations in which the test might be used, nor are they generalizable to prospective use.
When evaluating effectiveness, case studies are not as vigorous as other research designs. Prospective data on diagnostic accuracy is desirable in determining the applicability of a test. In the studies which were found, the lack of this type of data raises significant concerns.
However, as shown previously, patients who have PPC share many characteristics with patients who have ovarian carcinoma, including elevation of CA-125 levels and response to similar therapy. Both also share similar genetic predisposition. Some authors have reasonably suggested that both conditions are manifestations of the same underlying disease process since the mesothelium of the peritoneum and the germinal epithelium of the ovary arise from the same embryologic origin (Efiom-Ekaha, 2003; Eltabbakh, Piver et al. 1998).
Studies that involve the use of CA-125 for patients with PPC have used this diagnostic test for monitoring therapy related to this condition. Though one study described the use of CA-125 in diagnosing and monitoring patients with PPC (Skates, Troino, Knapp, 2003), all other studies have used this test only for monitoring therapy related to this condition. Physicians use this test to determine the most appropriate course of therapy for patients with PPC. Given the potential risks of currently available therapeutic options (e.g., surgery, chemotherapy) this is a critical decision with significant impact on the patient. This is the same rationale for the use of CA-125 for monitoring therapy for the treatment of ovarian cancer.
As noted in the introduction section, PPC is very similar to epithelial ovarian cancer in terms of its embryonic origin, microscopic appearance, symptoms, pattern of spread, treatment, and prognosis. Medicare currently covers the use CA-125 for monitoring of response to therapy related to ovarian cancer. Based on this strong association between these two conditions, we believe that CA-125 testing is reasonable and necessary for monitoring response to therapy for PPC.
The negotiated clinical diagnostic laboratory NCDs have lists ICD-9-CM codes associated with them. The list entitled “ICD-9-CM Codes Covered by the Medicare Program” contains codes that flow from the narrative indications within each NCD. Thus, in adding monitoring of treatment of PPC to the list of covered indications for tumor antigen by immunoassay CA 125 we are also adding the following ICD-9-CM codes to the list of codes covered by Medicare for this test:
158.8, Malignant neoplasms of specified part of peritoneum
158.9, Malignant neoplasms of peritoneum, unspecified
The requester also suggested ICD-9-CM code 159.8, Malignant neoplasms of other sites of digestive system and intra-abdominal organs. However, we believe that this code is not specific to PPC and inclusion can result in coverage of malignant neoplasms related to digestive and other organs rather than the peritoneum. Consequently, we are not adding ICD-9-CM code 159.8 to the list of covered codes for tumor antigen by immunoassay CA 125 at this time. .
IX. Conclusion
The Centers for Medicare and Medicaid Services (CMS) has determined that there is sufficient evidence to conclude that CA-125 testing is reasonable and necessary for the surveillance of Primary Peritoneal Carcinoma (PPC) in Medicare beneficiaries following treatment.
We will add PPC to the indications described in the narrative NCD for tumor antigen by immunoassay CA-125. We will also propose add ICD-9-CM codes 158.8 and 158.9 to the list of covered codes associated with the NCD for tumor antigen by immunoassay CA-125.
APPENDIX A [PDF, 58KB]
APPENDIX B
General Methodological Principles of Study Design
(Section VI of the Decision Memorandum)
In general, when making national coverage determinations, CMS evaluates relevant clinical evidence to determine whether or not the evidence is of sufficient quality to support a finding that an item or service is reasonable and necessary. The overall objective for the critical appraisal of the evidence is to determine to what degree we are confident that: 1) the specific assessment questions can be answered conclusively; and 2) the intervention will improve net health outcomes for patients.
We divide the assessment of clinical evidence into three stages: 1) the quality of the individual studies; 2) the generalizability of findings from individual studies to the Medicare population; and 3) overarching conclusions that can be drawn from the body of the evidence on the direction and magnitude of the intervention’s potential risks and benefits.
The methodological principles described below represent a broad discussion of the issues we consider when reviewing clinical evidence. However, it should be noted that each coverage determination has its unique methodological aspects.
Assessing Individual Studies
Methodologists have developed criteria to determine weaknesses and strengths of clinical research. Strength of evidence generally refers to: 1) the scientific validity underlying study findings regarding causal relationships between health care interventions and health outcomes; and 2) the reduction of bias. In general, some of the methodological attributes associated with stronger evidence include those listed below:
- Use of randomization (allocation of patients to either intervention or control group) in order to minimize bias.
- Use of contemporaneous control groups (rather than historical controls) in order to ensure comparability between the intervention and control groups.
- Prospective (rather than retrospective) studies to ensure a more thorough and systematical assessment of factors related to outcomes.
- Larger sample sizes in studies to demonstrate both statistically significant as well as clinically significant outcomes that can be extrapolated to the Medicare population. Sample size should be large enough to make chance an unlikely explanation for what was found.
- Masking (blinding) to ensure patients and investigators do not know to which group patients were assigned (intervention or control). This is important especially in subjective outcomes, such as pain or quality of life, where enthusiasm and psychological factors may lead to an improved perceived outcome by either the patient or assessor.
Regardless of whether the design of a study is a randomized controlled trial, a non-randomized controlled trial, a cohort study or a case-control study, the primary criterion for methodological strength or quality is the extent to which differences between intervention and control groups can be attributed to the intervention studied. This is known as internal validity. Various types of bias can undermine internal validity. These include:
- Different characteristics between patients participating and those theoretically eligible for study but not participating (selection bias).
- Co-interventions or provision of care apart from the intervention under evaluation (performance bias).
- Differential assessment of outcome (detection bias).
- Occurrence and reporting of patients who do not complete the study (attrition bias).
In principle, rankings of research design have been based on the ability of each study design category to minimize these biases. A randomized controlled trial minimizes systematic bias (in theory) by selecting a sample of participants from a particular population and allocating them randomly to the intervention and control groups. Thus, in general, randomized controlled studies have been typically assigned the greatest strength, followed by non-randomized clinical trials and controlled observational studies. The design, conduct and analysis of trials are important factors as well. For example, a well designed and conducted observational study with a large sample size may provide stronger evidence than a poorly designed and conducted randomized controlled trial with a small sample size. The following is a representative list of study designs (some of which have alternative names) ranked from most to least methodologically rigorous in their potential ability to minimize systematic bias:
- Randomized controlled trials
- Non-randomized controlled trials
- Prospective cohort studies
- Retrospective case control studies
- Cross-sectional studies
- Surveillance studies (e.g., using registries or surveys)
- Consecutive case series
- Single case reports
When there are merely associations but not causal relationships between a study’s variables and outcomes, it is important not to draw causal inferences. Confounding refers to independent variables that systematically vary with the causal variable. This distorts measurement of the outcome of interest because its effect size is mixed with the effects of other extraneous factors. For observational, and in some cases randomized controlled trials, the method in which confounding factors are handled (either through stratification or appropriate statistical modeling) are of particular concern. For example, in order to interpret and generalize conclusions to our population of Medicare patients, it may be necessary for studies to match or stratify their intervention and control groups by patient age or co-morbidities.
Methodological strength is, therefore, a multidimensional concept that relates to the design, implementation and analysis of a clinical study. In addition, thorough documentation of the conduct of the research, particularly study selection criteria, rate of attrition and process for data collection, is essential for CMS to adequately assess and consider the evidence.
Generalizability of Clinical Evidence to the Medicare Population
The applicability of the results of a study to other populations, settings, treatment regimens and outcomes assessed is known as external validity. Even well-designed and well-conducted trials may not supply the evidence needed if the results of a study are not applicable to the Medicare population. Evidence that provides accurate information about a population or setting not well represented in the Medicare program would be considered but would suffer from limited generalizability.
The extent to which the results of a trial are applicable to other circumstances is often a matter of judgment that depends on specific study characteristics, primarily the patient population studied (age, sex, severity of disease and presence of co-morbidities) and the care setting (primary to tertiary level of care, as well as the experience and specialization of the care provider). Additional relevant variables are treatment regimens (dosage, timing and route of administration), co-interventions or concomitant therapies, and type of outcome and length of follow-up.
The level of care and the experience of the providers in the study are other crucial elements in assessing a study’s external validity. Trial participants in an academic medical center may receive more or different attention than is typically available in non-tertiary settings. For example, an investigator’s lengthy and detailed explanations of the potential benefits of the intervention and/or the use of new equipment provided to the academic center by the study sponsor may raise doubts about the applicability of study findings to community practice.
Given the evidence available in the research literature, some degree of generalization about an intervention’s potential benefits and harms is invariably required in making coverage determinations for the Medicare population. Conditions that assist us in making reasonable generalizations are biologic plausibility, similarities between the populations studied and Medicare patients (age, sex, ethnicity and clinical presentation) and similarities of the intervention studied to those that would be routinely available in community practice.
A study’s selected outcomes are an important consideration in generalizing available clinical evidence to Medicare coverage determinations. One of the goals of our determination process is to assess net health outcomes. These outcomes include resultant risks and benefits such as increased or decreased morbidity and mortality. In order to make this determination, it is often necessary to evaluate whether the strength of the evidence is adequate to draw conclusions about the direction and magnitude of each individual outcome relevant to the intervention under study. In addition, it is important that an intervention’s benefits are clinically significant and durable, rather than marginal or short-lived. Generally, an intervention is not reasonable and necessary if its risks outweigh its benefits.
If key health outcomes have not been studied or the direction of clinical effect is inconclusive, we may also evaluate the strength and adequacy of indirect evidence linking intermediate or surrogate outcomes to our outcomes of interest.
Assessing the Relative Magnitude of Risks and Benefits
An intervention is not reasonable and necessary if its risks outweigh its benefits. Net health outcomes is one of several considerations in determining whether an item or service is reasonable and necessary. CMS places greater emphasis on health outcomes actually experienced by patients, such as quality of life, functional status, duration of disability, morbidity and mortality, and less emphasis on outcomes that patients do not directly experience, such as intermediate outcomes, surrogate outcomes, and laboratory or radiographic responses. The direction, magnitude, and consistency of the risks and benefits across studies are also important considerations. Based on the analysis of the strength of the evidence, CMS assesses the relative magnitude of an intervention or technology’s benefits and risk of harm to Medicare beneficiaries.
