Autologous Stem Cell Transplantation (AuSCT) for Amyloidosis

The Centers for Medicare and Medicaid Services (CMS) proposes the following:

The evidence presented in this decision memorandum is adequate and suggests that when recognized clinical risk factors are employed to select patients for transplantation, high dose melphalan and autologous stem cell transplantation (HDM/AuSCT) can provide a net health benefit for Medicare beneficiaries of any age group with primary AL amyloidosis. HDM/AuSCT is reasonable and necessary for patients with primary AL Amyloidosis who meet the following criteria:

• amyloid deposition in 2 or fewer organs,
• serum creatinine of 2.0 mg/dL or less, and
• cardiac left ventricular ejection fraction (EF) of 55% or greater.

CMS commends those practitioners who enroll their HDM/AuSCT patients with primary AL amyloidosis in a database (registry). CMS strongly recommends that the sponsors and principal investigators of future HDM/AuSCT trials engage an independent, reputable research center to pool the entire database from each of their respective trials and conduct analyses to identify patient selection, procedure related issues, and other research questions. CMS believes that for optimal patient care, a registry should include criteria that ensure:

1. Hospitals and providers are certified as competent in HDM/AuSCT.
2. Participating hospitals and providers report data on all patients undergoing HDM/AuSCT.
3. Hospitals and providers who do not comply with the data collection requirements are removed from the system.
4. The data set includes elements with the following characteristics:
   • Baseline patient characteristics,
   • Facility and provider characteristics,
   • Extent of disease progression, and
   • Long-term patient outcomes.
5. Specific hypotheses are addressed.

CMS is requesting public comments on this proposed decision memorandum pursuant to Section 731 of the Medicare Modernization Act. After considering the public comments, we will issue a final decision memorandum.

To:		Administrative File: CAG-00050R 
 
From:	Steve E. Phurrough, MD, MPA 
		Director, Coverage and Analysis Group 
 
		Marcel Salive, MD, MPH 
		Director, Division of Medical and Surgical Services 
 
		Samantha Richardson 
		Analyst, Division of Medical and Surgical Services 
 
		Susan Harrison 
		Analyst, Division of Medical and Surgical Services 
 
		Lori Paserchia, MD 
		Medical Officer, Division of Medical and Surgical Services 
 
Subject:	Proposed Coverage Decision Memorandum for Reconsideration Request for Autologous Stem Cell  
		Transplantation (AuSCT) for primary amyloid light chain (AL) Amyloidosis 
 
Date:		December 15, 2004

I.      Proposed Decision

The Centers for Medicare and Medicaid Services (CMS) proposes the following:

The evidence presented in this decision memorandum is adequate and suggests that when recognized clinical risk factors are employed to select patients for transplantation, high dose melphalan and autologous stem cell transplantation (HDM/AuSCT) can provide a net health benefit for Medicare beneficiaries of any age group with primary AL amyloidosis. HDM/AuSCT is reasonable and necessary for patients with primary AL Amyloidosis who meet the following criteria:

• amyloid deposition in 2 or fewer organs,
• serum creatinine of 2.0 mg/dL or less, and
• cardiac left ventricular ejection fraction (EF) of 55% or greater.

CMS commends those practitioners who enroll their HDM/AuSCT patients with primary AL amyloidosis in a database (registry). CMS strongly recommends that the sponsors and principal investigators of future HDM/AuSCT trials engage an independent, reputable research center to pool the entire database from each of their respective trials and conduct analyses to identify patient selection, procedure related issues, and other research questions. CMS believes that for optimal patient care, a registry should include criteria that ensure:

1. Hospitals and providers are certified as competent in HDM/AuSCT.
2. Participating hospitals and providers report data on all patients undergoing HDM/AuSCT.
3. Hospitals and providers who do not comply with the data collection requirements are removed from the system.
4. The data set includes elements with the following characteristics:
   • Baseline patient characteristics,
   • Facility and provider characteristics,
   • Extent of disease progression, and
   • Long-term patient outcomes.
5. Specific hypotheses are addressed.

CMS is requesting public comments on this proposed decision memorandum pursuant to Section 731 of the Medicare Modernization Act. After considering the public comments, we will issue a final decision memorandum.

II.     Background

On June 15, 2004, CMS began a reconsideration of the national coverage determination (NCD) on high dose melphalan and autologous stem cell transplantation (HDM/AuSCT) for primary amyloid light chain (AL) amyloidosis.

AL amyloidosis is a blood disorder, associated with an abnormal condition of blood cells, in which extra-cellular insoluble protein (amyloid) fibrils accumulate in various tissues and organs throughout the body. These amyloid fibrils are formed by an errant group of plasma cells in the bone marrow and consist of abnormal immunoglobulin (Ig) light chain proteins. Except for those within the central nervous system, amyloid fibrils can affect any major organ in the body.

AL amyloidosis is a rare disease; only 1200 to 3200 new cases are reported each year in the United States.¹ Two thirds of patients with primary AL amyloidosis are male and less than 5% of patients are under 40 years of age. Both the causes and origins of primary AL amyloidosis along with the mechanism of amyloid deposition remain poorly understood.

The organs most often affected in primary AL amyloidosis include the heart, kidneys, peripheral nervous system, and gastrointestinal tract. Amyloid deposits in these organs can cause shortness of breath, fatigue, edema (swelling of ankles and legs), dizziness upon standing, a feeling of fullness in the stomach (especially after eating), diarrhea, weight loss, enlarged tongue, numbness of the legs and arms, and protein in the urine.

The clinical course of primary AL amyloidosis is usually associated with rapid disease progression, involvement of multiple organ systems, and short survival periods. Extensive organ system impairment, secondary to amyloid deposits, often results in death. Due to the rapid progression of primary AL amyloidosis, median survival from diagnosis is between one to two years, depending on which organ systems are affected. Patients with cardiac amyloid involvement have an even poorer prognosis with a median survival of less than six months, thus accounting for almost one half of deaths from primary AL amyloidosis.¹

Because of its similarity to multiple myeloma, another plasma cell dyscrasia where plasma cells produce abnormal protein deposits, the treatment of primary AL amyloidosis has followed the same general path. The early treatment of primary AL amyloidosis largely focused around oral chemotherapy regimens. Patients were treated with standard doses of drugs such as melphalan, prednisone, and/or colchicine. Research suggested that multiple drug regimens can produce better response rates then single drug regimens. However, response rates to standard chemotherapy are quite low. For example, many patients do not live long enough to receive enough cycles of melphalan to actually benefit from treatment.

The poor response rates experienced with only chemotherapy prompted the use of HDM/AuSCT. HDM/AuSCT consists of a number of stages. The first stage is called mobilization where the patient is given a granulocyte colony-stimulating factor (G-CSF) or a granulocyte-macrophage colony-stimulating factor (GM-CSF) to stimulate the release of the stem cells from storage sites within the body prior to harvesting via leukapheresis (or bone marrow biopsy). The next stage is called conditioning where the patient is given a high dose of a chemotherapy agent, typically melphalan. In the final stage the harvested stem cells are administered along with supportive medical care.

Early on, patients with AL amyloidosis who underwent HDM/AuSCT experienced a reduction in amyloid-related outcomes; however, the transplantation-related mortality was higher compared to patients with multiple myeloma. The presence and severity of amyloid-associated organ dysfunction, which can be extensive in patients with AL amyloidosis and minimal to nonexistent in patients with multiple myeloma, were determined to be the reason for the mortality discrepancy. Subsequent use of HDM/AuSCT in medical practice and studies in research trials permitted the identification of risk factors that guide the selection of patients who are most appropriate to receive HDM/AuSCT. The main factors found to be associated with a significant increased risk of morbidity or mortality include the baseline cardiac and renal status of the patient and the extent of amyloid organ involvement (Comenzo, 2002). Cardiac ejection fraction (EF) was the clinical parameter most commonly used as an inclusion criterion in the clinical trials reviewed for this decision memorandum while the ventricular septal thickness was used less frequently. The serum creatinine level was most commonly used as an inclusion criterion in trials to identify the renal status.

III.     History of Medicare Coverage

CMS has determined that autologous stem cell transplantation falls within the benefit category of inpatient hospital services under Part A and physicians’ services under Part B. See §1812 (a)(1); §1832; §1861(s)(2); §1861(b).

On January 14, 2000, HCFA (now CMS) issued the first decision memorandum (CAG-00050N) which determined that a sufficient body of evidence does not exist to justify a national coverage decision in favor of AuSCT for patients with primary AL amyloidosis. CMS stated that we would reconsider if additional evidence was submitted in the future.

Section 110.8.1 of the National Coverage Determination (NCD) Manual states, in pertinent part, that stem cell transplantation is a process in which stem cells are harvested from either a patient's or donor’s bone marrow or peripheral blood for intravenous infusion. The transplant can be used to effect hematopoietic reconstitution following severely myelotoxic doses of chemotherapy and/or radiotherapy used to treat various malignancies.

Autologous stem cell transplantation is considered reasonable and necessary under §1862(a)(1)(A) of the Act for the following conditions and is covered under Medicare for patients with:

• Acute leukemia in remission who have a high probability of relapse and who have no human leucocyte antigens (HLA)-matched;
• Resistant non-Hodgkin's or those presenting with poor prognostic features following an initial response;
• Recurrent or refractory neuroblastoma; or
• Advanced Hodgkin's disease who have failed conventional therapy and have no HLA-matched donor.

Effective October 1, 2000, single AuSCT is only covered for Durie-Salmon Stage II or III patients that fit the following requirement:

• Newly diagnosed or responsive multiple myeloma. This includes those patients with previously untreated disease, those with at least a partial response to prior chemotherapy (defined as a 50 percent decrease either in measurable paraprotein [serum and/or urine] or in bone marrow infiltration, sustained for at least 1 month), and those in responsive relapse; and
• Adequate cardiac, renal, pulmonary, and hepatic function.

NOTE: Tandem transplantation for multiple myeloma remains noncovered.

Insufficient data exist to establish definite conclusions regarding the efficacy of autologous stem cell transplantation for the following conditions:

• Acute leukemia not in remission;
• Chronic granulocytic leukemia;
• Solid tumors (other than neuroblastoma);
• Up to October 1, 2000, multiple myeloma;
• Tandem transplantation (multiple rounds of autologous stem cell transplantation) for patients with multiple myeloma;
• Effective October 1, 2000, non-primary (AL) amyloidosis; and
• Effective October 1, 2000, primary (AL) amyloidosis for Medicare beneficiaries age 64 or older.

In these cases, autologous stem cell transplantation is not considered reasonable and necessary within the meaning of §1862(a)(1)(A) of the Act and is not covered under Medicare.

IV.     Timeline of Recent Activities

On June 15, 2004 CMS accepted Legacy Good Samaritan Hospital & Medical Center’s formal request and initiated review.

On July 26, 2004, Initial public comments were posted to the tracking sheet available electronically at: http://www.cms.hhs.gov/coverage/download/id126a.pdf [PDF, 15KB].

V.     FDA Status

Autologous stem cell transplantation is regulated under 21 C.F.R. §1271.3. Section 1271.3(a) defines the term autologous use as “the implantation, transportation, infusion, or transfer of human cells or tissue back into the individual from whom the cells or tissue were recovered.”

Section 1271.3(d)(2) defines human cells, tissues, or cellular or tissue-based products (HCT/P’s) as “ articles containing or consisting of human cells or tissues that are intended for implantation, transplantation, infusion or transfer into a human recipient. Examples of HCT/P’s include, but are not limited to, bone, ligament, skin, dura matter, heart valve, cornea, hematopoietic stem cells derived from peripheral and cord blood, manipulated autologous chondrocytes, epithelial cells on a synthetic matrix, and semen or other reproductive tissue.”

VI.     General Methodological Principles

When making national coverage decisions, CMS evaluates relevant clinical evidence to determine whether or not the evidence is of sufficient quality to support a finding that a service or item is reasonable and necessary for the diagnosis or treatment of illness or injury or to improve the functioning of a malformed body member. The overall objective for the critical appraisal of the evidence is to determine to what degree we are confident that: 1) specific clinical questions relevant to the coverage request can be answered conclusively; and 2) the extent to which we are confident that the intervention will improve net health outcomes for patients. Evidence may consist of external technology assessments, internal review of published and unpublished studies, recommendations from the Medicare Coverage Advisory Committee, evidence-based guidelines, professional society position statements, expert opinion, and public comments. A fully detailed account of “General Methodological Principles of Study Design” that CMS staff utilizes to assess the relevant literature on the therapeutic or diagnostic item or service for specific conditions follows the conclusion and references for this decision memorandum (see Appendix A).

VII.     Evidence

A.    Introduction

In order to appraise the net health outcomes of HDM/AuSCT for patients with primary AL amyloidosis in comparison with standard medical therapy consisting of melphalan and prednisone and to identify any relevant patient and facility selection criteria, CMS sought to address the following question:

Survival, both short-term and long-term, is the primary outcome of interest. In the articles reviewed, short term survival was also referred to as treatment-related mortality, which was defined by the various study investigators as death within 90 to 100 days of HDM/AuSCT.

Hematologic response was also an outcome of interest. In clinical studies, a complete hematologic response was typically defined as a total absence of abnormal plasma cells on bone marrow examination or a lack of detection of a monoclonal gammopathy in the blood and urine.

The goal of one of the clinical studies reviewed for this decision memorandum (Seldin, 2004) was to measure changes in quality of life (QoL). This outcome was assessed using a validated instrument called the Medical Outcomes Study 36-item Short Form General Health Survey (SF-36). The SF-36 includes 36 questions to assess 8 scales of health status that are combined into 2 types of composite scores: a physical component and a mental component. The composite scores are standardized; a score of 50 corresponds to the average for the US population (standard deviation of 10). A score change of 2 to 3 units is considered to be significant.

In Seldin, 2004, the authors noted that QoL instruments exist that are specific for patients undergoing stem cell transplantation. Since patients with primary AL amyloidosis can have a wide range of symptoms due to multi-organ involvement, the SF-36 was selected because it is validated for use in a variety of diseases and patient populations.

B.    Discussion of evidence reviewed

The evidence reviewed includes summaries of CMS’s 2000 AuSCT decision memorandum, CMS’s internal technology assessment of new or reconsidered evidence, as well as professional society position statements and expert opinion.

1.  Prior CMS Decision Memorandums for AuSCT

2.   External technology assessments

3. Internal technology assessment

4.  MCAC

5.  Evidence-based guidelines

6.  Professional Society Position Statements

7. Expert Opinion

8.   Public Comments

VIII.      CMS Analysis

National coverage determinations (NCDs) are determinations by the Secretary with respect to whether or not a particular item or service is covered nationally under title XVIII of the Social Security Act § 1869(f)(1)(B). In order to be covered by Medicare, an item or service must fall within one or more benefit categories contained within Part A or Part B, and must not be otherwise excluded from coverage. Moreover, with limited exceptions, the expenses incurred for items or services must be “reasonable and necessary for the diagnosis or treatment of illness or injury or to improve the functioning of a malformed body member.” § 1862(a) (1) (A).

The quality of the studies of HDM/AuSCT conducted in patients with primary AL amyloidosis since 2000 continues to be less than robust for the Medicare elderly population. Comparative evidence from randomized, controlled trials is not available. While the majority of the studies were prospectively conducted, the majority also were non-randomized. None of the studies were blinded. A control group, when used, was either case-matched (Dispenzieri, 2001; Dispenzieri, 2004) or inappropriate (Sanchorawala, 2003; Seldin, 2004). The requestors, and a number of article authors, state that a randomized, controlled trial will probably never be conducted due to the rarity of the disease and the lack of insurance coverage for patients of Medicare age.

There is some evidence on net health outcomes for patients older than 63 years. Many studies either ultimately did not enroll many patients older than 70 years (Casserly, 2003; Dispenzieri, 2004; Gertz, 2000; Gertz, 2002; Seldin, 2004), intentionally limited enrollment to patients 70 years old or less (Dispenzieri, 2001), or did not specifically note the number of patients older than 63 years (Dember, 2001; Sanchorawala, 2003; Skinner, 2004). The age-related analysis of outcomes performed by the requestor for patients 65 years and older, by our request, provides the most evidence. The majority of patients, however, were 65 to 70 years old; only 19% of the 61 patients were older than 70 years and only 1 was 80 years old.

While most of the studies did not separately report results for patients older than 63 years, the evidence does not suggest worse survival in these patients. The Boston University data from 61 patients who were 65 years and older showed a treatment-related mortality rate of 10%, which compares favorably to the 13% treatment-related mortality rate reported for all 701 patients in Skinner, 2004. A similar overall treatment-related mortality rate was reported in Dember, 2001 (9%), Gertz, 2002 (14%), Casserly, 2003 (13%), and Dispenzieri, 2004 (13%). Dispenzieri, 2004 noted that the survival of the 4 case-matched pairs of patients 65 years old or older was slightly better for the transplant patients than for the non-transplant patients. Compared to the wide range of treatment-related mortality noted in the 2000 decision memorandum, this mortality range across studies is narrower.

Skinner, 2004 reported a median survival of 4.9 years for the subset of patients who were 65 years old and older. This finding was not statistically significantly different from the 4.6 year median survival of the younger patients. Both results contrast with the overall median survival for patients treated with standard chemotherapy of 1-1.5 years (Gertz, 2000). Hence, HDM/AuSCT for patients 63 years of age and older appears to provide a longer survival compared to the currently available treatment.

The univariate and multivariate analyses reported by Dispenzieri, 2001 provide supportive evidence since they found no statistically significant survival difference among the age groups (≥50 years, 51-60 years, and 61-70 years). The multivariate analysis reported in Gertz, 2002 had similar results. Serum creatinine and the number of involved organs, but not age, were found to be independently associated with mortality. The sole study to hint at a survival difference based on the age group was Dember, 2001. Here, a comparison of the survivors and non-survivors at 1-year showed that the survivors were statistically significantly younger.

In contrast to age, the evidence does highlight the relationships between extent of organ system involvement and HDM/AuSCT-related mortality, and between baseline renal status and mortality. Results from analyses performed in Dember, 2001, Dispenzieri, 2001, and Gertz, 2002 point to greater than 2 organ involvement with amyloidosis as strongly associated with mortality, and hence a predictor of a poor prognosis. The Gertz, 2002 analysis found that baseline serum creatinine is independently associated with mortality. These findings serve to support the inclusion and exclusion criteria commonly used to select patients for HDM/AuSCT.

The majority of the studies reviewed for this decision memorandum were conducted in the Stem Cell Transplant Program and the Amyloid Treatment and Research Program at the Boston University School of Medicine in Boston, MA or in the Division of Hematology and Internal Medicine at the Mayo Clinic in Rochester, MN. Additional institutions included the Bone Marrow Transplantation and Leukemia Program at the Washington University School of Medicine in St. Louis, MO; the National Amyloidosis Centre at the Royal Free and University College Medical School in London, UK; the Medical College of Wisconsin in Milwaukee, WI; and the Hematology Service of the Department of Medicine at the Memorial Sloan-Kettering Cancer Center in NY, NY.

The preponderance of major academic institutions in the medical literature signifies the diversity and magnitude of resources necessary to appropriately care for patients with primary AL amyloidosis who undergo treatment with HDM/AuSCT. This point is highlighted by the creation in 1996 of the Foundation for the Accreditation of Cellular Therapy (FACT; www.factwebsite.org) to establish and maintain standards for the safe collection, processing, and administration of hematopoietic cells. It is unlikely that the evidence and results reviewed in this decision memorandum can be readily generalized to facilities with significantly less resources than that typically found at an institution with a hematopoietic transplantation program.

Finally, we desire to ensure that HDM/AuSCT only occurs in those patients who are most likely to benefit and that the procedures are done only by competent providers in facilities with a history of good outcomes and a quality assessment/improvement program to identify providers with poor outcomes and other areas for improvement. As mentioned above, we are concerned that the available evidence does not allow providers to target this therapy to patients who will clearly derive benefit. In order to provide maximum protection to our beneficiaries, CMS will require that reimbursement for HDM/AuSCT occur only if the beneficiary with AL amyloidosis receiving the therapy enrolled in either a FDA approved clinical trial or a qualifying national database (registry).

The submission of surveillance data on patients receiving HDM/AuSCT for AL amyloidosis to a national registry is reasonable and necessary to assure patient safety and protection. Data from the registry will help identify the appropriate patients to receive HDM/AuSCT for AL amyloidosis and help reduce the incidence of inappropriate therapy. These patient protections and safeguards would only be available to the extent that registry data can be made available in some form to providers and practitioners to inform their decisions, monitor performance quality, benchmark and identify best practices. We do not set forth precise standards for data sharing practices. But we do require that the collection and distribution of health information be consistent with the Standards for Privacy of Individually Identifiable Health Information.³

The national registry for HDM/AuSCT for AL amyloidosis must meet several operational criteria for facility certification, assessment and data completeness. The national registry must include criteria to ensure that hospitals and providers are certified as competent in the AuSCT procedure. Participating hospitals and providers must be required to report data on all patients undergoing HDM/AuSCT for AL amyloidosis. Also, hospitals or providers who do not comply with the data collection requirements must be removed from the system. Complete prospective systematic data collection will ensure the registry’s ability to achieve the objectives. Data elements in the national registry should address baseline patient characteristics, facility and provider characteristics, extent of disease progression, and long-term patient outcomes. After the appropriate objectives and hypotheses are developed, the minimum data necessary to answer the hypotheses can be identified, and simple processes for data collection and submission developed.

The registry must be designed to address specific hypotheses, some of which may come from the pooled data analysis of the studies reviewed above. Potential registry hypotheses can center on patient safety-related issues such as:

• the use of risk-based criteria for patient selection as suggested in the Comenzo, 2002 article, and
• the greater degree of G-CSF or GM-CSF associated toxicity seen during the mobilization stage of HDM/AuSCT.

IX.      Proposed Decision

The Centers for Medicare and Medicaid Services (CMS) proposes the following:

The evidence presented in this decision memorandum is adequate and suggests that when recognized clinical risk factors are employed to select patients for transplantation, HDM/AuSCT can provide a net health benefit for Medicare beneficiaries of any age group with primary AL amyloidosis. Based upon the above findings, HDM/AuSCT is reasonable and necessary for patients with primary AL Amyloidosis who meet the following criteria:

• amyloid deposition in 2 or fewer organs,
• serum creatinine of 2.0 mg/dL or less, and
• cardiac left ventricular EF of 55% or greater.

CMS commends those practitioners who enroll their HDM/AuSCT patients with primary AL amyloidosis in a database (registry). CMS strongly recommends that the sponsors and principal investigators of future HDM/AuSCT trials engage an independent, reputable research center to pool the entire database from each of their respective trials and conduct analyses to identify patient selection, procedure related issues, and other research questions. CMS believes that for optimal patient care, a registry should include criteria that ensure:

1. Hospitals and providers are certified as competent in HDM/AuSCT.
2. Participating hospitals and providers report data on all patients undergoing HDM/AuSCT.
3. Hospitals and providers who do not comply with the data collection requirements are removed from the system.
4. The data set includes elements with the following characteristics:
   • Baseline patient characteristics,
   • Facility and provider characteristics,
   • Extent of disease progression, and
   • Long-term patient outcomes.
5. Specific hypotheses are addressed.

CMS is requesting public comments on this proposed decision memorandum pursuant to Section 731 of the Medicare Modernization Act. After considering the public comments, we will issue a final decision memorandum.

Appendix A: General Methodological Principles

We divide the assessment of clinical evidence into three stages: 1) the quality of the individual studies; 2) the relevance of findings from individual studies to the Medicare population; and 3) overarching conclusions that can be drawn from the body of the evidence on the direction and magnitude of the intervention’s risks and benefits.

The issues presented here represent a broad discussion of the issues we consider when reviewing clinical evidence. However, it should be noted that each coverage determination has unique methodological aspects.

1. Assessing Individual Studies

Methodologists have developed criteria to determine weaknesses and strengths of clinical research. Strength of evidence generally refers to: 1) the scientific validity underlying study findings regarding causal relationships between health care interventions and health outcomes; and 2) the reduction of bias. In general, some of the methodological attributes associated with stronger evidence include those listed below:

• Use of randomization (allocation of patients to either intervention or control group) in order to minimize bias;
• Use of contemporaneous control groups (rather than historical controls) in order to ensure comparability between the intervention and control groups;
• Prospective (rather than retrospective) studies to ensure a more thorough and systematical assessment of factors related to outcomes;
• Larger sample sizes in studies to help ensure adequate numbers of patients are enrolled to demonstrate both statistically significant as well as clinically significant outcomes that can be extrapolated to the Medicare population. Sample size should be large enough to make chance an unlikely explanation for what was found;
• Masking (blinding) to ensure patients and investigators do not know to which group patients were assigned (intervention or control). This is important especially in subjective outcomes, such as pain or quality of life, where enthusiasm and psychological factors may lead to an improved perceived outcome by either the patient or assessor.

Regardless of whether the design of a study is a randomized controlled trial, a non-randomized controlled trial, a cohort study or a case-control study, the primary criterion for methodological strength or quality is the extent to which differences between intervention and control groups can be attributed to the intervention studied. This is known as internal validity. Various types of bias can undermine internal validity. These include:

• Different characteristics between patients participating and those theoretically eligible for study but not participating (selection bias);
• Co-interventions or provision of care apart from the intervention under evaluation (confounding);
• Differential assessment of outcome (detection bias);
• Occurrence and reporting of patients who do not complete the study (attrition bias).

In principle, rankings of research design have been based on the ability of each study design category to minimize these biases. A randomized controlled trial minimizes systematic bias (in theory) by selecting a sample of participants from a particular population and allocating them randomly to the intervention and control groups. Thus, randomized controlled studies have been typically assigned the greatest strength, followed by non-randomized clinical trials and controlled observational studies. The following is a representative list of study designs (some of which have alternative names) ranked from most to least methodologically rigorous in their potential ability to minimize systematic bias:

• Randomized controlled trials;
• Non-randomized controlled trials;
• Prospective cohort studies;
• Retrospective case control studies;
• Cross-sectional studies;
• Surveillance studies (e.g., using registries or surveys);
• Consecutive case series;
• Single case reports.

When there are merely associations but not causal relationships between a study’s variables and outcomes, it is important not to draw causal inferences. Confounding refers to independent variables that systematically vary with the causal variable. This distorts measurement of the outcome of interest because its effect size is mixed with the effects of other extraneous factors. For observational, and in some cases randomized controlled trials, the method in which confounding factors are handled (either through stratification or appropriate statistical modeling) are of particular concern. For example, in order to interpret and generalize conclusions to our population of Medicare patients, it may be necessary for studies to match or stratify their intervention and control groups by patient age or co-morbidities.

Methodological strength is, therefore, a multidimensional concept that relates to the design, implementation and analysis of a clinical study. In addition, thorough documentation of the conduct of the research, particularly study’s selection criteria, rate of attrition and process for data collection, is essential for CMS to adequately assess the evidence.

2. Generalizability of Clinical Evidence to the Medicare Population

The applicability of the results of a study to other populations, settings, treatment regimens and outcomes assessed is known as external validity. Even well-designed and well-conducted trials may not supply the evidence needed if the results of a study are not applicable to the Medicare population. Evidence that provides accurate information about a population or setting not well represented in the Medicare program would be considered but would suffer from limited generalizability.

The extent to which the results of a trial are applicable to other circumstances is often a matter of judgment that depends on specific study characteristics, primarily the patient population studied (age, sex, severity of disease and presence of co-morbidities) and the care setting (primary to tertiary level of care, as well as the experience and specialization of the care provider). Additional relevant variables are treatment regimens (dosage, timing and route of administration), co-interventions or concomitant therapies, and type of outcome and length of follow-up.

The level of care and the experience of the providers in the study are other crucial elements in assessing a study’s external validity. Trial participants in an academic medical center may receive more or different attention than is typically available in non-tertiary settings. For example, an investigator’s lengthy and detailed explanations of the potential benefits of the intervention and/or the use of new equipment provided to the academic center by the study sponsor may raise doubts about the applicability of study findings to community practice.

Given the evidence available in the research literature, some degree of generalization about an intervention’s potential benefits and harms is invariably required in making coverage decisions for the Medicare population. Conditions that assist us in making reasonable generalizations are biologic plausibility, similarities between the populations studied and Medicare patients (age, sex, ethnicity and clinical presentation) and similarities of the intervention studied to those that would be routinely available in community practice.

A study’s selected outcomes are an important consideration in generalizing available clinical evidence to Medicare coverage determinations. One of the goals of our determination process is to assess net health outcomes, and we are interested in the results of changed patient management not just altered management. These outcomes include resultant risks and benefits such as increased or decreased morbidity and mortality. In order to make this determination, it is often necessary to evaluate whether the strength of the evidence is adequate to draw conclusions about the direction and magnitude of each individual outcome relevant to the intervention under study. In addition, it is important that an intervention’s benefits are clinically significant and durable, rather than marginal or short-lived.

If key health outcomes have not been studied or the direction of clinical effect is inconclusive, we may also evaluate the strength and adequacy of indirect evidence linking intermediate or surrogate outcomes to our outcomes of interest.

3. Assessing the Relative Magnitude of Risks and Benefits

An intervention is not reasonable and necessary if its risks outweigh its benefits. Among other things, CMS evaluates whether reported benefits translate into improved net health outcomes. The direction, magnitude and consistency of the risks and benefits across studies are important considerations. Based on the analysis of the strength of the evidence, CMS assesses whether an intervention or technology's benefits to Medicare beneficiaries outweigh its harms.

Appendix B: Evidence Table [PDF, 219KB]

¹ http://amyloidosis.org/treatment/primary.asp#primary

² http://www.cms.hhs.gov/mcd/viewdecisionmemo.asp?id=9

³ Privacy Rule – Health Insurance Portability and Accountability Act of 1996. (http://www.os.dhhs.gov/ocr/privacysummary.pdf)

Blum W, et al. Primary amyloidosis (AL) patients with significant organ dysfunction treated with conventional chemotherapy followed by single-dose total body irradiation (TBI) and autologous peripheral blood stem cell transplant (PBSC) then IFN-maintenance: tolerance and efficacy. Blood 2001;98(11) abst. 2862:684a.

Casserly LF, et al. High-dose intravenous melphalan with autologous stem cell transplantation in AL amyloidosis-associated end-stage renal disease. Kidney International 2003;63:1051-1057.

Comenzo R, Gerts MA. Autologous stem cell transplantation for primary systemic amyloidosis. Blood 2002;99(12):4276-4282.

Dember LM, et al. Effect of dose-intensive intravenous melphalan and autologous blood stem-cell transplantation on AL amyloidosis-associated renal disease. Ann Intern Med 2001;134:746-753.

Dispenzieri A, et al. Eligibility for hematopoietic stem-cell transplantation for primary systemic amyloidosis is a favorable prognostic factor for survival. J Clin Oncol 2001;19:3350-3356.

Dispenzieri A, et al. Survival in patients with primary systemic amyloidosis and raised serum cardiac troponins. Lancet 2003;361:1787-1789.

Dispenzieri A, et al. Superior survival in primary systemic amyloidosis patients undergoing peripheral blood stem cell transplantation: a case-control study. Blood 2004;103:3960-3963.

Gertz MA, et al. Blood stem cell transplantation as therapy for primary systemic amyloidosis (AL). Bone Marrow Transplantation 2000;26:963-969.

Gertz MA, et al. Stem cell transplantation for the management of primary systemic amyloidosis. Am J Med 2002;113:549-555.

Hayes-Lattin BM, et al. Toxic megacolon: a life-threatening complication of high-dose therapy and autologous stem cell transplantation among patients with AL amyloidosis. Bone Marrow Transplant 2002;30:279-285.

Kumar S, Dispenzieri A, Lacy MQ, Litzow MR, Gertz MA. High incidence of gastrointestinal tract bleeding after autologous stem cell transplant for primary systemic amyloidosis. Bone Marrow Transplantation 2001;28:381-385.

Lachmann HJ, Gillmore JD, Pepys MB, Hawkins PN. Outcome in systemic AL amyloidosis following stem cell transplantation or infusional chemotherapy. Blood 2002:100(11) abstr 788:210a.

Anderson KC, et al. National Comprehensive Cancer Network, Clinical Practice Guidelines in Oncology, Multiple Myeloma, v.1.2005.

Sanchorawala V, et al. High-dose intravenous melphalan and autologous stem cell transplantation as initial therapy or following two cycles of oral chemotherapy for the treatment of AL amyloidosis: results of a prospective randomized trial. Bone Marrow Transplant 2004;33:381-388.

Seldin DC, et al. Improvement in quality of life of patients with al amyloidosis treated with high-dose melphalan and autologous stem cell transplantation. Blood 2004;104:1888-1893.

Skinner M, et al. High-dose and autologous stem-cell transplantation in patients with AL amyloidosis: an 8-year study. Ann Intern Med 2004;140:85-93.

Vesole DH, Perez WS, Reece DE, Akasheh M, Horowitz MM. High dose therapy with autologous hematopoietic stem cell transplantation (HSCT) for patients with primary systemic amyloidosis (AL): results from the autologous blood and marrow transplant registry (ABMTR). Blood 2003:102(11) abst. 402:118a.