Factors CMS Considers in Commissioning External Technology Assessments

For questions regarding specific National Coverage Determinations, please call the Coverage and Analysis Group’s main number, at (410) 786-2281.

For questions regarding the National Coverage Determination Process, please contact Vadim Lubarsky, at (410) 786-0840.

This guidance represents the Centers for Medicare & Medicaid Services' (CMS's) current thinking on this topic. It does not create or confer any rights for or on any person and does not operate to bind CMS or the public. You can use an alternative approach if the approach satisfies the requirements of the applicable statutes and regulations. If you want to discuss an alternative approach, we encourage you to contact CMS staff at the listed on the title page of this guidance.

I.     Purpose of this Guidance Document

This document relates the factors CMS considers in requesting an external Technology Assessment (TA) to supplement the National Coverage Determination (NCD) process.

II.     Background

This document describes one key part of the national coverage determination (NCD) process concerning CMS’s evaluation of whether an item or service is reasonable and necessary. Under certain circumstances, an external TA is an integral part of the NCD process, and the completion and evaluation of the external TA affects the overall timeline for completing NCDs. Specifically, the statutorily mandated timeframe for releasing a proposed decision is six (6) months after the formal request date. If CMS requests an external TA, that timeframe is nine (9) months.

III.     General Function

CMS undertakes a number of activities designed to improve the health care provided to beneficiaries. Among these activities are coverage policy decisions that determine which services can be covered as “reasonable and necessary” under title XVIII of the Social Security Act. These decisions call for the best scientific and clinical evidence available concerning the effectiveness of various medical diagnostic procedures and therapies, and the highest attainable level of expertise to evaluate such evidence.

Thus, public and private health care purchasers utilize expert health technology assessment processes to meet particular policy or clinical objectives. A TA can involve the evaluation of a technology’s performance characteristics, safety, efficacy, effectiveness, outcomes, appropriateness and economic impacts. TAs that systematically evaluate available evidence are the most highly regarded assessments. These types of assessments include a range of related activities such as identifying and prioritizing technologies for assessment, collecting and analyzing data, synthesizing and grading evidence, and disseminating findings and recommendations. CMS has typically used TAs to assist in the review of evidence in its NCD process of determining whether a particular technology is reasonable and necessary and, in some cases, to identify those areas that need further evidence development.

While a TA can evaluate many aspects of a technology, interpretation and critical appraisal of the evidence on patients’ health outcomes constitutes its key component. Accordingly, the performance of a systematic review of the evidence from the medical literature is at the core of every TA, whether undertaken internally or commissioned externally by CMS.

Systematic reviews are scientific investigations that synthesize the results of multiple primary investigations on a given clinical question. The evidence is then appraised to assess its validity (how credible it is), usefulness (its clinical applicability), and importance (magnitude of effect). To minimize bias, systematic reviews emphasize a comprehensive search of all potentially relevant articles and the use of explicit, reproducible criteria in the selection of articles for review. Primary research designs and study characteristics are appraised, data are synthesized, and results are interpreted.¹

By incorporating methods to assemble, sort through and integrate clinical evidence, the systematic review embedded in a TA represents a rigorous compilation of scientific evidence to answer clinical questions. It enables other parties to understand, replicate, and judge the collection, selection and analysis of evidence and is particularly useful when the medical research literature is complex or extensive.

TAs are, and will continue to be, focused on clinical factors pertinent to beneficiaries’ health outcomes. Furthermore, while economic considerations may be a factor discussed in a technology assessment, the primary purpose of a TA is to evaluate the clinical and scientific evidence pertaining to the clinical benefits and risks of the technology, and cost is not a factor in our review or decision to cover a particular technology.

At times, we may decide to refer an issue together with the relevant TA for further review to the Medicare Coverage Advisory Committee (MCAC). This step involves further evaluation of the TA. While the purpose of a TA is to collect and evaluate evidence, the role of the MCAC is to evaluate that assessment and any other pertinent evidence that addresses the technology in question.

IV.     Criteria for Requesting an External TA

During the NCD process, we may determine that we need assistance in evaluating the evidence. In many cases, this will be following the opening of an NCD (see Guidance Document on Opening an NCD). In other cases, we may determine that we need a TA to evaluate the available evidence prior to deciding on the need for an NCD. Also, there may be instances where a TA will help inform us on the status of the evidence on certain topics of interest to the Agency.

In general, we may request an external TA if one of the following conditions applies:

• The body of evidence to review is extensive, making it difficult to complete an internal technology assessment by CMS within the 6-month statutory timeframe;
• An independent formulation of the appropriate assessment questions and methodological approach to an issue is desirable given the complexity or conflicting nature of the medical and scientific literature available;
• Significant differences in opinion among experts concerning the relevant evidence or in the interpretation of data suggest that an independent analysis of all relevant literature will be of value;
• The review requires unique technical and/or clinical expertise not available within CMS staff at the time of the review;
• The review calls for specialized methods (e.g., decision modeling, meta-analysis) in health technology assessment;
• The topic under consideration will be referred for consideration to the MCAC; or
• Relevant non-proprietary but unpublished data could be collected and analyzed.

V.     Process

If CMS determines that it needs an external TA, we will obtain it from an entity with the requisite experience in TA methods and evidence-based medicine to ensure the technical competence and fairness of the TA report. Currently, we contract with the Agency for Healthcare Research and Quality (AHRQ) for the acquisition of TA reports via an interagency agreement. With CMS’ concurrence, AHRQ may conduct the assessment in-house; select an Evidence-Based Practice Center (EPC) for the task; or utilize another qualified entity. CMS is primarily concerned with which of these approaches will lead to a comprehensive expert assessment in the shortest time period.

Once we determine that an external TA is necessary, we will explore with AHRQ the TA feasibility and design. The discussion will include the assessment questions to be addressed by the TA, deliverables and suggested timeline, and additional material relevant to the TA. We will conduct conference calls and meetings as necessary with AHRQ to address assessment questions, scope, and other aspects of the study (e.g., the proposed analytic model with pathways by which the intervention may affect the disease process) to ensure that we reach agreement on the key elements of the TA production.

The completed TA will ordinarily include the following deliverables:

• Final assessment questions and TA framework;
• A summary of the literature search strategy, with inclusion and exclusion criteria, including a bibliography as well as a copy of the set of articles to be systematically reviewed;
• A peer-review-ready draft report including appropriate evidence tables; and
• A final TA report.

If we receive a formal request for coverage on an item or service for which a TA is already underway, we will inform the requestor of the status of the pending TA, as well as an estimated time for completion. Requests for a TA, whether new or in progress, are usually reflected on our website tracking sheet, followed by posting of the final TA report.

We primarily issue TAs in advance of MCAC meetings or as part of the NCD process. We will generally post TA final reports to the coverage website and welcome public comments. In addition, when we commission a TA as part of the NCD process, we will generally make the TA questions available to the public as the TA is under development. While posting of TA questions is for informational purposes, the public is encouraged to submit suggestions and comments. We also encourage input at any time on whether a TA should or should not be performed, and suggestions on current and future assessments. This may be accomplished via the public comment function on the coverage web site.

VI.     How to Access CMS’s Home Page

Users can access the CMS home page at http://www.cms.hhs.gov. The website includes more detailed information about health technology assessment activities.

¹ D Cook et al. Synthesis of best evidence for clinical decisions. In C Mulrow and D Cook eds., Systematic Reviews, Synthesis of Best Evidence for Health Care Decisions. American College of Physicians. 1998.