Response to Comments: Biomarker Testing for Neuroendocrine Tumors LCD L37851

A comment was received that, “The NETest gene signature has been confirmed and validated as a marker signature for neuroendocrine cancer (independent evaluation of >10,000 cancers encompassing 32 different types of tumors and normal samples, NIH-funded study.) The NETest has been independently validated as a diagnostic for NET and is more sensitive than Chromogranin A. The diagnostic accuracy of the NETest is ≥96%. NETest scores are highly reproducible 97%. The NCI has determined that progression free survival (PFS) is an appropriate surrogate outcome marker. PFS as a surrogate marker for outcome in NET studies has been independently validated. The NETest has been demonstrated to predict PFS (outcome) and functions as a monitor for PFS. Interventions alter the NETest which is then reflected by changes in PFS. NETest has demonstrable clinical utility.” Other issues included progression-free-survival and Mammaprint.

Summated Response. The submitted medical literature does not support this contention. The multiple articles submitted do not show robust evidence that the NETest improves overall patient survival and that it has gained overall acceptance in the United States. The issue is not merely comparing it to the chromogranin A which currently is accepted as the biomarker standard. The NETest clinical utility still has to be proven. The issue to progression-free-survival and Mammaprint are addressed in further comments below.

A comment was received that,"There are two peer-reviewed manuscripts that independently validate the NETest. (Chen et al., and Liu et al., 2018). The NETest has been independently and externally validated by the clinical-scientific regulators of the sates of Connecticut, New York and California. The test is undertaken in a professionally regulated clinical laboratory environment consistent with the requirements of five states including New York and Connecticut."

The issue is not any laboratory’s license. The Chen and Liu papers are discussed in the LCD. However, the Chen paper does not discuss its role in clinical management of patients. Unfortunately, in the Liu paper, the specificity and sensitivity of the test were not provided nor could they be accurately calculated with the data made available. However, the overall concordance with disease status reported (85) was well below 100%, a value necessary to ensure that not a single patient is misdiagnosed and therefore denied the most appropriate treatment. Understandably, reaching perfection is almost impossible in the real world, and this principle holds true especially in the complex management of patients with neuroendocrine neoplasia. Nevertheless, a clear definition of the sensitivity and specificity of NET is needed before the assay is widely adopted in clinical practice. Furthermore, whether the test performs identically regardless of tumour grade, tumour volume and/or treatment history remains unclear. Finally, the presented patient numbers are too small and the selected cohort was too heterogeneous. We do not know the relationship of all the authors, but in the article by “Modlin IM, Aslanian H, Bodei L, Drozdov I, Kidd M. A PCR blood test outperforms chromogranin A in carcinoid detection and is unaffected by proton pump inhibitors. Endocr Connect. 2014;3(4):215-223. doi: 10.1530/EC-14-0100,” Dr. Lisa Bodei is listed as being both from Wren Laboratories and Yale University.

A comment was received that, "There are 20 peer reviewed manuscripts that provide information about the NETest. Eighteen of the papers include Modlin and Kidd as co-authors, two of the manuscripts do not (Chen et al. 2018 and Liu et al. 2018). The twenty studies were undertaken at diverse medical facilities. All papers including the two manuscripts by Chen and Liu (that do not include Modlin and Kidd) favor the NETest.”

We have read and commented on all the submitted papers sent in support of NETesting and found others we found on several literature searches. We do not agree that at present robust evidence supports that the premise that the NETest has demonstrated clinical utility including increased overall survival. The Chen and Liu papers are discussed in the LCD.

A comment was received that NGS stated in the draft LCD that all of the favorable papers have Dr. Irvin Modlin and/or Dr. Mark Kidd as authors and that Dr. Kidd is an employee of Wren Laboratories. They stated that “There are other non-Wren manuscripts that were not reviewed by NGS and were not included in the draft that are favorable to the NETest (e.g., Chen et al. and Liu et al). These papers validate the NETest and confirm the results of the earlier clinical studies (that include Wren Laboratories employees and affiliates). This independent confirmation removes any intimation of bias/favoritism as suggested by NGS to be associated with papers from Wren Laboratories.” Also, the fact that the National Comprehensive Cancer Network (NCCN) does not cite the NETest in its guidelines was stated to be not relevant.

We are not questioning anyone’s integrity especially of any of the authors. We realize that it can be difficult to get research funding without other parties who do have an interest in the results. There is no really independent verification of the results and no evidence that NETest has proven clinical utility as supported by prolonged overall survival. The Chen et al. and Liu et al papers are included and reviewed in the LCD. NCCN is used by all Medicare contractors as a mandated reference. Failure to include NETest in the NCCN Guidelines is a major concern. Related, NCCN does reference Dr. Modlin’s other work with neuroendocrine tumors, but does not have any mention of NETest.

Again we received a comment concerning, “all of the favorable papers have Dr. Irvin Modlin and/or Dr. Mark Kidd as authors. Equally important is the fact that Dr. Kidd is an employee of Wren Laboratories, which has the proprietary rights to NETest. Dr. Modlin is listed on most of the papers as a consultant for Wren Laboratories. On some, he is listed as an employee of Wren Laboratories. Many of the other papers have other co-authors who are listed as employees of Wren Laboratories. Many of the articles are sponsored by Clifton Life Sciences, which is the parent company of Wren Laboratories. There is much perceived conflict of interest that appears to permeate through these publications.” The commenter then stated that “this rationale has no basis in fact since there are other “non-Wren “manuscripts (that were not reviewed by the NGS and not included in the draft LCD) that are favorable to the NETest (e.g. Chen et al. And Liu et al. These studies are not associated with Wren laboratories. The above two papers validate the NETest and confirm the results of the earlier clinical studies (that include Wren Laboratories employees or affiliates). This independent confirmation removes any intimation of bias/favoritism as suggest by NGS to be associated with papers from Wren Laboratories.” There were also comments about other contractors’ genetic testing LCDs and conflicts of interest of authors.

NGS performed several literature searches through PubMed. This will identify all medical literature indexed in PubMed of the US National Library of Medicine, National Institutes of Health. It will not identify others. Also, literature can be indexed after the draft was written. In addition, all medical literature that previously was submitted was reviewed. We did receive 111 medical articles during the comment period. All received a medical publications (and all other documents) were reviewed by NGS. Again, no one is questioning anyone’s integrity. Of note, all the publications reviewed by NGS in this LCD comment on disclosures and conflicts of interests. Disclosures and conflicts of interests and concerns of them were not limited to Wren Laboratories.

A comment was received that two publications provide credible independent external validation. “The first manuscript examined the NETest signature in ~ 10,000 cancer samples and confirmed that it is a valid marker for neuroendocrine cancers. The study was undertaken at the Baylor College of Medicine in Huston, Texas and includes authors from The Broad Institute (MIT) and Brigham and Woman’s Hospital in Boston Massachusetts. None of the authors have any relationship, academic or otherwise, to Modlin, Kidd, Wren Laboratories or Clifton Life Sciences. This is a NIH funded study.

The second paper highlights the findings from the NETest registry (NCT02270567). This paper provides an independent validation of the NETest as a diagnostic and prognostic marker. Moreover it defines the impact the NETest has on individual patient care and notes where Medicare-related cost-savings could occur in patient management.”

Again, we have read all the submitted papers sent in support of NETesting and found others to review. In the same journal issue that one of these two papers were referenced, there was an accompanying editorial criticizing the study and premature conclusions. We do not agree that at present robust evidence supports that NETest has demonstrated clinical utility including increased overall survival. Appropriate clinical studies are needed. The Chen paper does not address the clinical utility of the NETest.

Again, a comment was received that NCCN has not recommended the NETest and criticized NCCN. The authors of NCCN notes: “More research is required, however, before these and other new molecular assays are routinely used in a clinic. A multinational consensus meeting of experts concluded that, to date, no single currently available biomarker is sufficient as a diagnostic, prognostic or predictive marker in patients with neuroendocrine tumors.”

“This is a direct reference to Oberg et al. The NCCN neglected the following recommendation from the Oberg paper. The panel concluded that circulating multianalyte biomarkers provide the highest sensitivity and specificity necessary for minimum disease detection and that this type of biomarker has sufficient information to predict treatment effectiveness and prognosis. The panel also concluded that no monoanalyte biomarker of neuroendocrine tumors has yet fulfilled these criteria and there is insufficient information to support the clinical use of miRNA or circulating tumor cells as useful prognostic markers for this disease. The panel considered that trials measuring multianalytes (eg neuroendocrine gene transcripts) should also identify how such information can optimize the management of patients with neuroendocrine tumors.”

Again, we have read all the submitted papers sent in support of NETesting and found others to review. We do not agree that at present robust evidence supports that NETest has demonstrated clinical utility including increased overall survival. Appropriate clinical studies are needed to assess this. 

Beginning January 2008, Centers for Medicare and Medicaid Services (CMS) have specifically stated in chapter 15 of the Medicare Benefit Policy Manual the role of NCCN in the off-label drug use determination of cancer drugs. Thus, the value of NCCN for any Medicare contractor in making cancer decisions cannot be minimized. The Oberg papers have been reviewed multiple times.

A comment was received that “NGS suggests there are no clinical ramifications for early detection of disease. This is an unusual assertion contrary to the entire basis of modern medicine. Huge programs in every discipline of oncology have, as their principle goal, the early detection of disease. The premise being that therapy for cancer is more effective the earlier the disease is identified. A review of the literature identifies that early disease detection is a key component in follow-up and has been shown in multiple oncological settings to be associated with outcome improvements. There are innumerable clear and well-described clinical ramifications for early disease detection. The NETest has been demonstrated to provide an early indicator of disease progression and results in significant clinical benefit (patient outcome, resource utilization).”

Again, we have read all the submitted papers sent in support of NETesting and found others to review. We do not agree that at present robust evidence supports that NETest has demonstrated clinical utility including increased overall survival. Appropriate clinical studies are needed to assess this.

A comment was made that the NETest provides prognostic information that, like other gene expression assays, can impact treatment decision making.

Again, we have read all the submitted papers sent in support of NETesting and found others. We do not agree that at present robust evidence supports that the NETest has demonstrated clinical utility including increased overall survival. Appropriate clinical studies are needed. The issues about Oncotype DX or Mammaprint are not pertinent. Those tests are not NETest.

A comment was received that, “The evidence for clinical utility is substantial, has been peer-reviewed on multiple occasions and is accepted by the medical community. The test has been independently validated. There are 13 manuscripts that examine clinical utility and all demonstrate clearly that the NETest has substantial clinical utility. The study by Chen et al. has validated that the NETest as a bona fide biomarker for neuroendocrine cancers at a tissue level. They also demonstrated that these genes were not a component of and are not expressed in normal blood populations e.g., lymphocytes. Ipse facto, the blood source of the signature can thus only reflect a neuroendocrine tumor tissue source. This assertion has been confirmed in a series of matched: blood tissue measurements where the correlations between gene expression in tumor and blood is >80%. Furthermore a separate series of studies have demonstrated that the NETest is specific for neuroendocrine cancers and blood gene expression levels are only elevated when a tumor has neuroendocrine features.”

We do not share the opinion that “all demonstrate clearly that the NETest has substantial clinical utility.” The Chen paper really is not a clinical paper. It has been reviewed and is specifically discussed in the LCD. We do not share the opinion of the commentators.

A comment was received that the submitted studies provide unequivocal support for use of PFS as an appropriate endpoint to measure clinical utility in NETS. “Multiple studies, when examined in regard to impact on PFS and an examination on management, unequivocally demonstrate that the NETest exhibits robust, reproducible clinical utility. The value of the NETest for improving PFS and hence outcome has been demonstrated and validated.”

While we read the papers submitted on using progression-free-survival, this endpoint has not reached universal acceptance. We have also included in the LCD papers that criticize using progression-free-survival rather than overall survival. It has been especially studied in colorectal cancers, but not as much as in other cancers. Progression-free-survival has its major advantage in clinical drug trials where it also offers quickly seeing if the drug appears to be working with the additional advantage of saving pharmaceutical companies money.

A comment was received that “Using the NETest values even at their simplest levels (low or high) confirm disease stabilization or disease progression. Scores can be used to recommend intervention or treatment modification. It is not within standard practice for the NETest to be required to undergo a higher burden of proof. The outputs of low or high are easily discernible to any physician and not difficult to interpret.” Medicare payment for OncoType Dx and Mammaprint was mentioned. In addition “NGS has already acknowledged that these tests fall under the auspices of the FDA which do have ‘final guidance on the oversight of laboratory developed tests.’”

Again, we are not commenting on OncoType Dx and Mammaprint. Those are not really issues. NETest is the only pertinent issue here. The FDA does not have the same agenda as Medicare: FDA approval, clearance or other positive status does not mean Medicare coverage follows.

An ALJ hearing response in favor of the NETest was submitted as further support for the NETest.

As all laboratories, manufacturers, and attorneys know, ALJ decisions do not set any precedent. Submitting an ALJ favorable decision is not supportive evidence. Of note, any appeal that went to an ALJ means that the Qualified Independent Contractor (QIC) also felt that the NETesting was not payable under Medicare. Related, there are twenty-five cases (25) that an ALJ ruled unfavorable to Wren Laboratories concerning NETesting.

A comment was received that “Coverage policies are in place from other Contractors that support use and reimbursement of the NETest.”

One of the submitted coverage policies was a very old one from a company that does not do Medicare business in that jurisdiction any more. The other “coverage policies” appear to be default policies, rather than coverage policies based on evidence. NGS has contacted the medical directors involved and they were surprised that they were inadvertently covering NETest. One medical director had told NGS that the inclusion of NETest on the contractor’s website was in error and had thought it had previously been corrected. However, since these jurisdictions apparently had not received a single claim for NETest, they were not aware of this and it does not actually give much support for this.

NGS received a submitted list of references in support of the NETest.

This is a list of references. These are not actually the articles. These are not medical support. Of note, the LCD has 210 references in it.

A number of submitted letters and testimonies from doctors and beneficiaries in support of the NETest were received.

There are a number of letters of support. These are testimonials. None had medical articles sent with them. We have read all of them and appreciate their input, but they do not take the place of good peer-reviewed medical literature indexed in PubMed Data Base of medical literature, from the US National Library of Medicine, National Institutes of Health. Of note, several of the glowing recommendations for NETest were from authors whose articles had been reviewed and in those articles their conclusions published by them were not nearly as strong as those they espoused in their letters.

Various laboratory licensures were submitted.

The laboratory being licensed is neither the issue nor relevant.