Posts Tagged Evidence-based medicine

Industry Groups Challenge Readiness of ICER’s Value Framework

Hdecisionsealthcare is all about making decisions.

Who to treat? When to treat? How to treat? How long to treat? How much to spend to treat? The list goes on and on.

Fortunately, great strides have been made in healthcare decision analysis, but not without controversy.

On the plus side, decision analysis can provide objectivity and potentially improve the quality of the final choice.  On the downside, decision analysis is fraught with challenges, including methodological, process, outcomes, perspective, data sources, and how to present the data insights, to name a few.  ICER Logo

In this blog, we’ll discuss a few of the challenges of systematic decision approaches in the context of a topic that’s been in the news of late, pharmaceutical Value Assessment Frameworks. Specifically, we will focus on a relative newcomer on the block: the Institute for Clinical and Economic Review‘s (ICER) Value Assessment Framework and explore some areas of needed improvement. While doing so, we will also compare two ways of presenting these insights: Long-form content and a visual format called PepperSlice.

What’s the beef with ICER’s Value Assessment Framework?

For more than a year, ICER has used its value assessment framework to guide its evidence reports on new drugs and other interventions. Pharma industry and patient advocacy networks have reacted strongly (and by strongly, we mean mostly negatively) to the ICER framework. One of the most common responses to ICER is that their (and other) frameworks squash pharmaceutical innovation and prevent some patients from receiving needed care.

In addition, major industry groups have challenged the readiness of the ICER framework, questioning its appropriateness in several areas. To their credit, ICER has requested stakeholder comments in preparation for a 2017 revision of the framework.

The NPC (National Pharmaceutical Council), PhRMA (biopharma trade group), and AMCP (the Academy of Managed Care Pharmacy) have significant concerns about ICER’s methodology. In particular, they take issue with the following areas:

Real-world evidence.

NPC says “ICER [should] have a clear process for managing the evolution of evidence, especially in the case of emerging therapies…. [T]hese reviews will continue to be relied upon by other stakeholders even after additional data (e.g., real-world evidence) emerge.”

And AMCP suggests real-world evidence and patient-reported outcomes should be “re-examined to further enhance the utility and relevance of the value assessment framework.”

Budget impact.

ICER should not confuse budget impact with value. “Budget impact assessments — which are measures of resource use, not of value — should remain completely separate from value assessments,” says NPC.

And this from PhRMA: ICER should suspend “the use of budget impact estimates until more sound methods are developed and validated.”

Economic model transparency.

The information provided is “not sufficient to enable reviewers to reproduce the results and provide meaningful, real-time input. Full transparency — down to the equation level — is needed to enable reproducible results and support fully informed stakeholder collaboration.” NPC asks that ICER release the model to all stakeholders.


PhRMA asks for “Adjustment of the cost-effectiveness component of the framework to reflect the inherent and widely recognized limitations in traditional quality adjusted life years-based cost-effectiveness analysis (CEA), including capturing a wider range of benefits in CEA and presenting a range of care value estimates based on sound assumptions and varied approaches.”

Other responses to the ICER Value Assessment Framework include:

NPC: To guide future development, NPC published a set of Guiding Practices for Patient-Centered Value Assessment. Dan Leonard recently recapped NPC’s viewpoint on how frameworks should be developed.

PhRMA: Four specific recommendations are offered, intended to move ICER in a more “methodologically rigorous, patient-centered direction”. They request significantly more transparency into how it works with stakeholders. And they offer specific advice on How to Get Value Assessment Frameworks Right.

AMCP: The managed care and specialty pharmacists’ group expresses concern that the current framework “lacks a process for incorporating real-world evidence (RWE) and patient reported outcomes (PROs) into the catalog of evidence that informs the underlying economic models. [Doing so would] better represent the patient experience.”

ICER Value Framework Version 2.0.

In October, ICER convened a broad group of stakeholders to inform its planned update. Invitees included people from pharma, academia, payers, patient advocates, and trade groups. A revised framework will be posted for additional comments next month; ICER’s 2.0 version will likely become final in early 2017.

What’s next for value frameworks?

ICER is only one of several frameworks gaining traction in healthcare. Stakeholders are weighing in. To help establish best practices going forward, the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) has launched an initiative on US value assessment frameworks; more than 250 people attended the kickoff stakeholder conference. A task force is preparing a policy white paper on the appropriate definition and use of value assessment frameworks, expected Q1 2017. ISPOR is the sole funder of the effort.

Moving From Data Output to Data Insights.

Much of the data derived from decision analysis is challenging to present to decision-makers.  As an industry, we all need to get better at communicating complex evidence – simply. One tool, called PepperSlice, may help.  PepperSlice is designed to deliver insights and Evidence Graphs by structuring analytics and evidence in a simple way.

Capturing insights with PepperSlice.

Let’s look at how we can visually present the concerns with ICER’s Value Assessment Framework using PepperSlice insights and data visualization.

The images below depict the issues around RWE and the ICER framework, in easy-to-consume graphics that are transparent, show the relationship between the data inputs, and the data source behind the images.

The first “board” (Image 1 ) explains the issue, graphically on this topic: “ICER’s value assessment framework is not ready for widespread adoption by stakeholders”.

Image 1.

Below each board are “slices” (Image 2), designed as cards, and each of these slices offers an insight. In the PepperSlice platform, these slices can be pinned to a single board or multiple boards, can be saved, and are searchable and reusable.

Image 2.

A database provides the underlying structure to the insight slices (Image 3 below), and provides the relationship (reduces, lacks, improves, etc.) between the two inputs, the result, the analyst name (i.e., the “slicer”), and the evidence collection method.

Image 3.

Through this method of PepperSlicing, one can present insights with a click of a button, build and manage the inventory of evidence-based insights, use them for one or multiple analyses, and see how an insight was derived (What are the supporting data? How was the evidence analyzed? Who did the work?).   In just a few images, one can see inputs, outputs, and relationships.


In this blog, we’ve given you an overview of the ICER Value Assessment Framework and the major concerns of the decision framework according to key stakeholders.  We’ve also showed you, by example, how insights can be captured about this issue – or any decision/choice – using a new methodology called PepperSlice.

What do you think are the major concerns with Value Assessment Frameworks in general and the ICER framework, in particular?  Let us know what you think about the two presentations of the data: long-form vs short-form visual presentation using PepperSlice?

We’d like to hear from you.  Just leave us a comment below, and sign up to be informed of all blog posts from tHEORetically Speaking.

If you’d like to be a guest blogger, give us a shout at by emailing





tracy Tracy Allison Altman, PhD – Founder of Ugly Research   


 pattiPatti Peeples, PhD – CEO of HealthEconomics.Com

, , , , , , , , ,

No Comments

Good, Bad, or Indifferent on Drug Pricing: A Value Benchmark Approach for Newly Launched Drugs in the United States.

swarali tadwalker

Swarali Tadwalker, MPH

Hardly a day goes by without a new article being published in the mainstream press or a medical journal discussing drug pricing.  The cost of drugs is a hot potato topic at medical meetings, pharma conference rooms, government hearings, and dinner tables.  And this topic seems to generate high emotions and dogmatic views, typically resulting in a call for more data, reasoned approaches, and better insight.

For most newly launched drugs, the relationship of a drug’s price to its true value is critically important, yet persistently elusive.  We are all asking:  “Are they worth the price?”  Value-based pricing is one avenue to answer this question, but in so doing, must take into consideration the perspectives of various stakeholders: manufacturers, payers, patients, providers, and society.

In a bold effort to facilitate value-based drug pricing decision-making from a nationwide perspective, the non-profit Institute for Clinical and Economic Review (ICER) announced on July 21, 2015 the Emerging Therapy Assessment and Pricing (ETAP) program with the aim to provide a reliable source for information on drug cost-effectiveness to different stakeholders in the United States (US). The press release on the launch of ETAP can be viewed here.  Bolstered by a $5.2 million grant from a Houston philanthropist, ICER plans to double its staff and increase the production of up to 20 benchmark reports on the value of newly launched drugs in the United States.  The two major goals for ETAP are:

  1. Authoritative assessment and price benchmark reports; and,
  2. Public engagement with all stakeholders to enhance legitimacy, dissemination and impact.


HealthEconomics.Com’s President, Dr. Patti Peeples, conducted an in-depth interview in July 2015 with two driving forces of ETAP, Dr. Steve Pearson, President and Dr. Dan Ollendorf, Chief Review Officer of ICER, to explore the goals behind ETAP and the implications for this new benchmark approach to drug pricing.
 Pearson Steven D. Pearson, MD, MSc, FRCP is an internist and health services researcher who has a long history of academic work on comparative effectiveness research (CER) and bioethics.
 ollendorf1 Daniel A. Ollendorf, PhD is a multi-decade researcher focused on comparative effectiveness systematic reviews and health technology assessment (HTA) process.

Patti Peeples, RPh, PhD [PP]: Let’s start by hearing an overview of ETAP program and a description of the catalyst for the ETAP initiative. What can you tell us about ETAP?

Steve Pearson [SP]: In broad strokes, the Emerging Therapy Assessment and Pricing or ETAP program is meant icer logoto expand what ICER has been doing for several years, but to expand in a way that would really enhance its impact for greater usefulness for different decision makers.

First, let’s look at what ICER does:  ICER has traditionally focused on full systematic review of the comparative clinical effectiveness of emerging therapies, and this is typically integrated with a cost-effectiveness analysis (CEA) as well as an analysis of potential budget impact of shifting care to new treatment. More recently, ICER has been developing what we call a value-based price benchmark in which we basically anchor a price range to a new therapy along with the degree to which it improves patients’ health.

In addition, ICER looks at short-term affordability to better understand both the long-term and short-term value.  The idea here is to give insight into whether a drug will be brought into the healthcare system at a price and payment mechanism or fashion that can prove affordable in the long term.

Our new program – ETAP – is an expansion of ICER.  We had an early test drive of ETAP with our work on the Hepatitis C drugs, the results of which received a lot of notice, as well as use by insurers and others. ETAP will allow us to ramp up this activity so that we plan to have an ICER report available at or near the time of FDA approval for all significant new specialty drugs. Our target is to produce between 15-20 reports in the first two years.

[PP]: Steve, when you say that you are looking at both short-term as well as the long-term value, can you elaborate on those time horizons and also describe from whose perspective you are looking at this?

[SP]: Sure. For the long-term perspective in our reports we include cost-effectiveness analyses.  These analyses come from simulation models based on data from published literature and publicly available databases (e.g., from the US government), or manufacturer- or payer-provided data.  We typically take either a life-time or a very long-term horizon in our cost-effectiveness analyses so that we really capture the important downstream effect, both clinical and economic, on the entire healthcare system.

But we are quite sensitive to the fact that sometimes there are drugs that are relatively cost-effective in the long-term, but if these drugs are being used to treat a very large patient population and if a large number of patients shift to the new treatment in the short-term, it can really create affordability problem for patients and insurers. So, we have a methodology that links together both the long-term cost-effectiveness and the short-term budget impact over a two-year horizon to calculate our value-based price benchmark.


ICER Rating System Overview

The ICER Integrated Evidence Rating™ combines a rating for comparative clinical effectiveness and a rating for comparative value. The clinical effectiveness rating arises from a joint judgment of the level of confidence provided by the body of evidence and the magnitude of the net health benefit — the overall balance between benefits and harms. This method for rating the clinical effectiveness is modeled on the “Evidence- Based Medicine matrix” developed by a multi-stakeholder group convened by America’s Health Insurance Plans. More here.


[PP]: Steve and Dan, what thresholds do you use to determine cost-effectiveness and how are these data presented to stakeholders for evaluation and feedback?

[SP]: Good question. So, in some of our reviews in the past we provide, where possible, cost per Quality-Adjusted Life Year (QALY) data. We recognize that the cost per QALY metric is not very intuitive or applicable for many decision makers here in the United States.  Therefore, we also present costs for different kinds of intermediate clinical outcomes that may be more readily understandable or actionable.

But as far as CEA thresholds, we don’t use one single bright line in the sand in deciding “cost-effective or not”.  Importantly, our organization, ICER, is not the ultimate decision-maker on cost-effectiveness; we simply provide the information to decision-makers and facilitate a forum for discussion.  In addition, we typically model our results around the World Health Organization’s general approach to cost-effectiveness linked to Gross Domestic Product (GDP) per capita. Moreover, we integrate our considerations with elements of value that might lie outside the traditional cost for clinical outcomes framework.

These data are presented in reports and at public meetings comprised of independent panels of doctors and public representatives. Our expanded ETAP initiative will include working with external academic health economists as part of a virtual network to help us with our work. We think that this is a great opportunity to enhance the impact of much of the great work being done in health economics both in America and elsewhere.

[DO]: It is important to convey that this approach is intended to be collaborative with all stakeholders, including the pharmaceutical industry.  All parties are recognizing that trends are changing.  There’s been a significant change – even as recently as the last couple of years –  and we believe that this is one effort to try to have a clear and transparent way to recognize and reward drug and medical innovation.

[PP]: There are several currently available drugs that have garnered substantial publicity based on their price and clinical benefits (or lack thereof).  Will ETAP focus only on new drugs just being launched or will they also look at drugs already on the market?

[SP]: Yes, we most likely will look at existing drugs already on the market. Our grant funding specifies a focus on developing the infrastructure to be able to capture the incoming new drugs. But if there are drugs for which the evidence around their effectiveness in an applicable patient population or their price seems to be raising a lot of questions, we very much intend to have the potential to look at those drugs as well.

[PP]: What is your mechanism for public comment and review of your comparative effectiveness reports?  Will your results be “vetted” by interested stakeholders? 

[SP]: We have a comprehensive public comment process (see an example here). We have had and will continue to have multiple avenues for manufacturers, payers, patient groups, or anyone who wishes to comment on the scope of our reviews. They’ll get a chance to comment on the first draft and then we have public meetings where they can make public comments as well. We anticipate and welcome input from anybody on our processes as we go through them.  Our website also has an option to sign up for regular updates on our programs and activities, including calls for public comment on recently-released draft reports.

ICER’s existing core public programs, the New England Comparative Effectiveness Public Advisory Council (CEPAC) and the California Technology Assessment Forum (CTAF), offer simulcasts of their events.


The Institute for Clinical and Economic Review is launching a membership program in 2015 to allow a small, influential group of leaders from insurers, pharmacy benefit management firms, health technology assessment groups, and life science companies to work together, over time, to shape the future of evidence and coverage policy in the U.S. Learn more about membership benefits here.


For more information on ICER and ETAP, visit

Click for a larger view.

Click for a larger view.


In an infographic by Institute for Clinical and Economic Review, effects of high drug pricing were vividly illustrated. While many new therapies did not offer any new benefits from existing medications, in 2014, as many as 576,000 people had over $50,000 yearly medication costs with about a 60% increase. Further, high prices have led many patients to skip their dose or cut their dose.




Written by Swarali Tadwalker, MPH

Guest Blogger and Research Intern at HealthEconomics.Com

, , , , , , , , , ,

No Comments