Request for Proposals for Clinical Quality Measures to Improve Diagnosis
1. Project Description
The purpose of this funding opportunity is to provide assistance in the form of grants for the development of innovative clinical quality measures (defined in Appendix A) that promote excellence in diagnosis of three categories of disease – acute vascular events, infections and cancer.
The Diagnostic Excellence Initiative
In November 2018, the Moore Foundation announced its Diagnostic Excellence Initiative with a focus on diagnostic performance improvement. The initiative aims to reduce harm from erroneous or delayed diagnoses, reduce costs and redundancy in the diagnostic process, improve health outcomes and save lives. The initiative’s first area of focus is to develop and validate new measures for diagnostic performance. Examples of our first two cohorts of grants for diagnostic clinical quality measures are now available for review. Starting with measure development is important – currently, U.S. health care systems are unable to systematically measure diagnostic performance in real time, which limits the ability to quantify performance and guide improvements.
Defining the problem and gap analysis
Diagnosis is at the heart of the clinical practice of medicine; indeed, almost every action or intervention flows from the diagnosis. A wrong, delayed, or missed diagnosis allows illness or injury to persist or progress often with potentially preventable harm. Twelve million Americans experience a diagnostic error each year1,2 and diagnostic errors play a role in an estimated 40,000-80,000 deaths 3 annually in the U.S. alone. Diagnostic errors occur in both inpatient and outpatient settings, in both adult and pediatric populations, and across the health care system. Harm from diagnostic errors accounts for the highest proportion of malpractice cases and the largest settlements, suggesting they are a leading contributor to preventable injury or death.4 In fact, the true incidence of diagnostic failure isn’t known because studies are limited and health care centers do not routinely track such data.
There are many reasons why diagnostic error is common. First, diagnosis is difficult. There is inherent variability in disease presentation. Furthermore, diagnostic tests are less than perfect and clinical encounters inevitably have some lingering and irreducible uncertainty. Additionally, many health care systems are not optimally designed to support efficient and reliable diagnostic processes. There are systematic barriers for optimal diagnosis, including misaligned financial incentives and fragmented delivery systems. From a patient experience perspective, the diagnosis is often not adequately communicated or well understood.5
There is an urgent need to improve diagnosis. However, without an awareness of baseline performance, and standards against which to compare performance, there is no way to measure improvement or to gauge the results of interventions. Despite a lengthy and growing list of clinical quality measures in health care, few existing measures address diagnostic performance specifically.6 The challenge of finding meaningful clinical measures for diagnosis reflects the complexity of the diagnostic process. Much of the work of diagnosis is invisible to the outside reviewer and many diagnostic pathways involve a variable trajectory of thoughts and actions that can be difficult to capture or record. This work is made even more difficult in that there are few specific guidelines for what would constitute diagnostic excellence. Just how precise must a diagnosis be to be considered correct? What is timely? For stroke, every minute counts, but for cancer, a few days to weeks might be considered acceptable. Reasonable standards for one setting may be unrealistic for another. And finally, it can be difficult to find reliable sources of data on diagnosis. Large databases often lack sufficient granularity or include a patient’s full diagnostic journey. Available data from the electronic health record is typically optimized for billing and may not accurately capture patient symptoms, diagnostic reasoning, differential diagnoses, or diagnostic uncertainty.
With a growing awareness of diagnostic errors, the health care environment is ready for change. The National Academy of Medicine report in 2015 (Improving Diagnosis in Healthcare) helped galvanize action by declaring the need to improve diagnosis a “moral, professional, and public health imperative”.7 Recently, the Agency for Healthcare Research and Quality (AHRQ) listed diagnostic errors as one of the leading urgent priorities in health care for 2019 (AHRQ's Road Ahead: Seizing Opportunities in Three Essential Areas to Improve Patient Care) and announced a funding opportunity for work to better understand diagnostic errors. (With Increased Funding, AHRQ To Explore Scope and Causes of Diagnostic Errors). The movement to improve diagnosis is gaining traction, evidenced by the commitment of 60 medical societies and leading health care organizations in the Society to Improve Diagnosis (SIDM) Coalition to Improve Diagnosis. Additionally, the National Quality Forum has convened a committee to issue recommendations for measure development for diagnostic quality and recently completed their report, Reducing Diagnostic Error: Measurement Considerations. The need is evident and increasingly acknowledged, but the difficulty rests with determining how to tackle this multi-faceted problem.
The characteristics for diagnostic quality described by the National Academy of Medicine (‘diagnosis should be accurate, timely, and communicated’) may present competing aims, and they omit safety and cost efficiency.7 Achieving higher accuracy with break-neck speed may drive over-testing, generate unnecessary and confounding data, exhaust diagnostic resources, and even directly harm patients with unnecessary procedures.
The aim of the funded work on measurement will focus on enabling clinicians and systems around them to find an optimal balance between these competing aims. We begin with developing measures, because measurement is integral to improvement.
c) Project scope
To align with the foundation’s principles of supporting work that is important, measurable, and impactful, we have identified three categories of disease that comprise the most common and most harmful diagnostic errors, including acute vascular events (such as stroke and myocardial infarction), infections (such as sepsis and pneumonia), and cancer (such as lung cancer and colorectal cancer).8 Proposals must relate to one or more of these three broad categories.
d) Requirements and expected outcomes of grant
For this grant opportunity, there must be, at minimum, a proposed measure of diagnostic performance based on obtainable evidence in one or more of the three priority categories listed above. The expected work requires two interlinked activities: 1) development of the rationale for a measure and 2) operationalizing the measure into an algorithm (see Appendix B). Prior work in measure development is useful but not required.
The Moore Foundation is seeking measures that can eventually be developed into fully specified performance indicators that:
- address a performance improvement opportunity and fill a measurement gap;
- align with evidence (e.g., from the medical literature, clinical practice guidelines, or expert consensus);
- focus on outcomes (although process measures may be considered if they are particularly innovative and link to patient outcomes);
- are likely to be feasible—that is, the information can be easily and reliably retrieved or designed into commonly available data sources (such as the electronic health records or administrative claims) without imposing excessive burden on clinicians or patients;
- are likely to be high-value, that is, the challenges associated with developing or implementing the measure are outweighed by the potential benefits once implemented; and,
- rely on a data source (or sources) appropriate for pilot testing and accessible by the grantee for this purpose. The grantee need not be constrained to an existing data source if they have alternative methods or ideas for generating data, although their method must eventually be usable by others.
To optimize the likelihood of measure success, grantees are expected to seek input from multiple perspectives, including patients, working alongside technical experts as they develop and implement their measures. This can be satisfied with the formation of an advisory panel, or a series of panels, or ad hoc groups designed to focus on operationalizing a measure that meets its intended goals. The purpose of this requirement is to assure that the measure as imagined aligns with the measure as developed and implemented, and to assess the benefit and risks of implementation from multiple perspectives, including the patient, clinician, health care team, risk management, hospital organization, broader health care delivery system and the technical team comprised of informatics experts, data analysts, and others as needed. We will refer to this group(s) as the technical expert panel (TEP).
Grantees are expected to work with their advisory panel(s) to:
1. iteratively refine their measure to generate a high-value measure, and
2. iteratively operationalize the high-value measure into an algorithm (i.e., a set of steps that might involve collecting data, applying logic, and making calculations) to be pilot tested with a data source(s), and
3. implement the measure in real-time clinical settings, and
4. assess the success of their measure and revise as necessary.
- Rapid cycle evaluation and revision is typically required for successful measure development. We favor teams that are agile enough to test and refine, recognize failure early, and revise their project in rapid cycles. Learning what doesn’t work and demonstrating flexibility are desired features of this work and information about failed approaches is considered an important output to be understood and shared.
- •Grantees will receive technical assistance from Battelle, our consulting technical expert, to complete the grant deliverables described in Appendix C and to align their measures to specifications detailed in the CMS Measures Blueprint. This funding opportunity prioritizes ideas for measures that are likely to be impactful over the applicant’s experience in measure development. Creative and novel approaches are strongly encouraged.
- Participants will be invited to engage with other grantees in virtual or in-person meetings organized and funded by the Foundation to inform their work and mutually benefit from lessons learned from the cohort of grantees.
2. Award Information
a) Award amount
Up to eight projects will be awarded amounts varying from $250,000 to $500,000 for work done over 18 months.
Complexity of the measure proposed in the application will contribute to variations in the amount of the award. The measure type (outcome, process, patient-reported outcomes), scope (single setting, cross-setting, across specialties), and data source (e.g., electronic health records, registries, claims, multiple and/or linked data sources, novel approaches, etc.) will impact the assessment of the measure’s complexity.
b) Anticipated award dates
Project start date is approximately January 1, 2022.
c) Period of performance
This project is the first phase of a larger plan for measure development. Promising clinical measures may qualify for additional funding for successive phases of work to complete rigorous measure development and determine pathways for implementation.
3. Eligibility Information
a) Eligible applicants
Applicants should have an affiliation with an institution or sponsoring organization, including but not limited to academic institutions, health care delivery systems, medical and clinical specialty societies, patients and patient advocacy groups, medical liability and risk management organizations, independent research organizations, electronic health record vendors, and others with interest and/or expertise relevant to diagnosis measure development.
Successful applications will describe teams and partnerships that include a multidisciplinary group of experts, including clinicians with content expertise, individuals with appropriate analytic expertise (data science, statistics, measure development) and persons with experience using relevant data sources. An individual may satisfy more than one of these areas of expertise. Measure development expertise is helpful but not a requirement for funding.
b) Eligibility criteria
Applicants must be familiar with the U.S. health care system and have grant outputs feasible for implementation in the U.S.
Suitable measure concepts must be based on existing scientific evidence and/or clinical guidelines, not new or as yet untested diagnostic tests.
Our scope of funding is not directed at any of the following:
(1) development or evaluation of new diagnostic tests, products, or devices;
(2)development of new clinical guidelines or clinical prediction rules or clinical decision support, or
(3)clinical investigations designed to test a hypothesis.
Examples of previously funded projects can be viewed on the Moore Foundation website at Moore.org (see New projects aim to develop clinical quality measures to improve diagnosis and Second cohort aim to develop novel clinical quality measures to improve diagnosis).
4. Application Information
a) Content and form of applications
All applications must be submitted through the SurveyMonkey Apply online system. Applications will be limited to three applications for any given principal investigator.
b) Submission Dates
- This funding opportunity requires a multi-phased competitive application process. A summary of key dates and deadlines is shown below.
January 19, 2021
Online application opens
February 16, 2021
Informational webinar about RFP*
April 12, 2021
Informational webinar about RFP*
May 10, 2021
Deadline for receipt of applications
June 21, 2021
July 15, 2021
Required live meeting (virtual or in-person in Palo Alto, CA) for semi-finalists, by invitation only
July 26, 2021
Finalists notified and invited to provide supplemental application materials
August 23, 2021
Deadline for receipt of all application materials
January 1, 2022
Approximate start of projects
*Interested applicants should indicate their desire to attend a webinar in an email to diagnosis@Moore.org to receive log in instructions.
5. Application Review Information
a) Evaluation criteria
Detailed instructions for application questions and our evaluation criteria are provided in Appendix D.
b) Review and selection process
This funding announcement initiates a multi-phased competitive application process soliciting ideas and strategies for diagnostic clinical quality measures.
In the initial application process, we request a short description of the proposed clinical quality measure, its potential to improve patient outcomes, the intended data source that will be used to test the idea, and a brief explanation of methods planned to prepare the measure for implementation (beginning with feasibility and acceptability). Access to and ability to use an existing data source is necessary, however alternative and novel methods for testing will be considered. The application cycle will open on January 19, 2021 and close on May 10, 2021.
Semi-finalists will be announced by June 21, 2021. At that time, additional information will be solicited to advance to the second stage of the application process, including a preliminary budget, confirmation that applicants have notified their Office of Sponsored Research (or similar office), and attestation that they agree to participation in our cohort of grantees with our technical assistance partner in measure development. Additionally, the Moore Foundation policy on sharing data, HIPAA, and intellectual property should be reviewed.*
At least one member of the semi-finalist team is required to participate in a one-day, live meeting; depending upon safety concerns, the meeting may be virtual or in-person at the Gordon and Betty Moore Foundation offices in Palo Alto, California on July 15, 2021. If travel is required, travel expenses for up to two team members will be paid by the foundation. Semi-finalists will present their proposal before a technical review board comprised of experts in clinical measure development. Proposals will be assessed for the quality of their concept and likelihood of success.
The most promising proposals will be selected as Finalists and will be eligible for funding. Individual grants will be developed in collaboration with Moore Foundation staff. Finalists will be notified by July 26, 2021; all final application materials and supportive documents will be due by August 23, 2021.