08:10 - 09:50
Evaluation of decision support systems
Room: Bellecour 1 (Location: Cité internationale Lyon, Number of seats: 124)
Chair(s): Yasushi Matsumura
08:10 - 08:30
This document describes the development of a Business Intelligence (BI) dashboard for tracking the drug-drug interaction (DDI) alerts implemented as Clinical Decision Support Systems (CDSS) in Electronic Health Records (EHR). CDSS are known for their potential to reduce medical error. The use of requirements in the development of BI dashboards is crucial to obtain successful software. In this work, the requirements were analysed using a score methodology, considering the relevance of the indicators and visualization methods. CDSS effectiveness and acceptance have been questioned, so, it is fundamental to monitor their behaviour and performance. The dashboard was designed in order to satisfy the needed indicators. Using BI as a tool for monitoring the CDSS performance made it possible to operationalize the EHR content repository, maximizing the understanding in relation to the override and, by inference, to optimize the CDSS system by opening new lines of work.
08:30 - 08:50
We used formative evaluation to refine a decision and workflow support tool for liver cirrhosis with clinicians. We conducted semi-structured interviews using a prototype and clinical scenarios. Clinician recommendations were incorporated into the tool. Clinicians found the tool useful. A future study will assess the impact in clinical practice.
Keywords: clinical decision support; human factors engineering; liver cirrhosis; interview
Introduction Gaps exist in delivering evidence-based care for patients with chronic liver disease and cirrhosis when compared to evidence based guidelines . We designed and evaluated the CirrODS (Cirrhosis Order set and clinical Decision Support) tool with clinicians to improve clinical decision-making and workflow for liver cirrhosis [2,3,4].
Methods Clinicians participated in several rounds of formative evaluation using prototypes with subsequent redesign of CirrODS. Physicians (n=20) at three hospitals used clinical scenarios based on patients with cirrhosis and the workflow support tool. The admission orders made with and without the CirrODS tool were compared. Physician participants also described their experience using CirrODS and provided recommendations, which were coded into categories and themes and used to make modifications. We evaluated the clinical content, safety, and usability of CirrODS using both qualitative and quantitative methods.
Results We created an interactive CirrODS prototype that presents relevant patient data and recommendations for evidence-based tests and treatments to be ordered. Physicians viewed the tool positively and suggested that it would be most useful at the time of admission. CirrODS was perceived to serve clinical needs especially for less experienced providers. Interviews also suggest that it is likely important to include functionality so that individual users can tailor the timing of the use of the tool to their individual workflow preferences.
Discussion We iteratively evaluated and revised CirrODS with clinicians to improve the use of evidence-based treatment of cirrhotic patients during routine hospital practice by non-specialists. The results gathered during evaluation were promising, and end users expressed interest and appreciation for CirrODS. Overall, the tool maintained good usability while facilitating the ordering of a higher percentage of high-priority measures compared with those ordered without the tool. We also demonstrated the usefulness of formative evaluation based on user-centered design to develop EHR-based CDS tools.
Conclusions Using a formative approach with clinicians we evaluated and refined an innovative clinical decision-making and workflow support tool to facilitate the adoption of evidence for patients with cirrhosis. A future study will assess the impact on quality of care for patients with cirrhosis in clinical practice.
The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government or our academic affiliates. We thank the inter-view participants for their valuable contributions. This work was supported by VA Health Services Research and Development (HSR&D) grant IIR 13-052 and CIN 13-416.
Ms. Jennifer Garvin
The Ohio State University, The University of Utah, The US Department of Veterans Affairs
Dr. Garvin is currently the Division Director and Associate Professor in the Health Information Management and Systems Division in the School of Health and Rehabilitation Sciences. She develops applied informatics tools and methods to advance clinical and public health practice. She is currently a VA Research Health Scientist and an Affiliated Investigator with Center for Health Information and Communication at the Richard L. Roudebush VAMC, Indianapolis, IN. In terms of clinical practice, she and her team develop natural language processing tools to improve care for chronic systolic heart failure in the Department of Veterans Affairs (VA). Through several VA Health Services Research and Development grants she developed the Congestive Heart Failure Information Extraction Framework (CHIEF) which she used to automate heart failure quality measurement and she has extended the design to provide data for a decision support clinical reminder for titration of beta blocker therapy in primary care. As part of this research she also studies the contextual factors related to implementation of CHIEF using the Promoting Action on Research Implementation in Health Services framework, the Sociotechnical Model for Health Information Technology (HIT) Model, and User-centered Design to guide the research. These approaches allow her to study how to use HIT to facilitate the adoption of evidence into clinical practice, to partner with clinicians and other stakeholders in development, and to design and deliver clinical content so that it supports decision making.
08:50 - 09:10
Introduction: The opioid epidemic has reached crisis proportions in the United States, with the rate of overdose deaths currently exceeding that of traffic deaths. Physicians themselves have inadvertently contributed to the problem by prescribing more than the minimum required to control pain. Quality improvement initiatives and targeted continuing education often produce only modest changes in physician prescribing behavior, and traditional clinical decision support is often ignored. Instead, we have shown that redesigning an eprescribing interface to reduce the number of clicks needed to perform a recommended action (while also increasing the number of clicks needed to select a non-recommended action) made an immediate and lasting improvement in prescribing practices. We applied similar insights to redesign our eprescribing interface. The objectives of the current project were to use our electronic prescribing system to (a) conduct a retrospective study of opioid prescribing quantities during a series of local and state-level quality improvement initiatives intended to reduce opioid prescribing, and (b) implement and assess the electronic intervention to encourage prescribers to reduce the quantity prescribed for new opioid prescriptions.
Methods: The Weill Cornell Physician Organization is the multi-specialty faculty practice of the Weill Cornell Medical College in New York City, representing both private practices serving commercially insured patients and hospital-based clinics that accept Medicaid (public insurance for low-income Americans). The Institute for Family Health is a NYC-based community health center that serves all patients regardless of their insurance status. Both institutions use the Epic EHR. (a) Prescriptions of short-acting opioids by providers in the Weill Cornell internal medicine department were retrieved from the EHR database from 2016 through 2017. A first prescription of an opioid was defined as a prescription for a patient who had no recorded opioid prescriptions in the previous 12 months. (b) The EHR redesign intervention was implemented at Weill Cornell in March 2018 and at the Institute for Family Health in June 2018.
Results: The proportion of new opioid prescriptions complying with recommended 3-day supply rose modestly throughout 2016 and 2017, coinciding with initiatives that included hospital-based education about opioid risks, New York State Department of Health letters to high-volume opioid prescribers, and hospital quality improvement meetings with local high-volume prescribers. Numbers of new opioid prescriptions also declined slightly. The post-intervention data collection will be completed in December 2018 (9 months of post-intervention data for Weill Cornell and 6 months for the Institute for Family Health).
Conclusions: A two-year series of state and local quality improvement and educational initiatives was accompanied by a modest reduction in the quantity of pills dispensed to opioid-naive patients. Nevertheless, as of late 2017, less than 20% of new opioid prescriptions met the CDC recommendations of 3 days’ supply or less. This retrospective study is consistent with previous literature showing the difficulty of changing prescribing behavior through persuasion and supports the need for our human factors/behavioral economics intervention to reduce risky prescribing. The completed analyses showing the effects of the intervention will be available in summer 2019.
09:10 - 09:30
Seattle Children’s Hospital has a robust means for creating evidence-based clinical pathways consisting of clinical algorithms, decision support, and reports. Order sets are built in a systematic way , so that they are more usable and require less cognitive load to use than ad hoc created order sets . In November, 2015, we had a 10 day improvement retreat in Japan to streamline our creation of these pathways. The result of the retreat was that we should define population definitions earlier, then use analytic reports to inform the creation/update of pathways, to try and reduce cycles of rework (e.g. to reduce errors of omission). Subsequently, we piloted this new method of creating decision support on several pathways, with the distinct multidisciplinary teams that managed these pathways.
The teams would evaluate existing clinical decision support, and prototype decision support for these clinical pathways, then use results of orders analysis for patients with the specified diagnosis to determine whether orders were used at the expected time in the workflow, the origin of these orders, and to determine if orders should be added to the decision support.
In using this approach on several different pathways, different benefits were observed. Selected findings included:
Kawasaki Disease pathway: We found that these patients were written for nonsteroidal anti-inflammatory agents (NSAIDs) such as ibuprofen, sometimes even after the diagnosis was made – so the algorithm and decision support were updated to alert providers not to give NSAIDs while on aspirin. We also found that while IVIG and aspirin were in the Admission order set, these were rarely ordered as the diagnosis was often made later. As a result, these medications were moved to a separate order set to increase their use within decision support.
Tracheostomy pathway: We found that the respiratory therapists were ordering a “clinician respiratory communication” order that variably described characteristics of patient tracheostomy tubes. For safety, we created a new order that contained all standard tracheostomy tube details, as well as back up tube manufacturer, model, and size.
Intussusception pathway: We found that the preliminary imaging studies for this diagnosis were ordered from the “ED Abdominal Pain” order set and not the “ED Intussusception” order set. Making changes to the pathway in “pre-diagnosis” decision support allowed us to achieve good compliance with the new recommendation to diagnose by ultrasound.
Discussion and Conclusions
It is a relatively new to use analytical tools to prospectively inform the construction of new clinical decision support. Our pilot studies suggest that this method allows better understanding of provider ordering workflows in ways that support effective ordering, support creating tools to target patient safety concerns, and help to determine ordering patterns that may represent common errors which can be specifically targeted by educational efforts.
These efforts have had the additional benefit of allowing incorporation of these changes into the decision support specifications prior to analyst build, presumably resulting in reduced rework and provision of higher quality decision support at the time of initial go live.
Mr. Michael Leu
University of Washington
09:30 - 09:50
A fast and frugalgeneric tool can provide decision support to those making decisions about individual cases, particularly clinicians and clinical commissioners operating within the budget and time constraints of their practices. The multi-national Generic Rapid Evaluation Support Tool (GREST)is a standard preference-sensitive Multi-Criteria Decision Analysis-based tool, but innovatory insofar as anequity criterion is introduced as one of six. Equity impact reflects the number of population QALYs lost or gained in moving from Old (current intervention) to New (contemplated intervention). In the exemplar UK implementation Claxton’s NHS Willingness to Pay per QALY is the numeraire. Any weight from 0 to 100% may be assigned to the equity criterion but its presence affirms that it is persons-as-citizens who experience any opportunity harms or benefits arising from actions within the health service commons. A fully-operational but demonstration-only version is available on open access, as proof of concept and method.
Mr. Prof. Jack Dowie Professor Emeritus of Health Impact Analysis
London School of Hygiene and Tropical Medicine