JACR: Canadian rads dub decision support a nuisance
Implementation of Canadian Association of Radiology (CAR)-based decision support in a pediatric hospital showed little relevance of guidelines to clinical cases and even poorer compliance, with physicians admitting that they often “blatantly cheated” to avoid using the program, according to a study published in the April edition of the Journal of the American College of Radiology.
The use of clinical decision support (CDS) for imaging orders has grown considerably in North America in response to concerns of overutilization and inappropriate imaging. Following the widespread adoption of CDS by Minnesota providers, which saw reductions in utilization and cost, associations and providers in the U.S. are currently lobbying Medicare to consider electronic CDS.
The single-institution Canadian study performed at Children’s Hospital Winnipeg revealed mediocre results with CDS. Physicians and other caregivers at Children’s Hospital were trained in both classroom and clinical settings on the electronic program, which incorporated pediatric CAR guidelines (based on Royal College of Radiologists in England guidelines) into the CPOE process, giving clinicians the option of using CDS or sticking with paper.
Out of 9,925 imaging exams ordered between July 2006 and August 2007, 8,757 were placed using CDS. The program had relevant guidelines for 1,678 of these orders, largely because adult recommendations were not included, a weakness that is likely to be addressed in future pediatric criteria, according to the authors.
CDS evaluated 957 of the orders as inappropriate, recommending a different modality in 23 percent of these cases and cancelling the order in the remaining 77 percent. The program wielded little impact on physicians. The Canadian physicians changed their orders 19 times over the course of the study.
The impact on duplicate orders was minimal; the software alerted physicians to 367 identical studies performed within the last 30 days of the order. Physicians heeded the program’s advice in 40 cases.
The authors found that CDS recommendations were followed significantly more often for ultrasound and x-ray exams than all other study types, while modality and physician characteristics did not predict any other relationships.
Physicians and other clinicians were able to communicate their comments via monthly stakeholder committee meetings, consultations, focus groups and observation. “Although a few physicians found the decision support interesting and said that it had occasionally caused them to think about the orders they were placing, most did not, indicating that the decision support was ‘a nuisance’ or too ‘generic’ and not relevant to the complex or high-risk patients they had to treat,” explained Sarah Bowen, PhD, from the School of Public Health of the University of Alberta at Edmonton, Canada, and co-authors.
On the other hand, a majority of physicians viewed the CPOE element of the program favorably, primarily citing less room for error in order placement, electronic report viewing, less reliance on clerks to place orders and more legible acquisitions. “The CPOE system was likened to ‘internet banking’ and was described as ‘intuitive’ and ‘easy to use,’” the authors reported.
Physicians and technologists reported substantial variations in the time required for, and usefulness of, the CPOE and CDS. Many found order entry quicker. However, most complained that the system made recommendations after they had discussed the exam with the patient’s parents, at which point many physicians indicated they were unwilling to change their minds. Although information was more clear and legible to most physicians, there was widespread criticism of the limited amount of information provided by the system, which often made orders more difficult by omitting contexts for patient injuries and illnesses.
“As the quantitative results indicate, there was little enthusiasm for the decision support component of the software. Nearly all physicians believed that their ordering was appropriate and emphasized reliance on their radiology colleagues for advice when unsure of what to order,” Bowen and co-authors wrote.
In light of many positive experiences in the literature with CDS, the authors underscored the importance of this study in reinforcing the need to evaluate the implementation and impact of new health IT systems.
The use of clinical decision support (CDS) for imaging orders has grown considerably in North America in response to concerns of overutilization and inappropriate imaging. Following the widespread adoption of CDS by Minnesota providers, which saw reductions in utilization and cost, associations and providers in the U.S. are currently lobbying Medicare to consider electronic CDS.
The single-institution Canadian study performed at Children’s Hospital Winnipeg revealed mediocre results with CDS. Physicians and other caregivers at Children’s Hospital were trained in both classroom and clinical settings on the electronic program, which incorporated pediatric CAR guidelines (based on Royal College of Radiologists in England guidelines) into the CPOE process, giving clinicians the option of using CDS or sticking with paper.
Out of 9,925 imaging exams ordered between July 2006 and August 2007, 8,757 were placed using CDS. The program had relevant guidelines for 1,678 of these orders, largely because adult recommendations were not included, a weakness that is likely to be addressed in future pediatric criteria, according to the authors.
CDS evaluated 957 of the orders as inappropriate, recommending a different modality in 23 percent of these cases and cancelling the order in the remaining 77 percent. The program wielded little impact on physicians. The Canadian physicians changed their orders 19 times over the course of the study.
The impact on duplicate orders was minimal; the software alerted physicians to 367 identical studies performed within the last 30 days of the order. Physicians heeded the program’s advice in 40 cases.
The authors found that CDS recommendations were followed significantly more often for ultrasound and x-ray exams than all other study types, while modality and physician characteristics did not predict any other relationships.
Physicians and other clinicians were able to communicate their comments via monthly stakeholder committee meetings, consultations, focus groups and observation. “Although a few physicians found the decision support interesting and said that it had occasionally caused them to think about the orders they were placing, most did not, indicating that the decision support was ‘a nuisance’ or too ‘generic’ and not relevant to the complex or high-risk patients they had to treat,” explained Sarah Bowen, PhD, from the School of Public Health of the University of Alberta at Edmonton, Canada, and co-authors.
On the other hand, a majority of physicians viewed the CPOE element of the program favorably, primarily citing less room for error in order placement, electronic report viewing, less reliance on clerks to place orders and more legible acquisitions. “The CPOE system was likened to ‘internet banking’ and was described as ‘intuitive’ and ‘easy to use,’” the authors reported.
Physicians and technologists reported substantial variations in the time required for, and usefulness of, the CPOE and CDS. Many found order entry quicker. However, most complained that the system made recommendations after they had discussed the exam with the patient’s parents, at which point many physicians indicated they were unwilling to change their minds. Although information was more clear and legible to most physicians, there was widespread criticism of the limited amount of information provided by the system, which often made orders more difficult by omitting contexts for patient injuries and illnesses.
“As the quantitative results indicate, there was little enthusiasm for the decision support component of the software. Nearly all physicians believed that their ordering was appropriate and emphasized reliance on their radiology colleagues for advice when unsure of what to order,” Bowen and co-authors wrote.
In light of many positive experiences in the literature with CDS, the authors underscored the importance of this study in reinforcing the need to evaluate the implementation and impact of new health IT systems.