AJR: Training and testing boost rads CTC performance
CTC training and testing, provided as part of the National CT Colonography (CTC) Trial, improved radiologists’ performance and resulted in high sensitivity among inexperienced and experienced readers, according to a study published in this month's American Journal of Roentgenology.
Because of well-documented variability in CTC observer performance the National CTC Trial sought to minimize interobserver variability among participating radiologists by instituting a mandatory reader training and performance standards program, requiring radiologists to pass a qualification exam before study initiation.
All participants achieved sensitivity of approximately 90 percent, demonstrating that “CTC can be taught to others and performed with a high level of expertise by radiologists in many practice settings. However, in order to achieve this level of performance, physicians must immerse themselves in a carefully planned, hands-on course where they are reviewing cases and receiving feedback on their performance,” wrote lead author Joel G. Fletcher, MD, a radiologist from Mayo Clinic in Rochester, Minn.
Focused and tailored
The National CTC Trial required that ten radiologists who had interpreted fewer than 500 CTC exams with colonoscopic correlation participate in mandatory training before taking a qualification exam. Five experienced radiologists who had read more than 500 cases were allowed to take the exam without training.
The one day training session consisted of partial CTC exams of 27 cases with neoplasia and full review of 15 cases using primary 2D and 3D search. The initial phase of the training was designed to familiarize radiologists with the workstation, show the spectrum of colorectal neoplasia appearances and teach interactive problem-solving skills. Ten cases showed different cancer morphologies (annular, flat and polypoid), and 17 cases showed different morphologies and features of colorectal polyps (sessile, pedunculated, flat or villous).
In the final 15 cases, designed to teach colorectal lesion search methods using 2D and 3D fly-through techniques, radiologists interpreted full supine and prone colonography exams using primary 2D or 3D search.
All 15 radiologists took a qualification exam that included 20 CTC cases: 17 positive cases with 25 colonoscopy-verified polyps and three negative cases. Fifteen polyps were at least 10 mm in size, and 10 were at least 5 mm in size. Two experienced colonographers characterized polyps as easy-, moderate- or difficult-to-detect. Radiologists were required to detect 90 percent of the easy- and moderate-to-detect polyps at least 5 mm in size to pass the exam.
The eight radiologists who did not meet the initial exam threshold participated in a second day of training focused on problematic polyp characteristics and colonic conditions missed by more than 20 percent of radiologists. More than 80 percent of the missed polyps were sessile or flat, and one-quarter of missed polyps were located on a fold, were visible on one view only, had an undulating surface or resembled a bulbous or thickened fold.
A second exam was tailored for each radiologist and included all missed cases, a positive and negative case and a difficult-to-detect polyp. After the second exam, all 15 radiologists achieved a minimum of 90 percent sensitivity for easy- and moderate to detect polyps at least 5 mm in size. Sensitivity decreased as detection difficulty increased with mean sensitivities for easy-, moderate- and difficult-to-detect polyps at 92 percent, 76 percent and 42 percent, respectively.
An important message from the National CTC training experience is that CTC can be taught in a way that superior performance can be translated into a variety of practice settings, summed Fletcher. “Appropriate case selection for CTC teaching is critical, and courses for abdominal radiologists should include 50 endoscopically-validated cases with difficult-to -detect polyps and cancers that require a full search of the colorectum, with frequent feedback on radiologist performance,” the authors wrote.
Because of well-documented variability in CTC observer performance the National CTC Trial sought to minimize interobserver variability among participating radiologists by instituting a mandatory reader training and performance standards program, requiring radiologists to pass a qualification exam before study initiation.
All participants achieved sensitivity of approximately 90 percent, demonstrating that “CTC can be taught to others and performed with a high level of expertise by radiologists in many practice settings. However, in order to achieve this level of performance, physicians must immerse themselves in a carefully planned, hands-on course where they are reviewing cases and receiving feedback on their performance,” wrote lead author Joel G. Fletcher, MD, a radiologist from Mayo Clinic in Rochester, Minn.
Focused and tailored
The National CTC Trial required that ten radiologists who had interpreted fewer than 500 CTC exams with colonoscopic correlation participate in mandatory training before taking a qualification exam. Five experienced radiologists who had read more than 500 cases were allowed to take the exam without training.
The one day training session consisted of partial CTC exams of 27 cases with neoplasia and full review of 15 cases using primary 2D and 3D search. The initial phase of the training was designed to familiarize radiologists with the workstation, show the spectrum of colorectal neoplasia appearances and teach interactive problem-solving skills. Ten cases showed different cancer morphologies (annular, flat and polypoid), and 17 cases showed different morphologies and features of colorectal polyps (sessile, pedunculated, flat or villous).
In the final 15 cases, designed to teach colorectal lesion search methods using 2D and 3D fly-through techniques, radiologists interpreted full supine and prone colonography exams using primary 2D or 3D search.
All 15 radiologists took a qualification exam that included 20 CTC cases: 17 positive cases with 25 colonoscopy-verified polyps and three negative cases. Fifteen polyps were at least 10 mm in size, and 10 were at least 5 mm in size. Two experienced colonographers characterized polyps as easy-, moderate- or difficult-to-detect. Radiologists were required to detect 90 percent of the easy- and moderate-to-detect polyps at least 5 mm in size to pass the exam.
The eight radiologists who did not meet the initial exam threshold participated in a second day of training focused on problematic polyp characteristics and colonic conditions missed by more than 20 percent of radiologists. More than 80 percent of the missed polyps were sessile or flat, and one-quarter of missed polyps were located on a fold, were visible on one view only, had an undulating surface or resembled a bulbous or thickened fold.
A second exam was tailored for each radiologist and included all missed cases, a positive and negative case and a difficult-to-detect polyp. After the second exam, all 15 radiologists achieved a minimum of 90 percent sensitivity for easy- and moderate to detect polyps at least 5 mm in size. Sensitivity decreased as detection difficulty increased with mean sensitivities for easy-, moderate- and difficult-to-detect polyps at 92 percent, 76 percent and 42 percent, respectively.
An important message from the National CTC training experience is that CTC can be taught in a way that superior performance can be translated into a variety of practice settings, summed Fletcher. “Appropriate case selection for CTC teaching is critical, and courses for abdominal radiologists should include 50 endoscopically-validated cases with difficult-to -detect polyps and cancers that require a full search of the colorectum, with frequent feedback on radiologist performance,” the authors wrote.