Ability to read mammograms varies widely

Ability to read mammograms varies widely

April 20, 1998

The mammogram-reading skills of general radiologists vary enormously according to a study to be presented Monday, April 20, 1998, at the 28th American College of Radiology's National Conference on Breast Cancer in Washington, DC. Although the researchers who performed the research were impressed with the overall ability of most general radiologists, they were concerned about the magnitude of the variance.

Seventy-five general radiologists detected, on average, 70 percent of 50 breast cancers visible on 100 mammograms, about what previous studies would predict. Three expert mammographers detected 81 percent. All the experts had similar scores, ranging from 76 percent to 86 percent. For the generalists, however, scores ranged from a high of 98 percent--with a lot of false positives (normal exams rated as suspicious)--to a low of 8 percent.

"The generalists' scores were all over the map," said Robert Schmidt, MD, associate professor of radiology at the University of Chicago and director of the study, "rather than clustered around 70 percent. The average score was no surprise. Most previous studies suggest that 15 to 30 percent of cancers are not detected at the first opportunity. But the extreme range was an eye opener."

Moreover, there was only a weak correlation between the test results for each generalist and his or her self-assessed level of expertise. For example, some of those who classified themselves as having advanced skills scored lower than those who rated themselves as novices.

"Imaging technology has steadily improved and continues to get better," said Dr. Schmidt. "The human observer may now be the weakest link in the breast-imaging chain."

Most quality-of-care research is performed at academic centers and focuses of the talents of highly specialized experts who devote their careers to a few specific interests, such as mammography.

The overwhelming majority of screening mammograms, however, are read by general radiologists, who, by law, may read as few as two mammograms a day.

The researchers--from the University of Chicago, New York University, University of New Mexico at Albuquerque, and University of Illinois at Peoria--presented high-quality copy films from 100 mammography cases to 100 radiologists who were attending continuing medical education meetings, and to four mammography experts.

Each radiologist was given two-and-one-half hours to complete the exercise, conditions roughly comparable to a routine clinical situation, except for the far higher percentage of cancers. Ordinarily, there are only about five cancers per 1,000 screening mammograms.

The 100 films contained 55 normal cases and 45 cases with at least one cancer. (Five cases had two cancers). The observers were asked to rate the cases as normal or abnormal and to indicate the location of any suspicious lesion. The cancers were typical lesions found by routine screening mammograms, not unusually difficult or "tricky" cases.

Only 75 percent of each group of radiologists finished the task. Those who gave answers for fewer than 90 of the 100 cases were not included in the final assessment, "but they clearly would not have improved the scores," noted Schmidt, dryly. Those who did not complete the test found only 42 percent of the cancers in the cases they completed.

The experts detected 16 percent more cancers than generalists. Experts, however, were correspondingly more likely to rate normal cases as abnormal.

"Dedicated mammographers seem to have a lower threshold for what they find suspicious than generalists," said Schmidt. "They found significantly more cancers but wanted to take a closer look at more women who did not have cancers."

Although the 70-percent detection rate may seem low, "reading mammograms is extremely difficult, even for experts," cautioned Schmidt. It requires a lot of training and experience. "Very few people who don't do it full-time develop the skills to do it at the highest levels."

That's because reading screening mammograms is very different from other tasks performed by radiologists. Instead of looking for the cause of symptoms in a sick person, screening studies look for signs of minute, pre-symptomatic disease in apparently healthy patients. It requires a different mind-set, says Schmidt, and different sets of behaviors.

"The good news," added Schmidt, "is that this is a problem we know how to solve. We can teach generalists to read these images consistently and with a level of accuracy level closer to that of the experts. But we need to develop methods to make this training more accessible to all radiologists."

On one case, 83 percent of the generalists missed a cancer found by 75 percent of the experts. "As you might expect, we now use that as a teaching case," emphasized Dr. Schmidt.

Ongoing technology improvements will also help. Digital mammography units, which enable physicians to manipulate the images in new ways, are entering the market. The first such unit in Chicago is currently being installed in the University's mammography center. And researchers at the University of Chicago have developed the first computer that reads mammograms, providing a second opinion for the radiologist and cutting the error rate. This technology is expected to become commercially available within a year.

[Note: added in 1998. This technology had been licensed and commercialized by a company known as R2, which in April 1998 arranged with GE Medical Systems to implement their ImageChecker system as part of GEMS's digital mammography machines. An FDA advisory committee in May, 1998, recommended approval of the technology.] Until these technical advances become more common, Dr. Schmidt recommends that women seek breast programs at high-volume centers, such as academic institutions or specialized breast diagnostic centers with dedicated mammographers, who are more likely to have the advanced training and experience. Improved training in mammography reading could help tens of thousands of women each year.

The American Cancer Society estimates that 178,700 women will be diagnosed with invasive breast cancer in the United States in 1998 and 43,900 women will die from the disease. Mammography is the only known method that can find small, curable cancers before they can be felt and has been shown to significantly decrease deaths from breast cancer.

Summary:
The mammogram-reading skills of general radiologists vary enormously. In a test situation, 75 general radiologists detected, on average, 70 percent of breast cancers--about what previous studies would predict. However, their sensitivity showed tremendous variability. Expert mammographers detected 16 percent more cancers.