Physician Report Cards are Flawed

Many people wish to compare doctors against each other across quality metrics. However, report cards are flawed.

"FAIL" (Creative Commons)Americans have been concerned about the quality of their health care in recent decades. As part of the process to inform patients about the quality of health care providers, tools such Hospital Compare and the Leap Frog Group have evolved. However, the validity of reports cards allowing for the comparison of individual physicians remains in question.

This study looked within a single health care system encompassing 3 hospitals, 40 outpatient clinics, and over 1600 physicians. Investigators looked at quality indicators for patients with diabetes, congestive heart failure, ischemic vascular disease, and certain preventive services.  Using a multilevel logistic regression technique, they sought to determine if existing measures were reliable enough to determine differences in quality between individual physicians and entire practice groups.

The sample drew from 51,771 patients covered by 1 of 163 primary care physicians in 17 different clinics. The data revealed that there was measurable variation between physicians for many diabetes measures (11 of 14) and for preventive care (4 of 4). However, there was not significant variation among physicians for heart failure (0 of 9) or vascular disease measures (1 of 7).

After combining individual measures into disease-specific composite measures, enough variability was created to meaningfully distinguish quality for diabetes, vascular disease, and prevention but not for congestive heart failure.

While measurable variation existed for some metrics, fewer than 63% of physicians had an adequate panel size to consistently discover this variation. Using composite scores meant that about 75% of doctors had adequate panel size for diabetes and prevention metrics.

When looking at provider groups, neither individual nor composite quality scores for diabetes, heart failure, or vascular disease provided discriminatory power. Only for individual prevention measures of breast and cervical cancer screening, or the composite prevention measure, were provider groups distinguishable.

Commentary

The is a welcome phenomenon as many people inside and outside the health care system realize that care could be better coordinated, standardized, and safer for patients. However, how to best accomplish this complex task of improving quality remains elusive.

As we have discussed in many prior posts, yet lack consistent proof of effectiveness across multiple domains.

Health information technology offers benefits which anecdotally and sometimes empirically suggest that care processes might improve. Yet, have not necessarily been enjoyed in real practice. As most of these systems are designed to improve billing and not quality, this comes as no surprise to many familiar with the issue.

So what can be expected from physician report cards and web sites like Physician Compare in the future?

These new data suggest that the grades on physician report cards must be carefully selected to ensure that (1) quality measures adequately distinguish “high-quality” physicians from “low-quality” ones and (2) that sufficient data  is easily available to create a realistic score for each provider.

Smith, KA. Medical Care. 2013; 51 (3): 266-74.

by

Cedric Dark, MD, MPH