Deciphering The Medicare Hospital Star Rating System

CMS just released its newest star ratings for U.S. hospitals. The ratings were supposed to come out last summer but there was a delay because of problems with the methodology Medicare was using. The rating system attempts to compare hospitals’ performance with common conditions. Hospitals are rated with 1 – 5 stars, with 5 stars being the best and 3 stars being average.

Medicare uses a 6-step process for calculation of the number of stars awarded to each hospital:

Step 1: Selection and standardization of measures for inclusion in the Star Rating 

Step 2: Assignment of measures to groups 

Step 3: Calculation of latent variable model group scores 

Step 4: Calculation of hospital summary scores as a weighted average of group scores 

Step 5: Application of minimum thresholds for receiving a star rating 

Step 6: Application of clustering algorithm to categorize summary scores into star ratings

Medicare evaluates hospitals on up to 57 different measures. Not all hospitals have sufficient volumes of patients for each measure so the average hospital is rated on about 39 different measures. The measures can be grouped into seven different categories and then each category has several different measures:

1. Mortality (7 measures)
   1. Death rate from heart attack
   2. Death rate from CABG
   3. Death rate from COPD
   4. Death rate for CHF
   5. Death rate for pneumonia
   6. Death rate for Stroke
   7. Death rate for patients with serious complications from surgery
2. Safety of care (8 measures)
   1. Central line-associated bloodstream infections
   2. Catheter-associated urinary tract infections
   3. Surgical site infections from colon surgery
   4. Surgical site infections from abdominal hystectomy
   5. MRSA
   6. Clostridium difficile infections
   7. Knee/hip replacement complications
   8. Serious complications
3. Readmission (9 measures)
   1. Heart attack
   2. CABG
   3. COPD
   4. Heart failure
   5. Knee/hip surgery
   6. Pneumonia
   7. Stroke
   8. All discharged patients
   9. Unplanned hospital visits after outpatient colonoscopy
4. Patient experience (11 measures)
   1. Nurse communication
   2. Doctor communication
   3. Patients reporting receiving help as soon as they wanted it
   4. Pain control
   5. Patients reporting that staff explained about medications before giving it to them
   6. Room and bathroom cleanliness
   7. Hospital room was quiet at night
   8. Patients reporting that they were given information about what to do during their recovery at home
   9. Patients reporting that they understood their care when they left the hospital
   10. Overall patient rating of the hospital
   11. Patients reporting that they would recommend the hospital to a friend or family
5. Effectiveness of care (10 measures)
   1. Patients getting flu shots
   2. Healthcare workers getting flu shots
   3. Aspirin within 24 hours of arrival with chest pain or heart attack
   4. Patients leaving the ER without being seen
   5. Patients with stroke symptoms who got a CT within 45 minutes of arrival
   6. Percentage of patients receiving appropriate recommendations for follow-up screening colonoscopy
   7. Percentage of patients with history of colon polyps getting follow-up colonoscopy in appropriate timeframe
   8. Percent of mothers whose deliveries were scheduled 1-2 weeks too early
   9. Patients who developed a blood clot and did not get DVT prophylaxis
   10. Percentage of patients receiving appropriate radiation therapy for cancer that spread to the bone
6. Timeliness of care (7 measures)
   1. Time waiting in the ER for patients admitted to the hospital
   2. Time spent in the ER after the doctor decided to admit the patient to the hospital
   3. Minutes before patients with heart attack needing specialized care were transferred to another hospital
   4. Minutes from presentation to the ER to getting an EKG for patient with possible heart attack
   5. Total time in the ER for patients released (without admission)
   6. Time spent in the ER waiting to be seen by a healthcare professional
   7. Time waiting in the ER before getting pain medication for patients with broken bones
7. Efficient use of medical imaging (5 measures)
   1. Outpatients with low back pain getting an MRI without getting physical therapy first
   2. Outpatient CT scans of the abdomen that were combination scans
   3. Outpatient CT scans of the chest that were combination scans
   4. Outpatients getting cardiac stress tests before low-risk outpatient surgery
   5. Outpatients with brain CT scans go also got a sinus CT at the same time

The different categories are weighted differently in the final calculation: the first four categories are weighted 22% each and the last three categories are weighted 4% each. Therefore, they way to have your hospital perform well on the star system is to have a low mortality rate, have a low readmission rate, have a low incidence of hospital-acquired conditions, and have happy patients. The selection of the various measures within each of these categories becomes extremely important since different hospitals inevitably do better at some things than at others. Also, the time period that the measures are collected over varies with some measures collected from 2013-2016, others from 2015-2016, and still others from 2016-2017. The 57 measures can be found on the CMS Hospital Compare website. Some of these measures are rather general (e.g. percentage of patients who reported that their doctors communicated well) and others are rather specific (e.g. surgical site infections from abdominal hysterectomy). Some of the measures are directed toward patients (e.g. the percentage of patients assessed and given influenza vaccine) and some are directed toward the hospital employees (e.g. the percentage of healthcare workers assessed and given influenza vaccine).

Here is the overall star ratings for U.S. hospitals for 2016 and 2017. Notably, the percentage of both 5-star and 1-star hospitals increased since 2016.

The measures that CMS chooses end up being the “invisible hand” that dictates hospital priorities. Every year, hospital quality departments analyze whichever measures CMS has most recently chosen and hospitals focus on trying to do their best to keep the numbers up for those specific measures. Thus the CMS star system directs how hospitals will use their limited resources to improve their performance on those specific measures. This results in Medicare having a huge impact on how hospitals define quality and how they elect to improve quality outcomes. One of the challenges that hospitals face is that the measures look backwards for up to 3 years and the specific measures change from year to year so you never know exactly what you are going to be judged on from one year to the next and the data usually reflect your performance from more than a year ago.

So, does the CMS Hospital Compare star rating system really tell you which hospital in your area is the best one? Well… maybe. If you are being admitted for an elective hip replacement surgery, it probably doesn’t really matter to you how many minutes patients in the hospital’s emergency room spend before being discharged. But if the hospital performs well on the star system, then this indicates that the hospital is at least monitoring quality metrics and working to try to improve those metrics. So, if you are being admitted with gastroenteritis, then hopefully the things that the hospital is doing to reduce heart failure mortality will also spill over to keep gastroenteritis mortality down, also.

This year, our medical center did very well and got a 5-star rating and we are very proud of that accomplishment. I’d like to think that is because we provide the highest quality care possible but the reality is that a lot of hospitals that I know got 2, 3 or 4 star ratings even though they are very good hospitals that I would be comfortable being treated at. The star rating system is imperfect but consider it as a work in progress that is a first start at trying to objectively quantify medical quality.

January 18, 2018