Consumers have become less willing to blindly follow the advice of “experts” over the years. We’re more active and activist — taking the initiative to delve into researching everything from the best refrigerators to buy to the best options in big life decisions. And for many people, that includes checking out several different nursing homes before signing an admission agreement. But poring over websites or even touring the facilities can’t give a true picture of the quality of care that goes on behind closed doors.

In an attempt to address this concern, the Centers for Medicare and Medicaid Services (CMS) launched a website in 1998 that provided comparative information about nursing homes. The original site, jargon-filled and difficult to parse, was largely used only by industry insiders.

In 2007, two Congressional hearings were targeted on another factor thought to affect the quality of nursing homes: ownership. At that time, private investment firms and investment companies were beginning to gobble up nursing home chains, making it difficult to find out who actually owned the facilities or to hold anyone accountable for problems.

The situation provoked Senator Ron Wyden (D-Oregon) to make the oft-quoted observation: “Something is out of whack in this country when it’s a lot easier to find information about a washing machine than to find information about long-term care facilities.”

In response, two years later, CMS added a five-star rating system to the site, Nursing Home Compare in an effort to produce a type of online report card of nursing homes for consumers, their families, and caregivers. Every facility listed — which includes only those certified by Medicare or Medicaid — is given separate star ratings for:

• Health inspections — based on the last two onsite visits and complaint investigations conducted by state personnel;
• Staffing — the average number of hours of care nursing staff provides to residents each day; and
• Quality measures — based on 16 different physical and clinical measures such as percentage of residents who experience falls, urinary tract infections, or bedsores, and those who receive antipsychotic medications and who have symptoms of depression.

In addition, each nursing home is tagged with an overall quality rating from one star (much below average) to five stars (much above average).

The Faults in the Stars

Criticism of the site and its star rating system was fast and furious. Consumer advocates were quick to underscore that the underlying data seemed skewed. The information on health inspections, for example, is based on reports filed by state inspectors on assigned visits to facilities. In most states, the inspectors are a notoriously overworked, often undertrained bunch. (In my role as a former long-term care ombudsman, I observed firsthand many cursory “inspections,” marked by the inspectors’ tendencies to cozy up to staff while virtually ignoring the residents and the conditions around them.) And for the information on staffing and quality measures, CMS depends upon self-reports from facility administrators, who obviously have the most to gain by painting glowing pictures of themselves.

After an intensive examination of the rating system in 2014, The New York Times published a story with the accusatory headline: “Medicare Star Ratings Allow Nursing Homes to Game the System.” The story spotlighted one nursing home in a suburb of Sacramento, California that boasted a coveted five-star overall rating — a rating belied by the fact that more than 100 consumer complaints were filed against the facility during a four-year period, as well as about a dozen lawsuits from residents and their family members claiming substandard care. In accord with the CMS method, neither the complaints, nor the state fines imposed, nor the lawsuits were factored in when the overall quality ranking was computed.

The Times article also picked up on the questionable reliability of basing the nursing homes’ quality ratings on self-reported data, noting: “Of more than 50 nursing homes on a federal watch list for quality, nearly two-thirds hold four- or five-star ratings for their staff levels and quality statistics.” Both of those measures are reported by the facilities.

And as for “gaming the system,” the article noted that it is most commonly and most easily done when facility administrators play fast and loose with the staffing levels they report. Because adequate and attentive staff is considered so essential to good nursing home care, CMS automatically adds a star to the overall rating if a facility achieves either four or five stars for staffing. To pump up their scores, some facility administrators hire extra staff just before an inspection, then fire them shortly afterward, once the higher number is recorded. One administrator copped to committing the practice in an email, writing: “The staffing hours will be a little high for this week but will drop the following week.” His words came back to haunt him when they were used as evidence in a later lawsuit alleging falsification of records filed by the state’s attorney general.

More recently, researchers at the University of Chicago undertook a study of people who had placed a relative or a friend in a nursing home within a six-month period. They found that few consumers were aware of the CMS ratings, and when shown the website, most were skeptical about the source and accuracy of the data. In addition to urging transparency about the origins of the information used for the ratings, the researchers suggested the CMS site would be more helpful to consumers if it included information about costs, the activities available, and reviews and observations by actual residents.

Satisfaction v. Quality

Studies by CAHPS related to the CMS ratings took aim at the fact that the views of residents and family members are excluded. The CAHPS Nursing Home Surveys include one for those who live in a nursing home and another for those who have been discharged after a short stay. It sought to understand whether their levels of satisfaction with a facility relate to the star ratings given, and to answer the question of whether the government should be required to collect and report data about satisfaction levels. Only three states — Ohio, Minnesota, and Maryland — currently publish information related to nursing home satisfaction.

Scratching the Iceberg

Prior research findings have suggested that requiring states to collect and provide data about nursing home satisfaction may or may not be a good idea, as it takes money and effort to collect and may actually divert providers from giving more priority to easy fixes such as aesthetics rather than concentrating on providing better care.

However  arguably, the researchers could have been blindsided by the tangential finding that family members indicated much higher satisfaction levels with the nursing homes studied than the residents actually living there. This could be at least partially explained by family members’ desires to believe they had helped place their loved ones were in the best possible spots. But it raised interesting questions above and beyond the issues of quality and satisfaction.