How do researchers estimate regressions with patient satisfaction at the outcome? A brief review of practice

Recently, Anna Welander Tärneberg and I were doing research with patient satisfaction as the outcome, and we checked how other researchers had estimated these equations in the past. Here is what we found, as documented in the appendix of our recently published paper in the journal Health Economics.

chart

People use a lot of different methods, and many authors use multiple methods. But there is a rich history of using Ordinary Least Squares regressions to estimate impacts on patient satisfaction. In our paper, we used OLS but verified all the results with Probit and Logit regressions. To add to this list, Dunsch et al. (including me) have a new paper out last week on patient satisfaction in Nigeria, also using OLS as the main estimation method.

Advertisements

Are patients really satisfied?

Yesterday, my new paper — Bias in patient satisfaction surveys: a threat to measuring healthcare quality — was published at BMJ Global Health. It’s co-authored with Felipe Dunsch, Mario Macis, and Qiao Wang.

Here’s a quick rundown of what we learned.

Many health systems use patient satisfaction surveys to gauge one key element of health services: the patient experience. As part of evaluating an intervention to improve health care management in Nigeria, we piloted patient satisfaction questionnaires.

bad gus fring GIF-downsized

A common way to measure patient satisfaction is to give patients a series of statements and to ask if they agree or disagree: “The clinic was clean.” “Staff explained your condition well.” That kind of thing. We noticed that people seemed to be agreeing with everything.

sesame street nod GIF-downsized

Was that because the quality of care was good, or because saying yes is just easier?

neil degrasse tyson whatever GIF-downsized

So we randomly assigned some patients to get the standard statements, and others to get negatively framed statements. “The clinic was dirty.” “Staff were rude.” “Staff explained your condition poorly.”

questions asking GIF by Grandfathered-downsized

Reframing the questions negatively led to significantly lower reports of patient satisfaction on almost every item.

sad inside out GIF-downsized

Sometimes the drops were big, as high as 12 percentage points and 19 percentage points.

leaving gene kelly GIF-downsized

Our work was in Nigeria, but a lot of patient satisfaction questionnaires that we reviewed from a lot of countries use this positive framing.

everybody shrug GIF by Harlem Globetrotters-downsized

But that likely gives a falsely optimistic view of the patient experience.

happy will smith GIF-downsized

Patient satisfaction questionnaires that either mix positive and negative framing, or that avoid agree/disagree formats can do better.

better GIF-downsized

Of course, the quality of care is much more than patient satisfaction. But good health care systems can and should offer a positive patient experience.

parks and recreation thumbs up GIF by HULU-downsized

Check out the paper!

How Angus Deaton thinks you can make the world a better place

When Princeton students come to talk with me, bringing their deep moral commitment to helping make the world a better, richer place, it is these ideas that I like to discuss, steering them away from plans to tithe from their future incomes, and from using their often formidable talents of persuasion to increase the amounts of foreign aid. I tell them to work on and within their own governments, persuading them to stop policies that hurt poor people, and to support international policies that make globalization work for poor people, not against them.

This is (almost) the end of Deaton’s book The Great Escape: Health, Wealth, and the Origins of Inequality. This counsel reminds me of the Commitment to Development Index, which shows that there are many policies that rich countries can enact to help the poor beyond their borders besides providing foreign aid, such as easier migration rules and lower tariffs.

Does increasing teacher salaries improve student test scores?

Over at Development Impact, I just posted a piece — What do we learn from increasing teacher salaries in Indonesia? More than the students did — where I discuss recent work by de Ree et al. on an impressive policy experiment, where Indonesia doubled base pay for many civil service teachers.

Here’s the abstract of their paper:

How does a large unconditional increase in salary affect the performance of incumbent employees in the public sector? We present experimental evidence on this question in the context of a policy change in Indonesia that led to a permanent doubling of teacher base salaries. Using a large-scale randomized experiment across a representative sample of Indonesian schools that accelerated this pay increase for teachers in treated schools, we find that the large pay increase significantly improved teachers’ satisfaction with their income, reduced the incidence of teachers holding outside jobs, and reduced self-reported financial stress. Nevertheless, after two and three years, the increase in pay led to no improvement in student learning outcomes. The effects are precisely estimated, and we can rule out even modest positive impacts on test scores. Our results suggest that unconditional pay increases are unlikely to be an effective policy option for improving the effort and productivity of incumbent employees in public-sector settings.

 

 

What I’ve been producing

I’ve been away for a while: here’s a little bit of what I’ve been up to. Last week, I gave a talk at Stanford University, entitled “The Global Landscape of In-Service Teacher Professional Development Programs,” which you can watch below.

I’ve put up a few new blog posts on other blogs, in case you missed them:

 

And a couple of my blog posts have made it into other language: