What we learn from two new studies of patient satisfaction surveys

Over at Development Impact, I blog on two recent publications I had about how to better measure the patient experience in Nigeria.

Pitfalls of Patient Satisfaction Surveys and How to Avoid Them

A child has a fever. Her father rushes to his community’s clinic, his daughter in his arms. He waits. A nurse asks him questions and examines his child. She gives him advice and perhaps a prescription to get filled at a pharmacy. He leaves.

How do we measure the quality of care that this father and his daughter received? There are many ingredients: Was the clinic open? Was a nurse present? Was the patient attended to swiftly? Did the nurse know what she was talking about? Did she have access to needed equipment and supplies?

Both health systems and researchers have made efforts to measure the quality of each of these ingredients, with a range of tools. Interviewers pose hypothetical situations to doctors and nurses to test their knowledge. Inspectors examine the cleanliness and organization of the facility, or they make surprise visits to measure health worker attendance. Actors posing as patients test both the knowledge and the effort of health workers.

Read more…


For complex debates, an alternative to social media

There is an alternative to Twitter, Facebook and all those indignant op-eds that we use to confirm the superiority of our beliefs. It’s a flexible, troll-free, hacker-resistant platform on which complex social and moral questions can be carefully explored. It simultaneously engages our empathy and models the action of empathy for us. It’s called a novel.

That’s Ron Charles in the Washington Post.

I write it down so I don’t have to remember it

In the film Indiana Jones and the Last Crusade, Henry Jones — Indiana’s father — finds the key information needed to safely traverse a perilous journey to retrieve the Holy Grail.

Professor Henry Jones: Well, he who finds the Grail must face the final challenge.

Indiana Jones: What final challenge?

Professor Henry Jones: Three devices of such lethal cunning.

Indiana Jones: Booby traps?

Professor Henry Jones: Oh, yes. But I found the clues that will safely take us through them in the Chronicles of St. Anselm.

Indiana Jones: [pleased] Well, what are they?


Indiana Jones: Can’t you remember?

Professor Henry Jones: I wrote them down in my diary so that I wouldn’t *have* to remember.

[Dialogue is documented at IMDB.com.]

Just yesterday, I wrote to a colleague asking for information, and he pointed me to a blog post that I wrote two months ago. I often keep my research findings straight, but — to adapt from Henry Jones — I write them down so that I don’t have to remember them! As University of Chicago professor Linda Ginzel tells her students, “If you don’t write it down, it doesn’t exist.”

In honor of the senior Dr. Jones, I made this little reminder…

How do researchers estimate regressions with patient satisfaction at the outcome? A brief review of practice

Recently, Anna Welander Tärneberg and I were doing research with patient satisfaction as the outcome, and we checked how other researchers had estimated these equations in the past. Here is what we found, as documented in the appendix of our recently published paper in the journal Health Economics.


People use a lot of different methods, and many authors use multiple methods. But there is a rich history of using Ordinary Least Squares regressions to estimate impacts on patient satisfaction. In our paper, we used OLS but verified all the results with Probit and Logit regressions. To add to this list, Dunsch et al. (including me) have a new paper out last week on patient satisfaction in Nigeria, also using OLS as the main estimation method.

What were the best graphic books of 2017?

best graphicI’ve been reading more graphic books lately — graphic novels, graphic short stories, graphic memoirs, graphic biographies — and I’ve found that many outlets posted “best of 2017” lists.

I went through 15 “best of 2017” lists and identified the 100 graphic books that were recommended between them. A total of 10 books were recommended at least 3 “best of” lists. I’ve read half of the top 10, and they really are excellent, but there are many gems deeper down the list. So don’t stop digging.

The top 10 are listed below. The full list of 100 recommendations is available here. The numbering is funny because I use the same number — such 2a and 2b — if books received the same number of recommendations.

1. My Favorite Thing Is Monsters, by Ferris (recommended on 11 lists)

2a. The Best We Could Do: An Illustrated Memoir, by Bui (9 lists)

2b. Boundless, by Tamaki (9 lists)

4. Everything Is Flammable, by Bell (6 lists)

5. Hostage, by Delisle (5 lists)

6a. Imagine Only Wanting This, by Radtke (4 lists)

6b. Spinning, by Walden (4 lists)

8a. The Flintstones, Volume 1, by Russell & Pugh (3 lists)

8b. Shade the Changing Girl, Volume 1, by Castellucci & Zarcone (3 lists)

8c. You & a Bike & a Road, by Davis (3 lists)

Eight of the top ten are by women, which is cool (1, 2a, 2b, 4, 6a, 6b, 8b, and 8c).

Happy reading!


Are patients really satisfied?

Yesterday, my new paper — Bias in patient satisfaction surveys: a threat to measuring healthcare quality — was published at BMJ Global Health. It’s co-authored with Felipe Dunsch, Mario Macis, and Qiao Wang.

Here’s a quick rundown of what we learned.

Many health systems use patient satisfaction surveys to gauge one key element of health services: the patient experience. As part of evaluating an intervention to improve health care management in Nigeria, we piloted patient satisfaction questionnaires.

bad gus fring GIF-downsized

A common way to measure patient satisfaction is to give patients a series of statements and to ask if they agree or disagree: “The clinic was clean.” “Staff explained your condition well.” That kind of thing. We noticed that people seemed to be agreeing with everything.

sesame street nod GIF-downsized

Was that because the quality of care was good, or because saying yes is just easier?

neil degrasse tyson whatever GIF-downsized

So we randomly assigned some patients to get the standard statements, and others to get negatively framed statements. “The clinic was dirty.” “Staff were rude.” “Staff explained your condition poorly.”

questions asking GIF by Grandfathered-downsized

Reframing the questions negatively led to significantly lower reports of patient satisfaction on almost every item.

sad inside out GIF-downsized

Sometimes the drops were big, as high as 12 percentage points and 19 percentage points.

leaving gene kelly GIF-downsized

Our work was in Nigeria, but a lot of patient satisfaction questionnaires that we reviewed from a lot of countries use this positive framing.

everybody shrug GIF by Harlem Globetrotters-downsized

But that likely gives a falsely optimistic view of the patient experience.

happy will smith GIF-downsized

Patient satisfaction questionnaires that either mix positive and negative framing, or that avoid agree/disagree formats can do better.

better GIF-downsized

Of course, the quality of care is much more than patient satisfaction. But good health care systems can and should offer a positive patient experience.

parks and recreation thumbs up GIF by HULU-downsized

Check out the paper!

What I’ve been reading this month – March 2018

great escapeThe Great Escape: Health, Wealth, and the Origins of Inequality, by Angus Deaton. Deaton — winner of the 2015 Nobel Memorial Prize in Economic Sciences — offers a thorough and thoughtful history of inequalities in health and wealth. Within the broader analysis lies a wealth of gems: “The familiar concept of gross domestic product (GDP) is a good place to start (though it would be a very poor place to stop).” “In truth there are no experts on what a poor family ‘needs’ — except perhaps the poor family itself.” Much of the press coverage of this book focused on Deaton’s critique of foreign aid, which is interesting (whether you agree with it or not), but it’s a minority of the book.

best we could doThe Best We Could Do: An Illustrated Memoir, by Thi Bui — In this graphic memoir, a Vietnamese-American woman explores her family history and what it means for her own identity. She traces her parents’ stories during the Vietnam War, her family’s migration to the U.S. when she was a child, and their subsequent life as immigrants. It’s a beautiful and powerful and difficult tale. Abraham Riesman at Vulture: “Narratively intricate, intellectually fastidious, and visually stunning.” Publisher’s Weekly: A “mélange of comedy and tragedy, family love and brokenness.”

undoingThe Undoing Project: A Friendship that Changed Our Minds, by Michael Lewis — This is the fascinating story of an exceptional research collaboration that spanned decades, together with the resultant breakthroughs in psychology, with significant spillovers into economics and many other fields. Daniel Kahneman and Amos Tversky were pioneers in rethinking how we should think about how people think. Sunstein & Thaler in the New Yorker: Lewis tells “fascinating stories about intriguing people” and leaves “readers to make their own judgments about what lessons should be learned.” One great quote in the book — from Amos Tversky — points to an attribution problem that raises its head in development work constantly: “It is sometimes easier to make the world a better place than to prove you have made the world a better place.”

boundlessBoundless, by Jillian Tamaki — This wonderfully weird collection of short stories in comic format ends mid-word, fittingly. The most original and intriguing graphic work I remember reading. Michael Cavna in the Washington Post: “Organically disorienting, visually jarring — and, sometimes, sublime.” Rachel Cooke in the Guardian: “Fleeting as they are – most [of the stories] can be read in as long as it takes to order and receive a latte – each one is as indelible as it is singular.”

paperbacks from hellPaperbacks from Hell: The Twisted History of ’70s and ’80s Horror Fiction, by Grady Hendrix with Will Errickson — I don’t read horror fiction. When I was a teenager I read some Stephen King; that’s about it. But the genre experienced a heyday in the 70s and 80s, and Hendrix and Errickson document it in this entertaining cultural history. We stroll through the variety of sub-genres (possessed animals; crazy clowns; haunted houses). Hendrix and Errickson pay special attention to cover art, which is often innovative and just-as-often absurd. While many (or most) of the books he describes sound objectively pretty bad, they still testify to the boundless imagination of the authorial race. For example, here’s a roadmap through horror novels about bad kids: “Why do children act out? They might be nature creatures (Nursery Tale, Strange Seed), possessed by a vengeful spirit (Judgment Day), covering up a murder (The Little Girl Who Lives Down the Lane), or plotting a murder (Harriet Said…). Maybe they’re an ape-human hybrid (The Sendai), getting attacked by snakes (The Accursed), a hammer-wielding hellion (Mama’s Little Girl), capable of raising the dead (The Savior), or juggling a second personality that’s probably a demon (Smart as the Devil).” Below are a few of the covers featured in the book, in case you need more enticement.


sourdoughSourdough, by Robin Sloan — Robin Sloan is back! A few years ago he wrote a delightful debut novel — Mr. Penumbra’s 24-Hour Bookstore — and I’ve been excited to get to this, his new novel. Jason Sheehan at NPR put it well: “Robin Sloan’s new novel, Sourdough, is exactly like his first book, Mr. Penumbra’s 24-Hour Bookstore, except that it’s not about books (exactly), but is absolutely about San Francisco, geeks, nerds, coders, secret societies, bizarrely low-impact conspiracies that solely concern single-noun obsessives (food, in this case, rather than books), and also robots. And books, too, actually, now that I think about it.” It also involves a magical sourdough starter and warring colonies of microbes. It’s great fun.

baking cakesBaking Cakes in Kigali, by Gaile Parkin — Angel Tungaraza is a Tanzanian woman who has moved to Rwanda with her family and started a cake-baking business. She meets lots of interesting people and has interesting conversations with them. The novel is reminiscent of The No. 1 Ladies’ Detective Agency in that it tries to say something about current issues while not really focusing on plot and not having any serious conflict. The World Bank and the IMF both appear, with unflattering characterizations. And one exchange about the source of motivation for development workers inspired one of my recent blog posts: “Perhaps these big organisations needed to pay big salaries if they wanted to attract the right kind of people; but Sophie had said that they were the wrong kind of people if they would not do the work for less. Ultimately they had concluded that the desire to make the world a better place was not something that belonged in a person’s pocket. No, it belonged in a person’s heart.”

never let you goNever Let You Go, by Chevy Stevens — Thriller! Set in British Columbia! About a women, her daughter, and her recently-released-from-prison abusive ex-husband. Exciting! I did not predict the ending.

Fangirl, by Rainbow Rowell — Lovely coming-of-age story about an avid fangirlfan fiction author Cather over the course of her first year in college. I loved her writing professor and wish we’d seen more of her. I’ve never read any fan fiction (except a couple of pages of 50 Shades of Grey — which originated as Twilight fan fiction — over someone’s shoulder on the metro), but Cather’s sentiments are deeply relatable: “Maybe you think I’m a little crazy, but I only ever let people see the tip of my crazy iceberg. Underneath this veneer of slightly crazy and socially inept, I’m a complete disaster.”

Kids’ Stuff
waking the monstersHilo Book 4: Waking the Monsters, by Judd Winick — Girl’s mom wants her to be a cheerleader; she wants to be a ninja wizard instead. Robots! Aliens! So much awesomeness! Can’t wait for book 5! I love reading these with my kids.