Lies, Damned Lies, and Statistics (29): How (Not) to Frame Survey Questions, Ctd.

comments 5
discrimination and hate / equality / lies and statistics / statistics

Following up from an older post on the importance of survey questions, here’s a nice example of the way in which small modifications in survey questions can radically change survey results:

homosexual or gay importance of survey questions

(source, source, source)

Another example:

Our survey asked the following familiar question concerning the “right to die”: “When a person has a disease that cannot be cured and is living in severe pain, do you think doctors should or should not be allowed by law to assist the patient to commit suicide if the patient requests it?”

57 percent said “doctors should be allowed,” and 42 percent said “doctors should not be allowed.” As Joshua Green and Matthew Jarvis explore in their chapter in our book, the response patterns to euthanasia questions will often differ based on framing. Framing that refers to “severe pain” and “physicians” will often lead to higher support for ending the patient’s life, while including the word “suicide” will dramatically lower support. (source)

Similarly, seniors are willing to pay considerably more for “medications” than for “drugs” or “medicine” (source). Yet another example involves the use of “Wall Street”: there’s greater public support for banking reform when the issue is more specifically framed as regulating “Wall Street banks”.

survey wording effect

(source)

What’s the cause of this sensitivity? Difficult to tell. Cognitive bias probably has some effect, and the psychology of associations (“suicide” brings up images of blood and pain, whereas “physicians” brings up images of control; similarly “homosexual” evokes sleazy bars, “gay” evokes art and design types). Maybe the willingness not to offend the person asking the question. Anyway, the conclusion is that pollsters should be very careful when framing questions. One tactic could be to use as many different words and synonyms as possible in order to avoid a bias created by one particular word.

More on DADT and homosexuals in the military. More on assisted suicide. More on lying with statistics.

5 Comments

  1. Pingback: Lies, Damned Lies, and Statistics (31): Common Problems in Opinion Polls « P.A.P. Blog – Human Rights Etc.

  2. Pingback: Statistical Jokes (14): How Not to Frame Survey Questions « P.A.P. Blog – Human Rights Etc.

  3. Pingback: Lies, Damned Lies, and Statistics (32): The Questioner Matters | P.a.p.-Blog, Human Rights Etc.

  4. Pingback: Lies, Danmed Lies, and Statistics (37): When Surveyed, People Express Opinions They Don’t Hold | P.a.p.-Blog | Human Rights Etc.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s