It may be the case that the 20% “shift” in support was really a reflection of different question wordings and sample frames.
A year and a half ago, my colleagues posted these polling results showing that 28% of Utah voters support same-sex marriage (as of mid-2012). A few days ago, the Salt Lake Tribune published its own poll showing what appears to be a huge shift: 48% of Utahns supporting same-sex marriage. There is no doubt that Utahns’ support for same-sex marriage is growing (see here), but twenty percentage points? That’s a tremendous shift in a short time.
I have no quibbles with either poll, but be careful before comparing them too hastily. They differ in two key ways. First, the BYU poll sampled Utah voters, while the Tribune poll sampled Utah adults; those are different populations, and they will produce different results. Second, the two polls used different question wording, which can have large effects. Neither poll has a “better” wording or sampling frame; they are just different, and they produce a different look at public opinion in Utah.
You can read a much more detailed analysis of these two points in this post from Scott Riding at Y2 Analytics. (Disclosure: Scott is my former student, and two of my colleagues are closely involved with Y2 Analytics, though I am not.) Scott also reports the results of a new Y2 Analytics poll that replicates the BYU poll’s question wording and sampling method; the new poll does not find much movement. So it may be the case that the 20% “shift” in support was really a reflection of different question wordings and sample frames.
To illustrate the question wording effect, let’s look at two questions within the Tribune poll. Both asked about roughly the same thing, but they got different results. First, the Tribune asked this: “Should same-sex couples in Utah be allowed to form civil unions or domestic partnerships?” 25% said no. Later, they asked this: “Do you support or do you oppose any legal recognition of same-sex relationships?” 37% opposed.
Let’s think this through. 37 is 12 higher than 25. So apparently 12% of Utah adults oppose any legal recognition of same-sex relationships, yet they support civil unions or domestic partnerships. That makes no sense at all. This doesn’t mean that 12% of respondents are daft. It means that question wording can influence results, sometimes significantly.
Let me stress that I’m not taking sides whether the Tribune poll or the BYU/CSED poll is better. They both seem good to me. But they use different questions and samples, so we need to be careful when comparing them and identifying trends.
If you didn’t click that link to Scott’s post earlier, you should do it now. He has much more to add–especially the results of an additional (recent) poll.
Possibly related posts:
Possibly related posts (automatically generated)
Why pollsters should release their topline results (November 6, 2012)
Support for gay marriage is definitely growing, but not as fast as the pro-gay marriage movement wants people to believe. Pro-gay marriage groups were already arguing for gay marriage’s “inevitability” in 2012, when North Carolina voters approved a ban 61-39.