What the election shows: That social science works

Posted on November 7, 2012 by Adam Brown

Before any of the debates. Before SuperStorm Sandy. Before the last-minute barrage of campaign advertising. He predicted the result perfectly over a month ago.

Over the past several months, political scientists have consistently forecasted an Obama victory–a relatively narrow one, but a victory nonetheless. And, in the end, that’s exactly what happened yesterday. Every state but Florida has finished counting its votes. Assuming that Florida goes Democratic, then the following forecasters will all have correctly predicted EVERY state’s presidential vote:

Since this is a Utah-based blog, let’s look at Jay DeSart’s model for a minute. Over a month ago, Jay DeSart and his collaborator Thomas Holbrook posted this forecast:

This map was produced over a month ago!

Compare that forecast to the actual election result map. I took this one from the New York Times this morning:

The actual election result of of 9:00am Wednesday

Look carefully. You will not find a single difference. Working from here in Utah, Jay DeSart nailed it. Before any of the debates. Before SuperStorm Sandy. Before the last-minute barrage of campaign advertising. He predicted the result perfectly over a month ago.

Forecasts made far in advance of the election mostly relied on economic indicators, also called “the fundamentals” (which predicted a narrow victory for Obama). Forecasts made closer to election day mostly relied on polling data (which produced similar results). Either way, social science works. Consider what John Sides wrote today on every quantitative political scientist’s favorite blog, The Monkey Cage (the links are all his):

Barack Obama’s victory tonight is also a victory for the Moneyball approach to politics. It shows us that we can use systematic data–economic data, polling data–to separate momentum from no-mentum, to dispense with the gaseous emanations of pundits’ “guts” and ultimately to forecast the winner. The means and methods of political science, social science, and statistics, including polls, are not perfect, and Nate Silver is not our “algorithmic overlord” (a point I don’t think he would disagree with).

But 2012 has showed how useful and necessary these tools are for understanding how politics and elections work. Let’s hope that these tools become even more prominent in political journalism and punditry.

Randall Munroe put it more succinctly:

From xkcd.com/1131/

About Adam Brown: Adam Brown is an associate professor of political science at Brigham Young University and a research fellow with the Center for the Study of Elections and Democracy. You can learn more about him at his website.

Posted in Everything | Tagged , , , , , | Comments Off on What the election shows: That social science works

Utah Colleges Exit Poll Election Night

Posted on November 6, 2012 by Quin Monson

************Updated estimates as of 10:00pm

US President
Mitt Romney  72%
Barack Obama  25%
Someone else  3%
Projected winner: Mitt Romney

US Senate
Orrin Hatch  65%
Scott Howell  30%
Someone else  4%
Projected winner: Orrin Hatch

Governor
Gary Herbert  68%
Peter Cooke  28%
Someone else  4%
Projected winner:  Gary Herbert

Congressional District 1
Rob Bishop  70%
Donna McAleer  25%
Someone else  4%
Projected Winner:  Rob Bishop

Congressional District 2
Chris Stewart  58%
Jay Seegmiller  38%
Someone else  4%
Projected winner:  Chris Stewart

Congressional District 3
Jason Chaffetz  79%
Soren Simonsen  20%
Someone else  1%
Projected winner:  Jason Chaffetz

Congressional District 4
Mia Love  46%
Jim Matheson  50%
Someone else 4%
Projected winner:  **TOO CLOSE TO CALL**  The margin of error is approximately +- 4.5%.

Salt Lake County Mayor
Mark Crockett 38%
Ben McAdams 60%
Someone else 2%
Projected winner: **TOO CLOSE TO CALL**  We polled at only ten locations countywide, and even though we have a sample size of 818, the variability between polling places is too large to bring the margin between the candidates outside of the 95% confidence level.

Salt Lake County Council
Jim Bradley 56%
Joe Demma 42%
Someone else 4%
Projected winner: **TOO CLOSE TO CALL**

Salt Lake County Proposition 1
For 69%
Against 31%
Projected winner: For

*******************************

For the 16th time since 1982, the Utah Colleges Exit Poll has surveyed Utah voters as they left polling places in Utah throughout Election Day.  Today more than 500 student interviewers fanned out to 130 polling locations.

This year’s survey provided some different challenges.  First, with Governor Romney on the ballot, estimating turnout was difficult.  Second, the emphasis on early and absentee voting in Utah led us to conduct surveys of voters who voted before election day.

We’ve been entering data throughout the entire day. Below are the results for the statewide races and the four congressional districts as released at 8:00pm on KBYU.  We have also posted the estimates for some Salt Lake County races.  We’ve faced some obstacles in getting enough sample entered in the Salt Lake County races. When the margin of error (with 95% confidence) allows us to make a projection, we’ll indicate it below, otherwise we’ll indicate “TOO CLOSE TO CALL.”  With data continuing to come in throughout the night, these margins may change.  Check back here for updates.

US President
Mitt Romney  72%
Barack Obama  24%
Someone else  3%
Projected winner: Mitt Romney

US Senate
Orrin Hatch  66%
Scott Howell  30%
Someone else  4%
Projected winner: Orrin Hatch

Governor
Gary Herbert  69%
Peter Cooke  28%
Someone else  4%
Projected winner:  Gary Herbert

Congressional District 1
Rob Bishop  71%
Donna McAleer  25%
Someone else  4%
Projected Winner:  Rob Bishop

Congressional District 2
Chris Stewart  59%
Jay Seegmiller  38%
Someone else  4%
Projected winner:  Chris Stewart

Congressional District 3
Jason Chaffetz  80%
Soren Simonsen  19%
Someone else  1%
Projected winner:  Jason Chaffetz

Congressional District 4
Mia Love  46%
Jim Matheson  49%
Someone else 5%
Projected winner:  **TOO CLOSE TO CALL**

Salt Lake County Mayor
Mark Crockett 39%
Ben McAdams 58%
Someone else 2%
Projected winner: **TOO CLOSE TO CALL**

Salt Lake County Council
Jim Bradley 55%
Joe Demma 42%
Someone else 3%
Projected winner: **TOO CLOSE TO CALL**

Salt Lake County Proposition 1
For 68%
Against 32%
Projected winner: For

About Quin Monson: Quin Monson is Associate Professor of Political Science and a Senior Scholar with the Center for the Study of Elections and Democracy at Brigham Young University.

Posted in Everything | Comments Off on Utah Colleges Exit Poll Election Night

No, Utah will not have 80% turnout

Posted on November 6, 2012 by Adam Brown

Estimating Utah’s turnout by looking only at registered voters is like estimating Utah’s average wealth by looking only at those who are employed.

Important note (November 20, 2012). The turnout numbers below rely on data from Michael McDonald. He has updated his VEP numbers for Utah, so the turnout percentages I give below (and the denominator I give for 2012) would look somewhat different if I were to write this post with his updated data. I won’t go back and change the numbers in this post, but I will use McDonald’s updated numbers in future posts. If you conduct your own analysis, you should download the raw data from his website (linked below) rather than taking the numbers from this post. The broader points in this post about how to calculate turnout remain valid.

Newspapers this morning reported optimistic predictions from Utah’s election officials: “Based on early voting, I think we will be at least at 74 percent, and perhaps up to 80 percent” turnout among active registered voters. (Edited for clarity at 11:54am.)

There’s a major caveat there. Utah’s election administrators measure turnout in an unorthodox way that inflates the numbers. They report turnout as a percentage of registered voters. Meanwhile, reputable analysts measure turnout as a percentage of eligible voters.

How it changes the percentages

Take a look at some numbers. Utah’s voting age population (VAP) is the number of residents 18 years of age or older. Utah’s voting eligible population (VEP) removes institutionalized populations (e.g. felons) and other ineligible groups. VEP is the appropriate denominator when calculating turnout (read why). Utah’s estimated VAP and VEP reported below comes from Michael McDonald, the reigning expert on voter turnout in the United States.

Voting age population (VAP) in Utah 1,980,790
Voting eligible population (VEP) in Utah 1,835,666
Registered voters in Utah (as of June 2012) 1,287,892

Imagine how different these statistics will look if Utahns cast 1,000,000 ballots for president this year. The state elections office would report turnout of 77.6%. As a percentage of the voting eligible population, however, turnout would be only 54.5%. That’s a huge difference–23.1 percentage points.

There are two main reasons to report turnout as a percentage of VEP rather than as a percentage of registered voters.

Reason #1: It tells us what we’re actually interested in

In the United States, voting is a two-step process. First, you must register. Then, you must vote. The media report turnout statistics because people are curious to know how broadly American citizens have engaged the election. We all chuckle (or sigh) when we read that an odd-year municipal election received less than 10% turnout. But we all expect that a presidential election should engage the electorate and push turnout upward.

Turnout percentages are one measure of democracy’s health. If people believe that elections are important, they will register and vote. Estimating Utah’s turnout by looking only at registered voters is like estimating Utah’s average wealth by looking only at those who are employed.

Reason #2: It makes historical comparisons more accurate

Reporting turnout as a percentage of registered voters can also produce misleading trends. In 2010, Utah’s election officials claimed that turnout was the best for a midterm election in Utah in 16 years. With a little research, however, one learns the reason for this “improvement”: Fewer Utahns were registered to vote in 2010 than in previous years, inflating the turnout percentage when measured as a percentage of registered voters. When measured correctly (as a percentage of VEP), Utah’s turnout in 2010 was actually worse than every midterm election since 1994 except (barely) 2006.

Historical turnout data for Utah

For the record, here are the historical turnout percentages for Utah as a percentage of VEP. I include presidential years only. Data from Michael McDonald.

Year Utah’s VEP Votes cast for president Turnout as % of VEP
2012 1,835,666 We’ll see We’ll see
2008 1,746,298 952,370 54.5%
2004 1,574,463 927,844 58.9%
2000 1,431,668 770,754 53.8%
1996 1,326,919 665,629 50.2%
1992 1,162,363 743,999 64.0%
1988 1,043,170 647,008 62.0%
1984 998,820 629,656 63.0%
1980 915,484 604,222 66.0%

Punchline

It’s likely that tomorrow’s news will include coverage about the turnout rates in Utah. My plea to reporters: Calculate it correctly. You mislead the public if you go on the air with turnout percentages calculated as a percentage of registered voters. Get the number of ballots cast for president, and divide it by the voting eligible population (1,835,666).

Posted in Everything | Tagged , , , , , , | 3 Comments

Why pollsters should release their topline results

Posted on November 6, 2012 by Adam Brown

Polls can be done well or poorly. Releasing topline results aids the public in detecting poor polls.

The Tribune reported a Mason-Dixon poll last week suggesting a wide Republican advantage in the race for Salt Lake County mayor. Yesterday, the Tribune published a new story with “revised” results showing a tighter race. The initial error might have been avoided–or at least caught more quickly–if Mason-Dixon had been more transparent by releasing so-called “topline” results.

Polls can be done well or done poorly. When done well, a poll can provide a reasonable estimate of public opinion (taking account, of course, of the poll’s margin of error). There are three main ways that a poll can go wrong.

First, sampling error. If the pollster doesn’t contact a representative sample of voters, the poll will be off. That was apparently the problem with this Mason-Dixon poll. Their sample included a disproportionately large number of Republicans.

Second, non-response error. Not everybody contacted by the pollster will participate. If those who decline are systematically different from those who participate–for example, if more Republicans agree to take the poll than Democrats–that will also introduce error.

Third, measurement error. If questions are worded poorly, then you’ll get misleading results. Reputable pollsters are very careful to avoid measurement error. For example, the best polls will randomly rotate the order in which options appear. So, when asking about preferences in a presidential vote, some respondents will hear Obama’s name as the first option, while others will hear Romney’s name first. There are many ways to minimize measurement error, and reputable pollsters are mindful of them.

To aid the public in detecting faulty polls quickly, pollsters should transparently disclose their sampling method, their response rate, and their questionnaire. This disclosure is sometimes referred to as “topline results.” Topline results also include the pattern of responses to each question. If Mason-Dixon had released their topline results with the initial poll, readers might quickly have noticed that the mix of Republican and Democratic respondents seemed off.

(Incidentally, readers should always be suspicious when campaigns release their internal poll numbers, since those releases almost never include full topline results.)

If you search this site for “topline,” you’ll find that those who post poll results here (usually Quin Monson and Kelly Patterson) routinely post their topline results. At a minimum, when we write about polls, we include a methodological note at the bottom giving some information about the polling methodology for the benefit of polling nerds.

Polls can be done well or poorly. Releasing topline results aids the public in detecting poor polls.

Posted in Everything | Tagged , , , | Comments Off on Why pollsters should release their topline results

How Deep is Support for Romney Among Mormons Nationally?

Posted on November 6, 2012 by Quin Monson

The religious group affinity some Mormons appear to feel for Romney comes though a little more clearly among Republicans and independents where Romney receives nearly unanimous support from Mormon Republicans and enjoys a 2:1 advantage among Mormon independents.

Given their shared faith, it should come as no surprise that Mormons support Mitt Romney for president.  Among the nearly 1,500 Mormons in the 2012 YouGov/CCES national data (look below for details), 77 percent said they planned to vote (or already voted) for Romney.  With the first Mormon nominee for a major party presidential ticket, you might expect the historic moment to mean that Mormons of all political stripes would be more supportive of Romney’s candidacy compared to other recent Republican presidential candidates.

Support from 77 percent of Mormons for a Mormon presidential candidate is, in fact, about ten points higher than the support Mormons gave overall to John McCain (67 percent).1 Among Utah Mormons in the Utah Colleges Exit Poll data, 74 percent voted for McCain in 2008 and 86 percent voted for George W. Bush in 2004.   Comparing the 2008 national percentage of 67 percent to the 74 percent in Utah suggests that Utah Mormons may vote slightly more Republican than Mormons nationally.  If you assume the 2004 Utah estimate for Bush is a little higher than it was nationally, the 77 percent voting for Romney in 2012 among Mormons doesn’t look exceptional.

The figure below shows how Mormon McCain and Obama supporters from 2008 are casting their votes in 2012.  Romney holds the overwhelming majority of McCain voters from 2008.  However, Obama appears to lose about a quarter of his Mormon supporters to Romney.  To be sure, the consequences of this shift aren’t enough to affect any national results, but this is the kind of shift that would matter in a western battleground like Nevada in a close election.  What the shift from Obama 2008 to Romney 2012 among Mormon Democrats really suggests a religious group affinity for Romney that transcends party for some voters.

We know from a variety of research that American Mormons overwhelmingly identify as Republicans, a fact that goes a long way toward explaining the high overall level of support among Mormons for Romney.  In the CCES, when you include independents who “lean” toward a party as partisans, Mormons are 70 percent Republicans, 20 percent Democrats, and 10 percent independents. 2

The figure below shows the 2012 vote by self-identified partisan identification (with “leaners” classified as partisans).  Romney gets only 15 percent of Mormon Democrats instead of the 26 percent of Mormon Obama voters–a much less striking result.  In other words, Romney holds Obama under the 90 percent you might expect a presidential candidate to get among his own partisans, but just barely. The religious group affinity some Mormons appear to feel for Romney comes though a little more clearly among Republicans and independents where Romney receives nearly unanimous support from Mormon Republicans and enjoys a 2:1 advantage among Mormon independents.

Religiosity matters too.  The figure below displays the vote by party identification for two groups—those that attend church weekly, and those for whom church attendance is less regular (or who don’t attend at all).  The religious group affinity is stronger among more religiously active Mormons. This isn’t evident among Republicans, where there’s little room for Romney to improve, but a small difference appears between the two groups of Democrats and twenty point difference exists between the groups of independents.

Support for Romney among Mormons is quite strong overall, suggesting that a religious group affinity above and beyond party identification exists.  In the end, the religious ties don’t move many Mormons with existing partisan attachments.  Republicans have no room to improve and Democrats are difficult to pull away from President Obama. But among independents, where party attachment is not a factor, the religious group affinity appears to really matter.  So, how deep is support for Romney among Mormons?  Religious group affinity doesn’t appear to be deep enough to transcend strong party identification very often, even with one of their own at the pinnacle of the American political process, but it does affect some partisan voters and it can really move voters with no partisan attachments.

Survey Details

Plenty of data exists to take a good look at support for Mitt Romney among Utah Mormons (see here and here), but national survey data of a representative sample of Mormons in the 2012 campaign is hard to come by—until now.  YouGov—a professional survey research and consulting firm located in Palo Alto, California—conducts the Cooperative Congressional Election Study (CCES), a national survey involving many teams of scholars from leading universities around the U.S.  The cooperation part of the CCES pools resources enabling data collection from a very large sample.  In years past, the CCES has sampled well over 30,000 Americans.  This year’s CCES will be the largest ever.  The advantage of interviewing such a large sample is that the data collected so far includes interviews with 1,491 self-identified Mormons.

Posted in Everything | Tagged , , , , , , , , | Comments Off on How Deep is Support for Romney Among Mormons Nationally?

What’s up with Senator Mike Lee’s Favorability Rating? (plus Oct 2012 Key Research Toplines)

Posted on November 6, 2012 by Quin Monson

It’s not that Lee is really unpopular, it’s just that even after two years he’s still relatively unknown.  Lee’s statewide favorability resembles the statewide ratings of Utah’s Republican U.S. House members.

Key Research, a survey and market research company based in Provo, recently cooperated with faculty at BYU’s Center for the Study of Elections and Democracy to conduct a statewide survey.  We’ve already posted some of the results here and here.

Click here to download a topline report that includes the full survey questionnaire, frequencies for each question, and a detailed methodological report (including details about the sampling as well as response rates and cooperation rates).

What else is in the survey that we haven’t written about?  Let’s take a quick look at the approval ratings and favorability scores of our elected officials.  First, Governor Herbert enjoys an extremely high approval rating among Utah voters at 83%.  Herbert’s high approval rating goes a long way toward explaining why he’s cruising to reelection.  The legislature is doing quite well too, at 71%.  Generally, legislative bodies have lower approval ratings than the executive, and while the Utah Legislature’s approval typically goes down a few clicks during its session each year, they should collectively be pleased with the current approval level.  Finally, and not surprisingly, President Obama is not popular in Utah.

We also asked survey respondents to report the favorability of Utah elected officials with the following question: “For each of the following persons, please indicate whether you have a favorable or unfavorable impression.”  The percentages below report the percentages for the full statewide sample even for the three current incumbents to the U.S. House who each only currently represent a third of the state.

Governor Herbert and Mitt Romney both enjoy very high favorability ratings statewide.  Senator Hatch is about 10 points lower, but still respectable.  Members of Utah’s U.S. House delegation all have lower statewide favorability ratings.  But this is understandable as none of them represents a statewide constituency.  Notice that for both Rob Bishop and Jason Chaffetz, the proportion unable to give an opinion is between a quarter and a third.

That leaves the two most interesting ratings: Jim Matheson and Mike Lee.  Matheson’s is interesting because in our past polling, he has enjoyed extremely high favorability ratings along with high name recognition, even statewide.  Just back in June of this year, his favorability was 62% statewide.  In the March 2010 Utah Voter Poll, Matheson actually had a higher favorability statewide than Governor Herbert.  Matheson’s now lower statewide favorability demonstrates the perils of facing two consecutive close races,  especially when that includes having several million dollars spent against you in the statewide television market.

In Senator Mike Lee’s case it’s interesting to note that his favorability is relatively low compared to the other statewide elected officials.  Furthermore, the percentage expressing no opinion is relatively high (31%) compared to the other elected officials  It’s not that Lee is really unpopular, it’s just that even after two years he’s still relatively unknown.  Lee’s statewide favorability resembles the statewide ratings of Utah’s Republican U.S. House members.

In fact, when you put Lee’s statewide favorability side by side with the statewide and district favorability ratings for Utah’s U.S. House members, you find that Rob Bishop and especially Jason Chaffetz have high favorability ratings among their own constituents.  It’s still early for Mike Lee to be too worried, but within the next two years he should hope to shift most voters from “no opinion” toward the favorable group.  Otherwise we could see challengers inside and outside the Republican Party take notice and begin laying the groundwork for a strong challenge in 2016.  There, I’ve managed to mention 2016 this early without talking about the presidential campaign.  Fact:  Utah is much more likely to see another interesting U.S. Senate election than it is to see itself listed as a competitive presidential state.

Survey Methodology

For this survey 500 voters were sampled from the state’s file of active registered voters; 100 in each of the four congressional districts with an extra 100 oversampled in the 4th Congressional District.  For the statewide results, the numbers have been weighted so that each congressional district is equally represented.  Using the voter list for sampling allows information in the file to be used to make the sampling represent the target population of Utah voters more effectively and efficiently.

A model of general election turnout was estimated using age, party registration status, length of registration, and past election turnout–all shown in political science research to be related to voter turnout.  Using this model we produced an estimate of the predicted probability of voting in the 2012 general election for each individual registered voter in the file.  This produces a sampling pool of registered voters that can be randomly sampled based on their likelihood of voting. A Probability Proportionate to Size (PPS) sample was then drawn using this predicted turnout estimate such that voters with a higher probability of voting have a higher probability of being selected in the sample.   The sample was then matched to a database of telephone numbers and sampled voters were contacted and administered a questionnaire over the telephone by Key Research.

The survey field dates were October 9, 2012 – October 13, 2012.  The statewide sample of 500 produces a margin of sampling error of 4.4%.  The margin of error is larger for questions that some respondents chose not to answer or for the questions only asked of a subset of respondents.  Of course, sampling error is only one possible source of error in survey research.  Results can also be affected by measurement error (e.g. question wording and question order), coverage error (e.g. counting as “likely voters” survey respondents who will not vote), and non-response error (e.g. the people who responded to they survey are systematically different from people who refused or were not reachable).

Once again, our experience working with the team at Key Research has been extremely positive and we plan to work with them on their surveys on a regular basis moving forward.

Posted in Everything | Tagged , , , , , , , , , | Comments Off on What’s up with Senator Mike Lee’s Favorability Rating? (plus Oct 2012 Key Research Toplines)

What kind of Republican is Mia Love?

Posted on October 31, 2012 by Adam Brown

What kind of Republican is Mia Love? She’s not too far off from Chaffetz, Stewart, and Bishop.

Last week, I used Congressional voting data to assess whether Jim Matheson is a moderate or liberal Democrat. The unsurprising conclusion: He’s to the right of almost every other Democrat in the US House. It would be hard to call him a liberal based on his overall voting record. Whether that makes him a moderate, a conservative, a pollchaser, or a pragmatist is in the eye of the beholder.

Now let’s look at Mia Love. She doesn’t have a Congressional voting record to evaluate, but Boris Shor saves the day here. He’s a political scientist at the University of Chicago who has found a way to measure the ideology of Congressional challengers on the same scale that we use to measure the ideology of incumbents. Today, he released ideology scores for 722 of the 2012 candidates for US House, including both incumbents and challengers. (He also posted an explanation of how scores are calculated and some interesting aggregate analysis of the scores.)

Shor’s data places Mia Love somewhat right-of-center when compared to other Republican candidates for the US House this cycle. When compared to other Republicans running for the House, only 30% are more conservative than she is, with the other 70% somewhat less conservative. (Shor’s data identifies Jason Chaffetz and Chris Stewart as somewhat to Love’s left, with Rob Bishop somewhat to her right; notice that Bishop’s and Chaffetz’s positions were swapped in my previous post, when I used ideology scores calculated by different researchers, illustrating the difficulty of pinning down exact ideology scores.)

So what kind of Republican is Mia Love? She’s not too far off from Chaffetz, Stewart, and Bishop. They all appear to have ideologies somewhat more conservative than is typical for a Republican House candidate this year, but not so far right of average that they would appear extreme within their own party. If Republicans sweep Utah’s US House races this year, it’s likely that the four of them will get along well in Congress.

Posted in Everything | Tagged , , , , , , , , , | 1 Comment

What Are Utahns’ Top Issue Priorities?

Posted on October 26, 2012 by Damon Cann

Utahns’ issue priorities fall into three distinct groups with Education and Jobs being most important and Gun Rights and Public Lands being least important.

Note: This post is written by the POLS 6010 Graduate Research Methods class at Utah State University.  The post expresses the findings of the class and does not represent the official position of Utah State University.

Last week we posted trial-heat poll results from a poll conducted by our USU class.  In addition to the vote choice questions, respondents were asked to rate their priorities on 7 key state issues: Education, Jobs, State’s Rights, Energy, Immigration, Public Lands, and Gun Control.  Respondents were asked to rate how important these issues are to them on a scale from 1-10 with 10 representing very high importance and 1 representing very little importance.  Utahns’ issue priorities fall into three distinct groups with Education and Jobs being most important and Gun Rights and Public Lands being least important. We will report the estimated average importance and 95% confidence intervals (95% C.I.) to reflect uncertainty in our estimates due to sampling error.  The results are easily depicted graphically, and a textual description follows.

Education and Jobs

Education was the most important issue on the list with a mean score of 8.49 (95% CI: 8.23-8.75). Education is commonly a top priority among voters in all constituencies and the result came as no surprise.  Jobs trailed closely behind education with a mean score of 8.33 (95% CI: 8.08-8.59). The confidence intervals for education and jobs overlap, meaning we can’t distinguish between the importance of these two issues.  However, the confidence intervals for education and jobs do not overlap with the intervals for any of the other issues, making them the clearly most important issues.

State’s Rights, Energy, and Immigration

Poll respondents placed the importance of immigration, states rights, and energy in the middle. Immigration had a mean of 7.58 (95% CI: 7.28-7.88), states rights 7.57 (95% CI: 7.20-7.94), and energy had a mean of 7.46 (95% CI: 7.13-7.78). The confidence intervals for these three issues overlap, making them indistinguishable from each other in terms of importance to respondents.  Still, because these issues’ intervals do not overlap with the other issues, we can say they are more important than gun rights and public lands but less important than jobs or education.

Gun Rights and Public Lands

Gun rights and public lands were the least important issues to poll respondents. Gun rights had a mean issue importance of 6.65 (95% CI: 6.23-7.07) and public lands 6.73 (95% CI: 6.37-7.09). Once again the two had overlapping confidence intervals, making it impossible to determine which of the two is truly higher or lower. We can say with confidence, though that these two are the least important issues to poll respondents.

Comparison to the Utah Foundation Study

(Updated November 2nd to more accurately describe the Utah Foundation’s poll.)

Morgan Lyon Cotti of the Utah Foundation discussed the Utah Priorities Survey (conducted just before the 2012 caucuses) here. There are some slight differences in methodology–we asked respondents to rate the importance of a given set of 7 issues. The 2012 Utah Priorities Survey began by asking a sample of 400 people two open-ended questions: 1) What would you say is the greatest issue facing Utah? And 2) What is the next greatest issue? Those open-ended answers were then analyzed and grouped into 19 major topic areas, from which a larger survey was created and administered to 804 respondents statewide. Their surveys were conducted by phone during February 2012 and had a margin of error of +/- 3.5%.

Regardless of the method used, there are clear similarities between the results of the two surveys. Both identified jobs and education as top priorities of voters. Gun rights and control of public lands were lowest in importance in the USU poll and didn’t even make the top 10 list when considering the two most important issues to voters in the Utah Priorities Survey. Energy and Immigration placed in the top 10 but not the top 2 of the Utah Priorities Survey and in the mid-range of importance in the USU study. One possible explanation for the appearance of states rights in the mid-range of importance in the USU study but not the top 10 list from the Utah Priorities Survey could be that states rights is not a “top 2″ issue for many people but still has some level of importance (though not top-level importance) when people are asked to rate it.

Methodology

Results are based on telephone interviews conducted by students at Utah State University as part of a class assignment Oct. 8-13, 2012. The sample size is 221.  This sample size is larger than the previously reported survey because individuals who are not “likely voters” have not been screened out of these responses.   The survey was conducted from a random sample of individuals drawn from the state’s voter file. The sample is weighted by the probability of voting, party registration and age so the sample of respondents will reflect the population of active registered voters. The margins of error depicted in the figure are 95% confidence intervals that express the amount of sampling error.   The reported margins of error include the sampling design effects and incorporate the weights to adjust for coverage and non-response, but as with all polls, does not entirely account for other possible sources of error including measurement error, coverage error and nonresponse.

About Damon Cann: Damon Cann is an Assistant Professor of Political Science at Utah State University.

Posted in Everything | Tagged , , , , , , | Comments Off on What Are Utahns’ Top Issue Priorities?

Who Do Mormons Say Represents Their Faith Positively?

Posted on October 25, 2012 by Kelly Patterson

When you consider that Mormons overwhelmingly identify as Republicans, it is unsurprising that Mitt Romney is viewed by a large majority of Mormons as positively representing their faith.

In a previous post, we reported findings from the June Key Research poll showing that Utahns, especially Utah Mormons, have a great deal of enthusiasm about Mitt Romney’s presidential candidacy along with some concern about media coverage of their faith.  With the presidential race building to a crescendo of coverage, we repeated the questions on the latest Key Research poll.

We asked the question “Governor Mitt Romney is the first LDS (Mormon) candidate in history to win the presidential nomination of a major political party.  Reflecting on this accomplishment, do you think it is overall a good thing or a bad thing?”  In the latest poll, 73% of voters said it was “a good thing” compared to 68% in the June poll, a gain of five percentage points.  Among Mormons only, the proportion saying it is “a good thing” jumps to 78%, a number almost identical to the 79% in June.

Because Governor Romney’s candidacy has shined such a bright light on the LDS faith, we asked what sort of publicity individuals expected.  Among all Utahns, a solid majority (58%) believe that the presidential campaign would result in “both good and bad publicity” for the LDS Church. People who say “mostly good publicity” (28%) outnumbered those who say “mostly bad publicity” (7%) by four to one.

The mixed results for the publicity probably stem from the trust respondents have in the media.  57% said that they “do not trust the media to cover the LDS Church fairly,” while almost 30% do “trust the media to cover the LDS Church fairly.”   Only minimal differences emerge when restricting the sample to Mormons.

Media coverage often features prominent Mormons who have enjoyed success in politics, business, sports, or entertainment.  The figure below displays the extent to which the survey respondents believe that each individual is a positive representative of their faith.  These questions were only asked of those who self-identified as Mormons.   The percentages below combine the responses for “Always” and “Very Frequently.”1

There is substantial variation. When you consider that Mormons overwhelmingly identify as Republicans, it is unsurprising that Mitt Romney is viewed by a large majority of Mormons as positively representing their faith.  Romney is viewed on par with personalities such as football player Steve Young and singer David Archuleta (literally a rock star but currently a full-time Mormon missionary).

U.S. Senate Majority Leader Harry Reid, who has not hesitated to call out Romney on releasing his tax forms or even more directly about religion, receives a much lower rating.  Only 19% of Utah Mormons say he represents the faith positively “Always” or “Very Frequently.”

What predicts responses to Reid and Romney?  When you statistically control for party, ideology, religious activity, age, education, and gender simultaneously, it becomes clear that the political variables, not religious activity, are what really matter.2

For Reid ideology and age matter, while for Romney it is party and age.  Older and more conservative Mormons are less likely to say Reid represents their faith positively.  Older and more Republican Mormons are more likely to say that Romney represents their faith positively.  These results are similar to the data we’ve reported previously about Romney’s favorability rating among Mormon Democrats.

Should Romney be elected, his positive image among Mormons will likely go down.  Few politicians in the rough-and-tumble world of politics emerge with their image unsullied.

Survey Methodology
The sample was drawn from the publicly available file of Utah registered voters.  A model of general election turnout was estimated using age, party registration status, length of registration, and past election turnout. This model was used to estimate a probability of voting in the 2012 general election.  A Probability Proportionate to Size (PPS) sample was draw using this turnout estimate such that voters with a higher probability of voting have a higher probability of being selected in the sample.  This produces a sample of likely voters.  The sample was then matched to a database of telephone numbers and sampled voters were administered a questionnaire over the telephone by Key Research. The survey field dates were October 9, 2012 – October 13, 2012.  The statewide sample of 500 produces a margin of sampling error of 4.4%.  The margin of error is larger for questions that some respondents chose not to answer or for the questions only asked of a subset of respondents.  Of course, sampling error is only one possible source of error in survey research.  Results can also be affected by measurement error (e.g. question wording and question order), coverage error (e.g. counting as “likely voters” survey respondents who will not vote), and non-response error (e.g. the people who responded to they survey are systematically different from people who refused or were not reachable).

**Update: Click here to download a topline report that includes the full survey questionnaire, frequencies for each question, and a detailed methodological report (including details about the sampling as well as response rates and cooperation rates).

About Kelly Patterson: Kelly Patterson is a professor of political science at Brigham Young University and a senior scholar at the Center for the Study of Elections and Democracy.

Posted in Everything | Tagged , , , , , , , , , , | Comments Off on Who Do Mormons Say Represents Their Faith Positively?

Is Jim Matheson a liberal or a moderate?

Posted on October 23, 2012 by Adam Brown

Matheson’s Congressional votes often go against partisan expectations.

Jim Matheson presents himself to voters as a moderate willing to work with both parties. Critics from the right contend that he’s really a liberal at heart who will promptly ally with Pelosi and Obama when given the chance. It’s time for some hard data.

How we measure US Representatives’ ideology

Political scientists have worked for years to develop reliable ways of estimating the ideology of members of Congress. The best technique produces so-called DW-NOMINATE scores, available for download at VoteView.com. (Regular readers will recall that I used a similar method to estimate ideology scores of Utah’s state legislators.) The DW-NOMINATE algorithm uses all votes cast by members of Congress to line them up from most liberal to most conservative, with scores ranging from -1 (most liberal) to +1 (most conservative).

I stress that -1 does NOT mean “perfect Democrat” and +1 does NOT mean “perfect Republican,” as it’s entirely possible for a person to be far left of the Democrats or far right of the Republicans. These are relative scores, not absolute scores.

Below, you can see how the scores look for the current (112th) US House. The line goes up where there is a greater concentration of scores. You’ll see that there is a big concentration around -0.5 and also around +0.5. This reflects the polarization of the two major parties, with few Representatives in the center.

Distribution of ideology scores in the 112th US House

If I drew a separate line for each party, rather than using one line for all representatives, it would look like this:

Distribution of ideology scores by party in the 112th US House

The two lines don’t overlap at all. That’s because the most liberal Republican is somewhat more conservative than the most conservative Democrat. (It was common for the lines to overlap a couple decades ago, but that’s another story.)

How Matheson, Bishop, and Chaffetz compare to their parties

Now, let’s go one step further and mark Jim Matheson’s location. Just for comparison, we’ll also add in Rob Bishop and Jason Chaffetz.

Ideological location of Utah’s US Representatives relative to other Representatives

It is readily apparent that Matheson is among the most conservative Democrats in the US House. Of 194 Democrats presently serving, only 4 have a score more conservative than Matheson’s. Whether that makes him a “conservative” in absolute terms rather than relative terms is, of course, in the eye of the beholder.

Incidentally, Nancy Pelosi’s score of -0.533 places her well left of center even within the Democratic Party. Fewer than one-quarter of House Democrats are more liberal than Pelosi. There is noticeable ideological daylight between Pelosi and Matheson.

For discussion of Bishop and Chaffetz, read the footnote at the end of this sentence.1

How ideological (or predictable) is Matheson?

The DW-NOMINATE scores also give us an estimate of each Representative’s overall predictability. For most Representatives, you can do a pretty good job of predicting their vote on any particular bill simply by knowing the Representative’s DW-NOMINATE score (and also by knowing how the other 434 members of the House voted on the bill in question).

Usually, predictability is pretty high. As you can see from the chart below, you can predict Rob Bishop’s vote 92% of the time and Jason Chaffetz’s vote 93% of the time. These are typical scores. In fact, Chaffetz sits exactly on the median.

Data from the 112th US House

Jim Matheson, meanwhile, is hard to pin down. You can predict his vote only 78% of the time. That might sound high in absolute terms, but in the highly polarized environment of the US House, that’s a remarkably low score. Only 5 of the 435 Representatives have a lower predictability score. From a statistical standpoint, this low predictability score suggests that Matheson’s Congressional votes often go against partisan expectations.

Of course, this analysis does not attempt to address whether Matheson’s voting record is consistent with what voters in Utah’s fourth district desire. Only voters themselves can answer that question.

Posted in Everything | Tagged , , , , , , , | 3 Comments