Do I Mistrust this Poll? Let Me Count the Ways…

Posted on November 25, 2013 by Kelly Patterson

This post was jointly written by Quin Monson and Kelly Patterson.

Recently, the Libertas Institute posted a “survey” on their web site purporting to measure how Utah voters feel about a possible statewide anti-discrimination law to protect gay and transgender Utahns against housing and employment discrimination.

The survey has many issues that cast serious doubt on the validity of the results.  Here is a brief summary of some of the problems.

1. The questions are unbalanced and inaccurate.  For example, the two questions about an anti-discrimination law contain references to “jail time.”  However, jail time is not part of anyone’s anti-dicrimination ordinance or law.  What Libertas calls “contextualized” simply means asking loaded questions by introducing the threat of a non-existent punishment.

In addition to inaccurate questions about anti-discrimination, the question order is a serious concern.  The relevant questions are preceded by questions about free association and religion.  This order effectively biases the survey participants by having them consider reasons why they would not want a law right before they are asked about a possible law.

2.  The robocall methodology used by Libertas is not methodologically sound.  Robocalls cannot obtain a valid representative sample of any population of interest because they have no way to select or ask for a respondent within the household they are calling.  Many valid surveys ask for a respondent by name from a list.  Or, if the poll is conducted on a random list of phone numbers, a method is employed to select a person at random within the household.  For example, many pollsters ask to speak to the person who celebrated the most recent birthday. This poll could have been answered by someone’s 14 year old, and they would never know. The survey is supposed to represent voters, but this is virtually impossible to verify.  The poll contains no gender question.  Because they take whomever answers the phone, it is likely that the respondents are disproportionately female.  Voters in Utah are evenly split 50/50 between men and women.   Finally, a heavy proportion (37%) of the respondents are over 60 years old.  This too is disproportionately high.

3.  The survey also has some serious ethical and possible legal problems.  One serious issue is that they posted a data file on their web site that includes the actual phone numbers of the people they called.  This is a huge violation of trust.  The American Association of Public Opinion Research follows a code of professional ethics that you can find here. The relevant part that this survey violates says, “Unless the respondent explicitly waives confidentiality for specified uses, we shall hold as privileged and confidential all information that could be used, alone or in combination with other reasonably available information, to identify a respondent with his or her responses. We also shall not disclose or use the names of respondents or any other personally-identifying information for non-research purposes unless the respondents grant us permission to do so.”  They did not release the full script of the robocall, but we are guessing that they did not tell participants that they would be releasing their phone numbers publicly.  This would have led many people to hang up on the call and would make many of the existing respondents very angry.

We are also worried that they robocalled cell phones.  If so, this is actually illegal under federal law.  They could be in serious trouble if someone filed a complaint.

In their write up of the “survey,” Libertas claims that “These results validate our initial theory…”  Unfortunately, good social science does not validate theories, it simply rejects null hypotheses at certain levels of confidence.  We have no confidence in this survey.

About Kelly Patterson: Kelly Patterson is a professor of political science at Brigham Young University and a senior scholar at the Center for the Study of Elections and Democracy.

Posted in Everything | 10 Comments

What kind of Republican is Spencer Cox?

Posted on October 16, 2013 by Adam Brown

Spencer Cox is about as run-of-the-mill of a Republican as Utah can produce.

Governor Gary Herbert has nominated Spencer Cox to replace Greg Bell as Utah’s new Lieutenant Governor. Few outside the Legislature had heard Cox’s name before the governor’s announcement. Even within the Legislature, few knew his name before he began his legislative service less than one year ago.

Today the Legislature will consider the governor’s nomination and, in all likelihood, confirm him as Utah’s new Lieutenant Governor. The Tribune ran a good biographical profile of Cox today. I thought I would add a quick look at his Legislative voting record.

Because Cox served in the Utah House, we can use voting records to get a feel for what type of political ideology Cox might espouse. Using all the “ayes” and “nays” cast in the Legislature, we can apply a well-tested method to line up all the legislators from most liberal to most conservative. (You can read more about how these scores work here.)

On a scale from -100 (more liberal) to +100 (more conservative), Spencer Cox rates a +28.4. This places him almost exactly at the center of the House Republican caucus. For House Republicans, the average (mean) score was +28.7, and the median was +27.4. Cox’s voting record makes him most similar to Brad Dee (House Majority Leader), followed by Mike McKell (Cox’s brother-in-law) and Dean Sanpei (House Rules Chair). Indeed, Cox’s record is almost indistinguishable from any of these Representatives.

The chart below gives another way of considering how Cox’s voting ideology lines up with other legislators. I grouped ideology scores for the 75 Representatives who served in 2013 into bins 10 units wide (the left column in the table below). In the right column, I wrote one “D” or “R” for each legislator within each bin. For example, there were three Democratic Representatives with scores between -100 and -90, hence the “DDD” in the right column. Cox’s score is highlighted in red, in the politically safe area near the center of the House Republicans.

Bin Legislators
-100 to -90 DDD
-90.1 to -80 DDDD
-80.1 to -70 DDDDDD
-70.1 to -60 D
-60.1 to -50
-50.1 to -40
-40.1 to -30
-30.1 to -20 RRR
-20.1 to -10 R
-10.1 to 0 RRRRR
0.1 to 10 RRRRRRR
10.1 to 20 RRRRRRRRRR
20.1 to 30 RRRRRRRR
30.1 to 40 RRRRRR
40.1 to 50 RRRRRRRRR
50.1 to 60 R
60.1 to 70 RRRRRR
70.1 to 80 R
80.1 to 90 RRRR
90.1 to 100

What do we conclude? From his legislative voting record, it seems Spencer Cox is about as run-of-the-mill of a Republican as Utah can produce. Voters have no more reason to love or fear him than they would any typical Utah Republican official.

About Adam Brown: Adam Brown is an associate professor of political science at Brigham Young University and a research fellow with the Center for the Study of Elections and Democracy. You can learn more about him at his website.

Posted in Everything | Tagged , , , , , , , , | Comments Off on What kind of Republican is Spencer Cox?

New ideology ratings for the Utah Legislature

Posted on October 16, 2013 by Adam Brown

Two years ago I released ideology scores for each Utah legislator who served between 2007 and 2011. Today I’m releasing an update that extends the scores through 2013. (Thanks to my research assistant, Justin Chang, for invaluable help.) I’ve written in more detail about the methodology behind the scores in the past.

Please take a moment to understand the origins and limitations of these ideology scores.

  • I use the W-NOMINATE algorithm, originally developed to study the U.S. Congress, and widely used among political scientists for that purpose.
  • The W-NOMINATE algorithm takes account of every vote cast on the floor of the Utah House or Utah Senate.
  • Scores are relative. That means there is no score that means “liberal” or “conservative.” A score of +43 or -21 means nothing by itself. But a legislator with a score of +43 is relatively more conservative than a legislator with a score of +38, who is, in turn, relatively more conservative than a legislator with a score of -4. Higher scores are relatively more conservative than lower scores.
  • Scores are placed on an artificial score ranging from -100 (more liberal) to +100 (more conservative), but a score below 0 does not mean “liberal,” just as a score above 0 does not mean “conservative.” Rather, because scores are relative within the Legislature itself, then 0 simply indicates the legislator at the ideological center of his/her chamber. Because the Legislature is over 80% Republican, the 0 point will almost certainly be occupied by a Republican.
  • Scores are comparable only within a single chamber, and only within a single year. You cannot compare a House score to a Senate score, or a 2008 score to a 2012 score. Why? Because the scores are relative within each chamber and year. Even if every legislator, Republican or Democratic, moved dramatically to the between year X and year Y, the scores would still be forced by the W-NOMINATE algorithm to fall between -100 and +100 each year.

The complete scores are available on my personal website. You can start here: House (2013) and Senate (2013).

I’ll post some followups in coming days making use of these scores. The main purpose of this post is just to explain how the scores work so that I have something to link to when future posts make use of them.

Posted in Everything | Tagged , , , , , | Comments Off on New ideology ratings for the Utah Legislature

Senator Lee and the Shutdown

Posted on October 9, 2013 by Quin Monson

Lee has overwhelming support from the quarter of the population that doesn’t reject the Tea Party.  But the three-quarters of Utahns who do not identify with the Tea Party come to the exact opposite conclusion.

Senator Mike Lee has received considerable attention nationally, within Utah, and even among his fellow U.S. Senators for his role with Senator Ted Cruz in executing a strategy to defund the Affordable Care Act (widely known as Obamacare).  The tension continues to rise with the government shutdown and the possibility that this will lead to a failure to raise the debt ceiling.

So what do Utahns think about all of this?  Who views Senator Lee’s work favorably or unfavorably? And how does this all compare with the recent past?  The Center for the Study of Elections and Democracy at BYU just completed another Utah Voter Poll that allows us to answer these and other questions.

We asked respondents to choose between whether “Senator Mike Lee should stand by his principles, even when the result is a government shutdown” or “Senator Mike Lee should be more willing to compromise, even if that means passing a budget with funding for the Affordable Care Act.”

Lee Figure 1

First of all, 57% of Utahns overall would like Senator Lee to be “more willing to compromise” versus 43% who prefer that he “stand by his principles.” That result is not possible in Republican-dominated Utah without at least some Republicans preferring compromise.  In fact, 38% of all self-identified Republicans prefer that Lee compromise compared to 99% of all Democrats.  Independents side heavily with compromise at 65%.

A look at this question by active support for the Tea Party support yields absolutely stunning conclusions.  Among the 13% of respondents who identify as active Tea Party supporters, 90% say that Lee should stick to his principles and not compromise.  Among those who say they are not active supporters (74% of the sample), 73% say he should compromise.

The 13% who answer “Don’t Know” to the Tea Party question are also interesting.  Of this group, 80% say stick to his principles, 20% say compromise.  Those answering “Don’t Know” appear to be Tea Partiers hiding in the “Don’t Know” response option.

All told, then, Lee has overwhelming support from the quarter of the population that doesn’t reject the Tea Party.  But the three-quarters of Utahns who do not identify with the Tea Party come to the exact opposite conclusion.

But here’s the further complication:  among those who identify as Republicans but do not support the Tea Party, the split is 51/49 in favor of compromise.  Recall, that among all Republicans (Tea Party supporters or not), support for “standing by his principles” is very strong — 62/38.

So, how vulnerable is Lee because of his tactics on Obamacare and the shutdown?  On this issue he has near total support from the Tea Party and about half of the non-Tea Party affiliated Republicans.  He has generated some opposition within his own party because of his actions, but on balance and buoyed by the overwhelming support for Tea Partiers, his tactics in the budget fight still enjoy the support of most Republicans.

That said, Lee’s stance has carried over into his favorability ratings.  Compared to our June 2013 Utah Voter Poll, Senator Lee’s favorability fortunes have reversed course.  In October the favorability to unfavorability ratio is 40/51 while back in June 2013 it was 50/41.  More Utah voters now have an unfavorable impression of him.

Lee Figure 2

To put Lee’s current favorability numbers in the proper context, we’ve put them side by side with Jim Matheson’s current numbers, after all Matheson’s name always comes up in discussions of statewide elections. Just for fun, we’ve thrown in some favorability numbers for Senator Bob Bennett from the March 2010 Utah Voter Poll.   Matheson’s favorable to unfavorable ratio is 58/36, not great compared to Governor Herbert who is at 71/25 (not shown in the figure), but not bad for a Democrat amidst a sea of Republicans.  Notably, Lee is also now clearly behind the numbers of former Senator Bennett from our March 2010 Utah Voter Poll.

When you break the numbers down into “very” and “somewhat” favorable ratings, you see that Senator Lee is a polarizing figure.  His highest ratings are for “very unfavorable” (40%) followed by “very favorable” 26%.  Lee’s very favorable numbers are also virtually unchanged from June.  The change is from somewhat favorable to very unfavorable. Moderate Republicans who once gave Lee a lukewarm thumbs up are now giving him a strong thumbs down.  Matheson and Bennett, in contrast, are highest on the middle “somewhat” categories.

Lee Figure 3

So how is Senator Lee doing among members of his own party?  Back in June he was a rock star with a 71/22 favorable to unfavorable rating.  That’s now fallen off quite a bit, but it’s still at a respectable 57/33.  Is Senator Lee vulnerable to a challenge from the moderate wing of the Republican Party?  Maybe.  Check out former Senator Bennett’s numbers among Republicans in March 2010.  At 57/38 they look a lot like Lee’s do now.  If Lee draws a strong Republican challenger, it would be difficult to predict the outcome right now.  A lot would depend on who shows up to the caucus meetings and the composition of the delegates.  If the Count My Vote initiative is successful, all bets are off.

Lee Figure 4

A strong general election challenger presents a real danger spot for Senator Lee.  Notice Representative Matheson’s favorability/unfavorability ratio among Republicans statewide are is 52/40.  Matheson doesn’t need a majority of Republican votes to win a statewide election, he just needs between a quarter and a third.  That’s been his recipe for success for the last decade plus in his House elections.  That appears very doable right now.

Lee Figure 5

When the Republican only numbers are examined in more detail, once again Senator Lee’s “very favorable” numbers hold pretty constant from June to October.  The voters that always really like him haven’t changed their minds much and his current stance may simply reinforce those views, but he has driven away a group of voters that he will badly want back if he ever faces a strong challenger.  The difficulty for Democrats is finding a challenger with the capacity to win statewide who is willing to run.  The short list is indeed very short.

Altogether, these results are indicative of the larger difficulty within the Republican Party.  While Senator Lee enjoys intense support from a vocal minority and seems to be representing their perspective very well, he does so at his own peril.  The majority of Utah voters are looking for him to compromise and if he fails to do so, he could face electoral consequences.

Survey Methods

Click here to download a topline report that includes the full survey questionnaire, frequencies for each question, a detailed methodological report (including details about the sampling as well as response rates and cooperation rates) and information about the margin of sampling error.

 

About Quin Monson: Quin Monson is Associate Professor of Political Science and a Senior Scholar with the Center for the Study of Elections and Democracy at Brigham Young University.

Posted in Everything | Tagged , , , , , , , , , | 16 Comments

“Convicted? No, never Convicted.”

Posted on October 9, 2013 by Kelly Patterson

71% of Utah voters still think that the Attorney General should resign.  That number is down from 78% in June, but it still represents a sizable share of the population.

We take the title for this post from that classic patriotic movie Stripes.  In this movie, Bill Murray plays a down-on-his-luck character who decides to turn his fortunes around by joining the army.  Before he can enlist, he needs to answer a question from the recruiter: “Have you ever been convicted of a felony or misdemeanor?”  Murray’s character says “Convicted? No.” And his sidekick chimes in, “Never convicted.”

Such seems to be the current condition of Attorney General Swallow.  With the news that the Federal Government has decided not to charge him, he can say, like the characters in Stripes, “Charged? No, never charged.”   The salient question is whether or not that bit of news strengthens his position with Utah Voters.

In a June Utah Voter Poll, 34% of the respondents said that they thought he had done something illegal.  In the October poll, that proportion drops to 26%.  Many of those respondents from the June poll moved to the choice that “he has done nothing illegal, but he has done something unethical.”  That number increases from 62% in the June poll to 66% in the October poll.  The proportion that says he has done “nothing unethical” stays in single digits but increases from 4% in June to 8% in October.  Swallow’s lawyer might enthusiastically call this a 100 percent increase, but it is safer to say that it is within the margin of error.

Done Something Illegal

Though there has been some small movement away from the conclusion that Swallow has done something illegal, overall less than 10% of respondents believe that the attorney general has acted ethically – a precarious position for the state’s chief law enforcement officer.

Other results only emphasize that precariousness.  71% of Utah voters still think that the Attorney General should resign.  That number is down from 78% in June, but it still represents a sizable share of the population.

Should Resign

Scandals create complexity and nuance in public opinion.  Much of what respondents say about scandals depends to a large extent on what information respondents have to help them sort through the choices pollsters offer them (Shah et al. 2002; Zaller 1998; Funk 1996; Zaller 1992).  And not all information is created equal.  Negative often has a bigger impact than positive because it can be more memorable; respondents also tend to favor recently received information because it is generally easier to summon.

To test the extent to which knowledge of the Swallow scandal shaded public opinion, we inserted a framing experiment into the October poll.  Framing experiments allow the researcher to vary the information given to the respondent in order to assess the impact of different kinds of information.

Specifically, we wanted to know if the decision of the Federal Government not to charge AG Swallow would affect support for continuing the investigation undertaken by the Utah House of Representatives.

We used four treatments.  Voters were randomly assigned to either the control or one of the four treatments.  Each treatment contained slightly different information about the nature of the investigations facing the Attorney General.  The exact wording of these treatments is provided in the endnote.1

Remember that the control was the overall support for continuing the House investigation.  The control contained no additional information.  Overall, almost 74% in the control said it should “definitely” or “probably continue.”

However, the proportion that says that the investigation should continue depends on the information that the respondent has fresh in the mind.  The best news emerges in Treatments 1 and 2 where voters are told that the AG has either been “cleared of bribery accusations” (50%) or the Justice Department “would not seek bribery charges against him” (52%).  With that type of information in mind, the support for a House investigation declines.

Differences Across Treatments

The proportion that wants the House investigation to continue climbs higher in Treatments 3 and 4.  In Treatment 3, the stronger language of “cleared” is used along with a reminder of ongoing investigations by other entities.  Here, 57% want the House investigation to continue.  Treatment 4 uses softer language.  It does not refer to “cleared” but only to the Department’s decision not to pursue charges.  56% want the House investigation to continue under this treatment.2

Overall, even after directly reminding Utahns of the Justice Department’s decision not to proceed, a majority of respondents still prefer the House investigation to continue.  But the support for the House investigation is also sensitive to the contextual information that is provided.  What the public knows and recalls matters for the distribution of opinion.  The AG needs good news to reduce the large deficit of public trust he currently faces.

Methodology

Click here to download a topline report that includes the full survey questionnaire, frequencies for each question, a detailed methodological report (including details about the sampling as well as response rates and cooperation rates) and information about the margin of sampling error.

References

Funk, Carolyn L. 1996. “The Impact of Scandal on Candidate Evaluations: An Experimental Test of the Role of Candidate Traits.” Political Behavior 18(1): 1-24.

Shah, Dhavan V., Mark D. Wattts, David Domke, and David P. Fan. 2002. “News Framing and Cueing of Issue Regimes: Explaining Clinton’s Public Approval in Spite of Scandal.” Public Opinion Quartery 66(3): 339-370.

Zaller, John R. 1998. “Monica Lewinski’s Contribution to Political Science.” PS: Political Science and Politics 31(2): 182-189.

Zaller, John R. 1992. The Nature and Origins of Mass Opinion. New York: Cambridge University Press.

Posted in Everything | Tagged , , | Comments Off on “Convicted? No, never Convicted.”

A Clear Majority of Utah Voters Want the Legislature to Impeach Swallow

Posted on June 17, 2013 by Quin Monson

When a majority of strong partisans from a politician’s own party favor impeachment, that politician is in serious trouble

Update/Correction:  Although the numbers posted in the topline report were released correctly yesterday, the graphic and prose for the favorability numbers posted here contained an error.  The “favorable” numbers correctly joined “very” and “somewhat” favorable.  However, the “unfavorable” numbers only listed “very unfavorable” instead of adding the two together.  This mistake occurred for all of the candidates as I created the graphic and was then carried into the text.  I have now corrected the post to reflect the numbers from the topline and apologize for my error.  Thanks to Bob Bernick’s eagle eye for comparing the graphic to the topline and pointing out the discrepancy.

Numerous allegations have been made about Utah Attorney General John Swallow in recent weeks.  Public sentiment about the controversy has been largely absent from the discussion.  With Swallow mounting a more public defense in the last few days and legislators on the cusp of a meeting to discuss the charges and possibly move forward on impeachment, the Center for the Study of Elections and Democracy at BYU fielded a Utah Voter Poll that included a host of questions about the controversy. Detailed analysis comes below.  Here are the highlights:

  • John Swallow’s favorability is extremely low, now sitting at 12%  (combining “somewhat” and “strongly” favorable).  Less than 2% respond “strongly favorable.”
  • News of the accusations against Swallow is widespread among voters, and folks who have heard about it overwhelmingly believe that Swallow has committed ethical or legal infractions.  87% of Utah voters have “heard, read, or seen anything recently about the accusations,” and of this group, 34% think Swallow has done something illegal and 63% respond that he has done something unethical.  Less than 4% believe that Swallow has done nothing unethical.
  • A strong majority of voters favors beginning the impeachment process. After a brief introduction to the Utah impeachment process, 72% say the Utah House of Representatives should allow formal impeachment proceedings to begin.
  • When the roughly one quarter of Utah voters who do not want impeachment to begin are asked why not, 62% say that the legislature “should wait until the criminal investigations are complete.”  Only 4% say so because they “don’t believe the accusations.”
  •  The electoral consequences of beginning impeachment proceedings now are low. When asked if their own legislator voted to begin impeachment proceedings, a plurality of Utah voters (35%) say that doing so would increase their support for the legislator, while a majority (56%) say it would “neither increase nor decrease my support.”  Only 9% of voters respond that doing so would decrease their support.
  • Finally, at the end of the sequence of questions we asked, “Thinking about everything you have read or heard about the John Swallow situation, which of the following comes closest to your view of what should happen right now?”  78% say Swallow should resign and 22% say he should stay in office.

Favorability

Most of the time, elected officials are pleased when the ratio of favorable to unfavorable attitudes about them is at or above 2:1.  Governor Herbert enjoys a 3:1 ratio and Congressman Matheson is not far behind.  Utah’s two senators both sit near a 1:1 ratio that signals that most of their constituents are reasonably happy with their performance.   John Swallow’s ratio at nearly 1:7 is so backwards that it would cast a dark shadow over any elected official unlucky enough to have such low favorability.  Swallow’s 12% favorability is nearly 40 percentage points below that of Senators Lee and Hatch and over 60 percentage points below Governor Herbert.

Swallow Favorability CORRECTED

Do Republicans and Democrats differ on these questions?

When we examine the various questions about Swallow using partisan identification, two things become clear.  Not surprisingly, there are some partisan differences on questions about Swallow’s behavior generally as well as about both impeachment and resignation.  However, Utahns of all political stripes agree that elected officials should move forward to resolve the situation.  88% of Democrats and 75% of Independents want the legislature to begin impeachment proceedings now.  Republican support for impeachment is slightly lower, but even a large majority of Republicans (65%) want the legislature to begin the process.  This is also true for a clear majority (54%) of “strong Republicans” – who might be thought of as Swallow’s last line of defense.  Of the “strong Republicans” who do not favor impeachment, only 9% say it is because Swallow did nothing wrong or because the charges are not serious enough.  Of those strong Republicans who want to wait, the vast majority (72%) responds that they prefer the criminal investigations to be complete first.  When a majority of strong partisans from a politician’s own party favor impeachment, that politician is in serious trouble, especially when less than ten percent of those believe that the charges are minor or untrue.

Swallow Impeach

Swallow Resign

Additional Implications

We asked a few additional questions to try to unpack some of the other facets of this ongoing story.  Ethics and campaign finance reform constantly returns to the agenda of the state legislature and this scandal may yet bring us additional attempts at reform.  We get a taste of public preferences in the context of this particular story with an agree/disagree question that says, “The attorney general should not take campaign contributions from businesses he is supposed to regulate.”  More than 90% of voters agree with this statement and 76% strongly agree.

Some have defended Swallow by asserting that “he has been a victim of the media.”  A quarter of voters are ambivalent about this statement, while only about a third of voters agree and a plurality (40%) disagree.  In other words, most voters do not believe that Swallow has been the target of an unfair media campaign.

Voters harbor strong doubts about whether or not Swallow can “still be an effective attorney general.”  Only 21% of voters agree that Swallow can still be effective while 61% disagree.

Methodology

Click here to download a topline report that includes the full survey questionnaire, frequencies for each question, a detailed methodological report (including details about the sampling as well as response rates and cooperation rates) and information about the margin of sampling error.

Posted in Everything | Tagged , , , , , , , , , , | Comments Off on A Clear Majority of Utah Voters Want the Legislature to Impeach Swallow

Utah’s Supreme Court, where unanimity is the rule

Posted on June 10, 2013 by Adam Brown

Non-unanimous rulings from the Utah Supreme Court are very rare

The five Justices serving on Utah’s Supreme Court are the final word when it comes to interpreting Utah law and Utah’s Constitution, yet they labor mostly in obscurity, without attracting the sort of public attention that the U.S. Supreme Court attracts. Perhaps that’s because so many of the Utah Court’s rulings are issued unanimously, without the narrow-majority drama that we see in the U.S. Court. In fact, the Utah Court went through a period where non-unanimous decisions were rare–a period that has only recently begun to change.

Composition of the Court

Utah’s Supreme Court consists of five Justices. Utah employs a merit-based nominating system intended to favor expertise over ideology. Still, I’ll start with a simple graph showing how the composition of the Court has changed during the period I’ll be looking at (1997 through 2012). From 1997 through March 2000, four Justices were Democratic appointees (i.e. appointed by a Democratic governor, regardless of the Justice’s own partisanship); from March 2000 through May 2003, only two justices were Democratic appointees; since May 2003, only one Democratic appointee remained on the Court.

Partisan composition of Utah Supreme Court justices

Number of Utah Supreme Court justices appointed by governors of each party, by year

How much dissent is there?

Based on the previous figure, you might expect that 2000-2003 was a tumultuous time for the Court, with many narrowly divided rulings. You would be wrong. The chart below shows the percentage of rulings each year that were issued with anything less than unanimity (i.e. a 4-1 or 3-2 ruling). In 1997 and 1998, 20-25% of rulings triggered at least one dissent. Divided rulings fell sharply from 2000 to 2002 and were rare from 2003 through 2006.

Frequency of non-unanimous decisions by the Utah Supreme Court

Frequency of non-unanimous decisions by the Utah Supreme Court, by year

Even today, despite a modest rise in divided rulings, they still remain relatively rare. The Court hears only 80-100 cases in a typical year. In practical terms, that means we’re looking at only a handful of non-unanimous rulings each year, typically fewer than 10. Reporters take note: Non-unanimous rulings from the Utah Supreme Court are very rare–so rare that they probably deserve some media attention.

Which Justices dissent most often?

This leads us to a natural question: Which Justices are most willing to disagree with the majority? Ten Justices served at one point or another from 1997 through 2012. In the table below, I’ve listed how frequently each Justice dissents from the majority. (The Justices are sorted by the year they left the Court; if they are still on the Court, they are sorted by the year they joined it.) Daniel Stewart and Thomas Lee have been most willing to disagree with the majority’s opinion, although even they joined the majority roughly 90% of the time.

Justice Gov’s party Years Dissents
D. Stewart D 1979-2000 11%
M. Zimmerman D 1984-2000 5%
R. Howe D 1980-2003 4%
L. Russon R 1994-2003 4%
C. Durham D 1982- 4%
M. Wilkins R 2000-2010 4%
M. Durrant R 2000- 2%
J. Parrish R 2003- 1%
R. Nehring R 2003- 2%
T. Lee R 2010- 10%

Which Justices write the most opinions?

Every Court ruling requires a written majority opinion. (At their option, Justices can also release their own written concurring or dissenting opinion.) With 5 Justices on the Court, we would expect each Justice to write roughly 20% of the Court’s majority opinions, and that is roughly what we see. The only Justices outside the 18-22% range are Stewart (17%) and Lee (27%). (Side point: Given that Lee dissents relatively frequently, it is remarkable that he is also the most common author of majority opinions. His willingness to dissent has apparently not alienated his colleagues.)

What’s more interesting, though, is to see which Justices take the time to write concurring or dissenting opinions laying out their own thinking separately from the official majority opinion. Although every ruling needs a written majority opinion, nothing compels a Justice to release or concurring or dissenting opinion other than a desire to formally articulate his or her thoughts on a case. Thomas Lee is prolific in this respect. Whereas some justices release a concurring or dissenting opinion in only a handful of cases that they hear, Lee releases them in 16% of the cases he hears. When we combine his concurring and dissenting opinions with his majority opinions, we see that he releases a written opinion of one type or another in 43% of the cases he hears—almost half. Lee’s closest competition in the period under examination is Michael Zimmerman, who released written opinions one-third of the time.

The table below gives the data.1

Justice Majority opinion Other opinion Any opinion
Stewart 17% 10% 27%
Zimmerman 22% 11% 33%
Howe 22% 7% 29%
Russon 21% 5% 26%
Durham 21% 6% 26%
Wilkins 20% 5% 24%
Durrant 23% 2% 25%
Parrish 21% 0% 22%
Nehring 22% 3% 23%
Lee 27% 16% 43%

About the data

You can find all the Utah Supreme Court’s rulings since 1997 online. Over the winter, my research assistant Luke Bell, presently a CSED Student Research Fellow, combed through all the rulings from 1997-2012 and recorded information about them into a spreadsheet. It came out to over 1400 rulings. Occasionally, a Justice recuses or is absent and a lower court judge sits instead. I omit the small percentage of rulings where a sitting judge wrote an opinion–any opinion–but I keep rulings where a sitting judge joined without writing an opinion.

Posted in Everything | Tagged , , | Comments Off on Utah’s Supreme Court, where unanimity is the rule

Utahns don’t know much about Utah government

Posted on June 3, 2013 by Adam Brown

Utahns don’t know much about national politics, but they know even less about Utah politics.

Although Americans often take pride in living in the world’s oldest democracy, American voters tend to do a very poor job answering basic questions about our political system. (Think you’re an exception? Try this quiz.)

Of course, most political quizzes ask about national politics. I thought it would be fun to quiz Utahns on their knowledge of state-level politics and see how they do. After all, state politics matter far more in our daily lives than federal politics. (If you don’t believe me on that point, I’ll back it up in a moment.)

How much do Utahns know about Utah government?

I included a few political knowledge questions in the March 2013 wave of the Utah Voter Poll, a recurring poll of Utah voters run through BYU’s Center for the Study of Elections and Democracy. First, respondents saw a question that looked something like this:

You will have 60 seconds to answer this question. Which office is currently held by each of the following:

Respondents were shown three names: Joe Biden, John Boehner, and Harry Reid. Under each name, they were shown a list of six federal offices to choose from (it was the same list each time): US Vice President, US Senate Majority Leader, US Senate Minority Leader, Speaker of the US House, US Supreme Court Justice, Office Not Listed.

On the next screen, they saw exactly the same question, but the names shown were Greg Bell, Becky Lockhart, and Wayne Niederhauser. The six offices were changed to corresponding state offices: Utah Lieutenant Governor, Utah Senate President, Utah Senate Minority Leader, Speaker of the Utah House, Utah Supreme Court Justice, Office Not Listed.

Elsewhere in the survey, I also asked respondents how long US Senators serve (6 years) and how long Utah Senators serve (4 years). This one was open-ended, not multiple choice.

That brings us to a total of 4 factual questions about each level of government. Here’s how it came out:

US facts Utah facts
Second highest officer
(Biden vs Bell)		 87% correct 57% correct
Lower chamber leader
(Boehner vs Lockhart)		 65% correct 38% correct
Upper chamber leader
(Reid vs Niederhauser)		 67% correct 31% correct
Length of Senate term
(6 vs 4 years)		 56% correct 49% correct

Overall, the average respondent answered 69% of the US questions correctly and 44% of the Utah questions correctly. The 25 percentage point gap is impressive and disappointing. Utahns don’t know much about national politics, but they know even less about Utah politics.

We can look at this another way. Of all our respondents, 42% answered all four US questions correctly, while only 15% answered all four Utah questions correctly. Those 42% are our national political junkies. Our state political junkies appear to be missing in action.

Which voters performed the best?

I used some statistical analysis to assess which demographic groups performed better or worse with these knowledge questions overall. The method I used allows me to estimate the effect of one specific variable when other factors (age, education, income, gender, partisanship, marital status, religion) are held equal.1 Unsurprisingly, those with a college degree or a higher income performed better on both the US and Utah questions. What I really wanted to know was who did better on the Utah questions specifically, though.

Turns out that Democrats did. Democrats scored 0.44 points higher (out of 4 questions) than Republicans on Utah questions; on US questions, the effect was smaller (0.34 points) but still substantial. I find it interesting that the gap was largest on the Utah questions. Perhaps Republicans feel safe ignoring Utah politics, under the assumption that the Republican government will do Republican things. Meanwhile, Democratic voters might feel more threatened by the state’s Republican government and therefore pay more attention to it.

There was also a Mormon effect. Mormons outperform non-Mormons on Utah politics knowledge (by 0.25 points); the effect goes away on the US questions.

For those who wish they knew more about Utah politics, there is a simple remedy: Watch local news broadcasts, or read local newspapers. Those who report doing so score much higher (0.48 points) on the Utah questions. They also perform marginally better (0.22 points) on the US questions.

Why should Utahns care about Utah government?

Most Utahns don’t care much about Utah politics. I gave respondents a list of US and Utah political institutions and asked respondents to rate on a scale from 1 (“not at all”) to 5 (“a great deal”) how much each institution affects their “personal life or financial situation.” Here’s the average score for each institution:

US institutions Utah institutions
Legislative 4.19
(US Cong.)		 4.01
(Utah Leg.)
Executive 3.94
(US Pres.)		 3.71
(Utah Gov.)
Judicial 3.81
(US Sup. Ct.)		 3.38
(Utah Sup. Ct.)
Average rating 3.98 3.70

Respondents consistently rated Utah’s political institutions lower than the corresponding national institutions. When the three scores from each level are averaged together, respondents gave an average rating of 3.98 to the national government but only 3.70 to the state government. Here are two facts that call these judgments into question:

  • Each year, America’s 50 state legislatures pass roughly 100 times as many bills as Congress passes. Utah’s Legislature alone passes twice as many bills as Congress.
  • Each year, America’s 50 state court systems hear roughly 100 times as many cases as the federal court system. Utah’s state courts alone heard roughly as many cases (306,014 cases) in 2011 as the entire federal court system (344,642 cases).2

There is a reason that state governments pass WAY more laws and hear WAY more cases than the federal government: Most of the day-to-day governing happens in the states. Primary education, higher education, public safety, professional licensing, business regulation, land use and zoning, sales tax, property tax, public transit, incarceration, freeway construction, infrastructure development, water management, and most other policy matters are mostly handled at the state level. Unless you serve in the armed forces, work for the federal government, or depend on Social Security for day-to-day income, few federal actions have as much effect on your life as state actions. State governments matter WAY more in telling you what you can do and how you can do it.

Punchline

  • Far more governing happens in the states than in Washington. Utah’s legislative and judicial branches alone are as busy as the nation’s legislative and judicial branches.
  • Regardless, people believe that the federal government makes more decisions that affect their lives than their state government.
  • As a result, people pay more attention to national politics than to state politics.

Make of it what you will, but I found it interesting.

Methodological note

For question wording, sample size, margin of error, and other details, you can download the 2013 March Utah Voter Poll topline report here. [update: some errors were in the topline; it will be reposted soon.]

This post benefited from research assistance rendered by Justin Chang, a CSED student fellow. He dutifully accepted the grunt work assignment of calculating all the statistics shown here after I handed him the survey results. Any mistakes are my liability, of course, not his.

Posted in Everything | Tagged , , , , , , , , , , , , | Comments Off on Utahns don’t know much about Utah government

What is the future of the Republican nominating system?

Posted on April 17, 2013 by Kelly Patterson

The Republican Central Committee recently decided that it would not change the manner by which candidates gain access to the Republican primary ballot. The decision not to change the rules creates a possible showdown with Count My Vote. The group may seek to place an initiative on the ballot that gives candidates access to the primary ballot if they can gather enough signatures from voters. This procedure would allow candidates to gain access to the ballot even if they fail in the caucus-convention process.

It is difficult to predict the outcome of a ballot initiative so far in advance. Public opinion on ballot initiatives can be influenced by several factors. The intensity of the campaign and the kinds of arguments made to the public will matter.

The Key Research poll conducted at the end of January offers some clues about what the electorate might do. It contained the following question: “Would you favor or oppose making it possible for political candidates to participate in a primary by gathering voter signatures?”

One of the proposals being discussed would allow a candidate who did not prevail in the convention to gather a certain number of signatures and appear on the primary ballot. Fifty-eight percent of registered voters said they would “favor” or “strongly favor” such a proposal. When those answering “Don’t Know” are removed, the proportion of those favoring such a proposal increases to almost 67%.

Favor Access to the Primary Ballot through Signatures
Frequency Percent Valid Percent Cumulative Percent
Valid Favor 290 58.0 66.8 66.8
Oppose 144 28.8 33.2 100.0
Total 434 86.8 100.0
Missing System 66 13.2
Total 500 100.0

Partisans differ in their assessment of the proposal. Republicans view the proposal less favorably than Democrats.

Favor Access to the Primary Ballot With Signatures
Dem. Ind. Rep. Other Total
Favor 58
77.3%		 45
65.2%		 172
63.2%		 15
83.3%		 290
66.8%
Oppose 17
22.7%		 24
34.8%		 100
36.8%		 3
16.7%		 144
33.2%
Total 75
100.0%		 69
100.0%		 272
100.0%		 18
100.0%		 434
100.0%

If there is a ballot initiative, those who favor change start with an advantage. However, as Dr. David Magleby’s classic work on initiatives demonstrates, the “No” side can easily overcome these initial advantages.

Survey Methodology

Click here to download a topline report that includes the full survey questionnaire, frequencies for each question, and a detailed methodological report (including details about the sampling as well as response rates and cooperation rates).

Posted in Everything | Tagged , , , | 3 Comments

Recap: Looking back at the 2013 Utah Legislature

Posted on March 27, 2013 by Adam Brown

I’ve posted tons of stats about the Utah Legislature in the past couple days. To recap, here’s a quick summary of some of what you can now find here:

Do legislators work enough to justify their salary? If you take the number of hours they work and divide it into their pay, it’s not as glamorous as you might think.

Were Democrats able to pass their bills this year? Given that they control only 17-19% of the seats, they still passed a respectable 43% of their bills. However, the gap between the Democratic and Republican batting averages grew substantially this year.

The closest votes in the 2013 Legislature. Utah Legislators continued their norm of passing most things out by overwhelming 90+% majorities, but there were still some close votes. I list the 20 closest votes for each chamber here.

How quickly were bills passed in 2013? In 2011, legislators shifted toward introducing their bills later and, as a result, processing them much faster. That change has leveled off, not deepened or reversed, with 2013 looking a lot like 2011 and 2012.

Who voted “nay” most often in 2013? That’s right. I’ve ranked legislators by how often they did (or didn’t) vote “nay.” The overall winner was Dan McCay.

Who ran the most bills in the 2013 Legislature? Some legislators sponsored lots of bills; some passed lots of bills. The two lists don’t line up as much as you might think, though. I calculate an effectiveness score for each legislator (at least, those who sponsored at least a handful of bills) reflecting the percentage of their bills that they managed to pass.

Who missed the most votes in 2013? With so much to do in the brief 45-day session, legislators often leave the voting floor to tend to their other responsibilities. You’ll find the data here.

Bonus: My colleagues posted polling data a few weeks ago showing what Utah voters think about gun policy and about the John Swallow allegations.

That’s it for now. There may be more in a couple weeks. I’m sitting on other material that I just don’t have time to post about right now. I collect more statistics about the Utah Legislature than I blog about here. You can find all (well, most) of my statistics here: Utah Legislature statistics and trends.

Posted in Everything | Tagged , , , , , , , , , | 1 Comment