Interest group ratings are certainly useful, but it turns out there’s a much, much better way to figure out which legislators are most conservative and most liberal.
Credit where it’s due: Robert Richards, my undergraduate research assistant extraordinaire, contributed heavily to this research.
After each year’s legislative session, a handful of interest groups assign grades to each member of the Utah legislature. After collecting all the 2011 ratings and averaging across them, the Salt Lake Tribune recently published its scorecard for the 2011 Utah legislature, calling Rep. Carl Wimmer the most conservative legislator and Rep. Rebecca Chavez-Houck the most liberal legislator.
Interest group ratings are certainly useful, but it turns out there’s a much, much better way to figure out which legislators are most conservative and most liberal. Let’s take a look.
The trouble with interest group ratings
The Tribune’s ideological scores are based on rankings from the Utah Sierra Club, Parents for Choice in Education, the Utah Education Association, the Utah Taxpayers Association, and GrassRoots. Each group is looking only at certain issues as it evaluates each legislator. For example, the Sierra Club’s ratings look only at votes dealing with conservation and wildlife, while the PCE and UEA ratings look only at votes dealing with education.
Trouble is, there were 782 bills introduced during the 2011 legislative session, many of which have little to do with education or conservation. As such, ideology scores based on these interest group ratings will be less accurate than ideology scores that account for every vote cast.
Now, I’m not saying that interest group ratings are useless; after all, I’ve used them plenty of times in the past (for example, see here or here). But I am saying there might be a better, more thorough way to estimate each legislator’s ideology.
A better way to measure ideology
Political scientists have developed a way to measure each legislator’s ideology that accounts for every vote cast in the legislature. These are known as NOMINATE scores. (It’s an acronym, hence the capitol letters; see a history of NOMINATE scores here.) NOMINATE scores have been used to study Congressional ideology for years. However, we have not had NOMINATE scores available for Utah legislators in the past because they are so extremely time-consuming to calculate.
There is more than one version of the NOMINATE algorithm (we use W-NOMINATE), but the basic idea is to use all of the votes a legislator casts in a legislative session to assign a score between -100 (more liberal) and +100 (more conservative). The specific numbers have no meaning; just because a score comes out below 0 does not mean that a legislator is a “liberal.” The numbers only have meaning relative to one another. If your score is closer to -100 than mine is, then you are more liberal than me.
These caveats are important as we proceed:
- NOMINATE scores are comparable only within a single chamber within a single year. Since scores are calculated separately for the Utah Senate and Utah House, you cannot compare a Senator’s score to a Representative’s to see who is more liberal. Sorry. NOMINATE scores do not allow for that. (Frankly, interest group scores don’t allow for this either, and for the same reason: They are based on a different set of votes in each chamber, each year.)
- NOMINATE scores have no intrinsic meaning. There is not a number that means “conservative” or “liberal.” The scores are comparable only relative to other scores within the same chamber, within the same year.
- A lower number is more liberal, a higher number is more conservative. But remember, “0” doesn’t mean “moderate,” it just indicates somebody near the ideological center of the legislature. Given how conservative the legislature is overall, the center is also conservative by national standards.
Ideology scores for the Utah House
So how does Utah’s state legislature look based on a NOMINATE analysis? We ran the necessary calculations for Utah’s 2011 legislative session, and found results similar to the Tribune’s in many respects–but not all.1 Like the Tribune, we find that Reps. Rebecca Chavez-Houck and Carl Wimmer are the most ideologically extreme Utah legislators. Our rankings differ on other members, though. The table below compares our rankings for the Utah House to the Tribune’s rankings:
| Salt Lake Tribune | NOMINATE scores | |
| #1 liberal | Rebecca Chavez-Houck | Rebecca Chavez-Houck |
| #2 liberal | Brian King | Joel Briscoe |
| #3 liberal | Janice Fisher | Jackie Biskupski |
| … | ||
| #3 conservative | Keith Grover | Kenneth Sumsion |
| #2 conservative | John Dougall | John Dougall |
| #1 conservative | Carl Wimmer | Carl Wimmer |
In some cases, our rankings are very different from the Tribune’s. Consider Speaker Becky Lockhart and former Speaker David Clark. The Tribune ranks Rep. Clark the #4 most conservative Representative, with Lockhart down at #27, suggesting that they are far apart. But by taking account of every vote cast, we find that Rep. Clark comes in at #15 most conservative with Lockhart at #13–a trivial difference.
Here’s another difference. The Tribune found that the most liberal Republicans were to the left of the most conservative Democrats in the Utah House. By contrast, we found that every Republican was to the right of every Democrat in the Utah House. Click to view our complete list of ideology scores for Utah Representatives.
Ideology scores for the Utah Senate
Let’s move to the Senate. The Tribune identified Mark Madsen as the most conservative Senator, with Scott Jenkins somewhere closer to the middle. But by taking account of every vote cast in the 2011 session, we find that Scott Jenkins was actually the most conservative Senator, with Mark Madsen the #5 most conservative.
| Salt Lake Tribune | NOMINATE scores | |
| #1 liberal | Ben McAdams | Ross Romero |
| #2 liberal | Ross Romero | Luz Robles |
| #3 liberal | Gene Davis | Ben McAdams |
| … | ||
| #3 conservative | Howard Stephenson | Chris Buttars |
| #2 conservative | Wayne Niederhauser | Margaret Dayton |
| #1 conservative | Mark Madsen | Scott Jenkins |
Having spent plenty of time sitting in the legislature’s galleries over the past few years, the NOMINATE rankings just seem to ring more true than the Tribune rankings. I’m not sure I would have guessed that Senator Jenkins would be the most conservative, but I would certainly expect Senators Dayton and Buttars to be among the most extreme conservatives in the Senate. Click to view our complete list of ideology scores for the Utah Senate.
Final thoughts
Let me stress that I’m not trying to be harsh on the Tribune. I’ve used the exact same interest group scores in the past.
But why use NOMINATE scores instead of interest group ratings? Again, NOMINATE scores are more comprehensive than interest group scores, because they use every contested vote on every issue, rather than using only a few votes on a single issue. This also eliminates any bias involved in choosing which votes to count. In short, NOMINATE scores take into account everything a legislator does, producing a reliable, unbiased picture of where they stand on the ideological spectrum.
A few clarifications
(I added this update a week after publishing the preceding post. Some folks are misinterpreting these scores. Here’s hoping this update helps.)
It’s not necessarily an honor to be the most conservative or most liberal person. After all, if Lenin were in the legislature, he’d get a score much lower than any Utah legislator, and if Mussolini were there, he’d get a score much higher than any Utah legislator. A high score doesn’t mean “most loyal Republican,” it simply means “further right than anybody else.” Even if you are a very conservative (or very liberal) voter, you should not assume that the person with the most extreme score is the one with the views closest to your own.
Also, the rankings are less important than the actual scores. Many scores are clustered very close together, so that there is very little ideological difference between the #20 and #30 most conservative legislators. Don’t just look at the ranks; look at the actual scores. (To see the actual scores, follow the links given above.)
Possibly related posts:
Possibly related posts (automatically generated)
What kind of Republican is Spencer Cox? (October 16, 2013)
Very interesting. Thank You for posting. After watching almost 700 floor votes this year, certainly much more accurate than the SL Trib or any other system I have seen for overall scores. It would be helpful to number them from 1 to 75 vs just listing the score, as it is the order not the score that really means something.
If you follow the links given in the post, you can see all the ranks in order. But be careful to say that it’s the order and not the score that matters. It’s both. If two scores are very close together, then we need to treat those two legislators as essentially tied. If we focus only on the order, not the scores, then we will overlook that sort of detail.