Here at HighGround, we often say that accurate and unbiased survey data is one of the most important tools in your toolbox. Our team has more than a decade of survey experience and has become a statewide leader in determining and measuring statewide, regional, and local behavior. During that time, we have seen a lot of surveys – some good and some terrible. Our very own Sr. Vice President of Research and Strategy, Paul Bentz, joins the Arizona Capitol Times to discuss polling in their latest article: “Misunderstood cornerstone of elections, politics.”
Misunderstood cornerstone of elections, politics
Polls are flooding the world with who’s up, who’s down, and what’s on the mind of the people as the 2020 election is well underway.
And while polling is a cornerstone of politics and elections, teaching about polling isn’t a high priority of public schools, and the media can’t be counted on to fully understand and accurately report on the nuances of a particular survey.
Mike Noble, the managing partner of OH Predictive Insights and the most nationally recognized Arizona pollster, said he can tell not a lot of media members know what they’re doing with polls
“I love the media, I think they’re great, but I think it’s just kind of this new dynamic of this instant gratification … when you see a poll, and it kind of lines up with maybe a headline or view they were kind of feeling and thinking, then sometimes you get folks that will run and publish a poll without doing any due diligence,” Noble said.
Noble said he can tell who knows their stuff when they ask certain questions like if they conducted the poll themselves, the “top lines,” “cross tabs” or asking for a simple summary breakdown.
“I think most reporters are pretty good about it. But again, there’s just some that unfortunately, they’ll see the headline, and they’ll run it,” he said.
Schools don’t really teach a basic understanding of poll reading either. At Arizona State University, the focus tends to be on the mathematical element rather than political impact.
Gina Woodall, a senior lecturer at Arizona State University’s School of Politics and Global Studies, said she teaches polls in some of her classes, but focuses on methodology and statistics for the most part.
“When it comes to ‘reading polls’ one must understand what a ‘normal’ curve is, basic probability, standard deviations, confidence intervals, and sampling,” she wrote in an email. “A ‘good’ poll is always one where the sample size is chosen in a probabilistic way, and where the sample size is as large as it can be (law of large numbers)–given funding and time constraints.”
Woodall said polls are taught in some advanced placement level statistics courses at the high school level, but “the vast majority of students probably don’t get introduced to reading and analyzing polls until college.”
It’s not just everyday students who aren’t learning the importance of polls, but the people set to cover them in an election cycle aren’t learning how while in school in Arizona.
The Walter Cronkite School of Journalism and Mass Communication at ASU doesn’t offer specific courses dedicated to polling and elections, but has lessons in some classes.
“We do touch on it in several courses, although not in an in-depth way,” said Kristin Gilger, Cronkite’s interim dean. “Something for us to consider.”
Methodology
Conducting a poll can be considered art in the world of politics. Many groups and individuals conduct their own, there are wide interpretations of what they mean, and a lot of work goes into getting them right.
But reading a poll can be more challenging. There are certain elements to look for in every political poll — pollsters have their own go-tos, but generally they agree on a few certain principles.
The methodology — or how the poll is conducted — is the first part.
Paul Bentz, the senior vice president of research and strategy for HighGround Public Consulting, said the methodology is the “fundamental base” aspect everyone should look at.
There’s live caller, interactive voice response, or IVR, push button or a mixture of IVR and live caller. IVR and push button polls cannot call cell phones, because it’s illegal to robodial a cell phone.
“So any poll conducted solely by push button does not include a cell phone responding category,” Bentz said.
HighGround conducts their polls solely through live caller. Bentz said a major difference between live and push button is for the latter, “you don’t necessarily know who you’re calling. You think you know, but you don’t know who picked up the phone.”
Whereas if you ask for a person on the phone, he said of live calling, you know exactly who you are talking to. He said some pollsters swear by a mixed model, but using a push button does not automatically negate the poll, it’s just not his method.
“All push button polls should be met with a bit of skepticism,” he said.
Noble said good polls boil down to five or six elements called crosstabs. Noble conducts his polls with a mixed method.
He said the important crosstabs every good poll should have are gender, age, region, party affiliation and ethnicity, but he’s recently added the highest level of education to the mix saying it’s become important in recent elections.
Noble also said it’s important to proceed with caution when looking at a poll from an unrecognizable source. A lot of people are conducting their own polls to push a certain agenda, which he said hurts the reputation of the industry as a whole.
Noble and Bentz cited a recent poll from One America News Network, a right-leaning news channel and website, as an example of a “bad” poll, but said to not necessarily jump to conclusions because of the source. OAN’s poll, which was entirely push button and had many technical flaws, had U.S. Sen. Martha McSally and President Trump both up by 4 percentage points in their respective campaigns, while most other polling has found them to be trailing.
Demographics
Bentz said demographics is the next important factor. But not just including the right demos (or crosstabs), but making sure to ask the person on the line their demos. He likes live caller because he can look at some people’s voter registration files and know how they vote or register rather than what they say on the phone.
“People may not know how they’re registered to vote,” he said, adding that they could also be mad at their political party on that certain day and decide they are an independent rather than Republican or Democrat.
Arizona’s voter registration breakdown as of April 1 is roughly 35% Republican, 33% Democrat and 32% independent. These numbers are slightly skewed due to more independents registering as Democrat for the Presidential Preference Election in March. Polls should typically reflect that split, within a margin given that voters don’t reflect the exact split of party registration.
Bentz said that’s why it’s important to know someone’s party identity rather than rely on their response.
“There is a several point shift, if you ask people their party identity versus their actual party identity,” he said.
Bentz said the next important factor is the participation model, which is where you see who the sample size is: likely voters and registered voters are the most common types. Bentz does likely voters and looks at voting history as his base model.
He said he looks at voters who participated in two of the three most recent elections, while some may look at three of four. But that also changes depending on the age of the likely voter since some younger voters may not have been eligible to vote during a previous cycle.
“Part of the reason we do likely voters is that Republicans over participate in Arizona elections and Democrats participate right at about where they are, and then independents under participate,” Bentz said.
He said that’s how you can tell if Arizona is still considered a red state is how people participate in the election. Previous gubernatorial elections had a Republican advantage of plus 14 points, but in 2018 that number was cut in half to plus seven. Presidential elections typically see a smaller margin and Bentz expects 2020 to be around a four point gap with more independent participation.
Noble says he’s polling at Republicans plus five.
“When people talk about the ‘blue wave,’ what they think is, ‘oh, certainly there’s more Democrats than Republicans voting.’ And that’s just not the case. What actually is happening in Arizona is that the gap is closing. That more Democrats are participating and eroding or eating away at the Republican example,” Bentz said.