Welcome, my name is Floyd Ciruli

This roundtable is an additional component to the conference dialogue on the challenges faced by public opinion research, especially related to its image – its public face.

The Phoenix conference, in retrospect, may be seen as a tipping point – where AAPOR begins to aggressively address some growing weaknesses & threats.

The response, in my view, should not only be from the association, but should involve academic, commercial, nonprofit and media members taking independent actions and addressing specific topics.

One of the challenges that this panel addresses concerns the increasing incident of media polls being attacked as biased due to alleged bias of the publisher or broadcast network.

Of course, polls should be evaluated and criticized. That is why our panels have discussants; it is why papers are juried. List selection, interview techniques, question wording and interpretation are fair game, both at conferences with our peers and in a wider setting among partisan critics. But this discussion focuses on a newer aggressive feature in which poll results are attacked and debunked due to alleged bias of the newspaper, network or news outlet (the editorial positions of the paper or vague accusations of news bias are sufficient for critics). There may be some reference to a methodical flaw, but generally the critics simply dislike and don’t trust the paper or network.

The charge of bias among major media outlets has been around for more than a decade, largely repeated by influential talk radio hosts. However in the mid-to late-1990s, it was joined by such authors as Bernie Goldberg, and a new generation of talk show hosts such as Hugh Hewitt, in addition to Internet sites dedicated to criticizing media output. A network of talk show hosts, newspaper columnists and Internet bloggers have created a news cycle of instant attacks on alleged bias.

And not to be left out – the left has joined the chorus with books and websites attacking Fox News, Clear Channel and others for alleged conservative bias. In fact, liberals are starting a liberal radio network (and possibly TV network) to counter what they call the excessive power of the right.

And it’s having an effect. Pew reported in January that there are now more people who believe news organizations are biased (39%) than not biased (38%). The 38 percent “not biased” was a dramatic drop from 53 percent in 1996.

We have a panel this morning of pollsters who deal with these issues regularly as their polls are released, especially during the election cycle and on controversial topics.

They are:
Robert Daves of the Minneapolis Star Tribune, our conference chair who recommended the timely conference theme of “The Image of Public Opinion Research.”
Claudia Deane-Washington Post
Kathy Frankovic-CBS News
Susan Pinkus-Los Angeles Times

Let me conclude that, along with an examination of the phenomena with examples and a prognosis, the goal is to identify some best practices to proactively address the issue.

Observations and recommendations from panelists and the audience.

Observations

• Polls hold high interest and have immense power. They tend to frame news coverage and shape the actions and expectations of opinion leaders, candidates and campaign stakeholders. They are the primary topics of the “war room” and the “boys on the bus.”
  
• The Lewinski era accelerated the attacks on polls. Clinton’s low favorability but high approval rating was unusual and greatly irritated Clinton critics. National polls probably saved his presidency during the summer and fall of 1998 but became part of the story.
  
• The 24-hour news cycle has developed a format that requires liberal and conservative and pro and con perspectives. Polls are placed in the same context. The pollster is presented as speaking for the poll and someone else is criticizing it.
  
• Spinning polls is now a professional job in many campaigns, which either the campaign manager, press secretary, or pollster must fulfill. A new strategy is to anticipate polling results and begin spinning in advance.
  
• Entire websites are dedicated to criticizing newspaper coverage or news anchors. News organizations are labeled by color (red or blue) and polls are being colored by the labels.
  
• Nationally, there are now dozens of tracking and regular polls following the presidential election. Polls are published more frequently and from more organizations. There will be statistical and methodological differences from survey to survey. Any poll out of step will produce reaction even though it may be pointing to a new trend and/or simply be a product of random variation.
  
• State-level polls often receive more criticism than national polls. There are usually fewer polls and they can have a major impact on an election.
  
• Many critics have a political agenda and don’t care for dialogue or explanation. Hence, any response should be aimed at the broader public and not partisan critics.
  
• The punditocracy uses polls as weapons. Whenever a story reports “a poll shows” it is usually a miss-or over-interpretation of a poll’s findings.
  
• A beat reporter often is writing the lead and may not put the results in context.
  
• Recent research shows that bias is mostly in the eyes of the beholder. The major networks in fact use at least as many Republican vs Democratic sources in broadcasts.
  

Recommendations

The following were some of the recommendations from the discussion and question-and-answer period:

1. Develop a panel of advisors for polls.
   
2. Participate on local talk shows to explain polling and put polls in context.
   
3. React rapidly to criticism
   
4. Create a newsletter with recent data, comparative data, interpretation and methodological materials
   
5. Explain sampling with simple, understandable metaphors such as
   • Blood test
    Testing soup
   
6. Have pollsters write (or assist in writing) analysis of polls. Put results in historical perspective and use comparative data from other polls and periods.
   
7. Discuss any methodological issue in the news story such as over-or-under sampling of key sub-groups
   
8. Release sufficient sample information in the story or accompanying explanatory box.
   
9. Explain margin of error to facilitate accurate reports of comparative poll results.
   
10. Improve the AAPOR website to address the new more aggressive environment.
    
11. The association may need a spokesperson.
    
12. AAPOR should be strongly identified as upholding standards and offering leadership on quality polling and accurate reporting of results.

[top] [archives] [home] [send this page to a friend]