Over the past few years, political candidates and their parties have jumped on the bandwagon of cyberpolitics by creating Web sites to disseminate information and to take the pulse of the voters. The number of on-line polls has also increased dramatically. In surveying such polls on the Web, this article examines how they are conducted and how they are reported by the traditional news media.
Over the past few decades, polls have become ubiquitous in American society. During the 1992 election season, a rough estimate suggests that between five and ten thousand election polls were conducted (Traugott and Lavrakas 1996). The pervasiveness of polls is clearly demonstrated by their increased use in major news stories. Herbert Asher (1995) examined stories featured on the covers of the three major news magazines--Time, Newsweek, and U.S. News and World Report--and found a sizable increase in the use of public opinion polls as an integral part of cover stories between 1973 and 1993. For example, only about 15 percent of Time cover stories in 1973 used polls; in 1993 that number had increased to 46 percent (Asher 1995).
In addition to this development, in the past few years political candidates and their parties have jumped on the bandwagon of cyberpolitics, littering the Web with their sites to disseminate information, gain votes and volunteers with sound bites and pictures, and feel the pulse of the voters. In addition to the proliferation of traditional polls, there is also a growing number of on-line polls trying to monitor public opinion.
Criticisms of On-Line and Call-in Polls
Although on-line polling is relatively new, it has been criticized for similar problems as its long-discredited cousins, call-in and write-in polls. Some critics of such polls point out that they suffer terribly from major sources of survey error (Traugott and Lavrakas 1996). In particular, there is absolutely no control over who is "sampled" because respondents are self-selected and may log in or call in as many times as they choose. As a result, no one has any idea who is included in the sample. Therefore, there is no way to evaluate the external validity of the results (Gawiser and Witt 1994). Others argue that at times such polls simply invite respondents to express private prejudices and thus give such prejudice public standing and reinforce its legitimacy (Barber 1992). This is not to say that the results of call-in polls are always wrong; however, the people who conduct these polls have no way of knowing whether the results are right or wrong (Traugott and Lavrakas 1996).
Some critics have made more severe criticisms against call-in and on-line polls. Polling experts call them "an embarrassment, a horrible idea" (Mitofsky 1992:22). Sheldon Gawiser and G. Evans Witt dismiss them as "worthless, useless and misleading pseudo-polls" (1994:98). Norman Bradburn, former director of the National Opinion Research Center, has described these types of endeavors as SLOPS (self-selected listener-oriented public opinion surveys) (Gawiser and Witt 1994). However, "too often, far too often, SLOPS are reported as polls," Gawiser and Witt complain (1994: 98).
In defending the infamous CBS "America on the Line" news broadcast,[1] Kathleen Frankovic, director of surveys for CBS News, argues that the major criticisms of the program came from an assumption that much of the public cannot distinguish between call-in results, which cannot be generalized to the populace at large, and poll results, which can be generalized. "I'm confident that we can educate the public better on this distinction, even in the context of this audience participation device," she argues (Frankovic 1992:19).
Opponents disagree. They point out that Americans do not demonstrate much of an understanding of such concepts as margin of error. The results of a study by Paul Lavrakas and colleagues reinforce the view that Americans have a long way to go before they adequately understand the limitations of information generated via sample surveys (Lavrakas et al. 1991).
Some advocates of electronic and call-in polling often quote the number of the responses to the polls as evidence of their viability. In a 1995 America Online poll, 19,812 people responded, which led Adam Schoenfeld, vice-president for Jupiter Communications, to claim that "the size of the response was an impressive demonstration of how an online computer service can almost instantly measure public opinion on an issue."[2] Schoenfeld asserted that political pollsters might predict national and local elections based on on-line polls, as is already being done to gauge consumer responses to products.
Critics counter that even with hundreds or thousands of replies, these "straw polls" are still not representative. Despite their enormous numbers of respondents, both the infamous 1936 Literary Digest straw poll (based on about two million responses and incorrectly predicting Roosevelt's loss in the election) and the CBS call-in poll cannot be considered accurate measures of public opinion. This is because people who voluntarily choose to participate are likely to differ in important ways from the overall population. They may be more interested, informed, and concerned about the topic at hand and thus may hold views different from those of the overall population (Asher 1995), or they may have more strongly held views on the issue.
Web Users
Such discrepancies apparently are the result of the substantial demographic and socioeconomic differences between Web users and the general population.[3] Surveys have shown that the Web is a predominantly male domain and that its users are disproportionately white and young. Americans with higher levels of education are more likely to use the Web than their less-educated neighbors. A survey by SRI International on Web users has identified two Web audiences: "upstreamers" and the "other half." Upstreamers are predominantly male (77 percent) academics or professionals who have institutional Net accounts; 93 percent of them have at least some college education.[4] Web use is also related to income and race. Nearly half of all Web users (48 percent) make more than $ 50,000 annually (Birdsell et al. 1996), and the U.S. Census reports that only 14 percent of black households own personal computers,[5] compared with about 35 percent nationwide.
Those who use the Web for political purposes reflect roughly the same characteristics. In a recent Baruch-Harris survey, 90 percent of the users said that they voted in the 1992 presidential election (Birdsell et al. 1996), compared with 55 percent nationwide, placing them among the most politically active audiences using any form of communication.
On. Line Polls and Democracy
Another related issue of debate concerning on-line polls is whether these new communication technologies are good or bad for American democracy. Some feel that these new media may improve democracy by expanding the country's political conversations and citizens' sense of empowerment, considered by some theorists to be the essence of democratic practice. Others fear that these new media formats are ushering in an era of "hyperdemocracy," an inherently unstable form of direct democracy featuring "electronic town halls" and computer-TV interactive voting (Birdsell et al. 1996).
Opponents also argue that the new media are available principally to wealthy citizens, whites, and men, and they encourage in all users a vitriolic, uncivil rhetoric that may accelerate atomization rather than provide its cure. Government professor James S. Fishkin dismisses the new technology and politicians' attempts to enter into a real dialogue with voters. Citizens, he contends, are more disengaged from their political leaders than ever. Dismissing the proliferation of polls, Fishkin calls them isolated snapshots that "give us no information about how people would actually weigh tradeoffs among valued goals and among alternative means to achieving those goals."[6]
Some advocates of on-line polls have predicted that "e-campaigns" and online polls will change American politics and create an "online democracy" because of the "democratic and interactive" nature of the Internet. Richard Maisel and colleagues argue that interactive on-line services provide an inexpensive and efficient alternative to telephone interviewing (1995). They claim that online services might lend themselves to alternative approaches that have shown great potential when it isn't feasible to conduct precise sample surveys or when the cost of doing so would be prohibitive.
Because of the speed and frequency of call-in and on-line polls, some warn that people should expect what might seem to be more volatile public opinion (Crossen 1994). Daniel Yankelovich finds that the process of forming public opinion on complex issues can take more than ten years through several stages and that in the earliest stages, opinions tend to be vehement but extremely unstable (1992). Today's quick on-line polls are taken in the first stage and declared to be public opinion--until someone does another poll.
This article does not attempt to resolve many of these disputes. Instead, it presents evidence about polls on the Web and about the traditional news medias coverage of such polls.
Methods
This study was done in two stages. First, we studied on-line polls. We searched the World Wide Web for sites that would be likely to carry such polls. Due to the exponential growth of the Internet and the formidable number of Web sites, our search was not comprehensive. We searched for sites that are easier to find by major browsers, such as Yahoo, with keyword searches under "polls," and we focused on political polls. Altogether, we examined 57 Web sites that offer on-line polls (see Appendix A for a list of URLs). We also tried to vote in most of the polls to check the possibility of voting more than once, and we examined how the sites presented the results.
In the second stage, we examined the news media's coverage of on-line polls by conducting a search of all stories in Lexis-Nexis with the keyword online poll, ending on July 31, 1996. To be more exhaustive in our search, we searched repeatedly under all six categories: magazines (MAGS), major newspapers (MAJPAP), newspapers (PAPERS), transcripts (SCRIPT), wire services (WIRES), and newsletters (NWLTRS). The search identified 80 stories that either were totally devoted to the reporting of on-line polls or used some on-line poll results. We then grouped them by the ways they reported poll results and analyzed them using the guidelines of the American Association for Public Opinion Research (AAPOR) for reporting surveys and polls.
Findings
A Profile of On-Line Polls
Our study finds that on-line polls cover a range of topics, from politics and entertainment to marketing. Similar to most other political polls, the majority of on-line polls from the Web sites we checked (almost two-thirds of the total) could be classified as "horse-race" polls, followed by polls of general political topics, such as abortion issues (Table 1).
Our search also finds that most on-line polls allow multiple votes (18 of the total 23 political sites where we were able to cast our votes), which means that by clicking on "submit" or "vote," visitors of those sites can send as many votes as they wish. Many on-line polls also ask very detailed personal information of the user in the ballot, such as full name, home address, and e-mail address as well as gender, income, and education. Some will not let users vote unless they provide the requested information.
Some on-line polls push the users for opinions. For example, in a poll by SurveyNet, most questions include a choice of "no answer," but the site will not let users submit their vote unless they choose either "yes" or "no." Users receive a message telling them that if they do not have an opinion, they should not vote.
On the whole, these polling sites do not seem to be well managed. Most sites do not reveal the owner or sponsor of the site, the operator, or the purpose of the poll. Some sites give the poll results, and some never do. As we will discuss later, even if some on-line polls do present results, they often differ considerably from those found in major representative national polls and from actual election results. Some sites provide a link to a results page that either is not accessible because it is still "under construction" or is not updated regularly. Of those sites that do provide results, most do not give the closing date or the date the votes were counted. As we will discuss later, the wording of some polls, especially political ones, was clearly biased.
Media Coverage of On-Line Polls
The number of news stories that reported on-line poll results increased from merely five in 1992 to 39 in 1995 (see Table 2), including some by major news media, such as the New York Times, the Washington Post, Newsweek, and Advertising Age. This is one indicator of the increasing popularity of on-line polls, despite the complaints and controversies.
M. Mark Miller and Robert Hurd conducted a study of how well three newspapers--the Chicago Tribune, the Los Angeles Times, and the Atlanta Constitution-followed the AAPOR guidelines in reporting polls in general (1982). They found no marked trend indicating improved poll reporting over time (Asher 1995: 86). They speculate that in smaller daily and weekly newspapers that lack the resources to conduct in-house polls and employ their own survey research experts, poll coverage is probably much poorer (Asher 1995: 87). Our findings support their statement.
First, we find that news media reports of general polls have not improved much over the years since the Miller and Hurd study (1982). When the CommerceNet/Nielsen Internet Demographics Survey was released in early November 1995, almost all national newspapers in the United States reported the findings, including the NewYork Times, the Washington Post, the Los Angeles Times, USA Today, and the Boston Globe, as well as Time magazine. Of the 20 stories we found on Lexis-Nexis that reported the results, only 1 (by USA Today) mentioned the margin of error (4 percent), 9 gave the number of respondents, and 6 revealed that the survey was sponsored or paid for by CommerceNet. None questioned the methods used in the survey, although these major newspapers all have in-house pollsters and researchers.
As it turned out later, there were some serious methodological flaws in the survey, which were said to have exaggerated or skewed some of the demographics of Internet users.[7] Even after all of the disputes over the problems with the Nielsen/CommerceNet survey in early 1996, a report on Nielsen's second survey in August 1996 in the Washington Post still did not mention the margin of error.[8]
Our results show that the news media's reporting of on-line poll results leaves even more to be desired when evaluated against the disclosure standards advocated by the AAPOR, especially in reporting the methods used in the polls (Table 3). Few reports of on-line poll results follow the AAPOR's disclosure standards in explaining sampling error, sample selection procedure, question wording, and how the survey was done. On the whole, these reports do not give the public due warning that self-serving on-line polls may be produced by a candidate, party, or company. Only 2 of the 80 stories in our sample include statistical caveats contained in what some call the "nerd box" (Crossen 1994)-"About the Poll"--even though such "nerd boxes" do not begin to address all of the problems inherent in any on-line poll.
In many cases, no caveats were given about the polls. Most stories did not even bother to tell the audience how many people responded to the poll. Fewer than one-fifth of the reports included a short disclaimer--often with only one word--that the poll was "unscientific." Even those that did carry a disclaimer did not explain why the survey was not scientific. Some media did not actually interpret poll results, but instead presented simple, straightforward descriptive statements about them. Poll data were sometimes integrated into a news story to support the statements made in the story. The report sometimes jumped to conclusions without much evidence except some percentages from the on-line poll results.
It seems that some advocates of on-line polls are too easily carried away by some results. Commenting on a recent on-line poll in Hong Kong, which attracted 1,246 Hong Kong-based users, the managing director of Asia Online said, "This is the first survey of its kind in Hong Kong and proves that the Internet is a viable platform to conduct all sorts of profile surveys and opinion polls."[9] However, neither the wire service nor the director provided an explanation of how "viable" the on-line poll proved to be, or why.
In discussing the news media's performance in reporting polling results, John Brennan, survey director at the Los Angeles Times, complains that the media report scurrilous data that mislead rather than inform. Impressive-looking studies are treated as breaking news, and competitive pressures often push newspeople to "go with it now and ask questions later." The public--and much of the press--has a hard time discerning that fact. One might expect journalists to sound the alarm about the pitfalls of bad research, but the industry as a whole does a woefully inadequate job, Brennan argues.[10]
Although some stories did carry a caveat box or a disclaimer, which was supposed to inject a note of caution into the report, it sometimes did the reverse, giving the impression of a scientific seal of approval. In some cases, it also shifted the burden of responsibility to the readers to judge whether the polls were valid or not. Most members of the audience are unaware of the methodological details of polling or may not be so statistically sophisticated as to understand what the words really mean. Furthermore, even though most people do not understand the statistical principles under which polls are conducted, they believe that pollsters are doing a good job (Traugott 1992:136). Some news media also used misleading headlines in reports on the on-line polls, making unwarranted claims that may further confuse the audience (see Appendix B for some examples).
The results of our study suggest that secondary reporting is often poorer. For example, a Chicago Tribune story carried only one sentence indicating that the poll it reported was "unscientific" and had a margin of error that could not be calculated. When this story was picked up by another newspaper, however, even this short explanation was dropped. It may be that the editor who picked up the piece was not comfortable with statistics and did not understand what "margin of error" means, or he or she deliberately deleted the disclaimer to make the poll results appear more "scientific." Space does not seem to be a factor in this case, because the second story was not much shorter than the original one.
Privacy
Another issue of concern related to on-line polls is the privacy of the users. Such concerns may counterbalance the convenience that the technology can offer. The user often has to register before getting access to the interactive or voting pages. This is different from most traditional polls, which are usually anonymous. To start with, a user needs an account to access the Internet, which inevitably leaves a record of the sites he or she has visited. This may be enough to deter many people because several companies are providing software to track personal, industry, geographic, and registration data about who is visiting a given site, how long they spend at the site, and what they do there.[11] Besides, many sites ask very detailed information of the Web surfer, including sexual orientation.
The public is becoming more wary of giving out personal information. For example, only 63 percent of households in the 1990 census returned the form (compared with 85 percent in 1970) because of the fear of the government collecting personal information.[12] How willing users would be to vote onWeb sites is questionable because they have no idea who is collecting the information and for what purpose. It takes strong commitment or great interest about the issues and candidates to vote in on-line polls, which contributes to the following problem.
Generalizability
In discussing the problems with the infamous CBS call-in program "America on the Line," Warren Mitofsky, former director of the CBS News Survey Unit, argues that if CBS wanted to show how many thousands of people called them, it should just put the numbers up (Mitofsky 1992). Actual numbers, without percentages, will keep one from generalizing, he argues. Once you facilitate generalizing by turning the call-in results into percentages, people will generalize even when you tell them not to. Yet despite this advice, our results show that nearly all reports of on-line polls used percentages, and fewer than half ever gave the number of respondents. The same is true with results posted on the poll sites: Most of them just list the percentages without giving the number of votes.
On-line polls suffer most from the unrepresentativeness of their samples. Richard Maisel and colleagues found that compared to the general population, on-line users as a group are better educated, have a higher income, are more often male, and less often minority group members, and are more often registered voters--particularly registered Republicans (1995). Similarly, a 1993 Times Mirror poll showed that citizens who listened to and called radio talk shows were not representative of the overall citizenry; instead, they tended to be more Republican and more conservative (Kohut 1993, quoted in Asher 1995:12). Herbert Asher concludes that it is not surprising that the radio callin polls often generate results more conservative and pro-Republican than the outcomes obtained through scientific polling (1995).
Our findings also suggest that on-line voters are more likely to be conservative Republicans. Although in most national surveys taken during the first half of 1996, Bill Clinton led Bob Dole with considerable margins, on-line polls painted a different picture. For example, representative national polls conducted by Gallup, Harris, and other major polling institutions in June and July 1996 found Clinton well ahead of Dole (54 percent of the public would vote for Clinton versus 31 percent for Dole according to Gallup's July 18-21 poll, and 54 percent for Clinton versus 29 percent for Dole according to the Harris July 9-13 poll). However, of the 12 on-line polls that provided current poll results in our sample, Dole led in 8 of them, sometimes by a large margin, and tied in 1. For example, in an America Online poll, Dole was favored over Clinton 74.6 percent to 25.4 percent, and 2,086 voted for Dole and 1,236 for Clinton in an on-line USA Today Election '96 poll. As it turns out, Clinton defeated Dole by 49 percent to 41 percent. The results of some on-line polls were way off the actual election outcome.
A Summary of Problems
Following Claude Robinson's criticism of straw polls (1932), we can sum up the problems with on-line polls as follows:
1. Manipulation. The sponsors of an on-line poll may manipulate the returns to aid a candidate in whom they are interested. There is no way to check the results posted. Polls can also be manipulated through the wording of the questions. The questions in some on-line polls are politically tainted, as shown in the following question from one poll:
Under the Clinton Administration, funds for interdicting illegal drugs entering the United States have been dramatically cut. As a result, critics say, drug use is now skyrocketing after posting steady declines in the 1980's. Drug use among young people is up 55 percent, while seizures of illegal drugs are down 53 percent. Knowing these facts, would you be more or less likely to vote for Bill Clinton?[13]
The results of the Nielsen/CommerceNet 1995 survey highlight how an on-line survey can distort the overall picture. For example, in the random phone interview part of the survey, 40 percent said they found navigating the World Wide Web very challenging compared with only 16 percent of on-line users in the on-line poll part of the survey, and 22 percent surveyed by phone considered themselves very skilled in navigating the Web, compared with 52 percent surveyed at the Web site. The results of the Internet survey also showed more male users (about 73 percent) and fewer female users (about 27 percent) than phone surveys (about 66 percent male respondents and 34 percent female respondents). The differences in the two surveys are even more marked when respondents are asked how often they use the Internet: Almost 60 percent of the Internet respondents said they use it more than once a day, and only 15 percent of the phone respondents said they use it that much (New Media Age 1995). Although Nielsen/CommerceNet's report did caution readers about the limitations of information gathered from the Internet survey, none of the news media reports we examined heeded this warning.
2. Stuffing the ballot box. Some individuals and groups might make a determined effort to shape poll results by "stuffing the ballot box." As mentioned before, polling safeguards may be insufficient to prevent duplication in voting. For many polls, users can vote as many times as they like, just as one can make many phone calls in a call-in poll. Mitofsky admitted that he called 300 times during CBS's call-in poll (1992). Guy Kawasaki, the "evangelist" employed by Apple Computer to help reverse its declining fortunes by serving as cheerleader to the Macintosh faithful, once discovered an on-line poll about Apple's prospects. He suggested to his readers that they take part--resulting in a 96 percent vote of confidence in Apple in the poll. "No Internet poll is safe from us," he boasted.[14]
3. Class bias. With the skewed demographic and political characteristics of on-line users, the ballots may be gathered disproportionately from one economic or social class, gender, or party, thus distorting the poll results with the political views of this class.
4. Bias of selection in cooperation. Even if we assume that people from all classes were given an opportunity to participate in a poll, members of one class may cooperate with the sponsors more than those of another. We do not even know the size of the population of Internet users, although there are numerous user surveys. Self-selection in on-line polls presents a double problem. First, the users might not be representative of the broader population of which they are members. Second, those who actually complete the questionnaires and click the "submit" key to vote may not be reflective of all Web surfers or Internet users. We do not know how many have visited the site and decided not to vote. Nevertheless, as Asher points out, the results of these surveys typically receive a lot of media coverage (1995:13). The results of our study seem to support this argument.
5. Bias of participation-nonparticipation. People might participate in an online poll, but because of age, citizenship ineligibility, or disinclination, they might fail to vote in the official election.
Conclusion
George H. Gallup, an inventor of modern opinion polling, idealistically believed that surveys of the public would enhance democracy by providing leaders with a true picture of people's attitudes and concerns.[15] However, it seems that modern technology has proved to be both an enormous boon and a terrible drag on the quality of polls. Now that pollsters can do fast surveys on anything and get them published almost regardless of their quality, they do (Crossen 1994:123). It was not possible in 1996 to reach a representative sample of U.S. voters by this method, and readers should be wary of any "findings" from such polls because at this point in time, they cannot be representative of voters or the public in general.
Some hope that the possibilities of computerized surveying will increase as the availability of home computers increases, just as the increased penetration of telephones altered the standard means of data collection before the 1970s (Traugott and Lavrakas 1996:83). Others seem more pessimistic. Although the penetration rate of home computers has just reached 35 percent, the market has already shown signs of saturation, and sales are predicted to stall in 1997 and decline by 1998.[16] Besides, in a 1996 survey, only 24 percent of Internet users said that they had looked at a political candidate's site in the previous six months, and of those who had looked at a political Web site, only 4 percent said that information on the site had influenced their opinions.[17]
The approach used in this study to investigate the effectiveness of on-line polls is not definitive. Even though it is limited to the sites we sampled and reports that were uploaded in Lexis-Nexis, our findings suggest that on-line polls, like call-in and write-in polls, may be vehicles for engaging users and building their interest in politics and some other issues. On-line polls cannot, however, measure general public opinion or preference in any reliable or valid manner.
Knowing what role the new media will play requires elaborate empirical research not only into the circumstances of use, but into the content available and the behaviors enacted on-line as well. As David Birdsell and colleagues point out, those using the new media to improve political discourse will have to cope with skewed demographics, access limitations, and a user population that is difficult to understand (1996). Although on-line polls could become a potential force in the future, for now they are not much different from traditional call-in or write-in polls except that they reach an even more limited slice of the population. On-line polls are likely to be highly misleading, distorting our understanding of general public opinion. We are a long way from George Gallup's dream of survey-assisted democracy. On the contrary, it is average citizens who must be on guard to judge cautiously the dizzying array of statistics coming their way.
Appendix A: URLs of Web Sites Visited
1. http://www.berkeleyic.com/data/poll.cgi (GOP Presidential Primary Poll)
2. http: //WWW.IPT.COM/vote/vote.htm (Interactive Voting Booth)
3. http://www.96.com/cgi-96/survey.cgi? (Online Survey)
4. http://www.rtis.com/nat/pol/cyberpoll/(Presidential CyberPoll)
5. http://www.iguide.com/pol-govt/pres.htm (Presidential Preference poll)
6. http://www.formal.com/nov96/("The Race" for President 1996)
7. http://www.ring.com/olpohtm (The Ring! Online Internet Presidential Primary)
8. http://pathfinder.com/cgi-bin/GDML/gdmldb? (Virtual Election)
9. http://www.umr.edu/~sears/primary/vote.html (Virtual Primary: Information Headquarters for the Republican Primary)
10. http://server.Berkeley.EDU/herald/guest2.htm (Vote for President)
11. http://www.reagan.com/cgi-bin/main/primary.html (Vote in the Mock Republican Primary)
12. http://www.voxpop.org/strawpoll/(New site:http: // www.voxpop.org/weekly) (The Vox Pop Strawpoll)
13. http://www.cvp.com/cgi-win/bigvote.exe?politics (The BigVote)
14. http://www.survey.net/sv-po13.htm(Business & Politics Surveys)
15. http://www.dfw.net/~alans/inter/elect.html (New site: http: // www.deepend.com/) (Cyber Presidential Election '96)
16. http://www.netizen.com/netizenpoll (The Daily Poll)
17. http://www.boston.com/cgi-bin/vote.cgi?dolevp (The Dole VP Survey)
18. http://www.doonesbury.com/(The Doonesbury Straw Poll, from Newsweek)
19. http: //www.evote.com/polls / (eVote Polls, from USAinternet)
20. http: //www.usatoday.com/feedback/elxform.htm (How Would You Vote, from USA Today's Election '96)
21. http: //www.gallup.com/questionnaires/index.html (The Latest Gallup Organization Surveys)
22. http: //www.home-and-family.com/lhj-lwv/polling/polling.html (By Power the Vote, from the Ladies' Home Journal)
23. http://www.votelink.com/ns/POLITICS/POLITICS.htm(Politics Questions, from Votelink)
24. http: //www.iguide.com/pol-govt/showdown/forum.htm (Showdown '96 Plebiscite, from Showdown '96)
25. http: //www.uvote.com/ISSUELIST.html (UVOTE)
26. http: //www.csn.net/vote/ (Vote on Bills before Congress)
27. http: //www.georgemag.com/cgi-unprot/poll.pl (The Weekly Poll)
28. http: //www.internetpoll.com/
29. http://www.cvp.com/bigvote/
30. http://www.abortion.com/
31. http://www.cjr.org
32. http: //www.nw.com/zone/WWW/top.html
33. http: //www.politicsnow.com/
34. http: //allpolitics.com
35. http: //voter96.cqualert.com/
36. http: //www.electriciti.com/~lwvus/
37. http://www.vote-smart.org/
38. http: //policy.net/capweb/
39. http: //www.herald-sun.com/votebook/
40. http: //PoliticsUSA.com/
41. http: //www.townhall.com/
42. http: // www.networkers.com/campaign96/
43. http: //elecionamerica.com/home.html
44. http: //cnn.com/POLITICS/index.html
45. http: //www.clark.net/ccentral/
46. http: //www.fosters.com/
47. http: //www.oclc.org/VoteSmart/wv/wvhome.htm
48. http: //www.zdnet.com/macuser/poll.html/
49. http: //babs.prodigy.com/live/support/feedback/codie.htm/
50. http//www.lainet.com/drudge/
51. http://pathfinder.com/DSOflAcARgxTmrem/ reinventing/game2/public/gpo.html
52. http: //cybersight.com/cgi-bin/cs/ballot/oj/
53. http: //TBWA-chiat/Day/
54. http: //www.americanpresident.com
55. http: //www.cshopper.com
56. http: //netaxis.com/~ruffini/focus/
57. http: //www.vote-smart.org/other/participate.htm!
Appendix B: Examples of Misleading Headlines
1. "Colin Powell Will be Elected President Predicts Prodigy Online Poll; Bad News for Clinton: President Finishes Fourth in Three-Candidate Race," Business Wire, Sept. 20, 1995.
2. "How the Nation's Future Will Vote: Dole Holds Slim Lead in National College Student Poll," PR Newswire May 2, 1996.
3. "IBM/Lotus Merger Is Good for Business According to Information Week's Flash Poll," PR Newswire, June 16, 1995.
4. "Christmas Too Commercial, Says Generation X in Poll," Denver Post, Dec. 7, 1994:C2.
Notes
1. Following President Bush's State of the Union address and the Democratic response on January 28, 1992, CBS News aired a program during which viewers were invited to call an 800 number and register their responses to a short series of questions. The results, in which 315,000 out of 24.5 million attempted calls were tallied, were compared on the air to a separate postspeech survey in which 1,241 adults originally interviewed January 14 to 19 were reinterviewed.
2. Herb Frazier, "Citadel Survey a Glimpse of Online Future," Post and Courier, Sept. 5, 1995.
3. Despite the wide discrepancies of survey results about the number of Internet users, most surveys indicate similar user demographic characteristics, as discussed here.
4. Paul Andrews, "How Many Use the Internet? It's Hard to Say," Seattle Times, Jan. 14, 1996: sec. C.
5. Frazier, "Citadel Survey."
6. Richard L. Berke, "Polls Apart," NewYork Times, Jan. 14, 1996: sec. 7.
7. For details, see "In a Recount, Cyber Census Still Confounds," New York Times, Apr. 17, 1996, and Andrews, "How Many Use the Internet?
8. Washington Post, Aug. 18, 1996.
9. "Survey: Internet Booming in Hong Kong," BC Cycle, May 3, 1996.
10. John Brennan, "How it is that Scientific Surveys' Results Can be Polls Apart," Los Angeles Times, Apr. 17 1994, sec. D.
11. John Dodge, "Signs of the Times," Boston Globe, Nov. 22 1996; and "Powerful SearchTools Make it Hard to Hide on the Internet," Minneapolis St. Paul Star-Tribune, Mar. 21, 1996.
12. Steven Homes, "In a First, 2000 Census Is to Use Sampling," New York Times, Feb. 29, 1996.
13. From http://netaxis.com/~ruffini/focus/.
14. Lee Gomes, "An Airport Krishna? No, It's a Mac Zealot,"Austin American-Statesman, Jul. 30, 1996.
15. Brennan, "Scientific Surveys' Results."
16. Wall Street Journal, Mar. 19, 1996: sec. B.
17. USA Today, Apr. 15, 1996.
Topic of Poll N Percentage Presidential election 37 64 General politics 13 22 Entertainment 6 10 Marketing 2 3 Total 58 99[a]
a The total does not add up to 100 because of rounding.
Legend for Chart: A - Year B - Total C - Exclusive Reporting, N D - Exclusive Reporting, Percentage E - Integrated Reporting, N F - Integrated Reporting, Percentage A B C D E F 1992 5 2 40 3 60 1993 0 0 0 0 0 1994 15 8 53 7 47 1995 39 23 59 16 41 1996[a] 21 11 52 10 48 Total 80 44 55 36 45
Source: Lexis-Nexis.
a Up to July 31, 1996.
Disclosure Standard Number Conforming Percentage Response rate 1 1 Sampling error 2 3 Question wording 6 8 How interviews were done 9 11 Sample selection procedure 10 13 Use of disclaimer 13 16 Definition of population 16 20 Timing of interviews 22 28 Number of respondents 25 31 Sponsor[a] 56 70
N = 80.
a We assumed that polls reported to be conducted by some on-line services, such as Prodigy and America Online, were actually sponsored by these companies.
References
Asher, Herbert. 1995. Polling and the Public:What Every Citizen Should Know. 3rd Edition. Washington, D.C.: CQ Press.
Barber, Benjamin R. 1992. "Opinion Polls: Public Judgment or Private Prejudice?" Responsive Community 2 (Spring):406.
Birdsell, David S., Douglas Muzzio, Humphrey Taylor, and David Kran. 1996. "New Forms of Political Participation: A New Political Marketplace; The Web Snares Voters."Public Perspective 7(4):33-35.
Crossen, Cynthia. 1994. Tainted Truth: The Manipulation of Fact in America. NewYork: Simon and Schuster.
Frankovic, Kathleen. 1992. "The CBS News Call-in: 'Slipups in the Broadcast.'" Public Perspective 3(3): 19.
Gawiser, Sheldon R., and G. Evans Witt. 1994. A Journalist's Guide to Public Opinion Polls. Westport, CT: Praeger.
Lavrakas, Paul J., Jack K. Holley, and Peter V. Miller. 1991. "Public Reactions to Polling News during the 1988 Presidential Election Campaign." In Polling and Presidential Election Coverage, ed. Paul J. Lavrakas and Jack K. Holley. Newbury Park, CA: Sage.
Maisel, Richard, Katherine Robinson, and Jan Werner. 1995. "The Prodigy Experiment: Creating a Benchmark Using Online Polls." Public Perspective 6(2):8.
Miller, M. Mark, and Robert Hurd. 1982. "Conformity to AAPOR Standards in Newspaper Reporting of Public Opinion Polls." Public Opinion Quarterly 46(2):243-49.
Mitofsky, Warren J. 1992. "The CBS News Call-in: First and Foremost, Bad Information." Public Perspective. 3(3): 19.
New Media Age. 1995. "Survey Reveals Under-use of Internet." (Nov 9):9.
Robinson, Claude. 1932. StrawVotes:A Study of Political Prediction. NewYork: Columbia University Press.
Traugott, Michael W. 1992. "The Impact of Media Polls on Public." In Media Polls in American Politics, ed. Thomas E. Mann and Gary R. Orren. Washington, D.C.: Brookings Institution.
Traugott, Michael W., and Paul J. Lavrakas. 1996. The Voter's Guide to Election Polls. Chatham, NJ: Chatham House.
Yankelovich, Daniel. 1992. "How Public Opinion Really Works." Fortune, Oct. 5:102.
Biographical Notes
Paper submitted December 10, 1996; accepted for publication March 12, 1997.
~~~~~~~~
By Wei Wu and David Weaver
WeiWu is Lecturer in the Department of Organisational Behaviour at the National University of Singapore and is a former doctoral student in mass communication at Indiana University. Address: Department of Organisational Behaviour, Faculty of Business Administration, National University of Singapore, 10 Kent Ridge Crescent, Singapore 119260; phone: 65-7723004; fax: 65-775-5571; e-mail: fbawuw@nus.edu.sg
David Weaver is the Roy W. Howard Research Professor at the School of Journalism at indiana University. Address: School of Journalism at Indiana University, Ernie Pyle Hall 200, Bloomington, IN 47405-6201; phone: 812-855-9247; fax: 812-855-0901; e-mail: weaver@indiana.edu.