4. The Perils of Prediction
November 6, 1952
This is the witching hour. The Bomber Halloween sky looks down upon an entranced multitude. For months and Weeks, with increasing intensity, the election spell has been cast among the American people. The midnight hours tonight will break the spell and tomorrow's gray dawn will see men's faculties restored. But meanwhile the night is filled with imprecations, prayers, with high fever. And with prophecy. Yes, especially with prophecy. For weeks the country has been filling up with seers. Every man has been a soothsayer. He has been telling us who will win the presidential election.
His instrument of prediction varies. He may tell us by the feeling of his rheumatic lag, or by the results of an elaborate survey. Yet no matter the method he uses, we are interested. We are interested because we are excited. We cannot believe ourselves and cannot believe anyone else. Who knows who can foresee the future? Prophecy is a mysterious quality, like the touch of grace.
This very hour, however, is the last great hour of prediction. Tonight prediction meets the test of reality. It is too late to bother about new predictions. It is too late to do anything about our old predictions. There is nothing any of us can do save wait, and, as we wait, wonder something about the quality of our own predictions and those of others. We are in suspense and momentarily can know only that about half of us will be right and half of us wrong. Worse yet, the half that is right is probably right for the wrong reasons. And those that will be wrong in many cases will be unfortunately wrong. They deserved a better fate.
Nor need we believe that all those who are wrong in their predictions of our next president will be common people. They will have the good company of professors, of editors, of journalists, of ever so many candidates, of campaign managers, and yes, even perhaps of pollsters. But as each one of these groups of people has its own method of predicting the outcome of an election, each one has its own reasons for success and failure. Our great democracy, to paraphrase Justice Holmes, allows every forecaster to go to Hell in his own way. Understanding these ways, we may even perhaps reform ourselves for the next time. And in any event, reciting the face of others is soothing. So Napoleon found it at St. Helena. And so we shall be able to console ourselves in the early hours of the morning with the failures of others to do what we have tried and failed to do ourselves.
No matter how forecasters may predict, they seem to believe in prediction itself. That gives us something to begin with. Sometimes a man who has s good part of his income going into insurance, who never rides a plane because of a slight difference in accident rates, who is glad to take odds on Stanford versus Old Siwash in football, who wants laws against divorce because one marriage in three ends in divorce, who pays a fortune in advertising every year because surveys show that "advertising pays," who lays on extra help in advance for the Christmas crowds at his store, who goes to the office early to avoid the morning rush, and who buys wheat low to sell it high, will object strenuously to the notion that human behavior is predictable.
But the general run of American is a social scientist at heart. That is, he feels that human behavior can be predicted in many ways. The only problem that upsets him is the same one that upsets us all: how do you know that what you are trying to predict can presently be predicted? And we must admit that the prediction of the next president is an event that is by no means simple.
Reduced to its essence, the question is: who of two candidates will become president but a couple of times in our history, in 1800 and 1824, no one has become president as a result of the election. So we have a third possibility: that the election will be throw into the House of Representatives for lack of a majority of votes in the electoral college. This is not likely at the moment. There are only two strong candidates and a third candidate is not likely to gain the electoral vote of any state. Another possibility is the death of a chosen candidate before he becomes president, or of some crisis in the counting of the vote as happened in the Hayes-Tilden election contest of 1876.
In fact, the very calm with which we accept the notion of there being only two possibilities in the contest of today is slightly illogical. With our two major party system, we regard the election as a two possibility affair like the tossing of a coin -- either heads or tails must come up. A student of probability theory would tell us that the likelihood of a coin occasionally standing on end when tossed complicates considerably the prediction of the coin's behavior. This is especially true if we are predicting on the basis of only forty-one cases of election or anything else. I think, however, that we ought to be optimistic and avoid such conjectures tonight.
But let me make a point before we leave the subject of coin tossing. Accepting only two possibilities, could we settle the question of who will be president by tossing a coin. Past experience is friendly to the idea. In twenty elections since 1868, when the effects of the Civil War crisis had assumed a settled pattern, the Democrats have won nine times, the Republicans eleven. So those who say that the election is a toss-up may be saying more than they know.
But let me reassure you. Though the similar totals may be seductive, there is no further resemblance between predicting a coin's fall and predicting the winner of a presidential election. A presidential election, although it may be assumed to result in only the election of one of two men, is a simple choice based on a great many determining conditions. Instead of a coin, think of a war. One of the nations will win. But thousand of conditions determine which one will win. Leadership, organization, morale, manpower, history and other factors take their toll of errors in any prediction of the outcome. Thousands of conditions determine also who will be elected president. American history is full of terrifying examples of incidents that seemed to have influenced the outcome of an election. For example, there is some basis for believing that Hughes lost the 1916 election because he failed to make a clear, friendly gesture to Hiram Johnson when he visited California during the campaign. In addition to these seeming accidents, social scientists have discovered any number of forces, attitudes, and organizational and legal matters that influence the election results. Exact foresight seems impossible, but we know that the name of the successful candidate turns up in each election in a determined, rather than random fashion. In each election campaign -- at least in retrospect -- there is more to be said in favor of the result being on one side than on the other. A slight proof of this is the following fact: Instead of flipping a coin for each election since 1868, you bet on the party in power you would be correct in 13 cases, wrong in only seven. You can see that certain vague tides or cycles were at work a good part of the time.
In forecasting a presidential election the predictor tries to reduce the hundreds of determining conditions to the smallest number of conditions that he believes to be crucial. Sometimes he uses indicators, in addition to causal conditions, just as you can use straws in the wind or an analysis of the behavior of air masses to foretell weather, the first being an indicator, the second being determining conditions. Some leeway is granted the election forecaster of course. He has only to predict who will win and that is a lot easier than saying by exactly how much he will win. This advantage however, diminishes rapidly in a close election so that the prediction of the winner of a very close contest demands the qualities of a most exact science. In the case of an election, it is not too exaggerated to say that an entirely valid and reliable prediction of an election would mean that social science had achieved the stature of a most exact science. It could describe in advance the exact nature of the event.
Such a prediction is impossible. The most reliable predictions about political behavior are blunted predictions -- imprecise, fairly general, and vague -- not at all often what we would like to know about the future. This results from our knowing only partially the phenomena whose behavior we wish to foresee. And the less we know about the phenomena the more vague the predictions must be, until one gets the sort of predictions found in politics, or business cycles, and similar areas of life -- often they are so vague as to be almost self-contradictory. A mere glance at the mass of writing on elections shows that this is such a field. Hedging is to be seen everywhere. Phrases like "but on the other hand" or "if things do not change" or "barring a turn in the weather" or "if the trend continues" or "within a considerable margin of error" or other like phrases abound.
We must conclude that the field of election prediction is not a happy hunting-ground of political scientists. Left to their own devices they would not use such difficult predictions to test the value of their science. The most useful political predictions are found in other concerns of political science. We do not ask a psychologist to do not expect him to predict a breakdown at all. Yet we know that he can predict in general terms the behavior of the patient and tell us much about the behavior of the class of men to which the patient belongs.
Then why are we all predicting elections? We predict because we must. The most sophistic of ancient Athenians wondered whether the Delphic oracle talked pure nonsense or perhaps somewhat less than that. The most hardheaded of Romans worried a little when the entrails were examined with respect to the fate of his military campaign. The impulse to predict is irresistible. Man seeks assurance in the face of grave impending events. If we didn't try to predict elections, it probably would be a sign that we had little regard for the importance of elections. And so we must the unreasoning addiction to prediction.
We are fortunate in that we are not completely foolish in speculating about elections. The task is near enough to the margins of science to protect the speculators from violating their professional ethics. Election forecasting at its best has gone a respectable distance beyond intestinal prophecy and superstitious foreboding.
The question therefore becomes: what kinds of forecasts are the best? And to answer that we must ask: who is in the predicting business? What omens do they prefer? How skillfully do they pierce the veil of the future?
Take newspaper editors as our first case. You hear that they know who will be elected. The man they pick will win. After all , they even help elect the man they predict will win. But certainly this is not so. The newspapers for twenty years have been overwhelming Republican. For twenty years the country has been Democratic. Today 67 per cent of 1385 daily newspapers -- with 80 per cent of the total daily circulation -- support General Eisenhower. For each Stevenson newspaper that has been sold, four Eisenhower papers were sold. In nine states, no daily newspaper backed Stevenson. The best statistical techniques used to study the influence of the press show, not that the press is without influence on opinion and votes, but that the influence is a relatively minor factor in deciding elections.
But what about the press' predictions, even if it cannot help much to fulfill its own predictions? Well, even its predictions are not too good. In 1948 fifty of the nation's top political writers, surveyed by Newsweek magazine, unanimously predicted Dewey's election. They furthermore guessed he would get 366 electoral votes. Actually he received 189. Undismayed, the same magazine asked a similar group of 50 political writers the same question in 1952. This time 26 picked Stevenson and 24 Eisenhower. Their average guess was that Stevenson would get 267 electoral votes, Eisenhower 264. (This is statistical nonsense of course, but it is all grist for our mill.)
More ambitious than Newsweek, The Associated Press, in mid-October, polled some 2,000 newspaper editors and political writers about the election. The circumstances of the poll are scientifically suspect and the manner of computing the results is unknown. The gentlemen of the press thought Eisenhower would probably find 238 electoral votes, with 89 more wavering toward the Eisenhower camp; that Stevenson could rely upon 104 electoral votes with an additional 52 tending towards his side; and that 48 electoral votes were impossible to determine at that time. This poll of journalists really did not predict. It purported to represent a condition existing towards the end of the campaign. It cannot be shown to be right or wrong unless subsequent to the election a survey will show that very few people changed their minds in the last three weeks of the election campaign, a finding that is quite unlikely.
We will not be able to say that journalists have become better predictors but that they have become more cagey. As the article reports and I quote: "newsmen cautiously were refraining from jumping overboard with forecasts of Republican victory."
Of course individual newspapers throughout the country will continue to predict, and since most of them are supporting Eisenhower they will predict an Eisenhower victory Examination of a few such last-minute predictions reveals no new insights, no reliable techniques and analysis, nothing much more than the ordinary alert newspaper reader can do for himself. A jury of twelve newspaper publishers would probably give a worse prediction than a jury of twelve representative citizens. That is because very few people are newspapermen and therefore newspapermen form a biased sample of the population. Like normal people, they cannot control their own biases in making predictions. They come from upper income or educational groups and make successful predictions in Republican years. They tend furthermore to be localistic, viewing the country in terms of their newspaper circulation area. And they are not trained in the analysis of opinion as students in the institutes of journalism are today. Furthermore, editors have a kind of occupational disease. They exaggerate the effects of incidents in a campaign. Since they are interested in excitement for circulation reasons, they tend to believe that certain slogans, certain incidents, certain successful publicity stunts, certain campaign speeches of great merit, have more effect on the outcome than such incidents actually have.
Outstanding exceptions exist in the newspaper world. The highest form of journalism on election trends and results is represented in the writing of Samuel Lubell. Mr. Lubell has managed an interesting assortment of techniques combining the insights of journalists with the regular practices of social scientists. he runs about the country ringing doorbells in selected precincts, talking to people in carefully selected life situations such as in the home or at clubs or in stores or on the street. he is aware of the necessity of making many nice judgements about differences in intensity of support for a candidate among the different groups of the population. he worries conscientiously about the possibility that relatively more of one group than of another may turn out to vote or may stay home. Mr. Lubell is indeed a one man survey operation and merits attention. To refresh your memories, in one of his latest articles, Lubell wrote: "Strange as it may sound, Eisenhower today may well be on the verge of a landslide victory -- and yet he could awake on election morning and find himself cheated of the presidency by having lost several key states by small margins."
Granted Journalists on the whole are unreliable predictors, why can we not rely upon the candidate and his managers? They are in the middle of the fight. They are every day in contact with those who will cast their ballots. They are in touch with the machines who will get out the vote. They are immersed in public opinion. Can the candidate and his managers predict elections?
Certainly the candidate cannot. No one is so naive as to expect a valid result from asking all the candidates in the election who is going to win. For every objective and honest response on will get fifty that are not. I understand that in this campaign the Eisenhower campaigners do not bother to read or discuss election predictions. That is good. They would only fool themselves and waste their time. A man who has fallen into quick-sand is in no condition to analyze the vacuum principle.
With the campaign managers we should be able to do a little more. These men after all are often paid for being objective in viewing trends of sentiment. But campaign management is not yet a profession on America in the sense that managers are trained opinion analysts and propaganda analysts. A manager is sometimes a broken-down politician or a men who has two strikes on his in running for office. So the old chestnuts of election prediction are common among the managers as among the common people.
Many wise phrases have come out of the smoke-filled rooms. Some are dated, of course, like the slogan "You can't lick the press" which was fairly well accepted until the election of 1936. Now one has to be a fossil indeed to make such a statement. Press suppose guarantees nobody a victory. Other slogans survive, although precariously. "Look at the kind of people who are supporting us", it is said, "we are bound to win." But movie stars, industrial tycoons, star athletes, and Mrs. Eleanor Roosevelt do not forebode victory. They merely make the campaign interesting.
Some say you can predict a campaign's results by where the money is going. "You can't beat the money-bags," it is declared. Of course, this is often said by people who want to collect money for their campaign, even though they don't believe it. But many do believe it and the superficial facts are on their side. George Lundberg collect data on 156 elections in various parts of the country and found that, in 14 out of 15 cases, "Campaign expenditures, as reported, constitute an absolutely reliable index of the outcome of the election." If you think you have the magic formula here, I point out that money goes to a sure winner. Furthermore you cannot often tell who is getting the most money until the campaign is over.
Some politicians believe that the bigger the noise they make, the surer they are of victory. One aspect of this belief is the so-called band-wagon effect. It is believed that if you can predict loudly enough your own victory, a sufficient number of undecided or opposition voters will join your side to assure victory. An American Institute poll once asked people whether they knew what Republican candidate was leading in a poll of presidential candidates before the nominating convention. Then it asked the respondents whom they favored. Analysis revealed that those who knew the favorite in the poll were no different in their preference from those who did not know the favorite. Apparently the band-wagon effect is as often absent as present and a widespread belief that one side will win cannot be relied on as indicating the ultimate winner.
But what about crowds? Isn't a candidate who draws great crowds a likely man to win the election? Did the large crowds that greeted Dwight D. Eisenhower in his swing around the country during the political campaign indicate his election today? Did the relatively small crowds who attended Stevenson's speeches and greeted his appearance on the streets portend a defeat for the Democratic candidate? President Truman drew fair crowds in 1948, Governor Tom Dewey fine crowds. The Dewey audiences were enthusiastic the Truman audiences more reflective and calm. You can dismiss the idea that crowds foreshadow electoral success, unless we should know much more about the character and composition and planning of the crowds. The organized crowd turned out by well-discipline publicity machinery of a political organization, or the crowd that gathers to meet a General they have heard much of but not seen, or perhaps the crowd that comes for door prizes and for many other reasons is not necessarily a crowd that shows a spreading and gathering intensity of support for a man. By known techniques of social science one could estimate rather accurately the part these various motives play in the crowds and the technique could be used in following a candidate about the country. The expense of such a study, however, would be prohibitive. So the crowd phenomenon is left to guess work and can be taken to mean nothing.
However, we have not yet asked the managers to predict on the basis of their intimate knowledge of campaign organization. Granted that editors and candidates may be poor predictors. Do not the men who run the machines have a more objective view of the campaign? Can they not discern trends better and predict the outcome? Several studies have been made of the predictive ability of politicians. Dr. Claude E. Robinson collected the private predictions of politicians in 16 states on the outcome of the 1928 presidential election. Among the Democrats the average error in judging pluralities was 18 percent. Among the Republicans, 7 per cent. Some errors topped 41 per cent. Most errors went in the directions of the politicians' own wishes.
In two later studies, in 1944 and 1950, county chairmen of both parties were matched. In 1944 eight counties were used. In 1950 the chairmen were matched in thirteen counties. In all cases, the chairman was asked to estimate the vote of both parties in advance of the election. The counties were distributed among several sections of the country but nothing like a representative sample of counties was obtained, and there were undoubted some selection factors operating among those committee chairmen who replied to the questionnaire sent them. In all, we have the estimates of 42 chairmen. The average error of Democratic chairmen in 1944 was 4.1 per cent, of Republican chairmen, 6.8 per cent. These errors were compared with the national polls errors for the 1944 elections. The Gallup poll erred by 2.3 per cent, the Crossley poll by 1.8 per cent, and the Roper poll by two-tenths of one per cent. The American Institute of Public Opinion which in 1944 had, as I have already stated, an error of 2.3 per cent predicting the national vote, had an average error of 4 per cent in all elections from 1936 through 1950. In 1950 the average error of the Democratic chairmen was 9 per cent, of the Republican chairmen 5.9 per cent, and of the Gallup poll .7 per cent. Thus it would seem by this and other indications that the chairmen predicted their own county vote more poorly than the national survey organizations predicted the national vote. Lest you feel we are quibbling over a mere five or ten per cent of the vote, the errors of the individual chairmen resulted in fourteen out of the forty-two cases in wrong predictions of the final outcome of the election itself. The best individual predictors were chairmen of the stronger of the two parties in counties that were one-sided in their political affiliations. Assured and relaxed, they could watch their counties objectively.
A somewhat silly note may be introduced into the proceedings by inventing another index, which may be called the Persistence Index. In each of the 42 counties the vote of the county in the previous election was found. This was taken as the prediction of what the county's vote would be in the election under study, that is in 1944 and 1950. The average error of this simple persistence estimate was 3.5 percent in 1944 and 6.3 per cent in 1950. The average error of the 42 chairmen of both parties in both election in 6.7 percent. The average error of the persistence prediction for both elections is 5.2 per cent. In other words, if one had predicted for the 42 counties that the vote would be the same as last time he would have been more accurate than the average Democratic and Republican county chairman.
Now I have no information about the extent of party organization within these 42 counties. In the country as a whole very few counties are thoroughly organized for political warfare. Most of the more than 100,000 precincts of the nation lack workers of one or both parties. In 1948, 28,000 precincts had no Republican leadership, and 357 counties out of some 3,000 had not even a Republican county chairman. The exception lie mostly in metropolitan areas, where the old-fashioned precinct voting district and ward organization is maintained, is fed by patronage, by pressure groups and by volunteer workers, and is disciplined to get out the right king of vote and to report methodically and objectively the trend. If the whole country were organized in this fashion the predictions of professional politicians would be very accurate, perhaps better than the pollsters in the nation as a whole. Each precinct captain, and there may be over one hundred in each ward in a city like Chicago, vies with every other in predicting correctly his own precinct. Sometimes a prize is offered for the best prediction, regardless of whether the precinct turns in an absolute plurality for the party itself, so vital does an efficient organization regard such prediction. In cases like this one, each precinct captain is required to predict the vote of every single registered voter in his precinct, to predict whether the person will or will not cast his ballot, and to explain why he must give the verdict of undecided to any individual voter. Furthermore, he is asked to submit a budget of money that is needed to assure that his predictions regarding certain persons will not be defeated by a lack of funds. But again such reports about the few areas where political machines are strong are confidential, and would not be made available to the public.
If I were attempting to extract some useful predictions from the mass of profession politicians' predictions for an election, I should look to the political leaders of one sided electoral districts. Their predictions are somewhat more reliable. I would tend to believe the predictions made by Republican county chairmen in a solidly Republican Iowa county and the predictions made by, say, Jacob Arvey, regarding the 24th Ward in Chicago. There is not much point in these men blowing up by a few points a majority that may be already over 80 per cent or 85 per cent of the total vote. Then one could look at the past records of these areas to determine the character of the shifts in sentiment. Of course there would be many dangers in extending such local predictions to a larger area. Hence, it is best to ignore the professional campaign managers unless one can obtain their predictions privately.
Campaign managers and candidates as well as editors used to be fond of the expression "As Maine goes, so goes the nation", believing that the state of Maine in its September state and U. S. Senatorial elections would give an indication of the trend in sentiment in the country as a whole prior to the November elections. Interest has subsided in Maine since it failed to predict the overwhelming Roosevelt victories. In 1936 the expression was changed to "As Maine goes, so goes Vermont."
But reputable political scientists picked up the Maine case and examined it more carefully for the invariable kernel of truth. Dr. Louis Bean, in fact, has developed a method of examining not only Maine but any other state and declaring "As Maine goes, so goes the nation" or "As Iowa goes, so goes the nation" or "As California goes, so goes the nation." A much more careful student than the average person who uses the expression, Dr. Bean could see in a constituency of any kind the persistence of a behavior pattern peculiar to the constituency and the reflection of certain trends national in scope. Thus, although Maine is much more Republican than the nation, the size of the Republican vote there is significant. Incidentally, the Maine elections this year showed a drop in the Republican vote to where one might expect a Democratic majority in the country as a whole. But scandals shook the Republican regime and may explain the drop. Bean's method resembles somewhat the idea of the handicap in polo, racing, and other sports. The past behavior of a thing is observed to vary with some regularity from the larger entity of which it forms a part. That is, a curve depicting the electoral behavior of a given state or a given county or locality, will bear some regular relation to the curve representing the behavior of the nation as a whole in elections. If one were to know the behavior or be able to predict the behavior of a state or other electoral division with respect to a coming election under study, he would be able to have some idea of the behavior of the nation as a whole in at election. Thus California, like several other western and northern states, tends to be more Democratic than the country as a whole when the country votes Republican, and that difference is less when the country is evenly divided. When the country votes very closely in an election California tends to vote very closely, and as the margins increase either towards the Republican or towards the Democratic party, California increases by more than the national increase. In the present election, if the California poll were accurate in predicting as Eisenhower victory by as much as a five par cent plurality over Stevenson then it might be expected that the country as a whole go Republican.
However, the California poll indicates a percentage of undecided persons large enough to decide the election and, without knowing their intentions, such a prediction is useless. On the whole, the Bean method of trend analysis is more useful to theoretical political science than it is in predicting elections. The general principles that he has discovered are interesting and significant, but predicting a single event, as I pointed out earlier, is a complication and hazardous operation.
With all due respect to the seers that I have already discussed, I must say that I have saved the best for the last. The question now is: Can we believe the polls? The answer to the question cannot be a simple yes or no. There are good polls and bad poll and there are polls in between. No really first-rate national poll exists and we shall see why.
A poll, or more properly, a sample survey, is essentially a short-cut to the determination of the attitudes of a multitude by selecting for interview, on the attitudes in which you are interested, individuals that are characteristic of the multitude. Without a correct sample a poll is almost useless. The population is not homogeneous, and hence any one group of people are not representative of the population as a whole. The population is not like a bottle of homogenized milk in which the cream is distributed uniformly throughout the bottle. It is more like a bottle of non-homogenized milk in which, if you skim the top, you do not get the same cream content as when you draw a sample from the middle of the bottle.
There are two fashionable methods of drawing a sample from the population. There are also techniques that combine both of the general methods. One method is called the area method, the other the quota method. The area method requires a systematic subdividing of the population as it lives on the land into smaller and smaller districts by a completely random selection, a systematic selection of smaller and smaller units of residency, until finally the designer of the sample has reduced the whole population into a tiny number of persons randomly distributed over the surface of the population. The interviewer is then sent to these persons where they live, and they are interviewed.
The quota method is more generally used in election polls, is less expensive than the area method, and is in lower repute among social scientists. The quota method obtains its sample by determining from the census what proportions of the population are of different race, of different income, of each sex, of different ages, and of other visible characteristics. The sample will then have to reflect these proportions. Interviewers are assigned to question a quota of persons in each of the several categories of sex, race, age, income and the like. If there are six women to four men in the population to be studied, the interviewer interviews six women and four man and so on. A number of studies have shown that the major defect of the quota method occurs in interviewing. On cannot rely on underpaid and temporary interviewers to judge reliably the position of a person in the income classes or even the age classes of the population. If a poor man voted like a rich man, the quota method would give better results. But a poor man more often than not votes differently than a rich man; an interviewer does not get his proper quota of poor men because be doesn't know a poor man when he sees one in the upper economic and educational brackets. The result is usually a bias toward the Republican party in national elections, except in the South. Dr. Daniel Katz estimate that sampling contributed between 1/5 and 3/5 of the Gallup or Crossley errors in the 1948 elections.
This major error in administering a sample by quota method is accompanied by several other errors characteristic of all sample surveys unless elaborate safeguards are introduced. The questions asked of people must be carefully watched. Words do not have the same meaning to all people and to all groups of people. Questions about how one would vote today are not the same as question about how one will vote or intends to vote three weeks from now. Other administrative errors enter into a sample survey but we need not into them here.
We should mention, however, that the analysis of the interviews brought into the office allows of new errors. One of the most important of these errors is the placemen of people who say they do not know how they will vote. Until the present election it has been the practice of pollsters to assign the "undecided" in actual proportion to each major party. Recent post-election surveys have shown that the "undecided" do not split evenly. They have tended to be principally Democratic voters on election day. In discussing today's election the pollsters have been careful to explain the alternative was that the "undecided" can go. They have been chastened by past experience.
However, not placing the "undecided" becomes equivalent to not predicting who will be president. We know many interesting facts about the voting population and we are being deceived, but we are not allowed to draw erroneous conclusions nor are we allowed to know with any definiteness who will win. The elections of 1952, while they may represent a lack of good manners and may be otherwise disagreeable to many persons, do mark an improvement in the ethics of scientific reporting. The polling agencies are be congratulated for their honest account of their ignorance, even though we may be disappointed in the vagueness of the oracles.
Several theories have been advanced as to the character and probable behavior of the "undecided" in this election. The prevailing theory is that the "undecided" will have voted today like they voted before, that is, overwhelmingly Democratic. This the is weak, however. The "undecided" of 1948 are probably not the same3 people as the "undecided" of 1952. There is some possibility that the "don't knows" of 1952 will prove to have been reluctant Democrats tending towards Eisenhower. Again extensive and careful analysis of the character of the "undecided" would have allowed us to predict their behavior with fair reliability in today's election but that expense would be unsupportable by the present polling organizations.
At times a polling organization has tried to compensate for the known bias of its interviewers under the quota method by adding a couple of percentage points to the Democratic totals all down the line, on the theory that the under-sampling of the lowest income groups has introduced a Republican bias. This is a risky theory. It is possible that the lowest income groups may have reoriented themselves to a candidate such as Eisenhower, and any prediction based on such an adjustment from past experience would be dangerous in the present election. In fact, Elmo Roper observes a tendency for Eisenhower support to emanate from some young voters and organized labor. I agree with this analyst though it may surprise a great many professional and amateur seers.
Important last minute events and accidents of a campaign may also harm any prediction of the results of the election that has been based on a trend observable up to a week or two before the election. True, most people are irrevocably committed weeks before the election. But certain events do happen to change people's minds at the last moment. In this election there seems to be a good deal of voter restlessness. A survey following the 1948 election showed that 4 per cent of those who intended to vote for Dewey before the election did not vote and another 4 per cent switched to Truman. Of those who intend to vote for Truman before the election, 7 per cent did not vote and 1 per cent switched to Dewey. These last-minute switchers did benefit Truman in 1948. When one added to these last-minute switchers those who had been undecided before the election was nigh, most of whom finally voted for Truman, we can see that a considerable force was present on election day itself that was not predicted in the polls, and yet that did not represent an error in the poll's methods.
Finally, in all sample surveys polling there is a probable error. A perfectly representative sample does not exist. This error varies with the size of the sample and may be anywhere from 1 to 5 per cent. The major polling organizations strive to reduce this probable error, which occurs on top of all other errors, to about 3 per cent. What this 3 per cent probable errors means is that the results of the sample cannot be guaranteed to be representative in an exact sense of the attitudes being measured, but are almost sure to be exactly representative of some figure within 3 per cent more or less of the stated result.
One need not worry about the effect of this error in an election that is a solid victory for either side. But in an election in which the winner receives slightly over 50 per cent and the loser slightly under 50 per cent of the total vote, even the best sample survey is useless in predicting the winner. It is as likely to be wrong as not and yet these close contests occur in the crucial states with large electoral votes an often decide the results of American presidential elections. A handful of votes gave Wilson a plurality over Hughes over California in 1916, threw the entire California electoral vote to Wilson, and made him the winner. A few thousand votes in the present contest conceivably may decide the electoral vote of new York and of the election.
This is a grave burden upon the pollsters. Let us see what happened when the Gallup poll tried to predict the electoral behavior of the individual states in 1936, 1940, In all there were 144 individual prediction or cases. In ten states the actual Democratic percentage of the major party vote turned out to be between 35 and 44.9 per cent. In these ten solidly Republican states Dr. Gallup made ten correct predictions. He batted 1000. Fourteen states gave the Democrats between 45 and 49.9 per cent of the major vote. In those fourteen states Gallup again batted 1000. Thirty-six states gave the Democrats 50 to 54.9 per cent of the major party vote. In these closely contested states where the Democratic party eked out close victories, Dr. Gallup made 19 correct predictions. That is, 52.3 per cent of the time he predicted correctly. He erred in 17 cases, or 47.2 per cent of the time. In the 26 states where the Democrats obtained from 55 to 59.9 per cent of the major party vote he was correct in 22 cases or 84.6 per cent of the time, incorrect in four cases or 15.4 per cent of the time. In 58 cases in which the Democrats polled over 60 or more per cent of the major party vote, Dr. Gallup was correct 100 per cent of the time. The sad fact is that he was only slightly more than half correct in all judgements regarding states in which the Democrats achieved between 50 and 54.9 per cent of the major party vote. You will perhaps have noticed that the Gallup poll predicted correctly Republican victories even where the Republicans got between 50 and 54.9 per cent of the major party vote. This is because throughout his predictions he was over-estimating the Republican vote by some 4 per cent and we would say then that often the pollsters as represented in the present instance by Dr. Gallup, have often been weighted and biased on the side of the Republicans and have had a grim struggle to be correct more than 50 per cent of the time when an election was hotly contested. This record would have been worsened if the results in 1948 had been included in my analysis. In that year the Crossley poll underestimated the Truman vote by 7.9 per cent: the Gallup poll underestimated it by 5.4 per cent. Other polls underestimated the Democrats more or less. Gallup's American Institute of Public Opinion guessed closest in Texas where the actual vote was 66.4 per cent Democratic and the poll gave 66 per cent, a negligible difference. Another near miss was Rhode Island where Truman picked up 54.7 per cent of the vote. In the critical states of California, Illinois, Massachusetts, New York, Ohio, and Pennsylvania the Gallup predictions underestimated Truman's vote by respectively 5.6 per cent, 4.7 per cent, 10.3 per cent, 6.5 per cent, 7.7 per cent, and 3.2 per cent. In most of these cases the error made the difference between a Republican and a Democratic victory. Such evidence leads one to conclude that the polls are weakest where they ought to be strongest, least valid where we would hope them to be most valid.
Now you may wonder whether we cannot expect the national polls to continue their Republican bias. If this were the case, Stevenson would be an easy winner. But we cannot use the past as a guide. The Literary Digest bias in 1932 did not interfere with their prediction of a Roosevelt victory. The same bias in 1936 brought ludicrous results. So we cannot rely on the past biases of polls without intensive analysis of the meaning of the biases. It is even possible that certain polls may be biased against Eisenhower this time, in a strange manner that may not be discovered until later on.
From the information available to me, I would assert that the pollsters have not changed significantly their techniques since 1948. There is no reason to believe that the polls will have described today's events more accurately than the events of past election days. The major difference will be in the reporting. We have not been led up the garden path this year by rash predictions, as in years past. The lives of those who operate the polling agencies will be happier for this fact tomorrow and in the days ahead. There will be no popular clamor for their scalps, nor will congressional investigating committees descend upon them. Furthermore, they will not be plagued by the criticism of their colleagues in the academic cloisters. So long as we hold real elections in which everyone participates, rather than only a sample of the population, we will find that polls which are properly conducted and accurately reported are a useful device. They are much better than hunches in discerning trends of party opinion or party attachment. They can tell us who is more excited in the population, who is apathetic, who is undecided, what groups are siding with whom, and many other items of information basic to the rational conduct of a political campaign. To give sample surveys credit for this much is much better than to give them credit one day for superhuman foresight, and the next day to condemn them for bias and for trying to predict us inscrutable human beings.
The sample survey, by comparison with all the other forecasters we have examined, is most valid and reliable. If we were interested in determining how many voters favored one candidate over another as a purely technical question of counting, a perfected poll, using the latest techniques, with a larger sample, and with the proper controls, could give us a verdict as accurate in almost all cases as the election itself. A couple of millions of dollars would produce almost impeccable polls and save many millions of dollars of public money.
Now you may think this is going back on what I said about polling errors but that is not so. I said that polls cannot predict close elections but I did not say that polls could not be used to take a vote itself. In such a hypothetical case the poll itself would be the election. You would not have to worry about how many persons would be non-voters nor about what to do with the undecided persons. Rather, on a given day, the poll would be conducted and everyone who is sampled would be counted. We wouldn't have to worry about discrimination in the south against certain groups, about non-voters, about corrupt machines (or at least there would be only one corruptible machine, the polling agency), nor about weather not about many other features that distort the electoral decision when the mass of people must participate. Furthermore, many serious defects in the present polls are caused by cutting costs, by slipshod methods of taking the poll, and by poor reporting of results. These defects easily could be remedied.
In other words, the sample survey can count much better than it can predict. It can count just about as well as an election. In very close cases, an election could be taken to confirm or deny the results of a poll.
I introduce this proposal, however, without any serious intent. I bring it up solely to confirm the general high value of sample surveys correctly used. The larger character of the election process would make such a technical idea an undesirable and impossible one. An election in a democracy is much more than a technical abacus or adding-machine. It is more important than a mere counting of noses. It is, in social terms, part of a most significant structure of behavior patterns, emotion, and beliefs. Though our elections may seem massive and clumsy, their very bulk possesses utility.
An election embraces the many-sided phases and soul of a people. Its meanings cannot be reflected in a count although the count is essential to all other meanings. Its meanings are better reflected in the excitement we have felt, and in the impatience, well-controlled, that we feel at this moment -- and in the flush of elation with which we will all turn now to learn who shall be our rulers and servants.