This article investigates possible determinants of forecasting error for new prime-time network television programs. Each season, advertising industry forecasters attempt to predict the audience shares for new fall programs. Advertising expenditures are made on the basis of these projections,
The process of forecasting the size of television audiences has received extensive attention in both academic and industry circles (Cooper 1993; Rust and Eechambadi 1989). Any purchase of television advertising time involves purchasing a predicted audience. That is, advertisers buy time on the assumption that a certain size and type of audience will view the advertisement. However, a program's performance does not always meet advertisers' expectations.
The introduction of new prime-time network programs has provided perhaps the most visible display of this forecasting process. Each spring, advertising agencies and media buyers project the audience shares for new programs on the basis of pilots, clips, descriptions, and scheduling. On the basis of these projections, agencies typically purchase roughly 75% to 80% of the networks' ad inventory during the "upfront" period, which spans several months during the spring and summer (Mandese 1995). These audience projections are made in an environment filled with uncertainty, because these programs have yet to air a single episode and therefore have no previous track record that forecasters can use to guide their projections. Despite the uncertainty, the advertising industry and its clients rely heavily on these forecasts when planning ad schedules (DeCoursey 1995, p. S14). The question then becomes: Will the new programs that air in the fall fulfill the projections that guided the time purchases?
Unfortunately, these audience forecasts "have been notoriously inaccurate" (Rust and Eechambadi 1989, p. 13; see also Forkan 1986). For example, for the 1997-1998 television season, advertising industry forecasters incorrectly estimated the audience shares of 22 new prime-time programs and accurately predicted the audience shares for only 5 of the new fall programs (see Wells 1997). Typically, these forecasts overestimated, rather than underestimated, the performance of the new programs.
Time bought on the basis of an overestimate of the actual audience size means that the advertiser has overpaid for the number of viewers reached. Although the broadcast networks typically supply "make-goods" when a program fails to deliver its predicted audience size, these additional make-good slots may not fit the initial media plan as well as the original time slots purchased (Rust and Eechambadi 1989, p. 11). Consequently, it is in advertisers' best interests that the initial forecasts that guide their purchases be as accurate as possible and that the advertisers have a sense of the likelihood that a program will deliver its projected audience.
Although previous research has advanced understanding of the factors affecting the size of a program's actual audience, little attention has been paid to the factors affecting the accuracy of the predicted audience. Investigating the determinants of forecasting error opens a largely unexplored avenue of audience research into the "predictability" of various audience segments. Because advertisers will pay a premium for more reliable audience information (Fournier and Martin 1983; Webster and Phalen 1997), it is important that audience researchers identify when the forecasts that guide advertising buys are more likely to be reliable. Identifying the determinants of forecasting error can provide consumers of ratings forecasts with tools for evaluating the circumstances in which these forecasts are more likely to be accurate. Thus, advertisers evaluating a range of forecasts and seeking to place their advertising dollars with the programs most likely to deliver the predicted audience will have tools to help them identify higher-risk advertising buys. These variable levels of risk can then affect the ad pricing of different programs (see Fournier and Martin 1983).
This research represents a first step toward identifying those conditions in which audience behavior is more predictable and therefore when those audiences should be more highly valued. It focuses on new primetime programs aired by the Big Four broadcast networks (ABC, CBS, NBC, FOX) and on scheduling factors that can affect the degree to which audience shares can be predicted accurately.
Identifying "Uncertainty Factors"
Organizational decision-making research often has focused on how organizations cope with uncertainty when making decisions (e.g., March 1994; Stinchcombe 1990). Uncertainty has been defined as "imprecision in estimates of future consequences" (March 1994, p. 178). The key points that come out of this body of literature are that decision quality is inversely related to uncertainty and that uncertainty is reduced by the availability of information (March 1994). As Stinchcombe (1990, p. 5) says, "Uncertainty is reduced through news; then finally the residual uncertainty is transformed into risk and people make their bets." This statement accurately characterizes the process of forecasting audience shares for new television programs. When predicting audience shares for new programs, forecasters face a highly uncertain decision-making environment because they are unable to rely on a prior ratings history to guide their analysis.
Given the uncertainty inherent in this forecasting environment, it is essential that decision makers estimate the amount of risk present in any individual decision-making situation. Doing so requires investigating whether there are factors that affect the degree of risk (i.e., uncertainty) that surrounds individual forecasts. To investigate this issue, the sections that follow integrate decision-making theory and audience behavior theory in an effort to identify possible "uncertainty factors" and test their impact on audience forecasts.
Inheritance Effects
Research on inheritance effects focuses on the effects of a program's lead-in (the program preceding the focal program) and lead-out (the program following the focal program) on a program's audience size. The size of both a program's lead-in audience and its lead-out audience are often significant predictors of a program's audience size, because viewers have a tendency to remain with a single channel during a viewing session (Webster and Phalen 1997). Generally, the lead-in effect is more powerful than the lead-out effect (Cooper 1993). though both effects have diminished over time as program options have increased (Eastman 1998).
From a decision-making standpoint, if a program's lead-in or lead-out is new, there is less information available to forecasters when they make their predictions than if the lead-in or lead-out is a returning program with a ratings history. This leads to the following hypotheses:
H1: Programs with returning lead-ins will have less forecasting error than will programs without returning lead-ins.
H2: Programs with returning lead-outs will have less forecasting error than will programs without returning lead-outs.
Competitive Scheduling and Counterprogramming
Audience behavior research has demonstrated that the size of a program's audience is partly a function of the programs available on competing channels (Eastman 1998; Horen 1980). The common strategy of "counterprogramming" (placing a program that appeals to a very different audience segment against a competing program) arose because the audience size for a program cannot be understood purely in terms of the appeal and merits of that particular program (and its lead-in and lead-out). Rather, any program's audience size should be approached in terms of how it is likely to fare against competing programs.
From a decision-making standpoint, accounting for the effects of scheduling on a new program's audience size becomes more difficult as the available information about the likely performance of the competing programs declines. A program scheduled against new programs poses greater uncertainty than a program scheduled against returning programs. This leads to the following hypothesis:
H3: The greater the percentage of competing network time occupied by new programs, the greater the forecasting error will be.
New Program Quantity and Decision Making
When the time period available to make a specified number of decisions is held constant, we should expect better decision-making quality as the number of decisions that needs to be made in that time period decreases. The fewer decisions that need to be made, the greater is the amount of time that can be devoted to information gathering and processing for each individual decision. As the number of decisions that needs to be made in that time period increases, the amount of time that can be devoted to information gathering and processing for each individual decision decreases. Subsequently, the uncertainty level for each individual decision increases, and decision quality should decrease.
Applying this logic to television audience forecasting, those seasons in which a greater number of new programs are introduced impose greater information gathering and processing burdens on forecasters than do those seasons in which fewer new programs are introduced. The time period from program schedule introduction to forecast generation remains relatively constant, so the introduction of more new programs should result in greater levels of uncertainty and hence greater levels of forecasting error. This leads to the following hypothesis:
H4: The greater the number of new primetime network programs introduced in a new program's debut season, the greater the forecasting error will be.
Audience Behavior Theory and the New Media Environment
The enormous changes taking place in the media environment have complicated the process of predicting the behavior of television audiences (Eastman 1998). Developments such as the remote control device, the video cassette recorder, and the enormous channel capacity of cable television make it more difficult for programmers to attract and retain audiences using traditional programming strategies and for forecasters to anticipate how viewers are likely to be distributed across their program options. Recent additions to the media mix, such as direct broadcast satellite (DBS) and the Internet, further complicate the process of predicting audience behavior. These increasing complications and the associated uncertainty about exactly how they are likely to affect viewing behavior lead to the following hypothesis:
H5: Forecasting error for new prime-time network television programs will increase over time.
Programming History and Audience Homogeneity
Because this research focuses on the predicted audience shares for the Big Four networks' new primetime programs, it is important to recognize that the FOX network possesses some fundamental differences that may affect forecasting outcomes. First, FOX is comparatively new, having only begun broadcasting in prime time in 1987. The network still does not present a complete prime-time programming line-up (it leaves the 10:00 PM hour to the local affiliates). As a newer network, with a shorter programming record for forecasters to draw upon in their decision making, the FOX network appears to present a greater level of uncertainty than do the traditional Big Three (ABC, CBS, NBC). Second, the time period studied (1993-1998) represents a period of much greater volatility for FOX than for the other three networks. That is, FOX was still very much in its developmental stages, particularly in terms of expanding its program offerings and establishing affiliates. Thus, the status of the FOX network changed more substan tially from year to year than did the status of the Big Three networks. This situation introduces further uncertainty into predicting the audiences for FOX programs.
However, there is also reason to expect lower levels of forecasting error for FOX programs. Research by Tavakoli and Cave (1996, p. 78) indicates that "viewing variation tends to increase with the age of the viewers." That is, as viewers age they tend to consume a greater diversity of program types and become more difficult to associate with particular program-type preferences. This pattern suggests that the viewing behavior of younger viewers presents less uncertainty and should be easier to predict than the viewing behavior of older viewers. As FOX has always targeted a younger audience (Consoli 2000a), it is possible that predicting FOX program audiences poses lower levels of uncertainty than predicting audiences for the other networks' programs. These competing logical approaches lead to the following research question:
RQ1: Will FOX programs exhibit more or less forecasting error than Big Three programs?
Methodology
To investigate the hypotheses and research question outlined previously, data were collected on the predicted and actual shares of the total television viewing audience for new prime-time network television programs for the 1993-94 through 1997-98 broadcast seasons. Only the Big Four networks are included in this analysis, because the newer networks (UPN, WB, and, to a lesser degree, PAX) only offered a significant competitive presence during the last two seasons studied.
Audience share forecasts were compiled using the annual advertising industry survey published in Broadcasting & Cable. Each summer, Broadcasting & Cable publishes the results of a survey of the nation's major national advertising agencies. The survey includes the mean national audience share predictions for both new and returning prime-time network television programs. Thus, this annual survey essentially represents the advertising industry's consensus regarding the likely performance for each show in the prime-time network line-up.
Actual national audience shares were gathered from weekly Nielsen reports published in Broadcasting & Cable. To create a measure of a program's actual performance, a mean audience share was computed using national audience shares for the first four broadcasts (excluding the premiere) in which the program aired in its scheduled time slot, against its scheduled competition, and with its scheduled lead-in and lead-out (to reflect the scenario considered by the forecasters accurately). The premiere was excluded to reduce variation caused by differences in the amount of preseason promotion and because the premieres of many new programs run longer than their normal length. As one media buying report has noted, "Premiere performance is rarely an accurate predictor of overall season performance" (Consoli 2000b, p. 6).
Programs that were part of the schedule at the time the forecasts were generated but were removed from the fall broadcast schedule were excluded from the data set, as were programs that were scheduled in one time slot at the time the forecasts were generated and moved to a different time slot before the season commenced. Also, new programs that never actually ran against the competitive line-up they were scheduled to face or never ran with their scheduled lead-in or lead-out were excluded. As a result of these conditions, a total of 17 programs were removed from the analysis, leaving a total of 140 programs.
In some cases (52 programs), it was only possible to obtain one to three regularly scheduled broadcasts due to volatility in program schedules. However, these 52 programs were included in the analysis to avoid biasing the data set against programs that performed poorly (and were quickly removed from the line-up or rescheduled). When the programs with one to three regularly scheduled broadcasts were excluded from the analysis, there was virtually no difference in the overall explanatory power of the model or in the direction or magnitude of the relationships between the independent variables and the dependent variable.
From the predicted share and actual share data gathered from Broadcasting & Cable, a measure of forecasting error was created (ERROR). This measure was calculated as follows:
[absolute value of] (Predicted share-Actual share)/Actual share [absolute value of] x 100.
This measure serves as the dependent variable for this study. It expresses, as a percentage, the difference between a program's predicted audience share and its actual audience share. This measure is expressed in absolute value terms because uncertainty can result in either overestimation (positive error) or underestimation (negative error) of a program's audience.
To investigate Hypotheses 1 and 2, dummy variables were created for whether each new program's lead-in and lead-out were returning network programs (RLEADIN, RLEADOUT; 0=no, 1=yes). Some new programs started or concluded a network's primetime line-up (e.g., an 8:00 PM EST [7:00 PM EST on Sunday] start time; a 11:00 PM EST [10:00 PM EST for FOX] concluding time), in which case they lacked either a network lead-in or lead-out. To account for this distinction, additional dummy variables were created for whether each new program had any network lead-in or lead-out (LEADIN, LEADOUT; 0=no, 1=yes).
To investigate Hypothesis 3, the percentage of competing network time occupied by new programs (PERCNEW) was calculated for each new program as follows: First, the total number of half-hour blocks of Big Four network broadcast time encompassing the time of the new program was summed. Second, the total number of those blocks occupied by new programs was calculated. Third, this number was divided by the total number of half-hour network broadcast blocks. Mathematically, this can be represented as follows:
PERCNEW = Total number of competing network 1/2-hour blocks of new programming x 100/Total number of competing network 1/2-hour blocks of programming.
To investigate Hypotheses 4 and 5, the total number of new programs being introduced for each season (NEWSEAS) was recorded, as was the year each program was introduced (YEAR). Finally, with regard to Research Question 1, the network designation for each program was recorded. As was indicated, the primary concern here is whether the program aired on the FOX network. A dummy variable was created (FOX), in which those programs airing on FOX were coded as a 1 and those programs not airing on FOX were coded as a 0. Descriptions of all of the variables are presented in Table 1.
Results
The top portion of Table 2 presents the mean forecasting error, as well as the mean absolute value of forecasting error. As the table indicates, on average, forecasters overestimated the performance of new programs by approximately 15% (a negative mean would have indicated a tendency toward underestimation). This tendency toward overestimation is somewhat surprising, because advertisers ultimately must negotiate ad rates with the networks on the basis of these forecasts. However, the results presented here conform with the results of previous research (see Wells 1997). Perhaps this optimism is partly a result of forecasters' failure to account adequately for audience erosion to cable in their forecasting models (Wakshlag 1998), or perhaps that pilots of new programs often exhibit higher production values than regular season episodes leads forecasters to overestimate the prospects for new programs. When forecasting error is expressed in absolute value terms, the mean is more than 21%.
The lower portion of Table 2 displays the mean forecasting error (in absolute value terms) according to network. As the table indicates, the traditional Big Three networks have forecasting error means of between 20% and 25%, whereas FOX programs have a mean of roughly 14%. When FOX programs are compared with Big Three programs, the difference in the mean forecasting error is statistically significant (F=4.65; p[less than or equal to].05). These results suggest that forecasters are more accurate in predicting the audience shares for FOX programs.
Table 3 presents the results of the multiple regression model. The adjusted [R.sup.2] for the model is .19 (p=.00). RLEADIN, RLEADOUT, and YEAR are all statistically significant in the expected direction, providing support for Hypotheses 1, 2, and 5. As the table indicates, programs with a returning lead-in exhibit less forecasting error than programs without returning lead-ins (beta=-.35, p[less than].01). Similarly, programs with returning lead-outs exhibit less forecasting error than programs without returning lead-outs (beta=-.24, p[less than].05). Neither of the dummy variables designed to account for programs without network lead-ins or lead-outs (LEADIN, LEADOUT) was significant at the .05 level. The significant positive coefficient for YEAR (beta=.26; p[[less than].01) indicates that forecasting ereor has been increasing over time.
The coefficients for NEWSEAS and PERCNEW were not significant at the .05 level, thus providing no support for Hypotheses 3 and 4. Neither the number of new programs introduced in a season nor the percentage of competing network time composed of new programs had a significant effect on forecasting error. In terms of Research Question 1, though FOX programs exhibited significantly lower mean forecasting error than did Big Three programs, the FOX dummy variable was not significant within the multivariate analysis (beta =-.12; p[greater than].05).
Conclusion
This exploratory study represents a first step toward a greater understanding of the factors that affect the predictability of television audiences. This study finds that the presence of returning network lead-ins and lead-outs is significantly related to lower levels of forecasting error for new prime-time network programs. Just as lead-ins and lead-outs affect the size of program audiences, they also appear to affect their predictability. These results suggest that returning lead-ins and lead-outs significantly reduce the amount of uncertainty faced by forecasters, enabling them to predict the performance of new programs better. Thus, advertisers who use these forecasts to guide their time purchases will more likely reach their anticipated audience if they focus their purchases on programs with returning lead-ins and lead-outs.
With forecasters' tendency for overestimating the audience shares for new programs, the absence of returning network lead-ins or lead-outs is more likely to lead to overestimates of a program's audience rather than to underestimates. As was indicated at the outset, such overestimates can undermine the effectiveness of an advertising campaign, even when broadcasters supply make-goods to cover the difference between the predicted and actual audience.
This research also finds that forecasting error is increasing significantly overtime, which suggests that the dynamic and increasingly complicated media environment is making the task of predicting audience shares for new television programs more difficult. These results suggest that advertisers should be increasingly skeptical of the reliability of these audience forecasts and that their media planning should account for the growing likelihood that their up-front network buys will not produce the audiences that were initially predicted.
Other hypothesized uncertainty factors, such as the percentage of competing network time occupied by new programs and the number of new programs airing in a new program's debut season, were not significantly related to forecasting error. The nonsignificance of the number of new programs airing in a particular season may be due to a lack of variability in this measure, in that the number of new programs per season ranged between 27 and 36 during the years studied.
That the percentage of competing time occupied by new programs was not significant may indicate that structural factors such as competitive scheduling are decreasing in importance in terms of understanding the distribution of television audiences (see Adams 1993). As the number of program options increases, the degree to which viewing decisions are a function of alternative program options may be in decline, as viewing increasingly can be driven by program-type preferences (Youn 1994). Cable television is central to this transition; however, this study does not account for the possible impact of cable program schedules. Although broadcast programmers still generally focus on the programming line-ups of the competing broadcast networks in their scheduling decisions (Eastman 1998) and advertisers still primarily consider cable programming in the aggregate in terms of its effects on the size of broadcast audiences (Wakshlag 1998), specific variations in cable programming schedules can certainly affect the perfo rmance of individual broadcast network programs. Future research in this vein should expand to include cable programming and other emerging structural complexities (e.g., DBS and emerging networks such as WB and UPN) of the television environment.
With regard to the question of whether FOX programs exhibit more or less forecasting error than Big Three programs, the results are mixed. Mean comparisons indicate significantly lower levels of forecasting error for FOX programs. However, in the multivariate analysis, the FOX dummy variable was not statistically significant. The significant difference between the means, however, suggests important avenues for future research.
The reason for expecting lower forecasting error for FOX programs was the possibly greater ease in predicting the viewing behavior of younger versus older audiences. The results presented here provide some support for such a possibility, in that the network with the youngest audience (FOX) exhibited the lowest forecasting error and the network with the oldest audience (CBS) exhibited the highest forecasting error (see Table 2). Ratings analysis is growing less concerned with overall audience shares than with shares of particular demographic groups, and future research should investigate whether predictability varies in accordance with demographic factors. Previous research has found that advertisers will pay a premium for audiences whose predicted availability more closely matches their actual availability (Fourier and Martin 1983). Consequently, the possibility of associating "predictability" with certain demographic characteristics could introduce an important factor that could affect the valuation of part icular audiences.
Just as predictability of television audiences may be a function of the demographic characteristics of those audiences, it also may be a function of the qualitative dimensions of the individual programs and of how effectively these dimensions are assessed and correlated with audience appeal by forecasters. Therefore, future research should investigate content characteristics and examine whether certain program types exhibit greater consistency in their audience performance and whether this greater consistency translates into improved predictability in terms of audience shares. These recommended expansions also should look beyond new prime-time network programs to other program categories (e.g., returning programs, other dayparts) and program sources (e.g., cable). In summary, integration of structural factors with audience and content factors should significantly improve understanding of the conditions under which the viewing behavior of television audiences can be predicted effectively.
Philip M. Napoli (Ph.D., Northwestern University) is an Assistant Professor of Communications and Media Management, Graduate School of Business Administration, Fordham University.
This project was conducted with the assistance of grants from the National Association of Broadcasters and Fordham University's Graduate School of Business Administration.
References
Adams, William J. (1993), "TV Program Scheduling Strategies and Their Relationship to New Program Renewal Rates and Rating Changes," Journal of Broadcasting & Electronic Media, 37 (4), 465-474.
Consoli, John (2000a), "Broadcast's Demo Dilemma," Mediaweek, (March 6), 4-5.
----- (2000b), "For Prime Time, Reality Sets In," Mediaweek, (October 30), 6-7.
Cooper, Roger (1993), "An Expanded, Integrated Model for Determining Audience Exposure to Television," Journal of Broadcasting & Electronic Media, 37 (4), 401-418.
DeCoursey, Jennifer (1995), "Buyers Peg New-Season Winners & Losers," Advertising Age, (June 12), S14-S15.
Eastman, Susan T. (1998), "Programming Theory Under Stress: The Active Industry and the Active Audience," Communication Yearbook, 21, 323-378.
Forkan, James P. (1986), "Nielsen Waters Down TV Forecasters' Tea Leaves," Advertising Age, (April 7), 24, 79.
Fournier, Gary M. and Donald L. Martin (1983), "Does Government-Restricted Entry Produce Market Power?: New Evidence from the Market for Television Advertising," Bell Journal of Economics, 14 (Spring), 44-56.
Horen, Jeffrey H. (1980), "Scheduling of Network Television Programs," Management Science, 26 (4), 354-370.
Mandese, Joe (1995), "Supply and Demand, 'Upfront,' Ratings Points, Media Fragmentation: Oh How TV Has Changed," Advertising Age, (February 28), 20.
March, James G. (1994), A Primer on Decision Making: How Decisions Happen, New York: The Free Press.
Rust, Roland T. and Naras V. Eechambadi (1989), "Scheduling Network Television Programs: A Heuristic Audience Flow Approach to Maximizing Audience Share," Journal of Advertising, 18 (2), 11-18.
Stinchcombe, Arthur L. (1990), Information and Organizations, Berkeley, CA: University of California Press.
Tavakoli, Manouche and Martin Cave (1996), "Modeling Television Viewing Patterns," Journal of Advertising, 25 (4), 71-86.
Wakshlag, Jacob J. (1998), personal correspondence (February).
Webster, James G. and Patricia F. Phalen (1997), The Mass Audience: Rediscovering the Dominant Model, Mahwah, NJ: Lawrence Erlbaum Associates.
Wells, Melanie (1997), "New Fall Shows Disappointing Advertisers," USA Today, (November 17), 7B.
Youn, Sug-Min (1994), "Program Type Preference and Program Choice in a Multichannel Situation," Journal of Broadcasting & Electronic Media, 38 (4), 465-475.
Table 1 Independent and Dependent Variables
Variable Name Description
ERROR (DV) Absolute value of the percentage
of error:
[(Predicted share - Actual share)
/Actual share)] x 100.
LEADIN Does the program have a network
lead-in (0=no, 1=yes).
RLEADIN Does the program have a returning
network lead-in (0=no, 1=yes).
LEADOUT Does the program have a network
lead-out (0=no, 1=yes).
RLEADOUT Does the program have a returning
network lead-out (0=no, 1=yes).
PERCNEW Percentage of competing network
time occupied by new programs.
NEWSEAS Total number of new programs airing
in program's debut season.
YEAR Year program was introduced.
FOX Program aired on FOX network
(0=no, 1=yes).
Table 2 Means for Forecasting Error (N = 140)
Variable Mean Standard Deviation
Percentage of error 15.23 * 24.84
(difference/actual share)
Absolute value of percentage 21.35 19.78
of error (ERROR)
Network Mean N Standard Deviation
Mean Absolute Value of Percentage
of Forecasting Error by Network
NBC 20.48 36 21.84
ABC 23.84 39 22.41
CBS 25.11 36 19.26
FOX 14.38 29 11.11
Fox Versus Big Three
FOX 14.38 29 11.11
Big Three 23.16 111 21.14
Note: F = 1.90 (p[greater than].05).
Note: F = 4.65 (p[less than].05).
(*)Positive sign indicates that, on average, forecasts overestimated
rather than underestimated audience shares for new network programs.
Table 3 Summary of Simultaneous Regression Analysis for Variables
Predicting Forecasting Error (N=140)
Variable B Standard Error Beta
LEADIN 9.29 5.21 .20
RLEADIN -13.80 4.25 -.35 **
LEADOUT 9.18 5.80 .17
RLEADOUT -9.42 3.76 -.24 *
PERCNEW -.08 .06 -.10
NEWSEAS .51 .51 .08
YEAR 3.49 1.17 .26 **
FOX -5.76 4.06 -.12
Constant -6960.01 2334.86
Note: Adjusted [R.sup.2]= .19 (p = .00).
(*)p[less than].05.
(**)p[less than].01.