Empirical Research Published in Production and Operations Management (1992-2005): Trends and...

1. Introduction

Empirical: Relying on or derived from observation or experiment; verifiable or provable by means of observation or experiment; guided by practical experience or theory (adapted from Dictionary.Com 2006).

About 25 years ago, there was a strong perception among many researchers that the mainstream research in operations management and operations research was lacking a focus on practice (e.g., Ackoff 1987; Miser 1987; Corbett and Wassenhove 1993). A number of scholars (Buffa 1980; Andrew and Johnson 1982; Groff and Clark 1981; Meredith et al. 1989; Swamidass 1991; Flynn et al. 1990) made a case for empirical research, but for a while, the operations management community paid little attention. One can argue that most of the research in operations management at that time was primarily focused on abstract analytical techniques rather than practice-based modeling or empirical research. For example, Corbett and Wassenhove (1993, p. 638) stated that empirical research investigating the success of operations research techniques can be very valuable, but its use was not widespread. Similarly, Lee (1966) was quoted, stating that ". . . we need to know more about the real behavior of people in queues; until such time as we have that information, much of the theory that appears in the journals will remain no more than a collection of charming, mathematical acrostics. Mathematics, however ingenious, is not a proper substitute for knowledge (Miser 1987, p. 317)." Shubik (1987, p. 1521) stated that "Solving mathematical puzzles without worry concerning context can provide, for some, a satisfactory exercise, but the science and art of management calls for more. An application is when the context is understood, the theory is relevant, and the decision process is influenced."

The last decade of the twentieth century, however, witnessed a significant and welcome change towards both practice- and theory-driven empirical research in the operations management community. The Production and Operations Management Society (POMS) was created in 1989, and in the inaugural issue of its flagship research publication, Production and Operations Management (POM), Singhal (1992, p. 1) stated, "Our objective in publishing this journal is to improve practice." Alluding to the absence of outlets that did not cover certain topics and research paradigms, Singhal (1992, p. 2) further stated, "We hope that the existence of a journal with broad coverage will encourage authors to write articles on many nontraditional topics. The editors realize that authors have not written articles on certain topics or articles that use certain research paradigms simply because the outlets for publishing them have been limited. In Production and Operations Management, papers do not have to fit into an established mold, follow one of the traditional research paradigms, concern an accepted topic, or conform to conventional wisdom." Production and Operations Management took the lead by publishing in its inaugural issue five articles which were partially or completely based on empirical research.

According to Singhal (personal communication), Robert Hayes, Sunder Kekre, Hau Lee, Jack Meredith, Martin Starr, and Gene Woolsey were actively involved in formulating the editorial policy of POM, which placed a heavy emphasis on empirical research. Jack Meredith, George Shanthikumar, and Gene Woolsey served as editors of the empirical papers published in the journal's inaugural issue. Following his role in POM, Jack Meredith emphasized empirical research during his term as the editor of the Journal of Operations Management (JOM) during 1995 to 2001.

Hayes (1992, 1998, 2000, and 2002) articulated his vision of operations management in a series of essays in Production and Operations Management and emphasized the importance of different forms of empirical research. He (1992, p. 251) noted, "A theory that is based on the statistical analysis of large data sets may be appropriate when the external environment is relatively stable, since then the data points can be assumed to be drawn from the same population. But when the environment is changing rapidly, understanding what is going on comes less from ascertaining the central tendencies and trends of the data it generates than from a close examination of outliers, particularly if they are the most recent points. . . . Just as theory tends to focus on the underlying structure of a situation, cases tend to focus on big, pivotal moments in an organization's life. . . . The rigor-versusrelevance debate loses its power to divide us as soon as we expand the definition of rigor beyond its mathematical meaning to include careful observation and analysis."

Our purpose in this paper is to assess how far the authors of the papers published in POM have met the journal's stated objective of promoting empirical research. We also assess the diversity of articles in terms of the purposes of research, data collection approaches, and data analysis techniques. We review and evaluate empirical research papers published in Production and Operations Management during 1992 to 2005. While other review papers recently published in POM (Boyer, Swink, and Rosenzweig 2005; Buhman, Kekre, and Singhal 2005; Kleindorfer, Singhal, and Van Wassenhove 2005; Kouvelis, Chambers, and Yu 2005; Krishnan and Loch 2005; Schroeder, Linderman, and Zhang 2005) highlighted some of the empiricalresearch- based papers, we will present a systematic review and evaluation of all the empirical-researchbased articles published in the first 14 volumes of this journal. Specifically the purpose of this paper is to:

* study the evolution of empirical research published in POM from 1992 to 2005

* identify the mix of functional topics (strategy, quality, supply chain, etc.) and industrial focus (manufacturing vs. service) in empirical-researchbased articles

* classify the empirical research articles based on their primary purpose (theory building, theory verifying, application, and providing evidence), data collection approach (case study, qualitative research, archival research, survey-based research, laboratory research, and field research), and data analysis approach (descriptive statistics, various multivariate statistical techniques, and mathematical modeling); and

* provide directions for future empirical research in the operations management discipline.

The beginning of the 1990s represented a critical period of research in operations management when empirical research started to appear in substantial quantity in the mainstream operations management journals (e.g., JOM, POM). One can argue that this period represents the "growth" phase of empirical research in operations management. A few articles synthesized the findings of empirical research-based articles published in operations management journals before 1995. For example, Scudder and Hill (1998) analyzed empirical-research-based articles published in operations management journals from 1986 to 1995; Malhotra and Grover (1998) provided an assessment of 25 survey-based empirical research articles published from 1990 to 1995; and Wacker (1998) presented an assessment of empirical-research-based articles published from 1990 to 1995, primarily focusing on theory-building research. Other authors, such as Hensley (1999), Panniselvam et al. (1999), and Verma and Goodale (1995), analyzed some specific aspects of published empirical research primarily for the period of the late 1980s to mid-1990s. Since the publication of POM started only in 1992, these studies either did not cover POM or covered only its first few years of publication. We believe that empirical research within the discipline has evolved considerably since then and an analysis of empirically-based research articles published during the growth period that began around 1992 will provide valuable insights on the status of research in operations management. The purpose of our study is, therefore, to document the development and evolution of empirical research within the operations management discipline through an analysis of articles published during 1992 to 2005 in one of its top-tier journals.

The remainder of the paper is organized in the following manner. We first provide a brief background about different aspects of empirical research. We next describe our research approach, present the results, and discuss the trends in empirical research. Finally we offer concluding remarks and provide guidelines for future empirical research in the operations management area.

2. Background

During the last fifteen years or so, a number of papers have been published which describe in considerable detail many different aspects of conducting research based on empirical data (e.g., Swamidass 1991; Flynn et al. 1990; Boyer and Verma 2000). In this section, we describe papers from Production and Operations Management and highlight various aspects of empirical research.

2.1. Research Purpose

Previous research in the social sciences classifies empirical research according to two broad categories- theory building and theory testing (e.g., Flynn et.al. 1990; Swamidass 1991). However, operations management research also needs to be grounded in practice and should be able to demonstrate the utility of existing theories (e.g., Shubik 1987; Whetten 1989). Similarly, the researchers who develop analytical models in operations management might conduct empirical research to evaluate the validity of assumptions made in the modeling process. For example, are the interarrival times actually distributed according to the exponential distribution (Lee 1966)?

Therefore, we have classified empirical research papers in this study into four broad categories: theory building, theory verifying, providing evidence, and application. In theory-building papers, authors typically propose new sets of relationships, propositions, frameworks, and hypotheses which can be tested in future research studies (Whetten 1989). While it is common in many social science disciplines for theorybuilding papers to be primarily conceptual, within operations management, theory-building papers often also contain empirical data to provide motivation, early justification, or support for the proposed theory or framework. For example, in one of the early empirical studies, St. John and Young (1992) studied the patterns of tradeoffs and agreements between marketing and operations managers related to their competitive priorities, and more recently, Narasimhan, Jayaram, and Carter (2001) evaluated effectiveness of the purchasing function.

In theory-verifying, researchers test frameworks, hypotheses, or relationships proposed in earlier research (Flynn et al. 1990; Swamidass, 1999). For example, Rosenzweig and Roth (2004) tested and extended the "sand-cone model" of cumulative competitive capabilities and Leschke (1996) examined setup reduction in manufacturing processes.

The third type of empirical research which has appeared in the operations management literature is applications to existing theories, models, or frameworks in a variety of industries. For example, Boronico (1999) applied concepts from pricing, capacity, and quality management to postal services; Caro et al. (2003) applied operations management concepts to address environmental issues in Chilean pipelines; and Kapalka, Katircioglu, and Puterman (1999) presented an application of inventory control principles for a western Canadian retailer.

Finally, while a large portion of the operations management literature is based on analytical techniques, empirical studies can be conducted to provide evidence or to support assumptions made or conclusions drawn from model-based studies. For example, in a set of two papers, Christensen (1992a, 1992b) used data from the computer disk-drive industry to examine the S-curve framework used in new technology or product development. Noori and Chen (2003) applied a scenario-driven strategy to provide support for environmental management and product design concepts.

2.2. Data Collection Approaches

The diversity of industry and topic areas within the operations management discipline requires researchers to use many different types of data collection approaches. For example, an empirical study designed to verify the effectiveness of a proposed theory (e.g., quality management principles) is generally very different from one designed to uncover underlying characteristics of a specific approach (e.g., six-sigma quality improvement techniques) for a certain segment of companies (e.g., small service firm).

The data-collection approaches in social sciences can be classified in many different ways (e.g., Flynn et al. 1990; Singleton et al. 1993). Furthermore, it is possible that two different terms are used to describe the same approach (e.g., field research versus case study). Similarly, there might be a slight overlap between two data collection approaches.

We consider it appropriate to classify the data-collection approaches for empirical research in operations management into the following categories: qualitative research, case study, field research, laboratory research, archival research, and surveys. The qualitative approach consists of interviews, focus groups, and various other forms of non-quantitative data collected from subjects of a study (e.g., managers, customers, and employees). For example, Dube, Johnson, and Renaghan (1999) used a qualitative approach in their study of quality function deployment (QFD) for services; and Yeung, Cheng, and Lai (2005) used both qualitative and quantitative data for their work on quality management.

The case-study approach consists of data collection from one or more organizations (e.g., plants, business units, and companies) over extended periods of time. Case study data often include both qualitative and quantitative components and responses from more than one individual, work groups, or departments (Eisenhardt 1989). For example, Dostaler (2001) used the case-study approach to collect data from two British manufacturers to study cumulative versus synergies theories of manufacturing performance and Chinander (2001) assessed the internal drivers of firms' environmental awareness using a case study.

A field study requires one or more visits to the operations facilities for data collection, but unlike a case study, it does not necessarily involve extended periods of interaction (Cook and Campbell 1979; Bouchard 1976). Examples include Carrillo's (2005) study of industry clockspeed and new product development and the work of Fisher et al. (1997) on supply chain uncertainty.

While there is a potential for some overlap between the three data-collection approaches (qualitative, case study, field research) described above, they are not three different names for the same methodology. For example, while a case study to examine the scheduling policies in a manufacturing facility can be very quantitative, exploration of operations strategy in the same plant might require qualitative research. Similarly, qualitative research can be part of a case study but is not necessary because a researcher might interview managers in several different firms without the research being considered a case study. Similar overlaps between field studies and the other two approaches can also be identified.

Sometimes it is not possible to collect field or case study data for various reasons like cost, time, or other constraints (e.g., collecting data from pervious time periods). In such cases, researchers will often use archival data. This approach involves compiling data from existing sources of information such as government databases, financial reports, and consumer reports. Marshall Fisher, in his keynote address at the 2005 conference of the Production and Operations Management Society, strongly advocated this approach and cited many examples from other disciplines such as medicine, engineering, and finance on the usefulness of the archival-data-collection approach. POM has published a number of articles based on the archival-data-collection approach. These articles include Anderson, Daly, and Johnson (1999) which evaluated the impact of ISO-9000 on firm performance, and Kleindorfer and Saad (2005) and Hendricks and Singhal (2005) analyzed disruption risks in supply chains.

Some research questions cannot be addressed by existing sources of data, and for a variety of reasons, it may not be possible to conduct a case or field study. In such situations, researchers often collect primary data in a laboratory setting or via surveys. In a laboratory setting, researchers collect primary data from experiments conducted in a controlled environment (Campbell and Cook 1973). Laboratory experiments can also be used to identify causation (e.g., Fromkin and Streufert 1976). Examples of laboratory experiments include Mullens et al. (2005) and Croson and Donohue (2003). The applications of laboratory research in the operations management discipline is scarce as compared to other social science disciplines such as organizational behavior perhaps because of the complexities surrounding "real" operations or because of tradition. However, we do observe a large number of studies which use mail, phone, or internet surveys to collect primary data from subjects (managers, employees, or customers) using pre-structured questionnaires. For example, Vickery, Dorge, and Markland (1994) studied manufacturing competence and Rothenberg, Pil, and Maxwell (2001) examined the relationship between lean manufacturing practices and environmental performance using surveys. Surveys can also be used to conduct experiments for a broad audience using techniques such as conjoint or discrete choice analyses (e.g., Goodale, Verma, and Pullman 2003).

2.3. Analysis Techniques

The diversity of research focus and data-collection approaches implies that an even wider range of techniques would be used during the analysis phase. Therefore, we only describe broad categories of analysis approaches commonly used in operations management studies without the details of each individual technique.

The most common technique is descriptive statistics with such measures as mean, median, standard deviation, and frequency distribution which provide a broad description of the data (e.g., Stewart and Chase 1999; Upton and McAfee 1998). Bi-variate correlations (e.g., Melnyk, Sroufe, and Calantone 2003; Boyer and Lewis 2002), ANOVA (e.g., Lefebvre et al. 1992; Kathuria and Davis 2001), t-tests (e.g., Crandall and Markland 1996; Pagell and Handfield 2000), and chi-Square tests (e.g., Klassen, 2001; Anand and Ward 2004) have been used to explore the relationship between two sets of variables. Many operations problems require demonstrating (or testing) the association between two or more variables. Therefore, analysis techniques such as linear regression (e.g., Hays and Hill 2001; Klassen and Vachon 2003) and logistic regression (Keizers, Bertrand, and Wessels 2003; Lapre and Scudder 2004) have been used when testing for association. When the objective of research is to explore the association between several sets of variables in a complex structure, structural equation modeling and related analysis approaches are used (e.g., Ittner and MacDuffie 1995; Gupta and Somers 1996).

In our analysis of empirical research papers published in POM, we observed the use of a wide range of multivariate analysis techniques. These techniques include discriminant analysis (e.g., McDougall, Deane, and D'Souza, 1992; Kerkhoff, Eagar, and Utterback 1998), factor analysis (e.g., Sakakibara, Flynn, and Schroeder 1993; Dow, Samson, and Ford 1999), MANOVA (e.g., Tsikriktsis, Lanzolla, and Frohlich 2004; Nie and Kellogg 1999), MANACOVA (e.g., Karuppan and Schniederjans 1995), cluster analysis (e.g., Safizadeh, Ritzman, and Mallick 2000; Heim and Sinha 2002), and time series analysis (e.g., Alwan and Radson 1995; Inman and Leon 1993). We also observed that several authors used analytical approaches such as mathematical modeling (e.g., Sampson 2004; Goodale, Verma, and Pullman 2003) and simulation (e.g., Easton, Rossin, and Borders 1992; Malhotra and Ritzman 1994) for analysis of empirical data.

From the brief review of the articles presented above, it is clear that POM has attracted a diverse range of articles based on empirical research methodologies. The next section presents a systematic analysis of the empirical articles published from 1992 to 2005.

3. Research Methodology

We review the empirical research published in the Production and Operations Management during 1992 to 2005, which includes 56 issues in 14 volumes of the journal. We reviewed these journal issues consisting of a total of 399 articles, excluding editorials and introductions, to identify papers that completely or partially use empirical research methodologies. We classified articles as belonging to the Empirical Research category if it contained any form of "real" data. The papers not classified as empirical were divided into the following categories: Modeling and Analytical Methodologies- if the article is based only on mathematical derivations and/or simulated/created datasets; Conceptual and General-if the article does not include any data and is primarily based on logic and discussion of theoretical frameworks; Survey and Review-if the article is a review of the discipline, research topic(s), or methodology; and POM Education-if the article deals with issues related to teaching of operations management.

Out of the 399 articles we reviewed, a total of 153 papers fell into the empirical research category and the remaining were classified as modeling, conceptual, or education-related articles. Table 1 gives the percentages of papers in the various categories. Figure 1 shows the three-year moving average of the percentages of articles published in POM based on empirical research.

Table 1 and Figure 1 demonstrate that a signifi- cant proportion of articles published in POM are based on empirical research. Figure 1 shows an increasing trend in the percentage of articles based on empirical data.

After identifying the articles belonging to the empirical research category, we classified the articles based on several different criteria. While we did not formally calculate multiple-rater agreement indices (e.g., Boyer and Verma 2000), there was a high degree of consensus among the members of the research team. Therefore, there should be minimum impact of subjective assessment of an individual article on the overall findings of this research. Each article was classified according to the following criteria:

I. Manufacturing and/or service focus.

II. Operations management function, topic, or theme such as strategy, quality, and supply chain management.

III. Primary purpose (theory building, theory verifying, providing evidence, and application).

IV. Data-collection approach (qualitative research, case study, field research, laboratory research, archival reserach, and surveys).

V. Analysis approach used.

Given that research in operations management takes many different forms depending on the topic covered and the nature of the research question, we observed that several articles met more than one criteria within each category listed above. For example, it is possible that an article could be based on more than one data-collection approach and might also use more than one analysis approach.

Table 2 shows the breakdown of empirical articles published in POM during 1992 to 2005 into manufacturing and service sectors. Since the total number of published articles fluctuates from year to year, we have presented a three-year moving average of the percentage of manufacturing and service focused empirical research articles as a percentage of total empirical- research-based articles in Figure 2. The figure shows that the gap between the focus of articles within the two broad industrial segments narrowed steadily during the recent years, and it has now almost disappeared. We consider this result to be a very encouraging trend.

The empirical papers published in Production and Operations Management covered a variety of operations management functions, topics and themes. However we have grouped them under broad topical themes of operations management. The results are presented in Figure 3. The "other topics" category includes topics such as human resources and project management, each of which contributed a small percentage of articles.

Figure 4 shows the research purpose in four categories: theory building, theory verifying, application, and providing evidence. Theory verifying was the major purpose of research and accounted for 39.5% of the published papers followed by an equal emphasis for applications and providing evidence papers (24.8% each). Theory building scored the lowest rank in this category. It is interesting to note that within manufacturing-focused papers, the highest emphasis was on theory verifying (44.5%) and within service-focused papers, applications were most common (36.1%). Table 3 lists all 153 articles analyzed according to the four research purpose categories (theory building, theory verifying, application, and providing evidence).

Figure 5 shows the classification of all empiricalresearch- based articles based on the data-collection approaches used. A breakdown of these papers by manufacturing and service sectors is also included in this figure. About one-third of the published articles have used survey research, which is the most frequently used data-collection approach. Survey research is closely followed by field research (22%), which is then followed by archival research (15%). Case study and qualitative research collectively account for about 32% of the approaches. However, laboratory research has been used very rarely (about 2%) in published research.

We also classified the articles based on the dataanalysis approaches and techniques. We identified 16 different techniques that were used for data analysis. It was interesting to note that a majority of articles used more than one technique in the research process. A total of 559 incidences of using one of the 16 techniques within 153 papers were observed which means an average of 3.73 technique usage per paper. As shown in Figure 6, the range of techniques varied from simple descriptive analysis to a range of multivariate approaches (e.g., multiple linear regression, factor analysis, and structural equation modeling) and analytical approaches (mathematical modeling and simulation). Figure 6 shows the relative importance of the techniques used when measured as a percentage of the total incidences (559).

3.1. Cross-Tabulations

To explore the interrelationships amongst various facets of empirical research, we developed cross-tabulations presented in Tables 4 through 6. Tables 4-A and 4-B show the relationship between the purpose of the paper and the data-collection approach. In Table 4-A the column percentages add up to 100% whereas the row percentages add up to 100% in Table 4-B. As shown in Table 4-A, there appears to be a clear preference for the type of data-collection approach used for different study purposes. For example, case studies are primarily used for providing evidence (46.4%) followed by applications (32.1%). Qualitative research is primarily used for theory verifying (52.9%) followed by providing evidence (20.6%). Archival research shows a usage pattern similar to qualitative research-theory verifying (59.3%) and providing evidence (22.2%). Survey research is used most frequently for theory verification (64.8) followed by theory building (16.7%). Laboratory Research was found to be used for application (66.7%) and for providing evidence (33.3%). Field Research is primarily used for application (41.9%) and for providing evidence (34.9%).

The row percentages in Table 4-B show the relative usage of data-collection approaches for various purposes. The two major approaches that have been used for theory building are survey research (40.9%) and qualitative research (27.3%). Case study, archival research, and field research together account for 31.8%. Laboratory research has not been used for theory building. For theory verifying, again, the primary approach used was survey research (43.8%), followed by qualitative research (22.5%) and archival research (20.0%). Case study and field research together accounted for 13.8%. The use of laboratory research for theory verification was not reported in any paper. Field research appears to be the approach of choice for application (47.4%), followed by case studies (23.7%). Qualitative research, archival research, and survey research each accounted for 7.9%, followed by laboratory research at 5.3%. Field research (30.6%) and case study (26.5%) were the approaches of choice for providing evidence, followed by qualitative research and survey research at 14.3% each. Archival research accounts for 12.2%, followed by a low 2% for laboratory research.

Table 5 shows the cross-tabulation of research purpose with the data analysis approach used. Descriptive statistics is the most commonly used method for analyzing data for theory building (24.5%), followed by correlation (17.0%), factor analysis (13.2%), and linear regression (11.3%). Descriptive statistics is again the dominant method of analyzing data for theory verification (24%), followed by correlation (19.9%), linear regression (13.5%), and factor analysis (11.1%). Mathematical modeling at 72.9% is the favorite method of data analysis for applications. For papers which attempt to provide evidence, descriptive statistics scored 39.2%, followed by linear regression (13.7%), and modeling (11.8%).

Table 6 shows the cross-tabulation of the data-collection approach and the analysis method. For case study, modeling (30.2%), descriptive statistics (18.6%), and linear regression (14%) account for 62.6% of the techniques used. For qualitative research, only four techniques account for 72.0% of usage (descriptive statistics 25.3%, correlation 18.7%, linear regression 16.0%, and factor analysis 12.0%). Archival research primarily uses descriptive statistics (30.0%), correlation (13.3%), and linear regression (11.7%). These three techniques account for 55% of the usage. Four major techniques used in survey research account for 66.2% of usage. These include descriptive statistics (23.7%), correlation (20.1%), factor analysis (12.4%), and linear regression (10.1%). Researchers used modeling (40%), regression (20%), ANOVA (20%), and simulation (20%) for laboratory research. The dominant techniques for field research are modeling (37.0%), descriptive statistics (25.9%), and linear regression (13.0%) for a total of 75.9% usage.

4. Conclusion and Future Directions

Our conclusions are based on the analysis of 153 papers that were classified as empirical papers published in Production and Operations Management from 1992 to 2005. Overall, we observed that there is an increasing trend in the number of empirical-research-based articles published in POM over the last decade. During the fourteen-year period we covered in our analysis, articles based on empirical data have increased substantially from 30 to 50 percent of the published articles. This trend suggests that the operations management community enthusiastically responded to the mission of the journal by pursuing empirical methods in their research projects.

Manufacturing vs. services. While Karmarkar (2004) noted that service jobs currently account for over 80 percent of private-sector jobs in the United States, Pannirselvam et al. (1999) reviewed 1,754 articles published between 1992 and 1997 in seven journals with a "true" operations management focus and found that those pertaining to services accounted for only 2.71 percent. Although during 1992 to 1998, about three-fourths of the empirical-research-based articles published in POM focused on the manufacturing industry, the gap between the numbers of manufacturing- and service-focused articles published in POM has almost disappeared. POM has clearly taken a major leap in publishing empirical research in the rapidly growing area of service operations.

Research topics and interdisciplinary and interorganizational research. Until the mid-1990s, empirical research articles focused on a limited number of operations management functions, topics and themes. Most popular research topics include strategy, quality, supply chain, environmental issues, and product and service design. There is potential for pursuing empirical research on the interface between operations management and other areas such as accounting, finance, human resources management, information systems, and marketing (Buhman, Kekre, and Singhal 2005). Clearly, with the rapid proliferation of information technology, expansion of the service sector, and growth in global supply chain networks, interdisciplinary and interorganizational research is necessary for analyzing real-world operations management problems. Empirical research methodologies could also be applied to more traditional topics such as project management, location planning, layout planning, and assembly line balancing. Such studies could be used to confirm if the conventional wisdom gained from traditional modeling approaches is still valid or if they need adjustments.

Purposes of empirical research. We observed that among the four purposes of empirical research (theory building, theory verifying, application and providing evidence), theory verifying was the major purpose of empirical research. There are some noticeable differences between the purposes of empirical research for manufacturing and service sectors papers. Papers in the manufacturing sector report more on theory veri- fication whereas those in the service sector focus more on applications. This result might be due to the fact that historically problems within manufacturing industries have been studied more extensively than in the service sector. Therefore, during the time period studied, there was an abundance of manufacturing theories and frameworks which were tested by researchers. We also observed that empirical research in approximately one-fourth of the articles was conducted to validate analytical research. This result reinforces our conclusion that the operations management community has made a serious attempt during the last decade to pursue research based on real-world data.

Data-collection methodologies. A variety of datacollection methodologies were used in empirical research that included case study, qualitative, archival, survey-based, laboratory, and field research. The primary purpose of research drives the choice of the data-collection approach. Survey-based research was used most widely followed by field-based data collection, while laboratory research was used in rare cases. Therefore, we believe that there is a great potential for using laboratory research in operations management. Other disciplines, such as organizational behavior, marketing, and the physical and medical sciences often use laboratory research to test hypotheses in controlled environments.

Lack of triangulation. We also noticed that only a small number of papers were based on multiple datacollection approaches, pointing to a lack of triangulation. As empirical research matures within the operations management field, we think it is desirable to see more papers based on multiple data-collection approaches. Furthermore, we believe that the combined use of analytical and empirical research techniques should be further encouraged because it has the potential of offering greater insights into the issues being studied.

Methods of analysis. We also found that the choice of the method of analysis is influenced by the purpose of the study and the data-collection approach. We noticed that descriptive analysis approaches were used more frequently compared to advanced multivariate statistical techniques. This result might be related to the "newness" of empirical research within the operations management discipline. Nevertheless, we encourage researchers to use more advanced multivariate techniques so that we increase triangulation and draw stronger conclusions from empirical studies.

The empirical research published in Production and Operations Management has covered a rich diversity of articles in terms of the purpose of research, data collection approaches, and data analysis techniques. While Scudder and Hill (1998) assessed articles published in a range of operations management journals and found that almost all of the empirical articles were based on either surveys (61%) or case studies (35%), our results indicate that POM has published articles that were based on a much wider and more diverse range of data-collection approaches. Production and Operations Management has clearly established itself as a leading outlet for publishing empirical research in operations management.

The future is full of promise with new areas like global manufacturing and service networks, dynamic pricing and revenue management driven by the digital economy, new technologies like RFID, closed-loop supply chains and sustainable operations, and the increasing role of services in the economy. Production and Operations Management has just created two new editorial departments, one on behavioral issues in operations management to promote laboratory research, and the other on revenue management. The community has a wide range of opportunities to pursue traditional as well as new approaches for empirical research.

Acknowledgements

We want to thank four anonymous reviewers and a senior editor for their insightful comments on earlier drafts of the paper.