OR/MS Today - December 2002



The Paper Chase


The Paper Chase

Comparing the research productivity of quantitative/technical departments in schools of business

By Ravi Bapna and James R. Marsden


During the last two decades, as information systems/information technology developed in importance within business schools, a variety of department structures emerged. While in some cases, a new focused department was created to house IS/IT, a frequent tactic was to co-locate IS/IT with management science, operations management or a combination of the two. One argument for doing so was linked to the view that research in these areas was growing more and more integrated. Colloquially, such departments are also referred to as quantitative/IS departments, operations/IS departments or, in some instances, as technical/IS departments.

Given the emergence of these departments, we felt it was appropriate to ask how they are performing in the research arena. To address this question, we present a summary analysis comparing the research productivity of such departments in terms of publication activity in four key INFORMS journals that span the topics and frequently publish multi-disciplinary research from these areas. We chose the four INFORMS journals because of their prestige and because INFORMS, through its conferences and publications, represents high quality in quantitative/technical research. Our presentation is meant as an addition to available information and not as an exclusive or "ultimate" measure of research productivity.

To many of us, university administrators and external constituents seem overly interested in analytical comparisons. Yet we academics are hardly immune from this curiosity. We state our specific choice of "who," "what" and "over what period" to compare as follows:

WHAT TO COMPARE: publication productivity in four key INFORMS publications — Management Science, Information Systems Research, Operations Research and INFORMS Journal on Computing. These journals, with the exception of the INFORMS Journal on Computing, are part of the very limited and exclusive list used by BusinessWeek in evaluating the ranking of the best business schools.

WHO TO COMPARE: universities, business schools and departments combining information systems, management science and/or operations management.

OVER WHAT PERIOD TO COMPARE: we analyze publications in the four journals from 1990 to the present, a period covering or briefly following the formal establishment of the type of departments studied.
We begin by defining the variables "operationalized" for data collection. This is followed by a summary of our data collection activities and how we dealt with anomalies or data conflicts that arose. Data summaries are provided in a series of tables that include, for comparison and completeness, information of research productivity since 1990 by university, school/college and departments falling into our selected set.

Variables Utilized


For our purposes, the information that is required is the author's name, department/group and university/organization. For sake of completeness, we also recorded citation information such as article title, volume, issue, date and page numbers. Given that a vast majority of articles are co-authored (74.5 percent in our data set, shown later), we primarily base our comparisons on a weighting scheme that counts each article as an entity contributing one point to the pool. This point is then split up evenly across the number of authors, while also giving proportionate weighting to their departments. For instance, an article in Management Science that had three authors, two of which were from department A and another from department B, would credit department A with two-thirds of a point and B with one-third of a point.

Data Collection


In order to compare department productivity, we analyzed authorship of articles appearing in the four relevant flagship INFORMS journals (MS, OR, ISR, IJOC) from 1990 to the present. We created a normalized relational database in Microsoft Access that would allow us to record the complete affiliation details of all the authors of all the articles published in this time frame. However, in many instances information about department, school and university was only partially available. In several cases, only the university affiliation was presented. These inconsistencies were present across the four journals and over the years. We pursued several additional information gathering steps, detailed below, in order to resolve these inconsistencies:

Incomplete information resolution. The primary source of anomaly in the data is the lack of full affiliation information on the first page of each article. We considered the information to be full if it had three levels of information roughly corresponding to department, school and university. These levels may or may not exist, depending on the organizational structure of the particular university. If the department information for a particular author was missing, we deployed the following steps:

a) We looked to see if the author already existed in our database from a prior publication, and crosschecked to ensure that the author had the same university affiliation. If this was true then we considered it safe to assume that the person's department affiliation is the same.

b) If step (a) failed, we checked the school's Web site to determine whether it had distinct departments or academic groups and if the author belonged to these.

c) If steps (a) and (b) failed, we attempted to contact the author directly through e-mail.

d) If all three initial steps did not yield the information, we searched for the author's name using Google and tried to reach the author by telephone.

After all these steps, if we were able to determine that a particular school, such as the Sloan School of Management at MIT, did not organize themselves into such categories, we excluded it from the department level comparisons but included it in the school and university comparisons.

Identifying the comparison set of "IS/IT/Management Science" Departments. Given the broad appeal of journals such as Management Science, we wanted to narrow our comparison set by identifying departments that had an IS/IT and OR/MS focus and belonged to a business school. We did not want to compare such departments with, for instance, pure OR departments that exist in many mathematics programs or industrial engineering programs. In several cases, such a categorization is explicit in the department's name, such as the Operations and Information Management department at The Wharton School (University of Pennsylvania) and at University of Connecticut's School of Business. In other cases, we made the determination by visiting and examining departmental Web sites. For instance, the University of Washington Business School's Web site lists a Management Science Department, but indicates that it includes information systems, operations management and quantitative methods. Thus, the department comparison represents the relative productivity of those academic groups that have chosen to organize themselves into quantitative IS groups within business schools. As mentioned earlier, if such a departmental organization is not explicit or implicit, then we consider those authors in the school and university comparisons.

In summary, we constructed a database consisting of 3,338 articles published by 3,971 different authors belonging to 1,692 uniquely identifiable academic units. The comparison information derived from querying this database is presented below.

Presentation of Results


Joint authorship. The co-authorship distribution is presented in Chart 1.The unimodal distribution peaks at two and indicates that while co-authorship is widely prevalent, the vast majority of articles have, at most, three authors. Given the extensive level of co-authorship we also decided to investigate, within the departments that were productive, whether some departments had higher levels of intra-departmental collaboration than others. This was done using an absolute comparison scheme, the results of which are presented after the primary weighted comparisons.



Chart 1: Three-quarters of articles are co-authored.

Department comparisons (weighted measure). Table 1 presents the 36 departments achieving the highest total weighted measure linked to publishing in the four IS/IT related flagship INFORMS journals over the period from 1990 to the present. A bold line segregates the top 10 most productive performers from the rest.

Rank Weighted
Score
Deptartment Name School University
1 50.91 Operations and Information Management The Wharton School University of Pennsylvania
2 39.90 Management Science and Information Systems Graduate School of Business University of Texas at Austin
3 31.83 Operations Information and Technology Graduate School of Business Stanford University
4 21.58 Decision and Information Technologies Robert H. Smith School of Business University of Maryland
5 17.28 Management Sciences Graduate School of Business Administration University of Washington
6 17.17 Information and Operations Management Marshall School of Business University of Southern California
7 12.92 Operations and Informations Management School of Business University of Connecticut
8 12.67 Management Information Systems Commerce and Business Administration University of British Columbia
9 12.45 Management Information Systems K E Graduate School of Management University of Arizona
10 11.00 MIS Owen Graduate School of Management Vanderbilt University
11 10.83 Management Science and Information Systems School of Management University of Texas at Dallas
  10.83 Decision and Information Systems College of Business Arizona State University
13 10.43 Management Science Smeal College of Business Administration Pennsylvania State University
14 9.83 Decision Sciences and Information Systems College of Business Administration Florida International University
15 9.25 Information and Decision Sciences College of Business University of Illinois at Chicago
16 9.08 Management Science and Operations Management W E Simon Graduate School of Business Administration University of Rochester
17 7.42 Decision Sciences and Engineering Systems School of Management Rensselaer Polytechnic Institute
18 7.25 Decision Sciences Dept School of Business Administration National University of Singapore
19 6.83 Decision and Information Sciences College of Business Administration University of Florida
20 6.50 Information and Management Sciences College of Business Florida State University
21 6.33 Business Informations Systems School of Business and Management Hong Kong Univ. of Science and Tech.
22 6.25 Management Info Systems Dept Terry College of Business University of Georgia
23 6.17 Dept of Info Systems Weatherhead School of Management Case Western Reserve University
24 6.03 Management Science and Systems School of Management SUNY - Buffalo
25 5.95 Management Science College of Business Administration Northeastern University
26 5.75 Informations Systems L N Stern School of Business New York University
27 5.57 Decision and Information Sciences College of Business Administration University of Houston
28 5.42 Information and Decision Sciences Carlson School of Management University of Minnesota
29 4.53 Department of Computer Information System College of Business Admin Georgia State University
30 4.50 Management Information Systems School of Management Boston University
31 4.33 Information Systems and Decision Sciences College of Administration and Business Louisiana State University
32 3.85 Management Science College of Business Administration University of South Carolina
33 3.67 Quantitative Methods and MIS Department College of Business Administration University of Tulsa
34 3.58 Decision and Information Systems Kelley Graduate School of Business Indiana University
  3.58 Department of MS & IS School of Business Administration and Economics California State University-Fullerton
36 3.50 Department of Commerce Business School University of Queensland

Table 1: IS/IT/OR department comparisons (1990-2002).

It should be emphasized that Table 1 comparisons relate only to those academic units that have chosen, either explicitly or implicitly, to organize themselves as departments or groups. Notable highly productive research houses such as MIT's Sloan School and CRITO at the University of California-Irvine do not come under this umbrella, but figure prominently in the Business School comparisons shown below. [At Arizona State, the IS group was recently shifted into the Accounting Department but, for the majority of our time period, the departmental structure fit the grouping analyzed here. We included relevant publication from both Accounting and IS and Decision and IS Departments.]

In addition, we ran the queries for the first (1990-1996) and second half of the period, (1997-present) in order to see if productivity shifts had occurred. These results are reported in Tables 1a and 1b.

Rank Weighted
Score
Deptartment Name School University
1 28.37 Management Science and Information Systems Graduate School of Business University of Texas at Austin
2 23.92 Operations and Information Management The Wharton School University of Pennsylvania
3 16.67 Operations Information and Technology Graduate School of Business Stanford University
4 10.33 Decision and Information Systems College of Business Arizona State University
5 9.00 Information and Operations Management Marshall School of Business University of Southern California
6 8.50 Management Information Systems K E Graduate School of Management University of Arizona
7 7.83 Decision Sciences and Information Systems College of Business Administration Florida International University
8 7.83 Decision and Information Technologies Robert H. Smith School of Business University of Maryland
9 7.67 Management Information Systems Commerce and Business Administration University of British Columbia
10 7.50 MIS Owen Graduate School of Management Vanderbilt University
11 6.42 Decision Sciences and Engineering Systems School of Management Rensselaer Polytechnic Institute
  6.42 Information and Decision Sciences College of Business University of Illinois at Chicago
13 5.67 Operations and Informations Management School of Business University of Connecticut
14 5.60 Management Science Smeal College of Business Administration Pennsylvania State University
15 5.57 Decision and Information Sciences College of Business Administration University of Houston
16 5.50 Decision and Information Sciences College of Business Administration University of Florida
17 5.08 Management Science and Operations Management W E Simon Graduate School of Business Administration University of Rochester
18 4.75 Management Sciences Graduate School of Business Administration University of Washington
19 4.67 Information and Management Sciences College of Business Florida State University
20 4.50 Dept of Info Systems Weatherhead School of Management Case Western Reserve University
21 4.25 Informations Systems L N Stern School of Business New York University
22 4.00 Management Information Systems School of Management Boston University
23 3.78 Management Science and Systems School of Management SUNY - Buffalo
24 3.50 Management Science College of Business Administration Northeastern University
25 3.42 Information and Decision Sciences Carlson School of Management University of Minnesota
26 3.00 Department of Commerce Business School University of Queensland
  3.00 Information Systems J E Anderson Graduate School of Management University of California - L.A.
28 2.50 Management Info Systems Dept Terry College of Business University of Georgia
  2.50 Operations and Strategic Management Wallace E Carroll School of Management Boston College
30 2.33 Department of MS & IS School of Business Administration and Economics California State University-Fullerton
  2.33 Management Science College of Business Administration University of Tennessee
32 2.25 Decision Sciences Dept School of Business Administration National University of Singapore
  2.25 Department of Management Science School of Business Administration University of Miami
34 2.17 Information Systems and Decision Sciences College of Administration and Business Louisiana State University
35 2.08 Information Systems and Operations Management College of Business Administration University of Toledo
36 1.83 Management Science and Information Systems Faculty of Business McMaster University

Table 1a: IS/IT/OR department comparisons (1990-1996).

Rank Weighted
Score
Deptartment Name School University
1 27.00 Operations and Information Management The Wharton School University of Pennsylvania
2 15.17 Operations Information and Technology Graduate School of Business Stanford University
3 13.75 Decision and Information Technologies Robert H. Smith School of Business University of Maryland
4 12.53 Management Sciences Graduate School of Business Administration University of Washington
5 11.53 Management Science and Information Systems Graduate School of Business University of Texas at Austin
6 9.75 Management Science and Information Systems School of Management University of Texas at Dallas
7 8.17 Information and Operations Management Marshall School of Business University of Southern California
8 7.25 Operations and Informations Management School of Business University of Connecticut
9 5.00 Decision Sciences Dept School of Business Administration National University of Singapore
  5.00 Management Information Systems Commerce and Business Administration University of British Columbia
11 4.83 Management Science Smeal College of Business Administration Pennsylvania State University
12 4.50 Business Informations Systems School of Business and Management Hong Kong Univ. of Science and Tech.
13 4.00 Management Science and Operations Management W E Simon Graduate School of Business Administration University of Rochester
14 3.95 Management Information Systems K E Graduate School of Management University of Arizona
15 3.75 Management Info Systems Dept Terry College of Business University of Georgia
16 3.50 MIS Owen Graduate School of Management Vanderbilt University
17 3.42 Computer and Information Systems Dept Business School University of Michigan
18 2.83 Information and Decision Sciences College of Business University of Illinois at Chicago
19 2.78 Department of Computer Information System College of Business Admin Georgia State University
20 2.77 Management Science College of Business Administration University of South Carolina
21 2.67 Quantitative Methods and MIS Department College of Business Administration University of Tulsa
22 2.45 Management Science College of Business Administration Northeastern University
23 2.33 Decision and Information Systems Kelley Graduate School of Business Indiana University
24 2.25 Management Science and Systems School of Management SUNY - Buffalo
25 2.17 Information Systems and Decision Sciences College of Administration and Business Louisiana State University
26 2.00 Information and Decision Sciences Carlson School of Management University of Minnesota
  2.00 Decision Sciences and Information Systems College of Business Administration Florida International University
28 1.83 Information and Management Sciences College of Business Florida State University
29 1.67 Management Department College of Business Administration University of Notre Dame
  1.67 Dept of Info Systems Weatherhead School of Management Case Western Reserve University
31 1.58 Information Systems and Decision Sciences College of Business Administration University of South Florida
32 1.50 Informations Systems L N Stern School of Business New York University
33 1.33 Decision and Information Sciences College of Business Administration University of Florida
34 1.25 Department of MS & IS School of Business Administration and Economics California State University-Fullerton
35 1.17 Management Information Science Cox School of Business Southern Methodist University
36 1.08 Management Science and Information Systems School of Business SUNY - Albany

Table 1b: IS/IT/OR department comparisons (1997-present).

Observe that several departments made large steps forward in the productivity measure comparisons during the last six years. Several jumped into the top 10, including Management Science and Information Systems at the University of Texas at Dallas (from 43rd to sixth), Operations and Information Management at the University of Connecticut (from 13th to eighth), and Decision Sciences Department at the National University of Singapore (from 32nd to ninth).

Business school comparisons (weighted measure). Table 2 presents the top 36 most productive business schools publishing in the four IS/IT-related flagship INFORMS journals. These include entries from departments such as accounting, finance, etc. MIT's Sloan School of Management outshines all others in this comparison.

Rank Weighted
Score
Deptartment Name University
1 91.40 Sloan School of Management Massachusetts Institute of Technology
2 63.58 The Wharton School University of Pennsylvania
3 52.92 J E Anderson Graduate School of Management University of California - L.A.
4 52.67 Graduate School of Industrial Administration Carnegie Mellon University
5 50.23 Graduate School of Business University of Texas at Austin
6 47.92 Fuqua School of Business Duke University
7 35.75 Business School University of Michigan
8 31.83 Graduate School of Management University of California - Irvine
9 31.28 Graduate School of Business Stanford University
10 29.42 Graduate School of Business Columbia University
11 27.92 Robert H. Smith School of Business University of Maryland
12 27.78 Graduate School of Business Administration University of Washington
13 26.78 Harvard Business School Harvard University
14 26.65 Marshall School of Business University of Southern California
15 25.75 Commerce and Business Administration University of British Columbia
16 25.32 Krannert Graduate School of Management Purdue University
17 25.28 L N Stern School of Business New York University
18 21.42 Carlson School of Management University of Minnesota
19 21.42 Fisher College of Business Ohio State University
20 19.20 Kenan Flagler Business School University of North Carolina - Chapel Hill
21 18.17 Kellogg School of Management Northwestern University
22 17.65 K E Graduate School of Management University of Arizona
23 17.12 W E Simon Graduate School of Business Administration University of Rochester
24 16.92 J M Katz Graduate School of Business University of Pittsburgh
25 16.25 Weatherhead School of Management Case Western Reserve University
26 15.92 School of Business Administration University of Michigan
27 15.17 Walter A Haas School of Business University of California - Berkeley
28 14.50 Owen Graduate School of Management Vanderbilt University
29 14.25 School of Management Georgia Institute of Technology
30 14.20 School of Management Erasmus University Rotterdam
31 14.00 College of Business University of Colorado at Boulder
32 13.83 Olin School of Business Washington University
33 13.50 School of Management Boston University
34 13.23 School of Management University of Texas at Dallas
35 12.92 School of Business University of Connecticut
36 12.67 College of Business Arizona State University

Table 2: Business school comparisons (1990-2002).

University comparison (weighted measure). Table 3 presents the top 36 most productive universities publishing in the four IS/IT-related flagship INFORMS journals. These include entries from departments such as accounting and finance, as well as applied math, industrial engineering, etc.

Rank Weighted
Score
University
1 107.80 Massachusetts Institute of Technology
2 98.66 Columbia University
3 89.17 University of Pennsylvania
4 77.90 Carnegie Mellon University
5 76.65 Stanford University
6 62.35 Georgia Institute of Technology
7 61.02 University of Texas at Austin
8 58.77 University of California - L.A.
9 54.67 University of Michigan
10 52.67 Duke University
11 47.58 Northwestern University
12 35.17 University of Maryland
13 33.65 Purdue University
14 33.33 University of California - Berkeley
15 32.83 University of California - Irvine
16 32.70 AT&T Bell Labs
17 31.53 University of Washington
18 30.67 Ohio State University
19 29.17 University of Minnesota
20 28.13 Cornell University
21 27.92 Yale University
22 27.45 Tel Aviv University
23 27.28 Harvard University
24 27.15 University of Southern California
25 26.28 New York University
26 26.25 University of British Columbia
27 26.03 University of Wisconsin - Madison
28 25.45 INSEAD
29 24.62 University of Chicago
30 24.17 Universite de Montreal
31 22.35 University of Arizona
32 19.87 University of North Carolina - Chapel Hill
33 19.75 University of Pittsburgh
34 18.55 Pennsylvania State University
35 18.25 Naval Postgraduate School
36 18.08 University of Bologna

Table 3: University comparisons (1990-2002).

Comparisons using absolute measures. The comparisons above use a weighting scheme to account for co-authorship. For completeness, we also report comparisons using an absolute comparison mechanism where each author gets a point for being listed as an author of a published article in our dataset. In case of co-authored papers, this implies that each of the co-authors would contribute a point to the pool. The absolute measures incorporate a bias in favor of the importance of co-authored articles. For example, an article with five co-authors is assigned five points in the process while a single author article is assigned only one point in the process. Table 4 presents the results.

Rank Weighted
Score
Deptartment Name School University
1 98 Operations and Information Management The Wharton School University of Pennsylvania
2 91 Management Science and Information Systems Graduate School of Business University of Texas at Austin
3 65 Operations Information and Technology Graduate School of Business Stanford University
4 42 Decision and Information Technologies Robert H. Smith School of Business University of Maryland
5 35 Management Sciences Graduate School of Business Administration University of Washington
6 30 Information and Operations Management Marshall School of Business University of Southern California
7 29 Operations and Informations Management School of Business University of Connecticut
8 27 Management Information Systems Commerce and Business Administration University of British Columbia
9 25 Management Information Systems K E Graduate School of Management University of Arizona
  25 Management Science Smeal College of Business Administration Pennsylvania State University
  25 Management Science and Information Systems School of Management University of Texas at Dallas
12 21 MIS Owen Graduate School of Management Vanderbilt University
13 19 Management Science and Operations Management W E Simon Graduate School of Business Administration University of Rochester
  19 Management Info Systems Dept Terry College of Business University of Georgia
15 18 Decision Sciences and Information Systems College of Business Administration Florida International University
16 17 Decision and Information Systems College of Business Arizona State University
  17 Information and Decision Sciences College of Business University of Illinois at Chicago
18 16 Management Science and Systems School of Management SUNY - Buffalo
19 15 Management Science College of Business Administration Northeastern University
20 14 Decision Sciences Dept School of Business Administration National University of Singapore
  14 Decision and Information Sciences College of Business Administration University of Florida
  14 Informations Systems L N Stern School of Business New York University
23 13 Information and Management Sciences College of Business Florida State University
  13 Information and Decision Sciences Carlson School of Management University of Minnesota
  13 Dept of Info Systems Weatherhead School of Management Case Western Reserve University
  13 Management Science College of Business Administration University of South Carolina
27 12 Business Informations Systems School of Business and Management Hong Kong University of Science and Technology
  12 Decision Sciences and Engineering Systems School of Management Rensselaer Polytechnic Institute
29 11 Decision and Information Sciences College of Business Administration University of Houston
  11 Information Systems and Decision Sciences College of Administration and Business Louisiana State University
31 10 Management Information Systems School of Management Boston University
  10 Department of Computer Information System College of Business Admin Georgia State University
33 9 Information Systems J E Anderson Graduate School of Management University of California - L.A.
34 8 Department of Commerce Business School University of Queensland
  8 Decision and Information Systems Kelley Graduate School of Business Indiana University
  8 Information Systems and Operations Management College of Business Administration University of Toledo
  8 Operations and Strategic Management Wallace E. Carroll School of Management Boston College

Table 4: IS/IT/OR department comparisons (absolute measure, 1990-2002).

These comparison results are remarkably similar to the weighted measure results presented in Table 1. The departments at Penn State and University of Texas-Dallas move into the top 10, while Vanderbilt drops out.

Concluding Remarks


As we noted in our introductory remarks, the results contained here are presented as information for discussion. They are not an attempt at an exclusive comparison process, but rather as an initial look at how a set of relatively new department structures are performing in research output. We hope that the information provided in this article is useful to the readers and provides some insights into the research performance of the departments studied.

We have developed an accompanying Web site that will provide quarterly comparison updates and will include limited querying capability for our database. While we have made every effort to ensure the accuracy of the data and the department classification, it is to be expected in a project of this magnitude that some anomalies persist. We hope to work with the universities on resolving these, and expect that the Web site will be particularly useful in this regard. We are including individual measures in the Web site's design with the hope that researchers will not only look themselves up, but will also update us of any inconsistencies in the recording of the information. We also would like to suggest that INFORMS adopt a consistent affiliation reporting policy for all of its journals.



Jim Marsden (jimm@sba.uconn.edu) is Shenkman Family Chair and Head, Department of Operations and Information Management at the University of Connecticut. His research work has appeared in Management Science, IEEE Transactions, American Economic Review, Journal of Economic Theory, Journal of Political Economy and numerous other outlets. Ravi Bapna (rbapna@sba.uconn.edu) is an assistant professor at Department of Operations and Information Management at the University of Connecticut. His research work has appeared in Management Science, Decision Sciences, Naval Research Logistics, CACM, and Decision Support Systems, among other outlets.





  • Table of Contents

  • OR/MS Today Home Page


    OR/MS Today copyright © 2003 by the Institute for Operations Research and the Management Sciences. All rights reserved.


    Lionheart Publishing, Inc.
    506 Roswell Rd., Suite 220, Marietta, GA 30060 USA
    Phone: 770-431-0867 | Fax: 770-432-6969
    E-mail: lpi@lionhrtpub.com
    URL: http://www.lionhrtpub.com


    Web Site © Copyright 2003 by Lionheart Publishing, Inc. All rights reserved.