Home Free Lab ReportsOwing to the global economic crisis

Owing to the global economic crisis

Owing to the global economic crisis, many corporations’ IT management is confronted with an adverse
environment. In these difficult times, budgets tend to be cut back and IT management is required
to focus on efficiently running the systems needed to execute or support business processes. In this
context, corporate executives often require their IT managers to compare their cost and services to external
reference points in order to generate information on their own IT’s performance. The industrialization
and commoditization of IT is a trend that supports this development (Gartner 2008). In order
to provide such information for comparison, IT managers need key performance indicators (KPIs) that
are relative to their own data over time (longitudinal studies), to data of others (cross-sectional studies),
or to both (panel studies). Especially comparisons between companies are beneficial to IT managers,
as such information enable them to capture and account for their competitive environment. In
their search for this kind of information many IT executives are following a benchmarking approach.
As an established approach for organizations seeking information beyond their organizational boundaries,
benchmarking has evolved from product reverse engineering (as in the much noted Xerox case
described by Jacobson and Hillkirk 1986) to capture almost every aspect relevant to corporate management:
from simple KPIs to complex strategies (Ahmed and Rafiq 1998). Our experience of working
with a strategic IT/IS benchmarking initiative’s participants, however, shows that benchmarking’s
sustainable effects are rather limited. Practitioners often find it difficult to relate studies’ results back
to their own organization: an observation that is also supported by recent studies (Braadbaart 2007;
Moffett et al. 2008). Some companies attribute these difficulties to the limited comparability of data
(Hinton et al. 2000). This seems contradictory, especially in the light of the above-mentioned strong
trend towards standardization and commoditization of IT in corporations.
Despite its popularity in practice, there is little scientific coverage of benchmarking as an approach to
generate data and information on IT and its management in corporations (Cragg 2002). This calls for a
more detailed investigation of approaches, analyses, and the application of benchmarking in a corporate
context. Especially strategic benchmarking needs more scrutiny. Although its importance is widely
accepted, many companies seem to regard benchmarking as a new approach. Moffet et al. (2008)
revealed that only 29% of all their study participants used benchmarking at a strategic level.
This paper addresses two research questions: (1) understanding what factors impact a successful strategic
IT/IS benchmarking, that offers sustainable benefits for an organization, and (2) determining
how these factors can be translated into a model for explaining the success of IT/IS benchmarking to
foster our understanding of the phenomenon at hand and to guide benchmarking efforts in practice.
To present our research, we structure this paper as follows: the next section briefly reviews the most
important theoretical concepts that inform our research. We then present our methodological approach
for answering the research questions. After introducing the case studies on how benchmarking impacts
the management of IT organizations, we synthesize the cases into a model that explains how benchmarking
works and how it is impacted. We then discuss our limitations and research contributions.
2 THEORETICAL BACKGROUND
2.1 Benchmarking
Benchmarking in general can be defined as a continuous search for, and application of, significantly
better practices that lead to superior competitive performance (Watson 1993). As an important prerequisite
for this, data needs to be captured by measuring meaningful KPIs which, in our case, cover all
ITM related domains (Alshawi et al. 2003) relevant in a specific context. However, in ITM, benchmarking
has mainly focused on products and services – particularly cost and other quantitative
18th European Conference on Information Systems Page 2 of 12
measures – and has only shifted towards processes in recent years. Benchmarking for strategy has not
yet been fully embraced within IT (Cragg 2002).
While there is no consistent theoretical determination of benchmarking as of yet (Moriarty and
Smallman 2009), there are many different classifications of benchmarking approaches. According to
Carpinetti and Melo (2002), benchmarking can be classified according to its object. Consequently,
they distinguish between product, process, and strategic benchmarking. Furthermore, referring to
Camp (1989), they classify benchmarking approaches according to the benchmarking partner, i.e. internal,
competitive, functional, and generic benchmarking. The latter three can be subsumed as external
benchmarking. A wide spectrum of methods is available for benchmarking (Francis and Holloway
2007). Although their approaches differ slightly, they have several fundamental characteristics in
common. Drew (1997) identified five key activities: (1) determine what to benchmark, (2) form a benchmarking
team, (3) identify benchmark partners, (4) collect and analyze benchmark data, and (5) take
action. This process makes benchmarking a tool well suited for capturing rich datasets and relevant
KPIs, even in strategic settings.
These data form the heart of any benchmarking approach. We believe that the various characteristics
of benchmarking approaches mentioned above influence which data are needed and where they come
from. A solid benchmarking setup that incorporates these characteristics will, in turn, lead to more
meaningful results, i.e. produce a higher quality of the data a benchmarking project produces.
2.2 IT Management
IT management (ITM) provides the context in which benchmarking is applied. It influences benchmarking
as a management tool for ITM since it has to cover its contents. In addition, we believe that
the ITM process impacts how benchmarking can be applied and how it supplies information.
When examining the ITM content, its different domains should be taken into consideration, as well as
the relations and interdependencies between these domains. Various researchers have tried to identify
the domains of ITM, i.e. the fields of action that a holistic approach to managing IT/IS in a corporate
context needs to cover. Examples of such domains range from the management of a company’s portfolio
of computer-based applications (Segars et al. 1998) and its IT infrastructure (Mocker and Teubner
2005) to organizational considerations (Boddy et al. 2005). Some authors (e.g., Riempp et al. 2008)
bring those domains together and outline their relationships. This helps IT decision makers to assure
their approach’s comprehensiveness and accounts for the effects that management decisions in one
domain may have on others. ITM-related performance measurement needs to capture all these domains
and especially their relations in order to assure a thorough assessment of an IT organization’s
strengths and weaknesses and to account for potential inter-dependencies.
Looking at the ITM process, the questions of where and how benchmarking can be used to support
ITM arise. Building on studies that examine the strategy process specific to strategic ITM (e.g.,
Galliers and Sutherland 1994), Müller et al. (2009a) argue that ITM should be able to address the entire
cycle of strategic management specific to IT, i.e. strategic planning, strategy implementation, and
strategic positioning. Regarding the latter, they argue that an analysis of KPIs, contextual, and competitive
information is an important foundation for ITM and suggest benchmarking as a good source of
data. Following Müller et al. (2009a), a benchmark project can therefore very well feed data into the
strategic positioning process if it does account for these additional factors. This will increase the fit of
a benchmarking instrument to a particular context and provide important context information that is
needed to transfer insights back to a company’s specific environment.