TMCnet News
Bridging the Risk Gap: The Failure of Risk Management in Information Systems Projects [Research Technology Management](Research Technology Management Via Acquire Media NewsEdge) Project managers' tendency to focus on risks that are familiar, measurable, and controllable leaves projects vulnerable to risks. OVERVIEW: New product development projects are highly risky technical undertakings. Organizations frequently seek to manage the risk involved using standard risk management procedures, knowing that a company that better manages risks is less vulnerable. Nevertheless, NPD projects continue to fail to meet expectations for delivery time, budget, and outcomes. In this paper, we explore reasons why, despite employing self-evidently correct risk management procedures, adversities occurred in 19 major information systems projects. Project managers focused on the familiar, the measurable, the favorable, the noncommittal, and the controllable while excluding other risks that significantly affected their project performance. We have characterized this tendency as a series of five lures that leave projects vulnerable to risks. KEYWORDS: New product development, Project management, Risk management It is well understood that new product development (NPD) projects are risky endeavors (Cerpa and Verner 2009; Smith 1999; Nelson 2007) and that the management of risk is a vital task for managers in these types of projects. A plethora of canonical risk management standards has emerged (see, for instance, Project Management Institute 2013), and these are promoted as being self-evidently correct and entirely sufficient: "following these procedures, it is implied, will produce effectively managed projects" (Williams 2005, 498). In most cases, the prescription for risk management consists of a standard operating procedure comprised of often me- chanically performed activities. Typically, these processes revolve around a three-stage process: forecasting individual risks, assessing their importance, and identifying an appro- priate response. First, managers identify uncertainties that may affect the project's ability to meet its objectives. Then they assess the likelihood that the uncertainties will become real, as well as the consequences should they actually emerge. Finally, the risk management procedure prompts managers to formulate a response to address each significant risk. Thus, the overarching premise of risk management is a reduction- ist, predictive analysis that takes large, complex problems and reduces them to smaller problems that can be managed in isolation; this reductionist approach works well where lin- earity holds sway, but such situations are increasingly rare. However, for all the talk about the need to coordinate a repetitive set of organizational activities to manage risk, and for all the quantitative mechanisms available to risk manag- ers, there is growing evidence that risk management is often ineffective. Such ineffectiveness is often attributed to factors such as lack of knowledge or inadequate integration of stakeholders into risk management activities (Nelson 2007; Hubbard 2009). Our research adds to the understanding of why risk management is ineffective by showing the rationale project managers use for not following accepted processes for managing risks. As a consequence of these rationales, risks remained unmanaged-either unidentified, unassessed, or un-responded to. To respond effectively to risks, companies need to bridge the gap between ideal risk management pro- cedures and the actual conditions on the ground in NPD projects. To do this, the rationales managers use to exclude risks from active management must be identified and ad- dressed. Our work offers one approach to this problem. The Study Our study set out to investigate the gap between risk man- agement prescriptions and actual practice. In particular, we studied whether, to what extent, and why managers of NPD projects may not follow established prescriptions for risk management, thus allowing their projects to remain vul- nerable to risks. We invited 11 global computer services providers to participate in the study; participants were se- lected for their global reach and the comprehensiveness of their product offerings. Within these 11 organizations, we identified 19 informa- tion systems projects that experienced one or more critical incidents characterized by radical deviations from project plans in spite of complying with risk management frame- works. For example, one of the projects was the develop- ment of a major information system for the German Stock Exchange, involving transformation from a client-server infrastructure to a terminal-based solution. Halfway through the project, a major software incompatibility issue threatened to suspend the entire project, with significant ramifications for both the service provider and client. We wondered how such major risks slipped through the risk management processes and cascaded into such a critical in- cidents. We identified one or two critical incidents in each project to be studied in detail, yielding a total of 26 critical incidents for study. The critical incidents were researched retrospectively, in a stepwise manner. The initial step involved developing an understanding of the critical incident. The key decision makers-the project manager and key engineers-were asked to provide an account of the incident, including such factors as potential impact and potential ramifications. After having established the context of the incident, we asked the respondents to identify the risks that led to the incident. In each case, respondents articulated 5-10 risks associated with the incident. We then asked each respondent to further examine each individual risk. First, we established whether the risk was knowable-that is, was relevant information needed to iden- tify the risk available ahead of the incident. Had the respon- dent experienced such a risk before? That is, did he or she possess knowledge about that risk? If the answer was that the respondent had no knowledge about the risk, then the questioning moved on to the next risk. If indeed the risk was knowable from the respondent's perspective, then the line of questioning followed the steps of risk management: Had the risk been identified, assessed, and responded to? At any stage of the process of identifying, assessing, and responding to risk, was inaction and the rationale for it explored? This cog- nitive mapping exercise with each respondent revealed a pattern of managed risks but also showed signs of a process breaking down, with some risks not being identified, as- sessed, and ultimately responded to. Where inaction was identified, the rationale for the inac- tion, or for excluding the risk from mitigation action, was coded and classified. This analysis yielded patterns that en- capsulated why knowable risk had not been prevented from triggering a critical incident. We call these patterns lures. The Lures and Deterrents in Risk Management The occurrence of critical incidents points to one of two conditions: either there is epistemic uncertainty-a lack of knowledge that a risk exists-that rules out any form of risk management, or the risk is knowable but decision makers fail to act upon their knowledge. A project team may simply lack the knowledge to identify risks and thus be unable to respond to them. However, if similar incidents happened often enough in the past, then the team might be able to determine the likelihood of their occurrence in the future. Even so, if team members do not act upon this knowledge, the risk remains active. We explored the knowability of each risk by asking re- spondents what knowledge they had about a specific risk. We allowed the respondent to classify each risk on a Likert scale, from 1=knowable to 5=unknowable. This classifica- tion process revealed that only 2 percent of the 208 risks identified as precipitating or contributing to the 26 critical incidents studied were unknowable-which means 98 per- cent of the risks were knowable. Most incidents, then, stemmed not from a lack of knowledge but from the way knowable risks were managed (or not). We then tried to identify where the breakdown occurred-at the identification stage, where the goal is to identify as many potential risks to the project as possible; at the assessment stage, where the goal is to determine the severity of each risk and select the most significant risks for active management; or at the response stage, where the goal is to plan an appropriate response to each risk identi- fied for active management. At each stage, we determined whether a form of identification, assessment of response had been carried out and, if not, why. The rationales at each stage, what we call lures, provide a compelling map of the cognitive traps that leave risks unresponded to, and projects vulnerable to avoidable adversity. The Identification Stage: The Lure of the Familiar The identification stage involves determining which risks may affect project performance. Project managers are re- quired to filter out from a wide range of possible risks the ones that are relevant and likely enough to merit additional attention in the following stages. Of the 204 knowable risks associated with the 26 critical incidents we studied, 94 per- cent (192 risks) were identified in the formal risk manage- ment process, leaving 6 percent of knowable risks (12) excluded from further consideration and management. Why might this be? In our interviews, it became clear that managers tended to concentrate on risks they per- ceived to be "close." For instance, in one of the projects we studied, the project manager focused on and identified a number of technical risks; "When I did [the] risk assess- ment," he told us, "I looked at the technology involved." Yet, the critical incident was not triggered by technical inad- equacies but by a commercial issue-a failure of the techni- cal solution to be sufficiently user-friendly. We call this the "lure of the familiar"-managers tend to focus on com- monly recognized risks in areas with which they are famil- iar, but they ignore other areas of risk with which they find it more difficult to associate. The Assessment Stage: The Lure of the Measurable After risks are identified, they must be assessed to deter- mine which risks are most urgent and most in need of man- agement attention. Risks are prioritized according to their likelihood of occurrence and impact. A further 18 percent (34 risks) of the 192 identified risks were excluded from this formal assessment, dropping out of the management pro- cess. Regardless of the magnitude of the risk itself, whether a risk was retained or not was based on the perceived cred- ibility and accuracy of the estimates for the risk's probability and impact. Risks for which such estimates were perceived as credible and accurate were given more attention than risks where these estimates were perceived as ambiguous. In one critical incident we analyzed, in which the devel- opment and roll-out of a hardware/software solution suffered severe delays because of compatibility issues, the project manager acknowledged the role of assessing risks, saying "I think the probability assessment is the weakest link." Risks that were considered ambiguous or open to in- terpretation tended to be excluded from further manage- ment. In contrast, the risks that attracted the most attention were the ones that were easier to measure. All too often, risk management rests upon what can easily be counted-what we call the "lure of the measurable." Project managers tended to be attuned to risks whose prob- abilities and impacts could be easily assessed and whose re- sponses could be defined with confidence. They encountered difficulties assessing probabilities for a number of risks, not only those that were retrospectively singled out as the key risks associated with an incident. The Response Stage: The Lures of Positivity and Noncommitment and the Deterrent of Powerlessness The risk response stage is vital. Just identifying and assessing risk does not reduce risk exposure; an active response is re- quired to mitigate the risk. Yet, the biggest breakdown in risk management in the projects we studied occurred in the re- sponse stage, where specific risks should be targeted for ac- tive management. In our study, a further 28 percent (44 risks) of the risks that made it through the earlier stages were not actively managed, even though managers had already invested effort in identifying and assessing them. Our analy- sis of interview data identified three lures that explain this response gap. 1. The lure of positivity. One project manager explained his inaction with regard to key risks as follows: "Problems were kept to a minimum simply in order to come across as a competent provider." In calling attention to risks, project managers face the danger of undermining stake- holders' confidence in their ability to deliver. Giving stakeholders the answers they want provides the appear- ance of certainty or the perception of a safe and predict- able world with minimal risks. Few managers want to be seen as doomsayers, so they often succumb to the lure of positivity, causing discussion of risk responses to become taboo, something to be suppressed or deemphasized. 2. The lure of noncommitment. Evidence from the cases we studied also shows that project managers tended to defer commitments as long as possible, keeping their options open to such an extent that, in some cases, they did not act until a risk actually materialized. In one case, a proj- ect manager reflected on why the project team did not act preemptively on a range of key risks associated with a critical issue: "We did not think there was a need to respond because it had not happened yet." In the minds of some of the project managers we interviewed, risk is fiction until it materializes. Project managers seem to cling to inaction for two reasons. First, being proactive (by investing in risk mitigation) is seen as restricting a project manager's freedom. And investing resources to prevent problems that might not even happen is seen in some organizations as wasteful. Just like many Americans who decline to purchase personal health insurance, these managers are hoping that luck will be on their side. Risk responses are put on hold; project managers fall victim to the lure of noncommitment. But hope is not a strategy, and profes- sional practitioners of project management should not consider this to be appropriate due diligence. 3. The deterrent of powerlessness. A further aspect influencing managers' willingness to engage with risk is the sense of having too little control over responding to risks. Control refers to the extent to which individuals believe they can affect the risks to their projects. Project managers tended to believe that they lack the power to respond adequately to risk, perhaps due to a lack of resources attached to the risk response. For example, one project manager told us he felt powerless because "typical solutions which apply to these risks did not apply" in his case. It is interesting to note that once those critical incidents actually occurred, project managers were often provided with resources that they believed had previously been denied. These lures and deterrents all lead to the exclusion of risks from mitigation. The extent to which knowable risks were not managed is considerable. On average, 44 percent of all knowable risks were not actively managed (Figure 1). Project managers in our study acknowledged the ineffec- tiveness of applied risk management. Several interviewees admitted that risk management was for them little more than a "tick-box" exercise. As one project manager told us, "It becomes an administrative process, and as long as people feel there is a risk register somewhere and lip service is paid to it on a reasonably frequent basis, then they are managing risk." How To Fix It: Toward a More Comprehensive Approach to Risk Management The lures and deterrents have an unfortunate impact on project outcomes because they prevent actual or obtainable knowledge about risks from prompting appropriate action. However, with awareness and attention, they can be over- come, leading to a more comprehensive approach to risk management. For instance, diverse, cross-functional representation in the group performing risk identification can help teams avoid the lure of the familiar, by expanding the range of what might be familiar to someone on the team. Further, at this early stage, teams should consciously seek to err on the side of inclusiveness in identifying potential risks. Deter- mining which subset of the identified risks to focus on for active management should be saved for a later step in the process. Overcoming the lure of the measurable requires estimat- ing a real number for the probability and impact of each risk, no matter how challenging it may be to identify such a number. Reasonable estimates can often be obtained by in- vesting in the discovery of some evidence in terms of the drivers of the risk event and its impact (Smith and Merritt 2002). Providing a quantitative forecast even for hard-to- measure risks also enables ongoing, closed-loop improve- ment of forecasting accuracy (Hubbard 2009). Further, managers must be challenged to look beyond what can be measured easily or confidently. At the identi- fication stage, it may be helpful to encourage those assess- ing risks to incorporate intuition and "gut feeling" into their mental frame; despite the initial difficulty of attach- ing a probability, defining an impact, or predicting and al- locating a response to such feelings, the exercise can be productive. Extreme futures may be taken into con- sideration at this stage, not for the purpose of predicting the most measurable or likely outcomes, but to stimulate thinking about possible risks that should at least be considered. Addressing the lures of optimism and indecisiveness may require broader organizational change, altering the connotations attached to risk management. Risks should be seen as normal and expected, and good risk manage- ment as prudent practice, not doomsaying. Similarly, managers must be encouraged to be proactive in respond- ing to risk before it actually materializes in an incident that delays or kills a project, and they must be provided the resources to do so. A risk that does not materialize should not be taken as a sign that the risk never existed, but rather as the result of proactive responses successfully reducing the risk's likelihood. Hence, the rationale for executing risk responses should not focus on how much has been spent to prevent an incident that may not materialize, but rather must be driven by the assumption that any risk re- sponse has an impact beyond the immediately visible costs necessary to enact it. The perception of powerlessness is also an organizational issue. Project managers should be empowered to initiate im- portant activities such as risk responses. That empowerment may take the form of resources that give the decision maker the latitude to take action or of organizational authority and support for action. The first step in overcoming the lures and deterrents is recognizing the degree to which they are at play in the context of a particular team or project. We have created a tool to help managers assess the strength of the lures in their context, and to help initiate and guide a discussion on the role of the lures and deterrents in the team's risk management process (see "Assessing the Risk Gap," p. 31). As our study demonstrates, standard tools do not appear to prevent project managers from excluding knowable risks from the risk management process. Minimizing the role of the lures and deterrents-reducing the risk gap-requires a broader approach, using unconventional tools and meth- ods. One such tool is scenario planning, which can help fos- ter thinking beyond the familiar and the measurable. Scenario planning exercises encourage participants to de- fine and consider multiple possible and plausible futures that could constructively challenge each other (Ralston and Wilson 2006; Miller and Waller 2003; Mulvey, Rosenbaum, and Shetty 1997). In comparison with traditional risk man- agement processes, this approach does not aim to focus at- tention on individual, concrete risks that can be responded to in isolation; rather, it provides multiple, more abstract projections that can broaden the approach to risk and help reduce the effect of the five lures. And yet, as many have noted, project management with its set of rules and procedures-among them risk management-is being ap- plied with a waterfall approach (Raz and Michael 2001; Patanakul, Iewwongcharoen, and Milosevic 2010), with little attention to the potential benefits of more creative tools, such as scenario planning. We hasten to add that conventional risk management ap- proaches are not redundant. Augmenting conventional risk management approaches with tools such as scenario plan- ning can challenge a project manager's thinking and help overcome the influences of the five lures. They are not, how- ever, a substitute for proven processes that address identified risk and ensure that proper responses are executed. Taken together, conventional tools and more broad-based ap- proaches may provide a more effective risk management re- gime to make sense of and manage an inherently uncertain environment. Conclusion In our study of critical incidents in major NPD projects, we found that project managers tended to focus on the familiar and the measurable, and yet to some consider- able extent did not respond to those risks. Our insights into these managers' risk management approaches sug- gest that the current approach to risk management should be questioned. While the lures and deterrents we de- scribe provide some explanation for the gap between risk management standards and actual outcomes, they can- not offer a full explanation. Impediments to active risk management will, to some extent, depend on context, and they should be critically evaluated in every industry. Our work, however, and the insights it offers, may serve as a trigger for an important discussion about the exis- tence of a risk gap and the reasons it persists. The occurrence of critical incidents points to one of two conditions: either there is a lack of knowledge that a risk exists, or the risk is knowable but decision makers fail to act. All too often, risk management rests upon what can easily be counted-what we call the "lure of the measurable." Addressing the lures of optimism and indecisiveness may require broader organizational change, altering the connotations attached to risk management. References Cerpa, N., and Verner, J. M. 2009. Why did your project fail? Communications of the ACM 52(12): 130-134. Hubbard, D. W. 2009. The Failure of Risk Management. Hoboken, NJ: John Wiley & Sons. Miller, K. D., and Waller, H. G. 2003. Scenarios, real options and integrated risk management. Long Range Planning 36(1): 93-107. Mulvey, J. M., Rosenbaum, D. P., and Shetty, B. 1997. Strategic financial risk management and operations research. Eu- ropean Journal of Operational Research 97(1): 1-16. Nelson, R. R. 2007. IT project management: Infamous failures, classic mistakes, and best practices. MIS Quarterly Executive 6(2): 67-78. Patanakul, P., Iewwongcharoen, B., and Milosevic, D. 2010. An empirical study on the use of project management tools and techniques across project life-cycle and their impact on proj- ect success. Journal of General Management 35(3): 41-65. Project Management Institute. 2013. A Guide to the Project Man- agement Body of Knowledge. 5th ed. Newtown Square, PA: Project Management Institute. Ralston, B., and Wilson, I. 2006. The Scenario Planning Hand- book. Mason, OH: Thomson Higher Education. Raz, T., and Michael, E. 2001. Use and benefit of tools for proj- ect management. International Journal of Project Management 19(1): 9-17. Smith, P. G. 1999. Managing risk as product development sched- ules shrink. Research-Technology Management 42(5): 25-32. Smith, P. G., and Merritt, G. M. 2002. Proactive Risk Management. New York: Productivity Press. Williams, T. M. 2005. Assessing and moving on from the dominant project management discourse in the light of project overruns. IEEE Transactions on Engineering Management 52(4): 497-508. Elmar Kutsch is a lecturer in project program and project management at Cranfield School of Management, UK. His industrial experience includes managerial roles as a consultant in the IT industry. In a variety of project manager roles, he was responsible for the successful delivery of a number of large IT services projects, including major roll-out and outsourcing proj- ects. [email protected] Tyson R. Browning is an associate professor of operations management in the Neeley School of Business at Texas Christian University, where he con- ducts research on managing complex projects and teaches courses on proj- ect, operations, and risk management. He has over 50 papers published on these topics in a variety of journals and conferences. His industry experi- ence includes work with Lockheed Martin and consulting for Boeing, Gen- eral Motors, and other organizations. His recent work focuses on managing technical performance, resources, risk, and value in projects. He earned a BS in engineering physics from Abilene Christian University and two Mas- ter's degrees and a PhD from the Massachusetts Institute of Technology. [email protected] Mark Hall is a senior lecturer in operations and project management at the University of Bristol in the UK. He worked for several years in project man- agement, both in the United Kingdom and overseas, before returning to academia to complete a PhD in international management and cultural theory. Mark's research interests are focused around project management and public-sector service delivery. Current areas of research include the in- fluence of cultural theory in project environments with a focus on risk man- agement practices, performance measurement in the public sector, and sustainability. [email protected] DOI: 10.5437/08956308X5702133 (c) 2014 Industrial Research Institute, Inc |