Decisions Based on Analysis of Alternatives (AoA)
Dr. David G. Ullman 1 , January 2009 Analysis of Alternatives (AoA) is a term that has been adopted by the Office of Management and Budget (OMB) and the Department of Defense (DoD) to ensure that multiple alternatives have been analyzed prior to making costly investment decisions. AoA is an effort to move from the justification for a single alternative to the exploration of multiple alternatives so agencies have a basis for funding the best possible projects in a rational, defensible manner considering risk and uncertainty. As with other decisionmaking tools, techniques, and methodologies, AoA is most effectively used in a higherlevel ...view middle of the document...
• Level 2 – Propose multiple alternatives and provide multi-dimensional comparative analysis with weak inclusion of uncertainty effects. • Level 3 – Propose multiple alternatives, and provide multi-dimensional comparative analysis and support robust resource allocation decisions with the inclusion of uncertainty effects. These levels include measures about the number of alternatives considered, the inclusion of uncertainty in the analysis, and the level of decision support. More levels could be defined by considering these measures separately, but these four levels are sufficient for the current state-of-the-art.
Dr Ullman is President of Robust Decisions Inc (www.robustdecisions.com) and a consultant for Edge Consulting (http://www.edgeconsultinginc.com/). He can be reached at firstname.lastname@example.org or 1-541-754-3609.
This paper will show how OMB pushes funding approval from Level 0 to Level 1, and how DoD encourages Level 2 AoA. Further, the paper will show that current AoA methods only go part of the way to achieving the highest potential – i.e., Level 3 AoA. Keep in mind that the ultimate goal of an AoA exercise is to enable making the best possible decision about resource allocations where this decision is based on uncertain, incomplete, evolving, and conflicting estimates of cost, performance, and other critical measures. In this paper, the OMB and DoD approaches to AoA are discussed first, and then AoA’s promise will be explored. AoA’s Value Depends on Estimation and Risk The “analysis” in AoA refers to making estimates of future costs, performance and other critical measures in order to understand the risk of following a course of action. As Chapter 3 in Making Robust Decisions (Ullman, 2006) points out about estimates, “Where the past performance may be known, the present is obscured by its immediacy and the future is a best guess.” The best guess is clouded in uncertainty and uncertainty results in risk. Uncertainty comes from many sources (11 such sources are discussed in Making Robust Decisions), and these can be characterized by the types of resulting risks: technical risks, programmatic risks, operational risks and decision risks. The first three are often discussed; but the last one, decision risk, the risk of choosing the wrong alternative when performing an AoA, needs to also be known. Before discussing risks, some background on estimation accuracy is in order. Estimations about time, cost, or performance are notoriously inaccurate. In one government agency, cost overruns range from 31% (small projects) to 315% (very large projects). As another example of estimation inaccuracy, in the Chaos Report (Standish Group, 2000 and 2004) an annual analysis of IT projects, 51% of all IT projects were delivered late or over budget in 2004 and an additional 15% were cancelled. Further, projects completed by large companies had only 42% of the originally designed features and functions. It should be noted that the Chaos...