Comparison of Investment Analyses


Definition: Investment analyses take many different forms within the federal government and are applied to support several key decisions throughout the investment, acquisition, and program life cycles. Examples include analysis of alternatives, business case analysis, cost-benefit analysis, and economic analysis.

Keywords: analysis of alternatives, business case analysis, cost-benefit analysis, economic analysis, financial analysis, investment analysis

MITRE SE Roles and Expectations: MITRE systems engineers (SEs) are expected to understand key aspects of investment analyses to be performed at various stages of the life cycle, including decisions supported, primary objectives, and general approaches that can be applied. SEs will frequently team with economic/cost analysts to perform investment analyses, and their familiarity with key aspects of prevalent analytic approaches can improve the overall quality of completed analyses and the efficiency of supporting activities performed.

Background

Several different analysis approaches are designed to inform sponsor funding and expenditure decisions. These include analysis of alternatives, business case analysis, cost-benefit analysis, and economic analysis. This article provides an overview of the similarities, overlaps, and differences among prevalent business, investment, and economic analysis approaches and indicates which analyses are required by different government organizations.

Comparison of Key Investment Analysis Types

  • Analysis of Alternatives (AoA): An AoA is a technical assessment using distinct metrics and various decision criteria to objectively evaluate different potential courses of action (or alternatives). Typically, the focus is on an analysis of alternative technical approaches for meeting a given set of functional requirements or mission needs. Alternatives can, in special cases, be distinguished by different acquisition-related approaches if a real range of feasible technical alternatives does not exist. The assessment also includes a life-cycle cost estimate (LCCE) and risk assessment for each alternative and a preliminary recommendation(s).
  • Business Case Analysis (BCA): A BCA is used to determine if a new approach and overall acquisition should be undertaken. A BCA provides justification for investment (either to invest or not) based on a comparison of life-cycle costs, benefits, and the results of applying financially oriented calculations such as return on investment (ROI), net present value (NPV), and discounted payback period (DPP) for each alternative. An AoA establishes the alternatives to be evaluated. Based on the results of the financial analysis, a BCA helps determine if a potential new acquisition is warranted and if the effort should proceed.
  • Cost-Benefit Analysis (CBA): The primary objective of a CBA is to identify and obtain approval for the optimum way to solve a specific problem or capitalize on a specific improvement opportunity. A CBA documents the predicted effects (financial, operational, quantitative, and qualitative) of actions under consideration and describes the potential financial impacts and other business benefits to support a decision regarding adopting a particular solution before significant funds are invested. Agency-specific policies may differ, and CBAs within some organization may need to include a detailed budget estimate and identify funding sources for the preferred alternative. Many organizations consider a CBA a living document that should be updated as needed.
  • Economic Analysis (EA): An EA is a systematic approach to identifying, analyzing, and comparing costs and benefits of alternative courses of action. An EA should emphasize the identification of key variables that drive cost, benefit, risk, and uncertainty assessment results across competing alternatives. Depending on agency policy, an EA may be very similar to a CBA. In some government organizations, EAs include post-investment management considerations, for example, identifying key performance measures for monitoring and evaluating whether the initiatives ultimately achieved expected or desired results.

AoAs and BCAs are typically conducted early in the acquisition process for a new information technology (IT) initiative. For example, the Department of Defense (DoD) Joint Capabilities Integration and Development System (JCIDS) process specifies that they should be conducted prior to Milestone A. For each subsequent year, they should be validated against the baseline because costs, schedule, etc. may change [1].

In the past, many civilian agencies were not necessarily required to prepare distinct AoAs or BCAs but would, instead, rely on the alternatives analysis that was prepared in support of an Office of Management and Budget (OMB) Exhibit 300 submission, as required by OMB Circular A-11, Part 7. Although the E-300 is a fairly comprehensive document, a typical AoA or BCA will be more detailed and rigorous [2].

The DoD requires distinct and more comprehensive AoAs and BCAs primarily because DoD initiatives are often larger dollar investments than civilian IT initiatives. Civilian agencies must periodically check progress against the baseline and, as necessary, revise the E-300 annually. Most recently, several civilian agencies (e.g., Department of Homeland Security) have produced guidance for preparing AoAs or BCAs to support investment decision making [3].

IT acquisition professionals often have a clear understanding of AoAs and BCAs, but they are less clear about CBAs and EAs. Depending on the agency or individual, the terminology of these analyses can be used interchangeably. CBA and EA are basically similar analysis approaches.

Table 1 summarizes the general similarities and differences among the analytical approaches.

Table 1. Comparison of Analytical Approaches

Comparison of Analytical Approaches

Best Practices and Lessons Learned

Many of the following best practices and lessons learned that are identified for a particular investment analysis are also relevant for other investment analysis types.

Analysis of Alternatives (AoA)

Develop viable alternatives. Alternative courses of action that could feasibly be pursued to achieve the mission should be developed. An analysis should not use "throwaway" alternatives to justify a preconceived course of action.

Examine three or more alternatives in addition to any status quo (SQ) baseline. For example, new alternatives for traditional software development initiatives might include commercial off-the-shelf (COTS), custom software development, and COTS/software (SW) development hybrid. To meet certain requirements, it may be impractical to use all COTS or to solely embark on a custom software development. A well-developed AoA should include technically distinct, realistic alternatives. The SQ alternative should be viable; in particular, SQ does not necessarily mean "do nothing." For example, it may be necessary to include technology refresh in the future if it is essential to maintaining the required functionality.

Develop alternatives in collaboration with technical subject matter experts (SMEs). AoA/BCA authors should not invent alternatives for the purpose of checking a box and meeting a capital planning requirement. Consult with technical SMEs to think through the implications of how to best fulfill mission needs.

Alternatives can be differentiated by acquisition approaches. In certain cases, there may be few distinct technical approaches for fulfilling mission needs.

Establish evaluation criteria and scoring. Identify multiple, independent metrics and compare how each alternative affects them. Metrics may be quantitative or qualitative. Every metric does not need to be equally important. Weightings can be applied to reflect priorities.

Identify technical and cost risks. Risks with each alternative should be identified, including ease of implementation, relative difficulty of meeting certain challenging requirements, as well as an assessment of the probability that an alternative can meet cost and schedule goals.

Life-cycle cost estimating. An AoA should include a multiyear life-cycle estimate, which is generally broken out by acquisition, operations and support (O&S), and SQ phase-out costs. A typical DoD approach is to analyze O&S costs 10 years after the system capability is made fully operational. This would include at least one tech refresh cycle and enable the analysis to calculate financial metrics and benefits for each alternative over a longer time period.

Consider environmental changes in your recommendations. It is useful to recommend a specific alternative to pursue, and this can be done in conjunction with a subsequent BCA, as needed. In certain cases, it is useful to present a few recommendations that take into account likely changes to the surrounding environment (e.g., changes in the economic climate). For example, Alternative 1 may be the preferred recommendation if the program receives adequate funding but not if its funding is cut. Recommendations for specific alternatives can be tied to potential situations that may arise (which you should also document in the ground rules and assumptions).

Government/sponsor buy-in. Unfortunately, an AoA is sometimes viewed merely as a required check-the-box activity that is a necessary part of capital planning. For programs requiring large expenditures of funds, however, an AoA is a useful means for the program office to fully consider what must be done to move forward effectively and efficiently. The approach requires a degree of rigor that can often help the program succeed in the long term.

Business Case Analysis (BCA)

Identify potential benefits for each alternative. In contrast to an AoA, a BCA identifies measurable benefits for each alternative. An example of a measurable, or quantitative, benefit is an increase in productivity. Although qualitative benefits such as morale, better information sharing, and improved security cannot be readily measured, they may still be important to alternative selection.

Develop return on investment (ROI) statistics. ROI is often misunderstood in the federal government sector. Key financial metrics help gauge the ROI of alternative courses of action relative to the SQ alternative. No single calculation can fully describe ROI, and financial metrics typically calculated to support this description include net present value (NPV), rate of return (ROR), and discounted payback period (DPP). These three metrics take into consideration the "time value of money" (i.e., a dollar received today is worth more than a dollar received one year from now due to factors such as inflation). All calculations, except NPV, are typically performed relative to the SQ alternative. In other words, the calculations take into account how significantly an alternative course of action increases or decreases costs and benefits relative to the SQ.

  • The NPV for an alternative describes current and future net expected benefits (i.e., expected benefits less expected costs) over the life cycle for that alternative.
  • ROR, stated as a percentage, describes the discount rate at which the incremental current and future benefits of an alternative in comparison to the SQ equal the incremental costs of that alternative relative to the SQ baseline.
  • DPP, stated in units of time, describes how much time it will take for the cumulative incremental benefits of an alternative course of action relative to the SQ alternative to exceed the cumulative incremental costs of an alternative course of action.

Clarify the acquisition/implementation timeline. A clear timeline is important to fully understand and communicate how costs are, or should be, phased in over an investment and O&S period. Programs typically have two- to five-year investment periods, depending on the size of the initiative and the type of acquisition approach.

Identify the potential benefits, uncertainties, and risks. A BCA is specifically designed to highlight the value of proposed alternative approaches. It also presents an opportunity to articulate key uncertainties associated with benefit and cost estimates. Within a BCA, it is especially important to evaluate the likelihood and potential impact of ultimately not realizing certain benefits from an investment as initially anticipated. A BCA also presents an opportunity to evaluate the impact of risks that the LCCE will deviate from the baseline estimate. Specific major cost drivers that are highly uncertain (i.e., cost risk) should be called out. Software development labor is particularly prone to potential risk of underestimation. 

Cost Benefit Analysis (CBA)

Alternatives may differ from those in the AoA and BCA and should be analyzed in more detail. Depending on how much detail was provided in the AoA and BCA, a CBA should break down the preferred alternative into more specificity. A CBA may focus on how best to implement the preferred alternative, which requirements may yield the biggest "bang for the buck" (e.g., accelerated payback period), how different contracting approaches may reduce risk, etc.

CBA can analyze multiple alternatives. The standard rule of thumb for a BCA is three new alternatives plus the SQ (policies on the number of alternatives to evaluate may vary, and the number may be predicated on the problem being addressed). A CBA can offer many more, if necessary, with minor variations among each to help determine the best specific approach forward. It will still follow the basic format of a BCA, although the analysis for each alternative approach may not be as comprehensive as in the AoA/BCA.

CBA allows for incremental ROI results. A CBA is particularly useful for demonstrating the quantitative benefits that specific actions within a work breakdown structure (WBS) may yield. Therefore, a CBA is well suited for analyzing if one alternative offers a greater ROI than a competing alternative. Incremental financial analysis helps decision makers move forward by considering which alternative approach at a given point along an integrated master schedule (IMS) will yield a greater "bang for the buck" to justify an investment decision at a particular acquisition milestone or increment

Economic Analysis (EA)

Perform economic sensitivity analyses for key variables. EAs are often evaluated over the investment life cycle, and it is often unreasonable to assume perfect knowledge of what costs, benefits, and risks will be at all points in the future. For analysis variables that significantly drive analysis results and for which there is either considerable uncertainty or considerable impact if unlikely conditions actually occur, you should perform sensitivity analyses or uncertainty-weighted assessments to determine the potential impact of uncertainty on analysis results.

Evaluate all significant economic implications, but not necessarily in monetary terms. In the federal government, many investment costs, benefits, and risks are not readily translated into monetary terms. Regardless, if these implications are critically important to informing investment decisions, they should be evaluated. Consider whether other quantitative methods (e.g., comparison of numeric ratings for aspects of investment options) can be applied to assess the relative strengths and weaknesses of investment options.

References and Resources

  1. Defense Acquisition Portal, , accessed September 1, 2017.
  2. OMB Circular A-11, Part 7 (""), Section 300, June 2008, accessed September 1, 2017.
  3. Homeland Security Studies and Analysis Institute, January 22, 2014, , Ver. 3.0, accessed September 1, 2017.

Additional References and Resources

AFMAN 65-506, "," August 29, 2011, accessed September 1, 2017.

, April 24, 2013, accessed September 1, 2017.

Publications

Download the SEG

MITRE's Systems Engineering Guide

Download for EPUB
Download for Amazon Kindle
Download a PDF

Questions?
Contact the SEG Team