Parametric Modeling to Support System Acquisition

Requirements Change

The second source of change is actual requirement changes.  Here I’m talking about actual changes in the underlying requirements.  Perhaps there is a building code change that must be met.  Perhaps the legislature has changed a law which affects how the software should operate.  Perhaps the operational feedback from the field requires modifications in the armor plating.  The longer a project lasts, the more you can count on the requirements changing during the project.  For software projects, the rate of requirement change varies from 3% to 15% per year, with 12% being a common number.  The good news is that these changes are clear candidates for scope management through the change control process.  The bad news is that because they are changes to a project that is already underway, in most cases the only person who can implement the change is the company already doing the work.  This sets up a sole source bidder situation that is uncomfortable for the customer in the best case, and that can be used by the vendor to generate large profits at government expense in the worst case.

Figure 3:  Typical Change Request Processing

Let’s take a DoD project as an example.  The approach to managing change requests might look something like Figure 3.  A change request is prepared; an impact assessment plan (IAP) or some similar document is prepared; the vendor estimates costs; the government reviews the estimate.  If the costs are approved, the government then attempts to justify the additional funding and authorize the change.  If they do not agree with the estimate, or they cannot justify the expenditure, then they negotiate with the vendor.  At this point, there is often a significant amount of stress and finger pointing, with no-one very happy.

Figure 4: IGCE Oversight Estimate

The first attempt to fix the problem often looks like Figure 4.  Here, the government creates an independent government cost estimate (IGCE) for validation of the vendor estimate.  As long as the IGCE estimate matches the vendor estimates, this approach works swimmingly.  Unfortunately, when they don’t match, I have yet to meet a vendor that didn’t think their estimate was right and the IGCE estimate was wrong.  Again, lots of friction and finger pointing.  This time, because of the clear discrepancy between the two estimates, the almost inevitable result is that the entire process gets deadlocked.

Figure 5: Model Based Oversight

Now, let’s look at an alternative that does work consistently, on projects of all sizes and complexities (Figure 5).  The first step is that a parametric model based approach to estimation is agreed to by all stakeholders, configured, and validated.  This might be a software cost estimating model, a construction cost estimating database, or any other standard estimating method that is objective and verifiable.  This approach, and the underlying configuration, should be set up and agreed to with no money on the table, thereby allowing stakeholders to be objective.  Once it is finalized, then as an IAP is completed both the Vendor and an independent government expert review the IAP and use the common, agreed to estimating model to create independent estimates.  If they match, then the estimate is approved and the change control board can proceed with a cost benefit review of the change request.  If they do not match, then there is a reconciliation meeting to identify where and why the underlying assumptions do not match.  The key here is that the discussion is focused on technical aspects of the work to be done (e.g., HLO object definitiions), not on costs or efforts.  Once that is clarified, then both parties run the models again and compare results.  In my experience, 70% of the IAPs result in estimates that match and therefore do not need a reconciliation meeting at all; another 24% are resolved following one reconciliation meeting, and 6% either require another meeting or require that the original change request be clarified as to intent⁷.