Under what conditions will organizations derive the most benefit from these practices?
When an organization neglects the following factors, the effectiveness of the monitoring practices may be severely limited:
- Select capable contractors
Of course, whether they have experience in the domains covered by the program should be the primary consideration. Select
- those who have a history of measuring and analyzing their development activities, artifacts, and defects; and experience interpreting such measurements to control their projects
- those who have experience defining, using, and improving their development and sustainment processes, e.g., those who demonstrate compliance with various best practice standards (e.g., ISO 9000, CMMI)
- those who already have the capability to specify, use, analyze, and improve measures that directly or indirectly address the information needs of a program manager
- Staff a capable team
One key is to ensure that team members have the training and mentoring they need to make sound technical judgments. Teams must be empowered and encouraged to define their own work processes and regularly evaluate the quality of their work and gauge the progress made. Other enablers include having staff, not necessarily all staff, but one or more
- assigned and trained in how to interpret the dashboards
- experienced in interpreting trend charts, expertise in EVM systems, and using process measures, e.g., project yield, defect density, and defect escape
- who can, from a set of information needs, objectives, and associated measures, identify gaps in what those measures might cover
- experienced with reviewing artifacts similar to those that will be used on the program
- knowledgeable of process and quality system standards, both strengths and limitations
- experienced in change management, risk management, and quality planning
- experienced in peer reviews, unit tests, operational tests, functional tests, etc.
- familiar with assurance cases if program will be using same
- experienced in reviewing progress measurements during program reviews, probing for context, and interpreting the meaning of a change in one of those measures
- experienced in explaining measurements and their interpretation to stakeholders
- experienced in negotiating commitments with stakeholders
- Infrastructure for measurement data (on both sides)
A capable team needs a capable infrastructure. Use of a program dashboard requires collecting and organizing a lot of contractor data and generating charts relating these measures to time and resource objectives. A measurement infrastructure (e.g., database and chart generator) can assist in time stamping the data, linking the data to its source, organizing the data, projecting different views, storing the results, and incorporating into reports.
Richard Crume. “Who Is to Blame When Government Contracts Go Astray?” IACCM, 2008.http://www.iaccm.com/news/contractingexcellence/?storyid=548
Adrian Pitman, Elizabeth K. Clark, Bradford K. Clark, & Angela Tuffley. “An Overview of the Schedule Risk Assessment Methodology (SCRAM).” Journal of Cyber Security & Information Systems 1, 4 (2013).
Defense Acquisition University. Defense Acquisition Guidebook. DAU, 2013. https://dag.dau.mil/Pages/Default.aspx
Steve McConnell. Software Estimation: Demystifying the Black Art (Best Practices Series). Microsoft Press, 2006.
Software Program Managers’ Network. The Program Manager’s Guide to Software Acquisition Best Practices. Computers & Concepts Associates, 1998.
Copyright 2013 Carnegie Mellon University
Mr. Robert Ferguson is an experienced software developer and project manager who joined the SEI after 30 years in industry. His experience includes applications in real-time flight controls, manufacturing control systems, large databases, and systems integration projects. He has also led process improvement activities at two companies. At the SEI he supports customers with problems in estimation and measurement implementation.
Mr. Ferguson is a Senior Member of IEEE. He has Project Management Professional (PMP) certification from the Project Management Institute (PMI).