Case Study: Applying Agile Software Methods to Systems Engineering

Delivering a Software Intensive System (SIS) that is on time, within budget and with the required functionality with traditional systems processes has been problematic (Hagan 2011).  This problem will only increase as the complexity of SISs within the Department of Defense (DoD) grows and more functionality within systems is relegated to software (Force 2009, Group 2009).  Ultra-modern approaches— known as “agile” processes— have emerged to correlate with the rate of change encountered during software development.  Agility is “the speed of operations within an organization and speed in responding to customers (reduced cycle times)” (Daniels 2006).  The degree of agility when developing an IT system is the organization’s ability to respond to changing requirements and technology.  With quick technology refresh rates, long development cycles run the risk of placing a system in a state of obsolescence prior to initial release.  The need to change without notice throughout the development lifecycle is paramount to success in the ever-changing world of technology.

Functions performed by software in DoD aircraft has increased from 8 percent for the F-4 Phantom II in 1960 to 90 percent for the F-35 in 2006 (Ferguson 2001, Schmidt 2013).  With the proliferation of software within current systems, problems that were inherently software are evolving into system problems (Group 2009).  The issues of both system complexity and agility is not only recognized within the United States DoD, but also have been identified in the United Kingdom’s Ministry of Defense as some of the “next great systems thinking challenges” (Oxenham 2010) .

Since software has such a predominant influence on systems today, it seems natural to examine efforts within the software engineering community to control cost, schedule, and performance.  The balance of this paper describes an effort to apply software agile techniques at the systems level.  It describes, in the context of a case study an Agile Systems Engineering Framework, a technique developed specifically to help program managers be as agile and nimble as possible to their shifting environments.

Case Study

The company used in the case study has done so under the agreement that they shall remain anonymous.  To comply with this request the company will be referred to as “Juggernaut” for the purposes of this case study.  The name Juggernaut has no relation to the company in the study.  The product technical specifications, design documents and any detailed information that could be used to trace the case study back to the product or company will not be contained in this report.

Company Background

Juggernaut is an ISO 9001:2008 registered company with over a 100-year history and offices in multiple countries.  Juggernaut produces various Information Technology (IT) solutions to customers worldwide.

Juggernaut was initially a manufacturing organization that has expanded to include manufacturing, mechanical and software departments.  As their product line increased in complexity and software became a larger part of their systems, their traditional manufacturing top-down development methodology was found ineffective.  Their products were becoming routinely late, over budget and did not include the planned functionality.

Small changes to the waterfall-like manufacturing process were found to be ineffective and traditional agile software processes did not provide the framework needed to incorporate manufacturing, mechanical and software components into a single delivery.  Juggernaut soon realized a new development approach was required to allow for the rapid delivery of systems containing more than just software.  Agile software development had been successfully adopted within Juggernaut’s software department, but the need to expand those practices to include the entire systems engineering process was becoming evident.

Project Background

The project selected to utilize the Agile System Engineering Framework and Practices was comprised of hardware, firmware, software and manufacturing components.  The software component was already using agile software practices.  This effort was a major modification of an existing product and included the incorporation of new functionality and updates to the system’s hardware, firmware, software and manufacturing elements.

Internally, Juggernaut was comprised of several departments including quality control, engineering, operations, marketing / sales and manufacturing.  The engineering department consisted of multiple projects, of which the project involved in the case study was one of the ongoing projects.  Each engineering project had a series of sub-elements depending on the product under development.   The personnel assigned to each sub-element were not necessarily 100 percent dedicated to the one project, but were working on several projects simultaneously.  Each project within the engineering department was also competing for other company resources such as manufacturing, operations, marketing / sales and / or Quality Control (QC).  A system development organizational structure can be seen in Figure 1.

In addition to the internally developed hardware, firmware, software and manufacturing, portions of these components were outsourced to leverage external expertise in emerging technologies.  Roughly 50 percent of the design and mechanical components were outsourced.  These outsourced components needed to be accounted for and managed within the Agile Systems Engineering Framework to allow for external components to be tested at various integration points.  In addition to these outsourced components, Juggernaut used manufacturing facilities that were located both in and outside of the United States (US) to assemble the final product, which increased the coordination effort required during development.

Though this product was sold directly to customers, the product under development was also intended to be reused and integrated with two other products internally developed at Juggernaut.  The product needed to conform and / or be certified in several specifications, including American National Standards Institute (ANSI) and United States Military Standard (MIL-STD) specifications.  The combination of internally and externally developed components coupled with the certification process made the identification and definition of the interfaces and integration points paramount to the success of the project.

Figure 1 Juggernaut Organizational Structure

The project had a $1.3M budget for 13 months of development.  The internal team, team members working directly for Juggernaut, consisted of sixteen multidisciplinary members in the following specialties: Project Management, Firmware Development, Software Development, Hardware Development and Systems Testing.  The internal team worked together in the past on similar projects; however, they were utilizing a waterfall-type development methodology.  This effort was the first implementation of an agile system engineering methodology employed project-wide.

Past Performance

Juggernaut was able to provide complete past performance metrics including cost, schedule and functionality delivered on twelve projects developed by Juggernaut that were of similar size and scope.  Based on the twelve projects, Juggernaut was habitually behind schedule, over budget and not delivering the planned functionality.  The past performance metrics were calculated by taking the average of the estimated versus actual numbers for all three data points on the twelve projects.  The results can be seen in Table 1 Past Performance.

Table 1 Past Performance