• Home
  • Resources
    • Find Resources by Topic Tags
    • Cybersecurity Policy Chart
    • CSIAC Reports
    • Webinars
    • Podcasts
    • Cybersecurity Digest
    • Standards & Reference Docs
    • Journals
    • Certifications
    • Acronym DB
    • Cybersecurity Related Websites
  • Services
    • Free Technical Inquiry
    • Core Analysis Task (CAT) Program
    • Subject Matter Expert (SME) Network
    • Training
    • Contact Us
  • Community
    • Upcoming Events
    • Cybersecurity
    • Modeling & Simulation
    • Knowledge Management
    • Software Engineering
  • About
    • About the CSIAC
    • The CSIAC Team
    • Subject Matter Expert (SME) Support
    • DTIC’s IAC Program
    • DTIC’s R&E Gateway
    • DTIC STI Program
    • FAQs
  • Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer
Login / Register

CSIAC

Cyber Security and Information Systems Information Analysis Center

  • Resources
    • Find Resources by Topic Tags
    • Cybersecurity Policy Chart
    • CSIAC Reports
    • Webinars
    • Podcasts
    • Cybersecurity Digest
    • Standards & Reference Docs
    • Journals
    • Certifications
    • Acronym DB
    • Cybersecurity Websites
  • Services
    • Free Technical Inquiry
    • Core Analysis Task (CAT) Program
    • Subject Matter Expert (SME) Network
    • Training
    • Contact
  • Community
    • Upcoming Events
    • Cybersecurity
    • Modeling & Simulation
    • Knowledge Management
    • Software Engineering
  • About
    • About the CSIAC
    • The CSIAC Team
    • Subject Matter Expert (SME) Support
    • DTIC’s IAC Program
    • DTIC’s R&E Gateway
    • DTIC STI Program
    • FAQs
  • Cybersecurity
  • Modeling & Simulation
  • Knowledge Management
  • Software Engineering
/ Journal Issues / Supervisory Control and Data Acquisition / Case Study: Applying Agile Software Methods to Systems Engineering

Case Study: Applying Agile Software Methods to Systems Engineering

Published in Journal of Cyber Security and Information Systems
Volume: 1 Number: 3 - Supervisory Control and Data Acquisition

Author: Matthew Kennedy
Posted: 02/10/2016 | Leave a Comment

Delivering a Software Intensive System (SIS) that is on time, within budget and with the required functionality with traditional systems processes has been problematic (Hagan 2011).  This problem will only increase as the complexity of SISs within the Department of Defense (DoD) grows and more functionality within systems is relegated to software (Force 2009, Group 2009).  Ultra-modern approaches— known as “agile” processes— have emerged to correlate with the rate of change encountered during software development.  Agility is “the speed of operations within an organization and speed in responding to customers (reduced cycle times)” (Daniels 2006).  The degree of agility when developing an IT system is the organization’s ability to respond to changing requirements and technology.  With quick technology refresh rates, long development cycles run the risk of placing a system in a state of obsolescence prior to initial release.  The need to change without notice throughout the development lifecycle is paramount to success in the ever-changing world of technology.

Functions performed by software in DoD aircraft has increased from 8 percent for the F-4 Phantom II in 1960 to 90 percent for the F-35 in 2006 (Ferguson 2001, Schmidt 2013).  With the proliferation of software within current systems, problems that were inherently software are evolving into system problems (Group 2009).  The issues of both system complexity and agility is not only recognized within the United States DoD, but also have been identified in the United Kingdom’s Ministry of Defense as some of the “next great systems thinking challenges” (Oxenham 2010) .

Since software has such a predominant influence on systems today, it seems natural to examine efforts within the software engineering community to control cost, schedule, and performance.  The balance of this paper describes an effort to apply software agile techniques at the systems level.  It describes, in the context of a case study an Agile Systems Engineering Framework, a technique developed specifically to help program managers be as agile and nimble as possible to their shifting environments.

Case Study

The company used in the case study has done so under the agreement that they shall remain anonymous.  To comply with this request the company will be referred to as “Juggernaut” for the purposes of this case study.  The name Juggernaut has no relation to the company in the study.  The product technical specifications, design documents and any detailed information that could be used to trace the case study back to the product or company will not be contained in this report.

Company Background

Juggernaut is an ISO 9001:2008 registered company with over a 100-year history and offices in multiple countries.  Juggernaut produces various Information Technology (IT) solutions to customers worldwide.

Juggernaut was initially a manufacturing organization that has expanded to include manufacturing, mechanical and software departments.  As their product line increased in complexity and software became a larger part of their systems, their traditional manufacturing top-down development methodology was found ineffective.  Their products were becoming routinely late, over budget and did not include the planned functionality.

Small changes to the waterfall-like manufacturing process were found to be ineffective and traditional agile software processes did not provide the framework needed to incorporate manufacturing, mechanical and software components into a single delivery.  Juggernaut soon realized a new development approach was required to allow for the rapid delivery of systems containing more than just software.  Agile software development had been successfully adopted within Juggernaut’s software department, but the need to expand those practices to include the entire systems engineering process was becoming evident.

Project Background

The project selected to utilize the Agile System Engineering Framework and Practices was comprised of hardware, firmware, software and manufacturing components.  The software component was already using agile software practices.  This effort was a major modification of an existing product and included the incorporation of new functionality and updates to the system’s hardware, firmware, software and manufacturing elements.

Internally, Juggernaut was comprised of several departments including quality control, engineering, operations, marketing / sales and manufacturing.  The engineering department consisted of multiple projects, of which the project involved in the case study was one of the ongoing projects.  Each engineering project had a series of sub-elements depending on the product under development.   The personnel assigned to each sub-element were not necessarily 100 percent dedicated to the one project, but were working on several projects simultaneously.  Each project within the engineering department was also competing for other company resources such as manufacturing, operations, marketing / sales and / or Quality Control (QC).  A system development organizational structure can be seen in Figure 1.

In addition to the internally developed hardware, firmware, software and manufacturing, portions of these components were outsourced to leverage external expertise in emerging technologies.  Roughly 50 percent of the design and mechanical components were outsourced.  These outsourced components needed to be accounted for and managed within the Agile Systems Engineering Framework to allow for external components to be tested at various integration points.  In addition to these outsourced components, Juggernaut used manufacturing facilities that were located both in and outside of the United States (US) to assemble the final product, which increased the coordination effort required during development.

Though this product was sold directly to customers, the product under development was also intended to be reused and integrated with two other products internally developed at Juggernaut.  The product needed to conform and / or be certified in several specifications, including American National Standards Institute (ANSI) and United States Military Standard (MIL-STD) specifications.  The combination of internally and externally developed components coupled with the certification process made the identification and definition of the interfaces and integration points paramount to the success of the project.

JuggernautOrganizationalStructure

Figure 1 Juggernaut Organizational Structure

The project had a $1.3M budget for 13 months of development.  The internal team, team members working directly for Juggernaut, consisted of sixteen multidisciplinary members in the following specialties: Project Management, Firmware Development, Software Development, Hardware Development and Systems Testing.  The internal team worked together in the past on similar projects; however, they were utilizing a waterfall-type development methodology.  This effort was the first implementation of an agile system engineering methodology employed project-wide.

Past Performance

Juggernaut was able to provide complete past performance metrics including cost, schedule and functionality delivered on twelve projects developed by Juggernaut that were of similar size and scope.  Based on the twelve projects, Juggernaut was habitually behind schedule, over budget and not delivering the planned functionality.  The past performance metrics were calculated by taking the average of the estimated versus actual numbers for all three data points on the twelve projects.  The results can be seen in Table 1 Past Performance.

Average Cost Difference from Estimate +2.5%
Average Schedule Difference from Estimate +30%
Average Functionality difference from planned -5%

Table 1 Past Performance

Pages: Page 1 Page 2 Page 3 Page 4

Previous Article:
« Software Protection Against Side Channel Analysis Through...
Next Article:
Introduction »

References

Daniels, J. J. (2006). Review of Acquisition for Transformation, Modernization, and Recapitalization, U.S. Army War College. Master of Strategic Studies: 28.

Ferguson, J. (2001). “Crouching dragon, hidden software: software in DoD weapon systems.” Software, IEEE 18(4): 105-107.

Force, D. S. B. T. (2009). Department of Defense Policies and Procedures for the Acquisition of Information Technology. Washington, D.C. 20301-3140: 109.

Group, S. (2009, 04/23/2009). “Standish Newsroom - CHAOS 2009.” New Standish Group report shows more project failing and less successful projects. Retrieved 01/01/2011, 2011, from .

Hagan, G. (2011). Glossary of Defense Acquisition Acronyms and Terms. Fort Belvoir, Virginia 22060-5565, Defense Acquisition University Press: 346.

Hand, W. L. and G. C. Little (2012). Guide To The Business Capability Lifecycle For Department Of Defense ACAT III Programs, LMI Research Institute: 76.

Kennedy, M. R. and D. Ward (2012). “Inserting Agility in System Development.” Defense Acquisition Research Journal 19(3).

Oxenham, D. (2010). Agile approaches to meet complex system of system engineering challenges: A defence perspective. System of Systems Engineering (SoSE), 2010 5th International Conference on.

Schmidt, D. C. (2013). “The Growing Importance of Sustaining Software for the DoD.” Retrieved 02/16/2013, 2013, fromhttp://blog.sei.cmu.edu/post.cfm/the-growing-importance-of-sustaining-software-for-the-dod.

(Footnotes)

1 The cost variance was due to a vendor renegotiation and was not attributed to the Framework or Practices.

Author

Matthew Kennedy
Matthew Kennedy
Matthew R. Kennedy is a Professor of Software Engineering at Defense Acquisition University (DAU). Previously he was the Associate Director of Engineering at the National Cancer Institute’s Center for Biomedical Informatics and Information Technology (CBIIT). He served in the U.S. Air Force as a network intelligence analyst and has more than 13 years of experience in Information Technology. He has a Bachelor’s and Master’s degree in Computer Science and a PhD in Computer Science and Software Engineering.

Reader Interactions

Leave a Comment Cancel

You must be logged in to post a comment.

sidebar

Blog Sidebar

Featured Content

The DoD Cybersecurity Policy Chart

The DoD Cybersecurity Policy Chart

This chart captures the tremendous breadth of applicable policies, some of which many cybersecurity professionals may not even be aware, in a helpful organizational scheme.

View the Policy Chart

Featured Subject Matter Expert (SME): Daksha Bhasker

A dynamic CSIAC SME, Senior Principal Cybersecurity Architect, Daksha Bhasker has 20 years of experience in the telecommunications services provider industry. She has worked in systems security design and architecture in production environments of carriers, often leading multidisciplinary teams for cybersecurity integration, from conception to delivery of complex technical solutions. As a CSIAC SME, Daksha's contributions include several published CSIAC Journal articles and a webinar presentation on the sophiscated architectures that phone carriers use to stop robocalls.

View SME's Contributed Content

CSIAC Report - Smart Cities, Smart Bases and Secure Cloud Architecture for Resiliency by Design

Integration of Smart City Technologies to create Smart Bases for DoD will require due diligence with respect to the security of the data produced by Internet of Things (IOT) and Industrial Internet of Things (IIOT). This will increase more so with the rollout of 5G and increased automation "at the edge". Commercially, data will be moving to the cloud first, and then stored for process improvement analysis by end-users. As such, implementation of Secure Cloud Architectures is a must. This report provides some use cases and a description of a risk based approach to cloud data security. Clear understanding, adaptation, and implementation of a secure cloud framework will provide the military the means to make progress in becoming a smart military.

Read the Report

CSIAC Journal - Data-Centric Environment: Rise of Internet-Based Modern Warfare “iWar”

CSIAC Journal Cover Volume 7 Number 4

This journal addresses a collection of modern security concerns that range from social media attacks and internet-connected devices to a hypothetical defense strategy for private sector entities.

Read the Journal

CSIAC Journal M&S Special Edition - M&S Applied Across Broad Spectrum Defense and Federal Endeavors

CSIAC Journal Cover Volume 7 Number 3

This Special Edition of the CSIAC Journal highlights a broad array of modeling and simulation contributions – whether in training, testing, experimentation, research, engineering, or other endeavors.

Read the Journal

CSIAC Journal - Resilient Industrial Control Systems (ICS) & Cyber Physical Systems (CPS)

CSIAC Journal Cover Volume 7 Number 2

This edition of the CSIAC Journal focuses on the topic of cybersecurity of Cyber-Physical Systems (CPS), particularly those that make up Critical Infrastructure (CI).

Read the Journal

Recent Video Podcasts

  • A Brief Side-by-Side Comparison Between C++ and Rust – Part 3 Series: Programming Language Comparisons
  • A Brief Side-by-Side Comparison Between C++ and Rust – Part 2 Series: Programming Language Comparisons
  • A Brief Side-by-Side Comparison Between C++ and Rust – Part 1 Series: Programming Language Comparisons
  • Digital Engineering Implementation Progress and Plans Series: CSIAC Webinars
  • Assessing the Operational Risk Imposed by the Infrastructure Deployment Pipeline Series: The CSIAC Podcast
View all Podcasts

Upcoming Events

Thu 25

SANS Cyber Security East: Feb 2021

February 22 - February 27
Organizer: SANS Institute
Thu 25

19th USENIX Conference on File and Storage Technologies (FAST) 2021

February 23 - February 25
Organizer: USENIX
Jan 28

Data Privacy Day

January 28, 2022
Jan 28

Data Privacy Day

January 28, 2023
View all Events

Footer

CSIAC Products & Services

  • Free Technical Inquiry
  • Core Analysis Tasks (CATs)
  • Resources
  • Events Calendar
  • Frequently Asked Questions
  • Product Feedback Form

About CSIAC

The CSIAC is a DoD-sponsored Center of Excellence in the fields of Cybersecurity, Software Engineering, Modeling & Simulation, and Knowledge Management & Information Sharing.Learn More

Contact Us

Phone:800-214-7921
Email:info@csiac.org
Address:   266 Genesee St.
Utica, NY 13502
Send us a Message
US Department of Defense Logo USD(R&E) Logo DTIC Logo DoD IACs Logo

Copyright 2012-2021, Quanterion Solutions Incorporated

Sitemap | Privacy Policy | Terms of Use | Accessibility Information
Accessibility / Section 508 | FOIA | Link Disclaimer | No Fear Act | Policy Memoranda | Privacy, Security & Copyright | Recovery Act | USA.Gov

This website uses cookies to provide our services and to improve your experience. By using this site, you consent to the use of our cookies. To read more about the use of our site, please click "Read More". Otherwise, click "Dismiss" to hide this notice. Dismiss Read More
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled

Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.

Non-necessary

Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.

SAVE & ACCEPT