Experiences Testing Safety-Critical Software

This paper presents our experiences in testing critical software that supports flight systems developed by Lockheed Martin Astronautics (LMA in Denver, Colorado. This approach has not been proven in an academic sense, but has been demonstrated over the years to result in software that successfully performs missions. It is based on teams comprised of the correct skill balance in software and systems engineering, as well as using a defined process.

We explore generalized process that has been applied to several critical software programs at Lockheed Martin, some experiences in applying the process, and the importance of the engineering team. We concentrate in the area of testing or Verification and Validation (V&V) of these systems, set in the context of the overall development concepts (processes) employed at LMA.

Some authors and researchers would have you believe that the process is most important. You select the right or “latest & greatest” method; a good set of techniques; buy your support software (tools); add in schedule and budget to support these; and presto, you have a reliable and safe critical software system. From our experience, some of this is true, however, we advocate that you cannot overlook the human factor. Good engineers who think are necessary for the success of the process, and there is no substitute for human thoughtfulness. Additionally, no single engineer can understand all expects of these complex systems. This requires the use of teams with diverse skill sets, as well as a good process.

Click on the file attachment below to read the complete paper.