Testing Embedded Software
Embedded systems are everywhere. Modern appliances from TVs to mobile phones cannot function without them. Embedded software has become a vital part of a growing range of automotive, aerospace, biomedical, and military systems. As many of these applications are potentially life-threatening, the need for rigorous software testing has never been more compelling. This book provides a comprehensive and practical overview of testing strategies for embedded software.
Highlights of the book include:
- Practical advice on testing that works even when time and resources are limited
- An in-depth treatment of test design techniques that can be directly applied in your projects
- Detailed descriptions of roles, tasks, and responsibilities with organizational structures to help you establish a professional test organization
Testing Embedded Software is an invaluable guidebook for test engineers, programmers, project managers, and team leaders in the embedded systems industry. It is the first book to provide a complete, in-depth overview of embedded systems testing.
Review By: Jon Hagar
07/08/2010This book addresses a long-ignored area of software and systems testing. The industry has produced generic books on testing, web based testing, and other specialized areas, but this is the first book covering the area of embedded software. The book starts with a standard overview of test concepts and methodology, which are designed to assist testers in assembling test approaches and techniques for embedded software. The book's organization continues by covering both the ideas of a master test plan, and then detailed test plans covering lower levels of testing.
Next comes consideration of lifecycle issues, including both the traditional V and newer spiral models. Related to lifecycle issues, the authors consider the activities of both developers and independent test teams. Developer test activities are correlated to the lifecycle stages. Independent test team efforts follow the lifecycle stages explaining what testers are doing at each stage of planning, test preparation, test specification, execution, and completion.
Once the embedded lifecycle and test activities are covered, the book details techniques specifically related to embedded systems. Techniques considered in sections of the book include: risk analysis, testability, inspection, test design, and tester checklists. Each of these is presented with an introduction and then a "cookbook" based process outline. Particular detail is given to test design techniques and the all-important checklist with details specific to the embedded world.
After techniques, there is a section that details infrastructure considerations such as the test environment, which can be a large problem area for embedded systems, tools that again can be unique to embedded software, and automation concepts. The issues of embedded systems are further explored in the areas of input/output, in what the authors term "mixed signal analysis" and digital domain problems, both of which have aspects that are peculiar to the embedded domain.
Finally, there are sections detailing fault-tree analysis and failure-modes-and-effects analysis (FMEAs), both of which are important in testing critical embedded software systems. After dealing with some of the technologies and approaches for testing embedded software, the later sections of the book discuss the important issues of human relations and staffing. These include tester skills, team organizational structures, and finally, the importance of software configuration management in testing embedded systems.
A problem with any test book is what to cover and how. Because of size limitations there is a lot of material that must be skipped or summarized. This book is written as an introduction to the topic of testing such systems since it covers the important topics of planning, organization, and techniques that a tester will face when entering the embedded software world. Additionally, the book discusses in-depth issues that make the embedded tester's job different from that of a Web-based tester. Several sections and topics such as "Mixed Signals" and "Evolutionary Algorithms" treat topics thoroughly and are applied uniquely to the problems faced by embedded software testers.
The book does make a number of claims that are generally accepted in the testing community. But the book does not include data or references to support the claims. There was minimal coverage of topics such as test oracles (a judge or predictor of the outcome of a test); simulation and commercial tools that can be used to support embedded testing; or coverage of regression test issues that can complicate testing embedded software.
Many topic areas were probably limited due to space and size considerations. There were several sections that covered interesting "state of the art" topics. One topic is an interesting treatment of evolutionary algorithms, which are new to many of us and provided good "food for thought" reading. The book's coverage includes enough depth that a tester should be able to get a good start based on the information provided.
Another useful feature of the book is the writing style of some sections, which is such that a test group could pick up the book and "plug it into their environment." These procedures and plans can make the book useful to groups that lack standards and are looking to "jump start" the maturity of their organization. Additionally, the book has a lot of good lists and an information scheme. These features make the book very practical and a good starting point.
The end of the book contains a series of appendices dealing with various topics, including risk classifications, state based testing, automation blueprints, an interesting set of pseudocode covering evolutionary algorithms that can be used in generating test data for embedded software, an example test plan outline, and the section every test book should have—definitions of terms used in the book. In an appendix, the state chart section is presented as a tutorial on how to diagram state charts and apply them to embedded software. The appendix on "blueprint of an automated test suite" provides another "how to" covering producing a planner, reader, translator, test actions, initialization/synchronization, error recovery, and test reporting/checking.
The authors have filled the book with insightful comments and observations. People faced with having to start a project testing embedded systems will find the ideas presented in this book great reference material for their bookshelves. I found myself highlighting these gems, so that I can find them quickly. And, while the book may not cover every area that is being researched or developed in embedded system because of size or space limitations, there is a wealth of information and in-depth ideas.
Review By: James Diefenbach
07/08/2010"Testing Embedded Software" by Bart Broekman and Edwin Notenboom is about the importance of software to all industries, focusing on how software has become deeply embedded into all products. A key example highlights the auto industry that embeds systems in engine controls, safety devices, comfort controls, and entertainment devices. Embedded systems have become so complex and pervasive that the need for quality control through testing is more imperative than ever.
In response to this argument, the book focuses on how to control and organize complex testing, taking a more practical approach rather than theoretical. Scientific and engineering knowledge is used to design test approaches. The book talks about test automation and the need to thoroughly test complex embedded systems in methods that can't use manual testing approaches.
Overall I like the book because of its real life examples and how it points out the need for software testing. The appendices can be used as reference material and guidelines. It has checklists, examples, a blueprint test suite, and state transition charts. The writing style is much like a university textbook with examples, theories, and approaches.
The content's relevance to today's marketplace is appropriate. It points out how software has become embedded into almost every product available. This helps to emphasize the importance of proper and thorough testing; otherwise product quality and safety are compromised. The text relates to students beginning their study of software testing as well as professionals testing in the field. It would also be very helpful to development staffs trying to understand the full scope of the embedded systems designed. The book also provides a guideline of how to improve an organization by improving the testing strategies.