Recently I overheard a conversation between a test analyst and a business analyst about how a function should be tested. The response from the business analyst was, "If it is not breaking the application, it must be working fine!" Testing staff comes across such scenarios where a part or functionality of the application under test is not "testable." The tests they carry out are not conclusive enough to say that the functionality is working as specified. In this week's article, Ipsita Chatterjee defines testability and looks at the benefits of incorporating it in the products. Also discussed are simple ways to monitor the incorporation of this non-functional requirement in the software development life cycle and a few industry myths about testability.
What is Testability?
Testability is a non-functional requirement important to the testing team members and the users who are involved in user acceptance testing. It can be defined as the property that measures the ease of testing a piece of code or functionality, or a provision added in software so that test plans and scripts can be executed systematically.
Why is Testability necessary?
A typical software development lifecycle involves requirements gathering, analysis, design, coding, testing, implementation, and maintenance. A testable product ensures complete execution of the test scripts. Assuming that good test coverage is applied, most of the defects will be uncovered and fixed before the product is released. This insures customers will report a minimum number of defects. A lot of money is spent on supporting and maintaining a product after its development. Testable products are easy and less costly to maintain. The chances of achieving customer satisfaction with such products is are much higher. Hence testability is an important attribute to the maintainability of any software product.
How is this attribute measured and monitored?
Being able to test software, a piece of code or functionality, depends on what the user can see and control, known as observability and controllability.
Observability enables a tester or user to see the external and internal of the software. When a user receives the correct expected output, but the internal or the background processes are not quite what was specified in the requirements, defects are often found elsewhere. This is more important in the case of unit and integration testing rather than a simple black box testing.
Controllability is a measure of how easily a tester or a user can create difficult scenarios to test the software under extreme circumstances. For example, behavior of an application cannot be tested very easily when the hard disk is full or table overflow conditions exist.
Incorporating Testability into Software
There are so many methodologies of software development that it is difficult to list specific or stringent rules for creating testable software. Just like testing should occur from the very beginning of a project, project artifacts should be reviewed for testability from the beginning as well.
In a traditional waterfall model, the phases we have are software specifications, detailed design specifications, coding, testing, and implementation. Most practical usage of this model adds an "iterative" approach at each phase. Testability can be incorporated into the various stages as listed below. The list is in no way exhaustive. Practicing some of these steps can lead to more innovative ways of addressing the non-functional requirement.
- 1. Software Specifications Phase: During the review process of this document, the testing team should be specifically quizzed on their understanding of the requirements, and how the requirements form into several functionalities. Apart from the clear and straightforward requirements of what the application should do or is expected to do, the behavior under abnormal scenarios should also be documented clearly. Walking through the Use Cases used to formulate the requirements can also be very helpful. Checklists should facilitate addressing issues on how to create different scenarios.
- Detailed Design Phase: In order to incorporate testability in this phase inputs and expected outputs should be clearly stated. The checklists prepared during the last phase should be supplied to the designer. The design should elucidate the system path clearly so that testing team knows which programs are being touched in what scenario. As mentioned before, a correct expected output is not enough to ensure that the background processes are giving the correct results. Provisions for adding extra code or extra interfaces to test the application under "difficult