Teams and organizations often get excited when they hear the word “automation.” It’s like a new gadget that everyone wants a piece of—at first. But after a month or two, no one cares, and we shift our focus to other gadgets or, in this case, other project priorities and developments.
In order for automation to be successful instead of just a flash in the pan, companies need to approach it with realistic goals, the right attitude, and a willingness to make it work.
It is interesting to note that, no matter what domain or company I’ve worked for, there were some common problems that always affected automation projects. Here are six top reasons I’ve consistently seen automation projects fail. Keeping these pitfalls in mind will help you to avoid them and instead build stable automation frameworks, making the endeavor a collaborative experience so that the whole team owns automation.
1. People don’t understand what to automate (and what not to automate)
It is really important to understand what to automate and what not to automate.
Say you have a webpage. It is good to automate the different functionalities of the webpage, but it may be a bad idea to use automation to check rendering issues or location of elements on the page. This is because it is hard to know how the screen displays in different form factors due to so many different devices, browsers, and screen sizes. Also, if you start using x,y coordinates to verify element location on the page, the tests can become flaky when run on different screen sizes and resolutions. These scenarios are better off tested manually by human testers.
We always want to automate something that is stable, that is less prone to change, that has to be repeated multiple number of times, or that will help save valuable testing effort. By automating mundane and repetitive tasks, testers can spend their valuable testing effort on performing exploratory testing.
2. The team doesn’t have technically skilled people
Doing automation the right way requires testers to have a certain level of technical expertise. That being said, getting skilled people who have sufficient technical knowledge can be expensive and time-consuming. Not many companies, like start-ups, for example, can afford to hire automation engineers.
It would be worthwhile to hire a few technical testers who can kick-start automation projects and train others on the team to get involved in the automation effort. If that’s not an option, you can always invest in a tool that can help anyone do automation, irrespective of technical expertise.
3. There’s low visibility around automation
I’ve often seen teams where two to five people are in charge of automation and no one else has any clue of what is going on in terms of the automation efforts. This lack of visibility sets you up for failure because no one is going to take your work seriously, especially if you are not able to let people know how your efforts help to solve testing problems and provide value.
Fortunately, there are some easy ways to make your automation visible:
- Create an automation wiki page with information such as what features are being automated, what modules are being covered, and how your automation framework is set up
- Make the results of your automation visible to the whole team via email notifications and dashboards, and discuss the progress in daily stand-ups and other team meetings
- Hold lunch-and-learn sessions covering different trends, good practices, and tools related to automation
It is important to make automation a collaborative effort and involve the whole team in planning and writing it.
4. Applications are not easily testable
Teams do not often pay attention to the testability of their software. What do I mean by that? If you build an application, you also need to make sure it is easily testable from the unit level, system level, integration level, and acceptance level.
When you neglect this aspect, we end up having an application that is almost impossible to test from any level. Then we think automation is not working for the team, when the real problem is that some of the developers are building applications that are too complex and not testable.
Ensure you discuss the testability of a story, a feature, and requirements during backlog grooming and the sprint planning meeting, before the development effort starts on a feature. This will help to prevent problems later in the development lifecycle.
5. The goals of automation are vague
We start with the goal of building a robust automation framework that seamlessly integrates with the CI/CD pipeline, is maintainable, and runs in a stable and consistent way, giving quick feedback about the application. All these objectives are good on paper, but to get to this level of maturity, we need to start small.
This is where most automation projects fail: They start doing something complex with automation and end up having to refactor the entire framework after they are not getting the necessary value from it. This leads to wasting time, cost, and effort.
Start by identifying two or three high-level functionalities that are stable, and automate them first. Based on this, collect feedback on what went well and what did not. Then, once these tests run consistently and are stable, start identifying other tests to add.
It is also important to separate your automation suite into smoke tests and regression tests. Smoke tests would be the tests that run after every code check-in. They should typically finish running within five minutes to get quick feedback on new code changes. A regression test suite can run on a daily basis and covers different functionalities of the application. Your needs may vary based on the context of the project, but this is one approach that has worked well for me.
6. The team doesn’t have good processes to handle automation tasks
This is one of the most common problems with test automation: There is no proper process in place to handle automation tasks. Each team member has their own interpretation of the process, leading to chaos and failure of the automation project.
The process of automation should be streamlined and clearly defined right from the start to the finish of the lifecycle of a story. There should be no ambiguity about who has to do what and when during the software development lifecycle. Treat automation tasks as separate stories, just like how we implement different features as part of stories. This helps the team respect the automation tasks and makes them much easier to manage. Finally, make this process visible to the whole team by writing the steps on white boards and as checklists in stories, and constantly remind the team about them in the daily stand-ups and other team meetings.
Keeping these six common reasons for failure in mind will help you proactively avoid them when implementing an automation effort with your team.