Can software be too complex for end to end test automation?

James Barker's picture
James Barker asked on May 15, 2020 - 3:48am | Replies (1).

We have a very complex software product that is installed and runs on a Computer or Server.  The Software is managed by a Management Console in the Cloud. 

For years all of our End to End testing was manual.  But this was very resource intensive and time comsuming.  So a couple of years ago we moved to a more agule development approach and built a test automation system to run these End to End tests.  However, we quickly found limitations for example we couldn't have automated tests check what is being displayed in the UI running on the Computer, or check what is being displayed in the Management Console in the Cloud.  So we can never fully replicate in automation what the manual tests were doing.

The result is that we still run, some if not all, of the automated tests manually so check the parts that the automation can't.  So it hasn't really saved us any time or effort.  Also the reliability of the automated tests is an issue due to trhe complexity of the software and it having so many moving parts.  We often run into timing related issues.

Is it possible that some software does not make a suitable candidate for End to End test automation?  If so, what sort of approach should be taken in that situation?  I've been ready about test automation a lot and looked at a number of different tools and systems, but both the tools and the theory all seem to be based around website testing, rather than testing Desktop software.

Many thanks in advance for any thoughts and suggestions. 

1 Answer

Manish Mehta's picture

You have to break very complex software product, in question shared it would be front-end and back-end verification at minimum. Your usual automation tools in use for front-end and back-end would still work. Automation framework design would stitch together both (and multiple for very complex) to complete e2e automation (for e2e workflow).

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.