We have a very complex software product that is installed and runs on a Computer or Server. The Software is managed by a Management Console in the Cloud.
For years all of our End to End testing was manual. But this was very resource intensive and time comsuming. So a couple of years ago we moved to a more agule development approach and built a test automation system to run these End to End tests. However, we quickly found limitations for example we couldn't have automated tests check what is being displayed in the UI running on the Computer, or check what is being displayed in the Management Console in the Cloud. So we can never fully replicate in automation what the manual tests were doing.
The result is that we still run, some if not all, of the automated tests manually so check the parts that the automation can't. So it hasn't really saved us any time or effort. Also the reliability of the automated tests is an issue due to trhe complexity of the software and it having so many moving parts. We often run into timing related issues.
Is it possible that some software does not make a suitable candidate for End to End test automation? If so, what sort of approach should be taken in that situation? I've been ready about test automation a lot and looked at a number of different tools and systems, but both the tools and the theory all seem to be based around website testing, rather than testing Desktop software.
Many thanks in advance for any thoughts and suggestions.
