quality management Questions

We have a very complex software product that is installed and runs on a Computer or Server.  The Software is managed by a Management Console in the Cloud. 

For years all of our End to End testing was manual.  But this was very resource intensive and time comsuming.  So a couple of years ago we moved to a more agule development approach and built a test automation system to run these End to End tests.  However, we quickly found limitations for example we couldn't have automated tests check what is being displayed in the UI running on the Computer, or check what is being displayed in the Management Console in the Cloud.  So we can never fully replicate in automation what the manual tests were doing.

The result is that we still run, some if not all, of the automated tests manually so check the parts that the automation can't.  So it hasn't really saved us any time or effort.  Also the reliability of the automated tests is an issue due to trhe complexity of the software and it having so many moving parts.  We often run into timing related issues.

Is it possible that some software does not make a suitable candidate for End to End test automation?  If so, what sort of approach should be taken in that situation?  I've been ready about test automation a lot and looked at a number of different tools and systems, but both the tools and the theory all seem to be based around website testing, rather than testing Desktop software.

Many thanks in advance for any thoughts and suggestions. 

I would like to find out strategies used by other QA as well as which methods would work best in this environment as I am used to working in a dev environment and not financial institution 

I'm currently building out a Test PMO for an insurance company. We have a long term, fixed-price contract with a vendor using a waterfall model.  I'm currently working on KPIs and metrics which I will use to tell the testing story, which means both technical and business comsumers will need to find value in them. I'm conducting targeted interviews atm  but am interested in hearing if any of you have had experience in this arena.  

What kinds KPIs / metrics have you found valuable?

What kinds of chatring / mapping have you found to be confusing?

What considerations would you want to address?

Any additional comments?

I am on a team of two working on standardizing testing for the company. The culture here is very resistant to change, as I am sure many colleagues on here have experienced themselves. I want to continue pushing this elephant up the stairs, but I am afraid some of these templates (IEEE 829 standard templates) are too dense for most of them. My mentality right now is to take this in strides. I want to avoid creating a 12 part test plan that people need to follow if they are just going to get defensive and say they do not have the time to even test, or test this way. Of course we have plans to educate as much as we can, but this is going to take years. What are your suggestions, then, with creating templates? Right now I have identified three that need to be used across all teams (Test Strategy, Test Plan, and Test Summary). We have plans on creating a best practice document/resource for teams to educate them on the industry standards.