Traceability. There is more than just results.
Having test results is the first step. Running tests automatically to gather those results with less effort is the next. If your automation stops there you might have a problem.
Test results are not testing
Having passed test results a.k.a. a green light is amazing. But do make a common mistake. Functional end-to-end tests are not like unit-tests. Unit-tests are pure developer tests and the idea is all of them have to pass otherwise the code does not work. There, having a result is enough.
But in end-to-end tests it is more than just a few lines of code that get tested with every test. It is the full application with all of it dependencies. Without being able to tell exactly what went wrong your end-to-end test results do not give you any benefit.
Reports and traceability
An end-to-end test automation platform should provide a step-based results as a minimum. A real solution should provide step-based results, reports, and custom logs for your application. In addition, it should automatically build up a hierarchical system of tests that is fully traceable.
In TestResults.io you get for every step a trace to its test report, from the report to the configuration it was executed with, e.g. the software and the test environment, from the configuration to the test plan revision and from the test plan revision to the actual test case. For all of these stages you can see the individual results of the last ten executions in comparison all the time.
Try for Free
Give you confidence
Only this level of detail and structure gives you confidence in your software and in your decision. Traceability allows you to go back in time and prove that your decision was correct based on the available results. Audit proven. Right testing, best confidence.