Workday, Salesforce, Dynamics 365 Testing: Challenges, Tools and Testing Approach for Software-as-a-Service (SaaS) Applications.
Software-as-a-Service (SaaS) solves cost-intensive problems and brings countless benefits, the dream of every business decision-maker. However, a nightmare for the IT manager: The provider defines when updates are to be installed and company-specific roll-out plans can no longer be taken into account. The changes in the enterprise apps world due to SaaS also pose challenges for testing.
From an IT perspective, SaaS applications offer some advantages for enterprises, but the problems that can arise if quality is not ensured through proper and sufficient testing are serious.
With examples from the widely used SaaS solutions Workday, Salesforce and Microsoft Dynamics 365, we will show where the challenges lie and how you can solve them. However, the tips and information apply to all popular enterprise SaaS application.
4 Benefits of Enterprise SaaS Applications for IT
Even if every IT manager can list the problems of SaaS applications that are inevitable after the initial implementation while sleeping, the advantages outweigh the disadvantages for many enterprise customers.
- Recurring deployments and thus the development of an update & migration strategy, procurement and maintenance of hardware have been eliminated.
- Infrastructure costs are significantly reduced.
- Everything can be made available with a 99.999% availability from the web as software-as-a-service: ERP, HCM & CRM.
- From the cloud on the iPhone somewhere at the beach in Bora Bora.
Even in otherwise critical areas such as state administrations, military, etc. SaaS solutions are meanwhile accepted.
The challenges of software-as-a-service in quality assurance
What is usually seen with SaaS – but also often overlooked at the beginning – is the release cadence. The SaaS supplier defines the update cycles and the customer must comply.
Every user has the latest version of the software at all times. This leads to problems in quality assurance.
Using Workday, Salesforce and Microsoft Dynamics 365 as examples, I will show you what is important when testing SaaS applications. This is a topic that is often overlooked, even in the “as-a-service” world.
Does testing have to be carried out despite having a clear revisioning?
For testers, there are three different types of software
- Developed in-house, classic R&D department
- Purchased with adaptation/integration effort, classic implementation/consulting scenario
- Purchased and can be rolled out directly, classic IT operations scenario
Software-as-a-Service solutions such as Workday (HCM), Salesforce (ERP), Dynamics 365 (ERP) fall into the second category. A standard solution is purchased and is adapted to the company’s own processes by means of an implementation partner. In the past, as today, this is classic integration.
With one important exception: Some SaaS providers guarantee revision security. That is, own adaptations implemented on version 1 will work on version 2 without adaptation or new errors/side effects. Here, the SaaS provider takes over the complete regression testing, for changes it has introduced. This was not the case with on-premise installations, Microsoft Dynamics in particular should be mentioned here.
The assumption would be that once a SaaS implementation works, it will always work. This may be so if you never make a change to your customizations and luck is on your side.
Thus, revision security is only presumptive. To ensure that the solution still works as intended for you, including its implementations, it must be tested. This is also recommended by the SaaS companies themselves.
How are SaaS applications tested?
Revision security may be a relief in development, but it does not change much for the tester.
On the one hand, because your own extension often has to be adapted again, since the business process is constantly evolving. No company can afford to stand still.
On the other hand, a SaaS provider cannot guarantee one hundred percent freedom from errors. But you have to make sure that everything runs smoothly in your company. The more frequent and specific testing is, the more errors will be found.
Basically, the same applies to SaaS application testing as to other purchased on-premise software solutions such as SAP, Oracle or Microsoft in the past.
Not only can something go wrong with the extensions, but also with the SaaS application itself, as in the following example from Salesforce.
Testing pays off: The Salesforce Bug
In the “New Lead” dialog, the arrow from the scroll bar is hidden by the “Save” button and is therefore not clickable. Bugs like this often go unnoticed during testing, since technically all functions are correctly present, but are not usable for the user.
We recommend testing SaaS applications visually on the GUI. This ensures that technically everything works properly AND can be used by the user.
Here’s what a testing strategy for Workday might look like
When testing Workday, Salesforce or Microsoft Dynamics, the power of the solutions requires a structured approach. Focus on the following points in your testing strategy. Performance testing of SaaS applications should not be underestimated.
- Navigation & Search – Are the individual jump targets correct? Are the overview diagrams displayed correctly
- Security – Who has which access to what data? Is everyone seeing the right dashboards, the right reports?
- Data – Is the data loading correctly? Is the data being prepared correctly for the reports?
- Business process – Is the process running logically correctly? Are all rules executed correctly and steps processed in the correct order? Are all triggers fired correctly?
- Integrations – Is the configuration of the import/export interfaces correct? Is there a connection? Is the data transferred correctly?
- Reports – Are the customized reports displayed correctly? Are there too many Workday delivered reports? Do the right roles have access to the right reports?
- Performance – How is performance in the branch offices? Are Europe and China equally accessible?
Softwaretesting on-premise vs. Cloud (SaaS)
The basic tests have not changed from on-premise to SaaS. New is the increased update frequency, which is defined externally by the agile development of the providers.
SaaS solves the deployment problem of on-premise, but the high update cadence means that testing has to be done all the more frequently. How can you ensure that your tests are executed efficiently and after every update?
By automating the test cases. This way you run thousands of test cases simultaneously within a very short time. This is not only efficient, it also saves costs and manpower.
In some cases, validation is explicitly required, e.g. in regulated environments. In any environment, however, you should be aware that one error in the configuration is enough to drive the best-tested enterprise application solution from the manufacturer into the abyss.
Example of a configuration error in Microsoft Dynamics 365
Due to a configuration error, the decimal separator was used incorrectly in all interfaces in Microsoft Dynamics. Accordingly, incorrect numerical values are forwarded to the dependent systems.
Test automation is mandatory
To catch bugs like this before the software is deployed across the enterprise, test automation is essential in this agile SaaS environment.
Rely on a test automation solution that allows you to test different SaaS applications as well as other applications across multiple platforms, such as TestResults.io.
Book a free assessment call with one of our SaaS testing experts now.