Test Model: Before you write your first line of code

Blog

Test Model: Before you write your first line of code

Test Model: Before you write your first line of code

If you have landed on this page, it means that you are interested in building a robust and reliable model of your software for your test automation needs. In addition, you already know that you need to have a separate abstraction level between the test case definition and the test model itself to make your testing maintainable.

In this first article we will focus on five generic principals you should follow:

1) Know your audience – Who will use the model?
2) Planning, planning, planning
3) Checking the interaction correctness – How to avoid flaky tests
4) Building re-usability
5) Test Model logging

The first question you need to answer yourself is – who will use, develop and maintain the model.

1) Know your audience – Who will use the model?

There is a significant difference of creating a software model for test automation experts and for test automation rookies. Which leads to two possible approaches.
One uses static classes whereas the other ones requires you to instantiate the class. Therefore, if you are seeking a simple solution for rookies go with the static approach. It is less intimidating in the beginning, however if you want to use all the advantages of object-oriented programming rather use the second approach. Either way you need to choose an approach which will suit you the most and be the most beneficial for your test engineers/test automation engineers.

Let me illustrate that by showing you two different approaches based on sample code for the google search page.

Static approach:

public static class SearchWindow
{ 
  public static Button SearchButton {get; private set;}    
  public static TextBox SearchBox {get; private set;}
}  
 
public void SearchForContent(string content)
{ 
  SearchButton = new Button('image:SearchButton'); 
  SearchBox = new TextBox('image:SearchBox');  
  SearchBox.Enter(content); 
  SearchButton.Click(SearchBox.WaitForDisappear()); 
}
Class Definition: SearchWindow.cs

Object oriented approach

public class SearchWindow
{ 
  public Button SearchButton {get; private set;}   
  public TextBox SearchBox {get; private set;}
}
 
public SearchWindow()
{
  SearchButton = new Button('image:SearchButton');
  SearchBox = new TextBox('image:SearchBox');
}
 
public void SearchForContent(string content)
{
  SearchBox.Enter(content) 
  SearchButton.Click(SearchBox.WaitForDisappear());
}
Class Definition: SearchWindow.cs
public class ScreenOverview
{  
  public SearchWindow SearchScreen {get; set;}  
 
  public ScreenOverview() 
  {  
    SearchScreen = new ScreenWindow(); 
  }
}
Class Instantiation

2) Planning, planning, planning

There is no way around it. You need to plan on how you would like your model to look like. The usual approach is to build up the model as you are developing your test cases. That is fine if you have in mind how its structure shall look like. Naturally, it should reflect the Subject Under Test (SUT). Sounds simple, doesn’t it? Sadly, it is not. You need to plan quite a few things before starting, just to save time later. I will give you some hints on what you should watch out for, however please keep in mind that is just a small glance. 

How granular should the interaction with your SUT be?
One could write a method which would open a File Menu, select open and load file in one call (complex model) vs someone could say “I leave it to the test step definition” (simple model). Compare both approaches below 

Complex Model:

public void LoadFile(string fileName)
{ 
  Navigation.FileMenu().Open();
  FileDialog.Enter(fileName);
  FileDialog.ConfirmButton.Click(FileDialog.WaitForDissappear);
}
Method definition
[TestStep(1,TestInput = "Load the demo.txt file")]
public void Step1(ITester t)
{ 
  Model.MainScreen.LoadFile("demo.txt"); 
}
Call to model in the test case

Simple Model:

public void EnterFileName(string fileName)
{  
  FileDialog.Enter(fileName);
} 
 
public void ConfirmFileSelection()
{     
  FileDialog.ConfirmButton.Click(FileDialog.WaitForDissappear);
}
Methods definition 
[TestStep(1,TestInput = "Load the demo.txt file")]
public void Step1(ITester t)
{ 
  Model.Navigation.FileMenu().Open();
  Model.FileDialog.EnterFileName("demo.txt");
  Model.FileDialog.ConfirmFileSelection(); 
}
Call to model in the TestCase 

In one case you are leaving everything to the model and creating the whole flow on its own. That means that one method will handle the whole process for opening the menu, handling the dialog, entering the file name and confirming the selection. In the other you provide individual calls that must be combined in the test case. You should ask yourself, how much freedom should you give to the test designers. On one hand if you limit them to just use the flow you are safe in a sense that they cannot use the model in an uncontrolled way. On the flip side you are limiting the designers e.g. they cannot modify the sequence of events e.g. by adding a step in between. You really need to decide as early as possible who you would like your test designers to use the SUT test model. Otherwise you might end up in a mixture of both which can lead to future maintenance problems

How will you handle the interaction between screens?
This is quite important to establish at the very beginning. You need to be very careful when allowing each screen in the model to know about one another. Consider the example below 

public void SelectCustomer(string customerName)
{  
  CustomerList.Select(customerName); 
}
Class 1: CustomersOverview.cs 
public void EditCustomerName(string newName)
{ 
  CustomerName.Enter(newName);
}
Class 2: EditCustomerDetails.cs 
[TestStep(1,TestInput = "Modify an existing customer name")]
public void Step1(ITester t)
{ 
  Model.CustomersOverview.SelectCustomer("demo_shop");
  Model.EditCustomerDetails.EditCustomerName("demo_bodyshop"); 
}
Call to model in the TestCase 

With this call all seems clear. The first class and the second class know nothing about each other and let the test case drive the decision on what shall the next action be. Let’s modify both classes a bit just to see how nasty it may get.

public void SelectCustomer(string customerName)
{ 
  CustomerList.Select(customerName);
}
 
public void SelectCustomerAndModifyName(string customerName, string newName)
{ 
  CustomerList.Select(customerName);
  EditCustomerDetails.EditCustomerName(customerName, newName); 
}
Class 1: CustomersOverview.cs 
public void EditCustomerName(string customerName, string newName)
{ 
  CustomersOverview.SelectCustomer(customerName);
  CustomerName.Enter(newName);
}
Class 2: EditCustomerDetails.cs 

and now the call

[TestStep(1,TestInput = "Modify an existing customer name")]
public void Step1(ITester t)
{ 
  Model.CustomersOverview.SelectCustomer("demo_shop");
  Model.EditCustomerDetails.EditCustomerName("demo_shop","demo_bodyshop"); 
}
Call to model in the test case 

This is really an exaggerated example, but you can see that we can create a very nasty case where we try to perform the same action twice (SelectCustomer) from two different places, or even worse have a circular reference between different methods.  I am not completely against exposing all classes within the model to each other, however if you want to do it you need to be extremely careful. Otherwise whoever will use your test model might fall in a nasty trap. 

Will you grant the test case the access to the controls of the model e.g. Buttons or will you just have the reference methods? 
This is something you need to know when you start to define your first screen. Would you like to expose the controls from the model to test cases outside? What does it mean? Let’s look at it in an example

public class SearchWindow
{ 
  public Button SearchButton {get; private set;}   
  public TextBox SearchBox {get; private set;} 
 
  public SearchWindow(IAppBasics appBasics)
  {
    SearchButton = new Button(appBasics.Tester, nameof(SearchButton),     
    Images.SearchPage.SearchButton);
    SearchBox = new TextBox(appBasics.Tester, nameof(SearchBox),Images.SearchPage.SearchBox);
  } 
 
  public void SearchForContent(string content)
  {
    SearchBox.Enter(content); 
    SearchButton.Click(SearchBox.WaitForDisappear);
  }
}
Class Definition: SearchWindow.cs 

If we now access the model from the test case we can use it as follows:

[TestStep(1,TestInput = "Search for TestResults.io")]
public void Step1(ITester t)
{ 
  Model.SearchWindow.SearchForContent("TestResults.io");
}

or like that

[TestStep(1,TestInput = "Search for TestResults.io")]
public void Step1(ITester t)
{ 
  Model.SearchWindow.SearchBox.Enter("TestResults.io");
  Model.SearchWindow.SearchButton.Click(SearchBox.WaitForDisappear);
}

The key here are the controls that you have exposed to the test definition

  • public Button SearchButton {get; private set;}   
  • public TextBox SearchBox {get; private set;}

Since they are public, everyone using the test model will be able to use them within the test case itself. Is it good or bad? That depends on how well the test designers know the test model. If you trust that they cannot misuse it then leave the access opened. Otherwise make the controls private or internal.

How will you handle the test data? 
Your test cases will require different test data. That is a fact and there is no way around it. Therefore, you should establish the handling of test data within your test solution as soon as possible. Why is it important to plan it in the beginning? Imagine that you have already written 50 test cases against the model with some static values and now you decide to feed the system with randomly generated data? It would not be that efficient to go back to 50 TCs and modify each single step where the data is referenced. Therefore, decide early on how you want to handle it. Create a separate data handler which would feed the test cases. In the beginning it could be only a class which randomly selects an entry from an extremely long list of values :). Later, you can extend this to read from a database, XLS, CSV, XML files or whatever is required by your test cases. The bottom line is, the sooner you will have a concept on how you want to handle the data, the better. 

3) Checking the interaction correctness – How to avoid flaky tests

This is the holy grail of test automation and if somebody could give you a universal answer to “how to avoid flaky tests?”, it would be just plain brilliant. Unfortunately, it is not that easy. Flakiness can come from many different places, coding mistakes, wrong test data, problems in the test environments etc. What you can control though is how fast you will identify the problems and fix them. Why is it important? Imagine your test case aborts in step 8 as the data needed for the test were not loaded in step 1. Sounds trivial doesn’t it? And now imagine you are analyzing the logs and going through hundreds of lines just to see that “hey, I’ve messed up in step 1”. It would be a lot easier if you would have identified the problem at the correct step rather than 7 steps later. That is why it makes sense to check each interaction with the SUT within the test model. At TestResults.io we do it as follows:

public void ConfirmFileSelection()
{
  ClickButton(ValidateThatButtonWasClicked();
}

As you can see not only do, we click on the button, but we check if the interaction with the system was successful within the click itself. That is applicable to each single interaction which is done with the SUT. This approach will enforce a way of thinking that you need to check the outcome of each interaction. As a result, you will always think about what the reaction of the SUT to a certain interaction will be and hence your model will be more reliable. 

4) Building re-usability

There are several places in your model where you will be able to reuse parts of the code. If you are tempted to just do a copy-paste I have one simple answer for you. Don’t do it! If you see that some code can be reused, there is no other way than do extract it and reuse it in other places. That it not the end of the story. The common functions are just one thing you can reuse, but there are many others. Imagine you have pop-ups in your SUT which all have an OK and Cancel button but dynamically changing message. Create basic screens, basic pop-ups, basics dialogs you will be able to inherit from. Here is a short example of how it can be achieved. 

public class BasePopUp
{ 
  public Button OkButton {get; private set;}   
  public Button CancelButton {get; private set;} 
 
  public BasePopUp(IAppBasics appBasics)
  {
    OkButton = new Button(appBasics.Tester, nameof(OkButton), Images.PopUps.OkButton);
    CancelButton = new Button(appBasics.Tester,    
    nameof(CancelButton),Images.PopUps.CancelButton);
  } 
}
Class Definition: BasePopUp.cs 
public class WrongUser : BasePopUp
{     
  public bool VerifyMessage(string expectedMessage)   
  {
    // code to verify the message
  }
}
Class Definition: WrongUser.cs 

As you can see you don’t need to repeat the definition of the OkButton and CancelButton but just use the buttons from the base class.

5) Test Model logging

Finally – the test model logging. This is as important as the SUT logging itself. Keep in mind that your test model is just another piece of software (same as your SUT). That means it is also vulnerable to the same problems as each single SW project e.g. coding mistakes. Introducing logging to your test model allows you to easily debug the problems within your code after the execution is done. Log as much as possible where it makes sense. At minimum log the test environment, SUT version under test, test model version, each interaction along with the time stamps, test data etc. Keep in mind that the bigger your test model is, the harder it will be to maintain it, therefore good logging will significantly reduce the time needed for you to debug any problems that might arise. 


Summary

This is just the tip of the iceberg of the things you need to take care of when creating a reliable and robust test model. If you would like to know more about how to build your test model or about test automation in general, please read the free chapter of the Complete Guide To Test Automation E-Book here

Leave your thought here

Your email address will not be published. Required fields are marked *