Add Value to your Automation....

Simplify Automation Creation and Maintenance....

I am sure anyone involved in automation testing would have encountered the situation below:

Your team is working hard on an application, lots of automation scripts are created, the test suite is running all fine when suddenly one day you wake up to find that all scripts have failed..
Nothing changed in your code..right? Then what caused this catastrophic failure..
Everyone is behind you asking for the reasons why it failed and you yourself do not know what happened...Strange isn't it...? How can a person who has written the scripts be unaware about why they failed in the night..
I will tell you why....

1. The scripts were procedural test scripts which were just recorded and replayed with some checks in place, some loops, some if else statements and so on...

2. These kind of scripts are difficult to maintain when the application changes, and in development the application changes quite frequently from build to build. Some new objects get added to the application, some old objects are not present, some objects have their properties changed and so on....

3. Testers generally have an attitude that automation tools are just about record and replay and nothing more, they are unaware about the capabilities which they can bring into it.

How do you overcome all these issues then?

You can simplify maintenance by avoiding to fall in the pitfalls of poor record and replay scripting..which will not lead you to automation success.

Automation becomes of little value when it does not provides useful information which can be reviewed by testers and most importantly my managers (after all managers are interested in the number game, in the % age of bugs covered by automation etc)...

Never rely on the tools built in reporting features. Create a library of functions which can be then reused by each test. Using the library the output can be given in so many formats. Find a few examples here.

1. HTML Files: People love coloured HTML pages, plus on top of it if you can add a little bit of Java Script to it then it can be more presentable.
2. Excel Sheets: Excel sheets can be processed easily by machines and people can do lots of manipulations with the data just by using few mouse clicks.
3. Database: Results can also be posted directly to a database and which can then be used to comapre with a pre existing standard template.
4. XML File: Another very popular medium of output.

Reported results should contain enough information which can answer the following:

What happened to the application when error occured.
Exactly at which point did the error occured.
Which script and which line of code caused the error.
What action does the Framework took when error occured.
How many errors were application specific and how many errors were due to errors in scripts itself.
How many test cases ran totally?
How many passed and how many failed?

One more feature can be built into the framework..Snapshot of the application can be taken and stored whenever an error occurs so that it becomes more easy to debug the errors.
Snapshots also provide a real proof of where the error occured. Snapshots can also be customised with the Log File ouput, for example the time stamp in the log file for a particulat step can be appended to the Snapshot file name so that it becomes easier to find the actual state of the application at the point when error occured.

Most Importantly remember:
An automation effort which runs for several hours is of little importance if it takes another several hours to figure out what went wrong and to interpret the results..

And yes i will be posting many new ideas for automation..so keep reading...

Happy automation..
Madhukar..