Selenium Foundation Program - Introduction to Automation Testing

What is automation testing?

Automated testing tools are capable of executing tests, reporting outcomes and comparing results with earlier test runs. Tests carried out with these tools can be run repeatedly, at any time of day. The method or process being used to implement automation is called a test automation framework.

What is the difference between manual testing and automation testing?

Automation Testing	Manual Testing
Automation testing perform the same operation each time	Manual testing is not reliable. Using this method test execution is not accurate all the time.
Automation testing is very much helpful regressions in testing where code changes frequently.	To execute the test cases first time using manual testing will be very much useful. But it is not sure that it will catch the regression defects under frequently changing requirements.
Automation testing will be useful to execute the set of test cases frequently.	Manual testing will be useful when the test case only needs to run once or twice.
After making Automation test suites, fewer testers required to execute the test cases.	To execute the test cases every time tester requires the same amount of time.
Automation testing can also be done on different machine with different OS platform combination, concurrently.	Using manual testing, testing on different machine with different OS platform combination is not possible, concurrently. To execute such task different testers are required.
Using Automation testing, testers can test complicated application too.	It does not involve in programming task to fetch hidden information.
Automation runs test cases significantly faster than human resources.	Manual testing is slower than automation. Running tests manually can be very time consuming.
Some time it is not helpful in UI testing	It is very much helpful in UI testing
Automation testing is very useful for automating the Build Verification Testing (BVT) & it is not mundane and tiresome.	To execute the Build Verification Testing (BVT) is very mundane and tiresome in manual testing.
Initial cost of automation testing is more than manual testing but useful always.	Manual testing requires less cost than automation.

Why do we need automation testing?

Automated software testing is important due to following reasons:
Ø Testing manually all scenarios is time and cost consuming.
Ø It is difficult to test for multi lingual sites manually.
Ø Automation can run tests unattended. (No Human intervention)
Ø Automation increases speed of test execution
Ø Automation helps increase Test Coverage
Ø Manual Testing can become boring and hence error prone.

When to go for Automation?

Do the Automation testing in the following scenario of the Software:-

Ø Requirements do not change frequently
Ø Access the application for load and performance with many virtual users
Ø Steady Software with respect to manual testing
Ø Obtain-ability of time
Ø Projects that need to test the same areas often

What are the criteria for automation?

Ø Identify areas within software to automate
Ø Choose the appropriate tool for test automation
Ø Write test scripts
Ø Develop test suits
Ø Execute test scripts
Ø Build result reports
Ø Find possible bugs or performance issue

Which test scripts can be automated?

Test cases to be automated can be selected using the following criterion to increase the automation ROI:

Ø High Risk - Business Critical test cases
Ø Test cases that are executed repeatedly
Ø Test Cases that are very tedious or difficult to perform manually
Ø Test Cases which are time consuming

The following category of test cases is not suitable for automation:

Ø Test Cases that are newly designed and not executed manually at-least once
Ø Test Cases for which the requirements are changing frequently
Ø Test cases which are executed on ad-hoc basis.

Different automation tools:

There are many licensed and open-source automation testing tools available in the market. Here are few automation tools presently used in the organizations.

Selenium
Ø Open-source testing tool
Ø Used for Regression testing, Cross browser testing (only web applications)
Ø Supports different programming languages

Click here to know how to download Selenium and configure a test environment in local PC.

HP-QTP (HP-UFT)
Ø Licensed testing tool
Ø Used for functional and regression testing. (both Web and desktop application)
Ø Uses the concept of keyword driven testing.
Ø QTP supports .NET development environment

Click here to know how to download HP-UFT and configure a test environment in local PC.

IBM-Rational Functional Tester
Ø Licensed testing tool
Ø Used for functional and regression testing. (both Web and desktop application)
Ø Supports wide range of protocols and applications. (Java, HTML, .NET, Windows, SAP, VB)
Ø It supports custom controls through proxy SDK (Java/.Net)

Click here to know how to download IBM-RFT and configure a test environment in local PC.

Watir
Ø Open-source testing tool.
Ø Used for Regression testing. (only Web applications)
Ø Supports programming language like Ruby.

Click here to know how to download Watir and configure a test environment in local PC.

SilkTest
Ø Licensed testing tool.
Ø Used for Functional testing, Regression testing of e-Business applications.
Ø Supports programming language like C++.

Click here to know how to download Silk test and configure a test environment in local PC.

How to choose an automation tool?

Below are few details which need to be considered while implementing automation in project:

Ø Identify areas within software to automate
Ø Choose the appropriate tool for test automation
Ø Write test scripts
Ø Develop test suits
Ø Execute test scripts
Ø Build result reports
Ø Find possible bugs or performance issue

Also we need to have below details to automate our test scripts:

Ø Environment Support
Ø Testing of Database
Ø Object identification
Ø Scripting Language Used
Ø Support for various types of test - including functional, test management, mobile, etc...
Ø Support for multiple testing frameworks
Ø Easy to debug the automation software scripts
Ø Ability to recognize objects in any environment
Ø Extensive test reports and results
Ø Minimize training cost of selected tools

What are the advantages of automation testing?

Ø 70% faster than the manual testing
Ø Wider test coverage of application features
Ø Reliable in results
Ø Ensure Consistency
Ø Saves Time and Cost
Ø Improves accuracy
Ø Human Intervention is not required while execution
Ø Increases Efficiency
Ø Better speed in executing tests
Ø Re-usable test scripts
Ø Test Frequently and thoroughly
Ø More cycle of execution can be achieved through automation
Ø Early time to market

What are the different types of software testing that can be automated?

Ø Smoke Testing
Ø Unit Testing
Ø Integration Testing
Ø Functional Testing
Ø Keyword Testing
Ø Regression Testing
Ø Data Driven Testing
Ø Black Box Testing

What risks are involved in Automation testing?

One must consider below scenarios first before choosing automating test scripts:

Starting cost for Automation is very high –
Ø Automation tool purchasing cost,
Ø Automation testing tool training
Ø Maintenance of test scripts cost is very high.

Note : Spending lot of money in the initial stage without considering the risk factors doesn’t give better results.

Is 100% automation achievable – Automation testing cannot be 100% and don’t think of that. Surely you have areas like performance testing, regression testing, and load/stress testing where you can have scope of reaching near to 100% automation. Areas like User interface, documentation, installation, compatibility and recovery where testing must be done manually.

Unfixed modules should not be automated – Be careful before automating user interface. If user interface is changing always, cost associated with script maintenance will be very high. Basic UI automation is enough in such cases.

Is your application is stable enough to automate – It would be corrupt indication to automate testing work in early development cycle (Unless it is agile environment). Script maintenance cost will be very high in such cases.

Tester should have good programming knowledge – Good tester should have good programming knowledge, tester who really has good programming knowledge can use the Automation tool better and resolve applications issue better, otherwise it is just a time pass on tools.

How to get maximum ROI of automation testing?

To get maximum ROI of automation, observe the following

Ø Scope of Automation needs to be determined in detail before the start of the project. This sets expectations from Automation right.

Ø Select the right automation tool: A tool must not be selected based on its popularity but it's fit to the automation requirements.

Ø Choose appropriate framework

Ø Maintain Scripting Standards- Standards have to be followed while writing the scripts for automation.

Some of them are- Create uniform scripts, comments and indentation of the code

Ø Adequate Exception handling - How error is handled on system failure or unexpected behavior of the application. User defined messages should be coded or standardized for Error Logging for testers to understand.

Ø Measure metrics- Success of automation cannot be determined by comparing the manual effort with the automation effort but by also capturing the following metrics.

Ø Other things to note –
- Percent of defects found
- Time required for automation testing for each and every release cycle
- Minimal Time taken for release
- Customer satisfaction Index
- Productivity improvement

What are framework guidelines in automation testing?

A framework is set of automation guidelines which help in
Ø Maintaining consistency of Testing
Ø Improves test structuring
Ø Minimum usage of code
Ø Less Maintenance of code
Ø Improve re-usability
Ø Non Technical testers can be involved in code
Ø Training period of using the tool can be reduced
Ø Involves Data wherever appropriate

What types of frameworks used in automation?

There are four types of framework used in automation software testing:
Ø Data Driven Automation Framework
Ø Keyword Driven Automation Framework
Ø Modular Automation Framework
Ø Hybrid Automation Framework

Conclusion:

Choosing automation tool is very difficult and purchasing the right automation tools requires analysis and proper knowledge about business requirements of the application. Although automation can't fulfill 100% requirement coverage, it is very useful in automating the stable requirements (regression testing) within less time.