Success Story: Manual to Automated Testing Test Automation Framework Integrated with Azure DevOps
Who is the client?
The client is the Department of Records for a major U.S. city, responsible for maintaining records and fulfilling report requests.
Client was having difficulty identifying repetitive test cases
What was the client’s problem?
The client was having difficulty identifying repetitive legacy manual test cases and their migration to be able to adopt a test automation framework. They had a goal to reduce cost and increase productivity by integrating the automated testing process inside their CI/CD pipeline which would automate the testing, reporting and deployment of the application.
Challenges
- Little to none documnetation
- Not enough visibility of the manual test cases
- Lack of an end-to-end defined automation process
- Higher learning curve for test automation
Solutions
- Formal organizational project set up
- Maintaining of current manual test cases with various attribute like priority, complexity, fitness to automation etc.
- Workshop to identify the right applications for test automation
- Resourced with dedicated and skilled automation engineers
What services did Data-Core provide to solve the problem?
Data-Core decided the best option was to create a custom framework that would benefit the client, not just from the test automation perspective, but also help in reducing the manual effort in test execution and deployment of the application. Hence, the solution built comprises the following:- Combine several open source automation tools/libraries to weave the benefit of each while circumventing their weaknesses
- Develop commonly reusable features as much as possible, in order to reduce scripting effort
- Initialization of application under test
- Take screenshots
- Explicit waits
- Create automated reports
- Automate the test execution process each time a new build is available
- Publish the report in a dashboard
- Deploy the build where necessary
Framework benefits: automation,
execution & deployment
Tools and Technologies used:
- Java as a general programming language
- Selenium as a browser automation library
- TestNG as an automation framework
- Extent reports in conjunction with iTest Listeners for rich style reporting
- Jenkins server for continuous integration
- Microsoft Azure DevOps pipeline to automate the execution and deployment process
Implementation:
Data-Core designed a data driven framework in Selenium that had all test data managed from Microsoft Excel. This allowed execution of same test cases for various types of data without having to execute each of them manually. The commonly reusable features encourages reusability of test codes by reducing scripting time and effort. Consequently, plugging new features into the framework was greatly reduced.
Framework:
Azure Repos
In this project, we have utilized Azure Repos Git to store our project and test source codes. Both codes are located in their respective branches. The screen shows the automation codes residing in master branch of Azure repo.Pipeline/Release
The next step was to create a release pipeline in order to implement the CI/CD architecture. The basic steps in implementing the release pipeline are as follows:Configure the Azure Pipeline to use the Azure Repo where the automation codes reside Define the build process – either automated (triggered whenever new code is pushed to Repos) or manual Create Release – automation testing is triggered, result published in desired directory Deploy build to any target of choice – depending on test result, project source code is then tested and packaged to be deployed to any target, i.e. to production branch, etc.
Configure the Azure Pipeline
The first job of this build process is to configure a Pipeline in Azure. When it comes to automated testing, the pipeline consists of source codes for automation residing in Azure Repos. The next step is to queue the codes for execution in the CI pipeline and produce results. Once the results are published, the same is saved as an artifact to automatically trigger the next step, i.e. shipping application code to production branch.
Create a Release
The next step in this automated CI/CD pipeline is to create a release in order to test application code and deploy the same in production branch.
Logs
While the Pipeline jobs are running, Data-Core could go in and see the list of jobs and their respective logs.
Dashboard Reports
Azure DevOps has built-in dashboards to display the test execution result. Various graphical charts and presentations are available and can be shared with concerned stakeholders.
Benefits
- Excel for test data management: The test data is managed in Microsoft Excel which enables it to be easy to use, read and maintain, even for resources without knowledge of the automation tool used.
- Selective test execution: With the help of TestNG, test cases can be executed as a batch and can also be executed based on various attributes, e.g. Category, Priority etc.
- Execution in multiple nodes: The test cases are executed on various target machines with various software/hardware combinations, as deemed best for the project.
- HTML reporting: The framework allows logging of each test step and publishing the same as a rich html style report that can be shared with stakeholders.
- Easy maintenance of automation code: Data-Core used Microsoft Azure Repos as a version control system for maintaining automation codes. This allows the team of testers to continuously merge their automation codes in a single repository and on a regular basis.
- Automation of test execution and publishing report: With the implementation of Azure CI/CD pipeline, the entire process of test execution, the results published in dashboards and the deployment of the application has been automated within a single workflow.
Conclusion
The out-of-the-box approach Data-Core & the client embraced has allowed for verification with graphs and charts. This would have been impossible with a single automation tool alone. The automation effort covers areas that would have been difficult to test manually, considering the combinations of data, and thereby has resulted in at least 50% lesser testing cycle per release of the application.
