Automated Quality Assurance in modern product development — IT-Dimension
Ensuring the quality and reliability of software and digital products is paramount. Automated Quality Assurance (AQA) plays an important role in this process. In this blog post, we’ll delve into the theory, applications, and tools of AQA, exploring its significance in the modern software and product development.
Intro
Automated Quality Assurance, often referred to as AQA, is a systematic process of using automated tools and methodologies to assess the quality, functionality, and performance of software or digital products. AQA is used in various domains, including web and mobile applications, desktop software, and even embedded systems.
Ensuring the quality and reliability of software and digital products is paramount. Automated Quality Assurance (AQA) plays an important role in this process. In this blog post, we’ll delve into the theory, applications, and tools of AQA, exploring its significance in the modern software and product development.
Intro
Automated Quality Assurance, often referred to as AQA, is a systematic process of using automated tools and methodologies to assess the quality, functionality, and performance of software or digital products. AQA is used in various domains, including web and mobile applications, desktop software, and even embedded systems.
Quick look at the challenges the companies experience with reporting and accounting approaches, and what tools are now available for them.
Such technology simplifies testing by using special software to automatically check if a digital product works correctly. It’s like having a robot tester that clicks buttons, types, and looks for problems in the system. AQA tools give fast feedback, helping developers make sure their creations are error-free and perform as expected
The main difference between Automation QA and Manual QA is the one who executes the test cases. In AQA case, we deal with pre-developed automated test scripts. While in MQA, a human performs all the work. Accordingly, a number of both positive and negative outcomes follow from this (learn more in the next paragraphs).
Pros, cons, and important points
Automated testing offers numerous benefits over manual testing, which is why it’s widely adopted in the software development industry. Here are some of the key advantages:
- Repeatability: Automated tests can be run multiple times without any variation in the test conditions, making it easier to identify and reproduce bugs;
- Efficiency: Automated tests can execute test cases much faster than manual testing. This efficiency is especially valuable in projects with tight deadlines;
- Cost Savings: While there is an initial investment in creating automated test scripts, automation ultimately saves money in the long run;
- Faster Feedback: Automation provides quicker feedback to developers, allowing them to identify and fix issues early in the development process;
- Better Accuracy: Automated tests are less prone to human error, making them more accurate and reliable.
While automation testing offers many advantages, it’s important to know its disadvantages and limitations. Here are some of the potential drawbacks of automated testing:
- Initial Time and Cost: Setting up and implementing automation testing can be time-consuming and expensive;
- Maintenance Overhead: Automated tests require regular maintenance to keep them up to date with changes in the application;
- Lack of Creativity and Intuition: Automation cannot replicate the creativity, intuition, and critical thinking abilities of a human tester. It may miss subtle issues that a human tester could identify;
- Incompatibility with Legacy Systems: Older or legacy applications that lack modern interfaces and APIs may not be easily automated.
It’s important to strike a balance between manual and automated testing based on the specific needs and goals of the project. Effective testing strategies often combine both approaches to make the most of their respective strengths and mitigate their weaknesses.
Although the implementation of automated testing may seem like a fairly budget-intensive and time-consuming process, we should understand that this investment will pay for itself over time.
Another super important fact is that automated testing reduces dependence on specific individuals who know the project better than any other team members or documentation: auto-tests can be run by any authorized person at any time. Qualitatively written code can be maintained by any new automator.
With auto-tests, we also reject the normal human option of “blurred eyes” (means functional issues overlooking due to tester’s mood or familiarity with the app), which is usually experienced while having a repetitive testing of the same functionality over a long period of time.
However, it should be understood that writing auto-tests is not the creation of artificial intelligence that will completely replace the work of manual testers. Automated tests allow you to shift the complexities of regression testing, so this is passing a pre-written script, but not abandoning manual testing altogether.
Quick overview of popular QA Automation frameworks
When writing full-fledged E2E tests, a significant role in the market in 2023 is given to frameworks that can provide maximum realism of user behavior in the browser. Let’s briefly review frameworks written in the popular Javascript/Typescript programming language – namely their advantages and disadvantages.
The popularity statistics on downloads in 2023 speak for themselves
Cypress – is one of the most commonly used frameworks, quickly gaining popularity in the market due to advantages such as:
- Low entry threshold (easy to start writing auto-tests);
- Detailed documentation, with examples of correct/incorrect code writing;
- Presence of an interactive runner for launching tests;
- Great possibilities for API testing (interception and modification of requests/responses and their modification)
- Ability to save each action as a snapshot for detailed debugging.
Disadvantages:
- Supports only WEB platform + responsive for testing content on mobile devices. Not suitable for automating native mobile apps (requiring installation) and desktop solutions;
- Requires additional third-party plugins to extend capabilities;
- It has a number of technical limitations that are fundamentally unsupported by developers.
WebdriverIO – deep modernization of the popular Selenium framework, only with more features. From the advantages:
- Selenium framework allows you to solve almost any task;
- Multi-platform – suitable for web, native mobile and desktop automation;
- Large community with answers to any question;
- Good documentation and easy installation.
Re the cons:
- Not the lowest entry threshold;
- Lack of tests runner makes debugging of a dropped tests more difficult → need to watch logs;
- Sometimes there are errors with running tests due to incompatibility of browser engine versions.
Playwright – is a relatively new but fast-evolving framework from Microsoft Corporation. Combines the advantages of Cypress with the advanced features of WebdriverIO. Among other advantages:
- Constantly evolving;
- Has a powerful toolkit for launching and debugging tests right during the session;
The only disadvantages are low popularity on the market and, as a result, a small number of tutorials/community.
- Constantly evolving;
Puppeteer – the “evil twin brother” of the Playwright framework. This is because Playwright was originally Puppet and the development team was in Microsoft. Later, due to disagreements, the entire development staff moved to Google, so both frameworks had the same essence and capabilities up to a certain point, but are now being developed separately. Puppeteer has a great popularity in the western market, but the entry threshold is higher and worse suited for beginners. Also it has limited resources and tools than, for example, Cypress has.
To sum up. If you need a quick start of automation for a web, without much cost – choose Cypress or Playwright. Does your product also have native mobile apps? Then your choice is WebdriverIO. However, it is better to make the final choice after an expert estimation and QA Automation engineer’s audit.
Illustrative example of AQA integration from our experience
Speaking about a real case of implementation of automated testing from IT-Dimension’s experience, we can consider a complex project in the field of logistics called ZUUM. It is a comprehensive solution, which includes a wide range of functionality for carries, shippers, brokers, and drivers.
We started to cooperate with ZUUM with an active development stage. Large volumes of new functionality being developed required even more thorough testing and greatly expanded the required work time of manual testers. So it was decided to onboard a QA Automation engineer on the project, who would be able to assist manual testers and shift a significant part of the load during regression testing to auto-test scripts.
As a web testing automation framework – we chose Cypress. It had detailed documentation, a large community, and was constantly evolving, bringing to the market new releases. In addition, it had a convenient interactive runner and allowed saving actions step by step.
The testing framework was built from scratch using the page object model. New tests were added every week and used in each regression testing cycle. With active daily deployments, the auto-tests were automatically run early in the morning every day so that the automator could see the result by the start of their work.
All in all, we managed to cover frontend, backend, and load testing with great automation and, as a result, high efficiency. ZUUM team got a way to perform daily regression test runs with no need to spend a few days on it, as well as receive detailed test reports.
The used tech stack:
Web automation: Cypress / WebdriverIO / Playwright / Selenide
API automation: Cypress / Postman / Newman
Load testing: jMeter
CI/CD: AWS / Jenkins / Gitlab CI / Terraform
Management tools: Jira / Trello / TestRail
Reporting tools: Allure / Mochawesome
Web automation: Cypress / WebdriverIO / Playwright / Selenide
API automation: Cypress / Postman / Newman
Load testing: jMeter
CI/CD: AWS / Jenkins / Gitlab CI / Terraform
Management tools: Jira / Trello / TestRail
Reporting tools: Allure / Mochawesome
And here are some result numbers:
- 700+ test cases were covered by QA Automation over the course of the year (many of them involved a large number of actions (about 150) in a single case and were of high complexity to test);
- Another 214 API tests were written for isolated backend testing;
- The time for regression testing decreased from 3+ days for the manual testing team to 13 hours for the auto-tests;
- After the introduction of multithreading, the total testing time decreased to 5 hours.
The end result was as follows: instead of the required 3 days of meticulous manual testing, we got only 5 hours of QA Automation to perform a full regression. Impressive, isn’t it? And in addition, we removed the human factor from the testing process (chance of human error, “blurred eyes”, etc.).
Outro
Taking into account everything discussed above, we can undoubtedly say that QA Automation streamlines the testing process, improves the quality of the software, and enhances the overall efficiency of the development lifecycle. It is an essential component of modern software development practices, such as Agile.
Of course, the process of AQA implementation in the project must be properly planned. Here it’s worth taking the time to estimate the project as a whole, prepare test plans, and, most importantly, select the right technical stack for such integration so that everything fits your unique needs.
Contact us if you need a consultation on QA Automation planning, implementation, or audit. Let’s work together to improve the quality of the solutions we develop!
Such technology simplifies testing by using special software to automatically check if a digital product works correctly. It’s like having a robot tester that clicks buttons, types, and looks for problems in the system. AQA tools give fast feedback, helping developers make sure their creations are error-free and perform as expected
The main difference between Automation QA and Manual QA is the one who executes the test cases. In AQA case, we deal with pre-developed automated test scripts. While in MQA, a human performs all the work. Accordingly, a number of both positive and negative outcomes follow from this (learn more in the next paragraphs).
Pros, cons, and important points
Automated testing offers numerous benefits over manual testing, which is why it’s widely adopted in the software development industry. Here are some of the key advantages:
- Repeatability: Automated tests can be run multiple times without any variation in the test conditions, making it easier to identify and reproduce bugs;
- Efficiency: Automated tests can execute test cases much faster than manual testing. This efficiency is especially valuable in projects with tight deadlines;
- Cost Savings: While there is an initial investment in creating automated test scripts, automation ultimately saves money in the long run;
- Faster Feedback: Automation provides quicker feedback to developers, allowing them to identify and fix issues early in the development process;
- Better Accuracy: Automated tests are less prone to human error, making them more accurate and reliable.
While automation testing offers many advantages, it’s important to know its disadvantages and limitations. Here are some of the potential drawbacks of automated testing:
- Initial Time and Cost: Setting up and implementing automation testing can be time-consuming and expensive;
- Maintenance Overhead: Automated tests require regular maintenance to keep them up to date with changes in the application;
- Lack of Creativity and Intuition: Automation cannot replicate the creativity, intuition, and critical thinking abilities of a human tester. It may miss subtle issues that a human tester could identify;
- Incompatibility with Legacy Systems: Older or legacy applications that lack modern interfaces and APIs may not be easily automated.
It’s important to strike a balance between manual and automated testing based on the specific needs and goals of the project. Effective testing strategies often combine both approaches to make the most of their respective strengths and mitigate their weaknesses.
Although the implementation of automated testing may seem like a fairly budget-intensive and time-consuming process, we should understand that this investment will pay for itself over time.
The popularity statistics on downloads in 2023 speak for themselves
Cypress – is one of the most commonly used frameworks, quickly gaining popularity in the market due to advantages such as:
- Low entry threshold (easy to start writing auto-tests);
- Detailed documentation, with examples of correct/incorrect code writing;
- Presence of an interactive runner for launching tests;
- Great possibilities for API testing (interception and modification of requests/responses and their modification)
- Ability to save each action as a snapshot for detailed debugging.
Disadvantages:
- Supports only WEB platform + responsive for testing content on mobile devices. Not suitable for automating native mobile apps (requiring installation) and desktop solutions;
- Requires additional third-party plugins to extend capabilities;
- It has a number of technical limitations that are fundamentally unsupported by developers.
WebdriverIO – deep modernization of the popular Selenium framework, only with more features. From the advantages:
- Selenium framework allows you to solve almost any task;
- Multi-platform – suitable for web, native mobile and desktop automation;
- Large community with answers to any question;
- Good documentation and easy installation.
Re the cons:
- Not the lowest entry threshold;
- Lack of tests runner makes debugging of a dropped tests more difficult → need to watch logs;
- Sometimes there are errors with running tests due to incompatibility of browser engine versions.
Playwright – is a relatively new but fast-evolving framework from Microsoft Corporation. Combines the advantages of Cypress with the advanced features of WebdriverIO. Among other advantages:
- Constantly evolving;
- Has a powerful toolkit for launching and debugging tests right during the session;
The only disadvantages are low popularity on the market and, as a result, a small number of tutorials/community.
Puppeteer – the “evil twin brother” of the Playwright framework. This is because Playwright was originally Puppet and the development team was in Microsoft. Later, due to disagreements, the entire development staff moved to Google, so both frameworks had the same essence and capabilities up to a certain point, but are now being developed separately. Puppeteer has a great popularity in the western market, but the entry threshold is higher and worse suited for beginners. Also it has limited resources and tools than, for example, Cypress has.
To sum up. If you need a quick start of automation for a web, without much cost – choose Cypress or Playwright. Does your product also have native mobile apps? Then your choice is WebdriverIO. However, it is better to make the final choice after an expert estimation and QA Automation engineer’s audit.
Illustrative example of AQA integration from our experience
Speaking about a real case of implementation of automated testing from IT-Dimension’s experience, we can consider a complex project in the field of logistics called ZUUM. It is a comprehensive solution, which includes a wide range of functionality for carries, shippers, brokers, and drivers.
We started to cooperate with ZUUM with an active development stage. Large volumes of new functionality being developed required even more thorough testing and greatly expanded the required work time of manual testers. So it was decided to onboard a QA Automation engineer on the project, who would be able to assist manual testers and shift a significant part of the load during regression testing to auto-test scripts.
As a web testing automation framework – we chose Cypress. It had detailed documentation, a large community, and was constantly evolving, bringing to the market new releases. In addition, it had a convenient interactive runner and allowed saving actions step by step.
All in all, we managed to cover frontend, backend, and load testing with great automation and, as a result, high efficiency. ZUUM team got a way to perform daily regression test runs with no need to spend a few days on it, as well as receive detailed test reports.
The used tech stack:
Web automation: Cypress / WebdriverIO / Playwright / Selenide
API automation: Cypress / Postman / Newman
Load testing: jMeter
CI/CD: AWS / Jenkins / Gitlab CI / Terraform
Management tools: Jira / Trello / TestRail
Reporting tools: Allure / Mochawesome
And here are some result numbers:
- 700+ test cases were covered by QA Automation over the course of the year (many of them involved a large number of actions (about 150) in a single case and were of high complexity to test);
- Another 214 API tests were written for isolated backend testing;
- The time for regression testing decreased from 3+ days for the manual testing team to 13 hours for the auto-tests;
- After the introduction of multithreading, the total testing time decreased to 5 hours.
The end result was as follows: instead of the required 3 days of meticulous manual testing, we got only 5 hours of QA Automation to perform a full regression. Impressive, isn’t it? And in addition, we removed the human factor from the testing process (chance of human error, “blurred eyes”, etc.).
Outro
Taking into account everything discussed above, we can undoubtedly say that QA Automation streamlines the testing process, improves the quality of the software, and enhances the overall efficiency of the development lifecycle. It is an essential component of modern software development practices, such as Agile.
Of course, the process of AQA implementation in the project must be properly planned. Here it’s worth taking the time to estimate the project as a whole, prepare test plans, and, most importantly, select the right technical stack for such integration so that everything fits your unique needs.
Contact us if you need a consultation on QA Automation planning, implementation, or audit. Let’s work together to improve the quality of the solutions we develop!
Useful Blogposts
November 9, 2023
Automated Quality Assurance (AQA) in modern product development — IT-Dimension
Automated Quality Assurance in modern product development — IT-Dimension Ihor Martynenko, Valentin Minko ● 9 November 2023 8 minutes QA AQA Product development Automation Cypress Playwright Puppeteer Ihor Martynenko, Valentin Minko ● 9 November 2023 8 minutes QA AQA Product development Automation Cypress Playwright Puppeteer Ensuring the quality and reliability of software and digital products …
Automated Quality Assurance (AQA) in modern product development — IT-Dimension Read More »
AQAAutomation+5
October 6, 2023
The Manifest Names IT-Dimension as one of the Most Reviewed Software Developers in Ukraine
The Manifest Names IT-Dimension as one of the Most Reviewed Software Developers in Ukraine B2B Recognition Business Tools-and-Partnerships Business Improvement Company Announcements Industry Recognition The Manifest Award One of the most essential business drivers in the market today is software. Many companies have been utilizing software to further improve their processes, operations, services, and even …
AwardsB2B+1
September 12, 2023
Reporting and accounting automation with Google App Scripts and Chat bots
Reporting and accounting automation with Google App Scripts and Chat bots Desktop Google App Script Web Python JavaScript Valentin Minko ● 17 July 2023 6 minutes Desktop Google App Script Web Python JavaScript In today’s fast-paced business environment, reporting and accounting are still super important processes in company management and its efficiency, as well as …
Reporting and accounting automation with Google App Scripts and Chat bots Read More »
MobileWeb+3
September 7, 2023
Advanced interactive prototyping: more than just a design tool for the product development.
Advanced interactive prototyping: more than just a design tool for the product development. 17 July 2023 10 min read Web Design Desktop Figma Valentin Minko ● 17 July 2023 7 min read Web Design Desktop Figma In the fast-paced world of product development, the ability to visualise and test ideas efficiently is crucial — and …
DesignDesktop+2
September 7, 2023
(Short) Fleet Management Software: Solving Global Transportation Issues and Streamlining Business Success
(Short) Fleet Management Software: Solving Global Transportation Issues and Streamlining Business Success 17 July 2023 10 min read Web Front End Back End Mobile Angular .Net Core RxJS Swift Valentin Minko ● 17 July 2023 6 min read Web Front End Back End Mobile Angular .Net Core RxJS Swift The logistics and transportation industries have …
Back EndFront End+6
August 18, 2023
(Long) Fleet Management Software: Solving Global Transportation Issues and Streamlining Business Success
(Long) Fleet Management Software: Solving Global Transportation Issues and Streamlining Business Success 17 July 2023 10 min read Web Front End Back End Mobile Angular .Net Core RxJS Swift Valentin Minko ● 17 July 2023 10 min read Web Front End Back End Mobile Angular .Net Core RxJS Swift The logistics and transportation industries have …
Back EndFront End+6