Agile Testing Quadrants: Technology-Facing Tests That Critique the Product (Q4) | David Tzemach
The tests involved in quadrant four help both the business and development teams evaluate whether the product delivers the correct business value.
These tests look at the whole product rather than a specific feature or story. They are conducted to test the non-functional requirements (e.g., performance, security, recovery etc.).
The Main Challenges of Quadrant 4 Tests
In my experience, there are four main challenges to quadrant four tests:
What are the requirements?
In an Agile working environment, we do not always have the requirements for all aspects of the system that should be tested as part of this quadrant. This, of course, may lead to some of the most complex challenges for the development teams.
It demands specific expertise.
Another exciting aspect of quadrant four tests is automatic tools and specific expertise that help the team critique product characteristics. Automated tools are crucial for this type of test. They provide the team with the frameworks to simulate and monitor specific system behaviors that they cannot achieve using manual efforts.
Customers that do not understand the importance of these tests
Nonfunctional testing is just as crucial as functional tests. However, not all customers know and understand what these tests mean and their impact on the system.
Development teams that neglect these tests
Some of the biggest failures related to this quadrant were the results of wrong assumptions by customers that the development team would take care of these non-functional aspects as they did for the functional requirements. It takes a robust development team to respect these non-functional aspects and understand their importance and how they affect the practical aspects of the system.
Is It the Team’s Responsibility to Run these Tests?
In theory, cross-functional Agile teams should have all the skills and expertise to meet the challenges involved in the development process, including this kind of testing.
However, there is a big difference between theoretical literature and the practical world. Non-functional tests demand specific knowledge, experience, and skill that does not usually exist within the Agile team.
Take performance testing as an example. The team can run performance tests by building specific tools to support the testing effort as needed. However, in some cases, there will be testing challenges related to nonfunctional areas that demand particular tools that the team has neither the experience nor knowledge to use.
Sometimes, there is no time for the team to spend learning these tools, and it will be easier for the organization to use an external contractor who will take ownership of this specific area. Another option is to create exceptional performance or security teams that take full ownership of these tests.
Quadrant 4 Tests Are Complex. What Do We Do Now?
A great way is to start at the beginning of the project when the team determines the feature’s test strategy. Here, the team can identify the resources, skills, and tools they will need to take full ownership of quadrant four tests.
Regardless of whether the team is responsible for this test or uses external resources, the responsibility is on the team to ensure that there is no abuse of the functional requirements driven by nonfunctional issues.
When Should the Team Start these Tests?
Technology-facing tests that critique the product should start as early as possible. They should not be left to the very end of the development process. The main reason for this is that during the execution of these tests, the team will gain the vital information that can drive changes in the system architecture to support scaling or add security solutions that help improve the non-functional aspects of the product.