top of page

Agile Testing Quadrants: Business-Facing Tests That Critique the Product (Q3) | David Tzemach

Quadrant 3 is associated with testing that only a human can do (automated scripts can help) and covers many types of tests such as risk-based testing, usability testing, and exploratory testing.

Tests done in this quadrant are based on real customer scenarios, so the team should base their tests on the customer's perspective. To do so, the team will have to use guiding questions like:

  • How will the customer work with the product?

  • Which functionality is most important to the customer?

The main goal of business-facing tests that critique the product is to ensure that the customer receives a product that meets their expectations.

Three keys make the difference between success and failure to meet the challenges involved in Q3 testing:

  • The team has solid test coverage in quadrants 1 and 2.

  • The team must use their intuition, and critical thinking as these tests are based on them.

  • The involvement of customers is crucial; they can share quick feedback that ensures that the product meets their expectations.

Beware of a False Sense of Quality

In a traditional testing environment, tests in this quadrant are mostly done at the later phases of development and only after the coding is complete. Thus, test teams have enough time to test the product and meet user expectations.

However, in Agile development, the team does not have the privilege of waiting until the entire product is built. This particular point is important in Agile testing because it can explain what I call “the false sense of quality” driven by the assumption that the product is fully tested based on the tests in quadrant 1 (which focus on internal quality) and quadrant 2 (which confirm the intended behavior of the system).

I have tried to understand why Agile teams fail due to this false assumption in the last few years. The analysis has determined two main reasons:

  1. Lack of understanding of the Agile test quadrants' true meaning and what each quadrant represents.

  2. The false assumption is that all critical bugs have already been found in Q1 and Q2; there is no real reason to invest in Q3 tests.

To avoid the false sense of quality, Agile teams must remember that the customer may not know what he wants until he can see it (which, of course, is too late). For this reason, Q1 and Q2 testing cannot guarantee that the product meets customer expectations.

The biggest advantage of Q3 testing is that it is done once the product is already built. One of the key success factors related to this quadrant is the customer's involvement and feedback.

The team can ensure that the customer receives the product they need by using customer feedback. It allows the team to design complex test scenarios, adjust usability issues and determine the overall quality of the product.

The Importance of Automation to Quadrant 3 Tests

Business-facing tests that critique the product are mostly based on knowledge, instinct, intelligence, and experience. This makes it too difficult to automate them or just less effective. However, we can still use automated tools to save time and increase the testing process's overall efficiency. For example, generating test data, configuring testing environments, and more.

Automated tools make the testing effort in this quadrant much more effective compared to the alternative of doing it manually. Therefore, the team must choose the right tools to support their test activities.

Also, Q3 tests should only run once the team has created a solid automation test framework for quadrants 1 and 2. Please do not focus on this quadrant without solid automation to support it.

The Importance of Early Demonstrations to the Engineering Process

As you probably know, Customers are the company’s biggest asset, as they possess the power to determine if the team prospers or falls.

From an engineering perspective (especially for Q3 tests), the importance of customers is in their requirements, which are used to guide the development process. Therefore, it is logical to involve them as much as possible. Hence, they share their feedback early and often avoid development pitfalls related to incorrect interpretation or miscommunication between the team and the customer.

In an Agile environment, the team should get this feedback at the end of each sprint as part of the review meeting. There, the team can present their work during the sprint and gain customer feedback before marking the story as done. By using such demos, the team can avoid the classic declaration by customers "That's what I said, but it's not what I meant."

But what happens when the customer is not available to attend those meetings, and the Product Owner is not strong enough to replace them? One of the solutions is to share a working version of the product with the customer to use in his environment without a meeting.

By using an early demonstration of the product, the customer can share feedback about the quality of the work that the team can use to improve the product until it meets the exact customer needs. If you choose to work with this method of early demonstrations, you must find the balance regarding the number of demos that work for them, so the feedback loops provide real value and not overhead.

7 views0 comments
bottom of page