The Importance of the Agile Testing Mindset | David Tzemach
Updated: Mar 3, 2022
A tester in an Agile team is a key contributor to product quality. As key members, testers should have both the personal and technical ability to collaborate with team members and stakeholders. In order to collaborate well, testers must embrace an Agile testing mindset.
This is a new way of thinking about the day-to-day activities of testing and other quality efforts, which brings new challenges not encountered in the traditional working environment.
Testers and other team members (including programmers), must change their old habits in order to succeed as an Agile team. There are not really any shortcuts in transitioning to the Agile environment before they gain the familiarity with the process and challenges of Agile development.
On paper, it is easy to tell your teams that they must embrace a completely new mindset. What does it mean in the real, practical world? Below are seventeen core aspects of the Agile testing mindset:
The Agile Testing Mindset Should Be Adopted by Everyone
Do not think that the mindset shift is only for testers, it is important for the whole team. The Agile test mindset demands full collaboration between testers and developers to deliver high-quality products.
Testers Are Still Important
One of the biggest fears of testers is that they will lose their relevance once integrated into Agile teams. This fear grows when testers understand that most of the tests in this environment are going to be automated.
The Agile testing mindset allows testers to focus where they make a difference while ignoring the repetitive tasks of manual testing. Testers can shift their focus to improving the overall quality of the product and the day-to-day processes of coding and testing.
Testing Starts in Your Head
I have worked closely with hundreds of testers at different levels, from testers who focus only on basic testing through to testers who were experts in the field. One observation which distinguishes excellent testers from mediocre one is their different way of thinking.
The greatest testers use their thinking to interpret system behaviors unlike anyone else, making the right test design choices and describing the most complex problems in the simplest way so anyone can understand. This list can help you grow your testing mindset:
Develop tools to gather and assess evidence for your findings.
Develop a positive and common language with your team.
Work on your ability to make the right decisions under pressure.
Do not take anything for granted and question everything.
Be proactive, creative and always try to innovate.
Work on your ability to break complex challenges into small parts.
Learn to say whether the software quality is good enough or not.
Automate Whenever Possible
In Agile development, where the team is expected to deliver value to their customer in each sprint, there is simply less room for manual testing labor. By using the Agile testing mindset, the team can automate many of their tests in the earliest development phases to increase the overall quality of the product and reduce the manual testing effort.
However, you should never try to automate everything at once. Start in those areas with the highest ROI and proceed based on the value you get from adding new layers one at a time.
Testing Is About Preventing Bugs
The Agile testing mindset focuses on preventing the creation of bugs rather than finding them at the advanced phases of the development effort. By using different development techniques (TDD, ATDD, and BDD) the team will have both the mindset and the techniques to conduct a more efficient testing process.
Following this mindset shift, Agile teams benefit from involving their testers at the earliest phases of the development effort, as they can design and write tests that direct the coding activities even before a single line of code is ever written.
Continuous Testing from Day One
In a traditional testing environment, tests are conducted towards the final stages of the project and only once all development activities are done. However, Agile teams, and especially their testers, must adopt continuous testing, which is the only way for them to deliver real business value at the end of each sprint.
To do so, testers must develop their skills and adopt a new testing mindset and different test strategies, for example:
Perform a formal review of new requirements received from the PO to identify potential risks.
Automate any test that should be run more than once.
“Shift left” activities that ensure that the code gets written and tested from day one.
Use test techniques that provide the best ROI (e.g. risk-based testing, exploratory testing, etc.).
Prioritize your tests to ensure that the most important functionality is tested first.
The Lone Wolf Syndrome
One of the biggest changes that testers should adopt is the whole-team approach, as they are no longer the only ones with the responsibility to guarantee that the product meets the expected quality standards. Instead, they must become team players and work closely with the rest of the team in collaboration.
Question Everything – Without Pointing Fingers
Testing is mostly about exploration and it is impossible to explore effectively without asking the right questions. The testers’ ability to ask explicit questions about key artifacts of a project (e.g. requirements, design, timelines etc.) is a fundamental tool.
Testers that fail to do this will find out very quickly that they can’t determine a test strategy nor a simple test plan for their features. The ability of testers to ask the right questions is only part of the story, as it also matters how these questions are asked. Testers should know who they are talking to and ask their questions without a provocative attitude that will likely put the other side on the defensive.
You Can Never Find All the Bugs
There will always be escaped bugs no matter the time and effort invested in the testing process. If you think otherwise, you are either delusional or new to this field. That said, testers must use their time wisely throughout the project, investing their effort in areas that make a difference so that the bugs that do escape are small and will not have any real impact on their customers.
Quality Comes from the People Who Build the Product
Testers may think they are the guardians of quality just because they test the product, but they don’t create quality and therefore cannot take it away. Quality starts from the programmers who build the product, and that is sometimes a heavy burden for them to bear. Testers should help them deal with that burden more effectively.
For testers to succeed with this mission, they must start with the basics and embrace the mindset that they are not the only ones on the project who care about delivering a great product. Running tests and revealing bugs is a great way for testers to share important information that will improve the quality of the product, but it takes the effort of the entire team to promote the assurance of quality.
Following a Mission Based on Requirements
Agile teams usually have more programmers than testers. As such, testers must be able to take unique missions that cannot always be performed by their associates.
These missions vary by company, project, or even by the personalities of the other team members. For example, in one industry, testers deliver a detailed test plan to their customers as part of the project demands, while in other projects test plans are only lightly documented and then forgotten after the tests are executed. Here are a few examples of requirements that might define the tester’s mission:
Find critical bugs as early as possible.
Provide an assessment of the quality of the product at specific gates.
Ensure that the test plan is designed and executed based on specific quality standards.
Help the team improve its testing process.
Determine the risks involved in not executing specific tests.
Collaborate with customers to understand their expectations.
Clear requirements allow the tester to determine their missions without wasting time and effort on requirements that do not add value. Testers must know that their ability to negotiate their missions and set clear expectations determines their relevance to the project. These are basic tips that allow testers to conduct efficient and constructive negotiation:
Before setting your mission, ensure that the requirements are clear. If they are not, ask questions to add the missing information.
Remember that your ability to determine your missions is the foundation for anything you do.
Each testing mission should be clear both to yourself and to other stakeholders. This is the only way to set realistic expectations.
Ensure that you prioritize your mission based on ROI and in collaboration with the relevant stakeholders.
Testing Through Exploration
There are still some people with the narrow view that testing is all about executed test cases and comparing what happens against expected results. This view makes testing seems like a simple thing that any monkey can do in their free time.
Another thing to remember is the challenges that testers have in test creation that include boundaries of time, ambiguous/partial requirements and the lack of collaboration of key stakeholders. In addition to these, there is one more challenge that is the most difficult. This is related to the test design itself and the test designer who almost never has access to guides or best practices as to what should be tested, let alone what should be expected as a result.
To handle such a complex challenge, testers must base most of their test designs on their senses, inferences, and experience. The best testers are the ones who follow their instincts and have the confidence to embrace exploratory inference, one idea leading to another in ways you can’t predict in advance.
Focus on Product Evaluation Rather than Witnessing
One of the most epic mistakes that testers can make is to focus their attention on witnessing the product rather than devoting their time to evaluating it. By witnessing the product, testers can learn quite a lot about the product and how it works. This is invaluable, but it is not testing.
Testers should focus their attention on evaluating the product and the quality of the implementation. That will help them identify the quality gaps or problems that could not be predicted in advance.
Trust Others’ Work, But Know When Not to
Trust is always important and one of the main pillars of good teams. However, there are situations where although you trust a person, you still have to ask questions to ensure you understand the full picture before approving this person's findings.
As an example, let's take a test manager that should release an important patch to a customer. In most cases, the team will start to test the patch and approve it at the end of the testing effort. However, there are some situations where this approval should be double-checked, especially when the final approval comes down to "We are sure it works" or "We tried it and it works". If that the case, there are four basic questions that you must ask before approving the work:
What does the "it" refer to? What part of the solution are we talking about?
Which parts of the solution were checked and how?
When does the solution work? What range of architectures/topologies were covered?
How well does the solution work? Does it work great or just well enough to release?
By answering these questions, you have all the information to ensure that "it works" actually means something.
Know Your Limitations and Boundaries
One of the seven testing principles states that "Exhaustive testing is not possible". This makes sense as even the smallest testing project can include thousands and even millions of tests which it is almost never possible to cover within the time scope of the project. We cannot cover all the tests we want or need, so what can we do?
Even before techniques such as risk-based and exploratory testing is the understanding that no matter how many tests you run or how much time you take in your testing efforts, all you have is an impression of the product and its quality. Therefore, you must always ensure that you reflect this to other stakeholders when asked to share the status of product quality, including how you test it, the potential risks and the known limitations of your test process.
Trust Your Intuition, But Do Not Forget the Justification for Your Choices
Testers use their intuition in many phases and areas of the testing process. This includes the type of test data they use and the way they judge the output of specific tests. However, basing most of their work on gut feeling is not good enough and cannot work in the long run or when dealing with complex projects.
Apart from the fact that intuition is often strongly biased and limited to the boundaries of the specific tester, the real trouble comes when it is communicated to other stakeholders who now base their decision on gut feeling rather than solid facts.
Therefore, testers must use their intuition only where it is necessary, and then mostly as a guide. It should never be justification for their output. A great example of this issue is bug reports. Testers who use their intuition to open bugs tend to indicate that it's an obvious bug without really explaining it. They quickly find that their associates are most likely to dismiss it.
What Kind of Thinker Are You?
There are different categories of thinking that testers must use to succeed in their day-to-day challenges. The most important categories are:
Creative thinking – Creative thinkers think outside of the box and come up with innovations to solve their challenges. Testers with this type of thinking can break away from the norms of testing and use more of their imagination to test.
Analytical thinking – Analytical thinkers can separate a whole into its constituent parts in order to examine each part and their relationships. Software testers must use this type of thinking as it allows them to solve complex testing problems and handle each one in a structured and methodical way.
Critical thinking – Critical thinkers evaluate ideas and determine their value, accuracy, authenticity, and worth. They do so not just by breaking the idea into patterns, critical thinkers always try to explore other elements that could influence the obvious conclusion. Testers with this type of thinking know how to detect and minimize errors related to the testing process or to product observations on quality criteria.
Technical thinking – Technical thinkers understand technology, so they can understand the causes and effects in specific scenarios. Testers with this type of thinking have knowledge of relevant technologies and tools and provide technical facts and predict the behavior of their solutions on the product.
Practical thinking – Practical thinkers focus on the physical world, rather than the theoretical one. They prefer to have facts and statistics over philosophy to justify a solution. Software testers that almost always work in a narrow timeframe will benefit from this type of thinking as it allows them to apply technical solutions and effort within the scope of the project.
Each of these categories can be enough for one scenario or another, but when you master all of them, you will have the power to see things in other ways than when using any one category.