There are many challenges in any testing project, and testing in the Agile environment is no different. Let us review some of the most common challenges that Agile teams face as part of the development effort.
The Barrier of Automated Tests
One of the best practices of Agile testing is to automate everything you can. This is how we can reduce the regression effort, avoid accumulating technical debt and increase the team’s ability to deliver a high-quality product. So, what can go wrong?
Automating test cases consumes time and effort that is sometimes simply not available.
The team does not understand the long-term benefits of automating their tests and simply resists spending the time to implement an automated solution.
Developers that still think that automated tests are under the responsibility of testers and do not provide help when needed.
The team does not have the knowledge or experience of automated tools and therefore can’t start an effective automating process.
Poor automation design that consumes a huge effort to maintain.
Free some time for the team to automate. In extreme scenarios, you can even dedicate a whole sprint to this issue.
At least until you have a stable environment, do not invest in GUI testing that is usually very hard to maintain.
Start to automate at the earliest phases of development where developers can contribute the most.
Failure to Adopt the Agile Testing Mindset
Agile is a mindset that affects everyone involved in the project. It naturally affects testers that must adopt a new testing mindset. Adoption of the Agile approach for testing is not for everyone, I have seen too many testers fail to adopt this approach during the Agile transition who left their teams and even the organization.
Work with your testers to ensure they fully understand how Agile is different, and how it will affect their responsibilities.
Talk with your testers to understand their fears and what is holding them back from adopting this new mindset.
Give your testers the confidence and time to adapt to the new environment. The change will not come in the first sprints.
Unrealistic Expectations of Management
The challenge of setting the right expectations of quality during an Agile project is probably one of the biggest for an Agile consultant. It all starts with a wrong approach where teams are expected to follow the same methods as they did in the traditional test environment, although they are now working very differently.
Once, I was involved in a project where one of the managers asked the team to share the whole test plan for the four-month project. Of course, this doesn’t fit into the new Agile environment where the product is incrementally tested. This simple example can explain how Agile teams can become very frustrated, as they are pushed to work in Agile methods by the same management that is still living in a traditional world.
Work with management to establish goals and expectations that make sense in the Agile environment, but also fits the business needs.
Block, as much as you can, senior managers addressing your teams with demands and requests that you can defuse and filter.
It is easier for management to reduce their negative impact if they have full transparency of what is going on in the development process.
Show your managers how Agile testing is different from the traditional environment.
The Syndrome of the Lone Wolf
Agile follows a whole-team approach, but it’s very common that not everyone knows the essentials of testing and why it's no longer under the responsibility of testers. It is very common for developers to say that testing activities belong to testers and they will not lift a finger to contribute to the testing effort.
Testers should also know that they can no longer treat themselves as quality owners and that they will never be able to succeed without the full collaboration of the other team members.
No Time to Share Knowledge
You can see the maturity of testers embracing the Agile mindset just by looking at how they share their unique knowledge with other team members. That said, the maturity level in this area is not relevant if there is no one in the team that has the time and motivation to embrace it.
To be able to make a change, the team should first gain a true understanding of why they must be involved in the testing process and why it’s the only way for them to flourish.
Be patient, it takes time for developers and testers understand the Agile environment and what it takes to succeed in it.
The team should create specific stories for knowledge sharing. In my team, I ensure that there are at least 4-6 hours of stories dedicated to education.
Show the team (by using specific measurements) how their collaborative effort is contributing to their success.
Testing in a World of Uncertainty
The Agile development approach is based on delivering the highest value to the customer. To be able to do so, Agile teams work in a world of frequently changing requirements that in extreme scenarios can come even at the last minute of the sprint. From a testing perspective, working in a frequently changing environment is extremely challenging, as it affects the scope and effort of testing activities.
The team must strive to automate their tests as much as possible.
The team must be informed of any changes.
Advanced testing techniques must be implemented (Risk-Based and Exploratory Testing).
Developers also need to take an active part and become involved early in testing.
The team, and especially testers, should keep track of what is and is not done.
Frequent Builds that Increase Code Errors
Agile teams must embrace an automated testing approach that will reduce the efforts involved in conducting manual tests. To be able to do so, the team must use automated tools and supporting frameworks such as CI/CD that ensure that the team works with stable builds and quality releases.
This is great on paper, but it usually takes months until the team can actually achieve this goal. Therefore, until the team will succeed to establish such frameworks, they usually work in unstable environments where the code is changed hundreds of times a day without the ability to test it prior to producing builds.
Without the ability to test the code, the team will waste time on builds that are not suitable for testing in an environment that already uses a built-in time boxing for each activity.
Invest in building your CI/CD system even if it means fewer deliverables per sprint.
Design your tests based on the assumption that they will be automated (even if you will have to run them manually for one or two sprints).
Ensure that teams follow the TDD approach that reduces risks and increases quality even if there is no stable CI system.
Stories Before Bedtime
There is a reason why stories are written in a specific format and should follow specific models like INVEST that guarantee that the team will be able to implement them as intended by their creators.
In many cases, stories reflect an Idea or Vision, but severely fail when it comes to the technical details that the team needs to guide their development activities and testing in particular.
Testing activities based on such stories have a higher chance of failure, as the team have to use assumptions, which is the last thing that we want to do as a baseline for testing.
One of the biggest challenges that we want to avoid in Agile testing is teams that treat sprints as “mini-waterfalls” and wait for stories to come to them for testing only at the end of the sprint after the coding.
Clearly, “mini-waterfalls” are not suitable for the Agile environment. They add major risks to the team’s ability to deliver stories that meet the expected quality standards.
Invest in automated tests that guarantee that the code is continuously tested without a dependency on manual testing.
Testers must be involved in the testing process from the first part of the code that is available for testing.
Developers should start testing at the earliest phases of development to reduce manual effort.
The Ability to Keep Focus at All Times
Agile development sprints will always have their own unique complexity and challenges. As a result, test activities carried out by each one of the team members can be overwhelming and can result in testers losing focus. When writing this, I asked two of my testers to send me a quick list of their day-to-day activities that can explain why it is so easy for them to lose track, here are just a few examples from a list of more than twenty items:
Running ad-hoc exploratory sessions to validate new code.
Support other teams when they have technical questions.
Perform root-cause analysis for quality issues.
Execute tests as their main job.
Analyze automation results.
Setup test environments.
Design test scenarios and determine when and how they are going to be implemented.
Handle all bugs issues (reporting, reproducing and verifying bugs).
Ensure that testers raise a flag for any situation where they struggle to perform due to loss of focus, letting the team know they must become more involved.
Don’t let your testers to take tasks of other team members until they accomplish their own tasks. There is no reason for a single tester to perform tasks for 6-7 developers that can do them on their own.
Drive your team to take ownership of testing tasks that were under the strict responsibility of testers.
Applying the Values and Practices of Continuous Testing
Agile development promotes testing right from the beginning of the Sprint. To do so, the team must apply the approach of continuous testing. However, what looks great as a concept can become very hard when it comes to implementation.
There may be times when the testing effort is not moving as smoothly as expected. The causes may include a lack of communication between testers and developers, a lack of understanding of the requirements and missing technical experience for the different test activities.
Ensure that the team has the time and resources they need.
Testers must collaborate with the Product Owner to ensure their understanding of the requirements is correct.
Ensure that there is a plan for the test strategy at the beginning of the sprint.