Common Pitfalls to Avoid When Using Metrics in agile projects
These are some of the most common pitfalls of using metrics in the Agile environment.
What is the problem you are trying to resolve?
Metrics are often used just because they allow some measurement but without really putting forethought into what the actual problem is that these metrics are being used to resolve or track.
One key guideline I promote in my teams is that we only measure aspects based on the problem we want to solve. When the team understands the problem and what we need to measure to solve it, they become more motivated and by far more willing to contribute to the overall measurement process.
Measuring without asking the right questions
We always need to ask questions regarding metrics even if they are working for us. Not asking the right questions can lead to automatically using the same metrics that worked for us in one project but which are not suitable for the current one. In addition, it can lead teams to use metrics that will not help them achieve their objectives.
Some basic questions should be asked at the end of any major milestone (depending on the project’s timelines and goals):
Are the current metrics helping to promote team motivation and performance?
What are the problems that the metrics are trying to resolve or the goals they’re trying to help reach?
Is there any need to modify current metrics to fit new goals?
Are the metrics providing the value we expected to see?
Do we use metrics that help promote the project?
Do we use metrics that can help the team to grow?
Are the metrics clear to everyone?
Metrics that do not add real value
Metrics are important, but it is also important to only create metrics that really matter. Most often organizations will use metrics that are just useless, such as the number of test cases or the number of code lines written per day. Remember: if a metric does not contribute to your overall process it is most likely not worth the time spent on measuring it.
The human factor
We are dealing with people, who are involved in the measurement process and therefore affect its results. Based on my experience, no one likes to be measured regardless of his work pace and the value he provides.
For many people, measurements of their work will create discomfort, a feeling of lack of confidence from their superiors, unwanted pressure, and, crucially, focusing their entire effort to meet the metrics goal instead of doing what they are really supposed to. For example, a measurement that focuses on the number of tasks completed per sprint by individual team members risks pushing team members to commit to more tasks than they are capable of delivering. As a result, more tasks will be delivered with poor quality, which will make trouble and create technical debt in future sprints.
So, what can we do to remove the human factor? Well, we cannot, but there are many actions that we can take to reduce it, starting with metrics that do not involve human interaction. We can also informally review the metrics with team members to explain each metric and its purpose, etc.
Metrics used to judge individual team members
There is a good reason why we use metrics in projects and specifically in an Agile environment that promotes the agenda of continuous improvement. One thing that can put it all at risk is by using metrics in a way that will measure and judge an individual or specific team's performance. We have all seen the organization destroyed the confidence and trust between them and their teams, which led to catastrophic results and goes against the Agile mindset.