Acceptance Test-Driven Development (ATDD) | David Tzemach
Updated: Mar 3, 2022
Acceptance Test-Driven Development (ATDD) is a process that ideally involves all roles in the team contributing to the overall testing process using their perspective, knowledge, and experience. As part of an ATDD session, they all work together to review the acceptance criteria and determine the set of acceptance tests before development begins.
Naturally, the main contributors to this session are the development team, who usually focus on the questions “How will we test it and what can go wrong?” and the customer, who usually focuses on “What problem should we try to resolve?“
This practice of writing acceptance tests is based on the acceptance criteria, representing the user’s view of how the system functions. The team has all the information they need to design acceptance tests to ensure that the system operates.
What are acceptance tests?
Acceptance tests are written from the user's point of view, based on the acceptance criteria and technical information encapsulated in the user story. Acceptance tests are black-box system tests that the team uses for examining the effect on the system and the expected results, such as specifying the output of a system given a particular input or validating the interactions with other interfaces.
Acceptance tests are essential for regular development cycles. They become part of the regression suite that the team uses as part of their hardening sprints and before a production release. Moreover, acceptance tests may be used as part of the Definition of Done, as a user story is not considered complete unless it has passed its acceptance tests.
Each user story can have one or more acceptance tests. The number of tests is determined by the importance and complexity of the requirement. Generally, you create as many acceptance tests as needed to ensure functionality works.
The acceptance Test-Driven Development (ATDD) cycle
The acceptance test-driven development process is usually conducted as part of the planning meeting, but it may also be done in a dedicated meeting during the sprint. To understand how ATDD works in an Agile environment, we will review each step with an example.
Phase 1: Selecting a user story
The ATDD process begins with selecting a user story, which specifies the user’s requirements and what they expect to receive at the end of development. In this example, we will use the following story: “As a user, I want to be able to log in to my work computer so that I can perform my tasks.”
Phase 2: Review of the Acceptance criteria:
The user can access their computer from any location.
The authentication process must be encrypted.
The user should gain access only to their data on the server.
Phase 3: Discuss the requirement
Discussions are held between stakeholders who can ask questions and supply any missing details. The main goal of this discussion is to ensure that the team has all the necessary information to meet the acceptance criteria:
What does “any location” mean?
What devices can the user use to gain access?
What is the encryption level that we need to support?
How should we determine the user's access rights for the main server?
What is the password complexity required?
What is the expected behaviour of the authentication process if the account is locked?
During the discussion, we will most likely discover that what looks like a simple little requirement on paper becomes a complex development that may affect the design and testing efforts. Asking the right questions is incredibly important in this step, as they help the team and the business stakeholders set expectations, change priority or even split the story.
Phase 4: Sketching out acceptance tests
Although it’s part of the discussion process, it’s separate and should only be done when the team has gained all the information needed to start thinking about the acceptance tests. As part of this phase, the team creates an initial list of test cases to cover real user scenarios. For example:
Acceptance criteria: “What is the password complexity required?”
The information gained as part of the discussion:
Password Length should be a minimum of five chars.
The password must contain at least one capital letter.
The password must have at least one special char.
Test cases (positive path): P@ssw0rd, PaSSword!, PassFor@authentication
Test cases (negative path): Password, Pass, @!#$
Phase 5: Code implementation
The acceptance test criteria now become the basis for development. Here, the team usually follows the TDD model as a coding practice, which allows them to execute tests and refactor the code until the test cases have passed.
Benefits of Acceptance Test-Driven Development
Acceptance tests increase the overall efficiency of the development process. The main benefits are:
Increases collaboration between stakeholders – ATDD is a great way to improve the cooperation between stakeholders. It starts at the earliest point of the process, where user stories are reviewed and continue until the code is implemented and tested. To support this, all team members should work together throughout the development process to understand what needs to be achieved and what they should do to accomplish it.
Helps the team focus on customer needs – As user acceptance tests are based on the criteria that are the basis for the whole development effort, the customer goals remain in focus throughout development. Developers gain an early understanding of what they are expected to deliver and what the tests are. ATDD ensures that the team develops the code with the customer perspective as the main thing that guides them.
An integral part of the development – Acceptance testing is an essential part of the development effort. It is performed multiple times during coding to ensure that the new code meets the quality standards and has no negative effects. By running tests multiple times in a sprint, developers can identify and fix bugs before they become a significant risk.
It helps testers affect the quality process – ATDD is important for testers as it changes their work and influences overall quality efforts. ATDD promotes testers as key contributors as they no longer test the code at the end of the sprint. They can take ownership of most activities involved in the ATDD process.
How is ATDD different from TDD?
Both techniques support the concept of early testing, where test cases are written and executed before even a single line of code is implemented. The main difference is that ATDD focuses on testing based on business user functionality, while TDD is mainly used to automate unit tests that support development activities.