Acceptance Test-Driven Development (ATDD) is a process that ideally involves all roles in the team contributing to the overall testing process using their own perspective, knowledge, and experience. As part of an ATDD session, they all work together to review the acceptance criteria and determine the set of acceptance tests before development begins.
Naturally, the main contributors to this session are the development team, who usually focus on the questions “How will we test it and what can go wrong?” and the customer who usually focuses on “What problem should we try to resolve?“
This practice of writing acceptance tests is based on the acceptance criteria, which represent the user’s view on how the system is supposed to function. By doing so, the team has all the information they need in order to design acceptance tests which ensure that the system functions as it should.
What are acceptance tests?
Acceptance tests are written from the user's point of view, based on the acceptance criteria and technical information encapsulated in the user story. Acceptance tests are black box system tests that the team use for examining the effect on the system and the expected results, such as specifying the output of a system given a particular input or validating the interactions with other interfaces.
Acceptance tests are important for the regular development cycles, as they become part of the regression suite that the team use as part of their hardening sprints and before a production release. Moreover, acceptance tests may be used as part of the Definition of Done, as a user story is not considered complete unless it has passed its acceptance tests.
Each user story can have one or more acceptance tests. The number of tests is determined by the importance and complexity of the requirement. Generally, you create as many acceptance tests as needed to ensure that functionality works.
The acceptance Test-Driven Development (ATDD) cycle
The acceptance test-driven development process is usually conducted as part of the planning meeting, but it may also be done in a dedicated meeting during the sprint. To understand how ATDD works in an Agile environment, we will review each step with an example.
Phase 1: Selecting a user story
The ATDD process begins with the selection of a user story, which specifies the user’s requirements and what they expect to receive at the end of development. In this example, we will use the following story: “As a user, I want to be able to login to my work computer, so I can perform my tasks”.
Phase 2: Review of the Acceptance criteria:
The user can access their computer from any location.
The authentication process must be encrypted.
The user should gain access only to their own data on the server.
Phase 3: Discuss the requirement
Discussions are held between stakeholders who can ask questions and supply any missing details. The main goal of this discussion is to ensure that the team has all the necessary information to meet the acceptance criteria:
What does “any location” mean?
What devices can the user use to gain access?
What is the encryption level that we need to support?
How should we determine the access rights of the user for the main server?
What is the password complexity required?
What is the expected behavior of the authentication process if the account is locked?
During the discussion, we will most likely discover that what looks like a simple little requirement on paper become a complex development that may affect the design and testing efforts. Asking the right questions is incredibly important in this step, as they help the team and the business stakeholders set expectations, change priority or even split the story.
Phase 4: Sketching out acceptance tests
Although it’s part of the discussion process, it’s separate and should only be done when the team has gained all the information needed to start thinking of what the acceptance tests are. As part of this phase, the team creates an initial list of test cases to cover real user scenarios. For example:
Acceptance criteria: “What is the password complexity required?”
The information gained as part of the discussion:
Password length should be a minimum of five chars.
The password must contain at least one capital letter.
The password must contain at least one special char.
Test cases (positive path): P@ssw0rd, PaSSword!, PassFor@authentication
Test cases (negative path): Password, Pass, @!#$
Phase 5: Code implementation
The acceptance test criteria now become the basis for development. Here, the team usually follows the TDD model as a coding practice, which allows them to execute tests and refactor the code until the test cases have passed.
Benefits of Acceptance Test-Driven Development
Acceptance tests increase the overall efficiency of the development process. The main benefits are:
Increases collaboration between stakeholders – ATDD is a great way to improve collaboration between stakeholders as it starts at the earliest point of the process where user stories are reviewed and continues until the code is implemented and tested. To support this, all team members should work together throughout the development process as they must gain a clear picture of what needs to be achieved and what they should do to accomplish it.
Helps the team focus on customer needs – As user acceptance tests are based on the criteria that are the basis for the whole development effort, the customer goals remain in focus throughout development. Developers gain an early understanding of what they are expected to deliver and what the tests are. ATDD ensures that the team develops the code with the customer perspective as the main thing that guides them.
An integral part of the development – Acceptance testing is an integral part of the development effort and is performed multiple times during coding to ensure that the new code meets the quality standards and that there are no negative effects. By running tests multiple time in a sprint, developers can identify and fix bugs before they become a major risk.
It helps testers affect the quality process – ATDD is important for testers as it changes their work and increases their influence on the overall quality efforts. ATDD promotes testers as key contributors as they are no longer testing the code at the end of the sprint. In fact, they can take ownership of most activities involved in the ATDD process.
How is ATDD different from TDD?
Both techniques support the concept of early testing, where test cases are written and executed before even a single line of code is implemented. The main difference is that ATDD focuses on testing that is based on business user functionality, while TDD is mainly used to automate unit tests that support development activities.