In a on , wrote the following about the timing of writing acceptance criteria (acceptance tests) for user stories:

The Right Time to Define

I am frequently asked, “When should we write the acceptance criteria?” Sometimes programmers resist giving an effort estimate for a story unless they can see the ACs—sometimes detailed ACs, at that. However, there is little point in POs (and testers) spending time on ACs until stories are about to be scheduled. After all, if ACs take hours to write and the story is not scheduled, their time is wasted.

Also, if ACs are added but then the story doesn’t get scheduled for a year, by that time the story and the ACs may have changed. Because old criteria are in place, it can be easy to overlook these potentially important changes.

I would rather product owners did not write ACs until the last possible moment—even just before the planning meeting. At that point they should be fairly sure what they will request from the team. This would have the beneficial effect of forcing brevity on the PO.

Writing ACs inside the iteration neatly sidesteps the problem of postponed or canceled work. This means the team must be prepared to accept—and even estimate, if needed—stories without ACs. Because writing ACs might well be the first task in an iteration before any code or tests are written, any effort estimates given must be for the work required to “write ACs and deliver the code” rather than just “deliver the code.”

Another solution I have had some success with is writing the ACs within the planning meeting. At this point, teams know the stories to schedule. This will make the planning meeting longer, but on a small team there is unlikely to be many stories. A large team can split into smaller groups and work on stories separately.

(Test scripts based on ACs, however, are best created within the iteration, preferably before coding begins. Creating test scripts is part of the work of delivering a story.)

Following…

Now I’ve been following Allan for quite a long time. He has written some very interesting articles and a book entitled , which is focused on an aggregation of agile approaches, and I’ve found his thoughts to be largely interesting and quite sound.

But as I was reading this article, and specifically the advice called out above, I found that I disagreed with Allan. And not just a little…but a lot.

The Story

Earlier in his article Allan wrote the following about the level of detail for acceptance criteria:

The Level of Detail

Consider this example:

As a delivery company, I want to have the customer’s full postal address on packages so that I can deliver them correctly.

Such a story might have ACs like:

Customers must provide a valid postal address
Shipping labels must carry a correctly formatted address

Notice that a “valid postal address” is not defined. That kind of detail can be left until later, as ACs are not the place to elaborate. (An exception might be when some nonobvious detail is important, say, using all nine digits of the ZIP code.)

And when I saw this, I couldn’t have agreed more with his advice. The two acceptance criteria are short and nicely augment the definition (and understanding) of the story proper.

Different Advice

When I provide advice in classes around acceptance criteria, I usually use similar scope language. I normally give the following rules or advice around the criteria:

• Acceptance criteria are not equivalent to functional tests. I sometimes refer to them as mini – UAT for the story level (User Acceptance Tests). They surround acceptance vs. testing. I often ask Product Owners to “sign-off” on their stories and most of this entails running through and checking off the acceptance criteria.

• Acceptance criteria are typically a part of the Definition of Done.

• They focus on business value critical aspects of the story. Functionality that the Product Owners deems crucial for the customer to realize the value to the story. Often I characterize this is “business logic” for the story.

• They are defined and refined at the same time as the story evolves.

• I normally ask for 3-5 acceptance criteria to be defined. There is nothing special about those numbers. Zero-to-one would probably be too few. And fifteen-to-twenty would probably be too many, as you would lose the focus.

• They are a PART OF the user story. And by that, I’m referring to the 3-C’s notion from Ron Jeffries. That is a user story is composed of Card, Confirmation, and Confirmation. The acceptance criteria are considered to be Confirmation.

• I would not ask someone to estimate a user story without acceptance criteria. I don’t think you can have a full idea of the story without them. Now can they evolve during the lifetime of the story, of course! But they need to be defined in order for the story to be a candidate for estimation, backlog refinement, and story readiness.

• To me, acceptance criteria are intended to inspire thinking on the part of the team. For developers, they inspire design and impact thinking. For the tester, they inspire where and what to test thinking. In all cases, they focus the team on the “important bits” of the story. It’s like we were defining a user story for a house to be built in California. The acceptance criteria might focus on: verifying that the house is resilient to earthquakes. That criteria would permeate all of our design and testing work as we built the house. Or at least I hope it would…

Confused

As I read Allan’s guidance on timing I got quite confused. Much of his argument to define the criteria very late in the game is around the effort it takes to define them. However, if I look at the two criteria in his example, it couldn’t have taken very long at all to define them.

So if the effort is relatively minimalistic, then what’s the big deal surrounding deferral? I can’t really tell.

And there’s a missing value-based piece to the acceptance criteria that I think Allan essentially missed. It’s the conversations. I think the conversations around the story are so much more valuable with the criteria defined – including the conversations to define (and refine) them.

Again, if you’re writing a treatise as each acceptance criteria, then I can follow Allan’s advice. But that’s not the intent behind writing user stories anyway.

Wrapping up

I rarely so strongly disagree with a fellow agile coach or thought leader. But in this case, I believe Allan’s advice is completely wrong.

I want you to define and refine your acceptance criteria WITH your user stories – continuously during the lifecycle of your stories.

Don’t worry about the effort in time to define them. Instead, worry about the quality of your conversations…

Stay agile my friends,

Bob.