The Requirements Engineering Magazine appears quarterly. It is cost free and provides you with up-to-date articles reflecting the activities of the RE and BA community.
Simply sign up for being notified about new issues of the Requirements Engineering Magazine.
You may unregister at any time by sending a mail to firstname.lastname@example.org from the e-mail address which you have registered with.
I doubt whether I understand what “acceptance criteria” are. I have no doubts how I interpret the term. In my interpretation, they have nearly nothing to do with requirements and certainly nothing to do with requirements engineering. Looking around in the real world of requirements engineering, I encounter the term being used with a completely different meaning. I doubt that introducing this meaning to the term is needed and useful.
In my simple world of software engineering, there are requirements and there are test cases. The tests of the system against the requirements are often called “acceptance tests”. What matters for acceptance tests, is who carries them out, rather than whether they are against system requirements. The test of a GUI component by a prospective user can be declared an acceptance test. Hence, what is of importance is the consequence of the test result. If it is used to decide whether the subject under test is fit for productive use, then the test is an acceptance test. By the way, the supplier can never take this decision.
What role do acceptance criteria play in my simple world of software engineering?
Acceptance criteria are a kind of counterpart to the contract. The most abstract acceptance criterion would be: are all contractual agreements fulfilled? To arrive at this decision, a checklist is usually developed and used to determine whether the overall criterion is satisfied.
The checklist contains acceptance criteria concerning:
This list is not exhaustive. The checks are an investigation into whether the delivered items are o.k. and audits of the organisations that need to be ready.
We thus have a clear distinction between acceptance, acceptance test, and acceptance criteria. Acceptance is the decision whether the contractual obligations have been fulfilled. Acceptance test is a test with the purpose to find out whether the subject under test is fit for use. Its results will be used to decide upon product acceptance if they are subject of acceptance criteria. Acceptance criteria are the conditions that are checked in order to be able to take the decision to accept or reject (not yet accept) a product.
How are acceptance criteria used in the actual world of requirements engineering?
I encounter requirements engineering practices where “acceptance criteria” are a deliverable of the requirements engineer. My observations revealed in this context two uses for acceptance criteria:
The first case is rather unfortunate, since the requirement seems incomplete without the acceptance criteria. How should one decide what shall be included in the requirement and what shall be expressed as an acceptance criterion? Why do we need two vehicles for the same thing? What is the benefit of expressing acceptance criteria as an addendum to a specification versus a complete specification of the requirement?
The second case is more adequate, but not quite. In this case, the so-called acceptance criteria are nothing else than the list of test cases the requirements engineer thinks should be executed. Instead of declaring the required coverage of the requirement and letting the test designer determine the test cases, the requirements engineer determines the test cases (and usually fails to declare which coverage criterion was applied). I don’t mind the requirements engineer (partially) doing the job of the tester. But why should we call what he is doing specification of acceptance criteria and not specification of a minimal set of (acceptance) test cases? (Hint: The required coverage is actually a management decision – it has to do with risks and money.)
The second approach resembles Specification by Example, though it has a different order of activities and less rigor in expressing the test cases (the acceptance criteria).
I doubt whether I grasped all the uses of the term “acceptance criteria” out there in the field. I have no doubts that what I did observe (mostly the half-hearted description of “test cases”) does not justify the use of the term “acceptance criteria” instead of test cases.
By the way, it is a shame I could not find the original source of acceptance criteria in the literature, neither in “agile” nor elsewhere. Thus, I have another doubt: acceptance criteria have no theoretical foundation. They are a pragmatic invention that cause more confusion than necessary.
I will be happy if you prove my ignorance or otherwise dispel my doubts.
P.S: Thanks to Oliver Hoeffleur who shared first my doubts and helped me to sharpen them