The Requirements Engineering Magazine appears quarterly. It is cost free and provides you with up-to-date articles reflecting the activities of the RE and BA community.
Simply sign up for being notified about new issues of the Requirements Engineering Magazine.
You may unregister at any time by sending a mail to firstname.lastname@example.org from the e-mail address which you have registered with.
In a previous article , we claimed that at least in certain circumstances, for certain types of systems, the requirements management discipline is still necessary in an agile framework. “But what is the impact of agility?” we asked. Should requirements management be done differently, and how? This article explores the necessary changes to the lifecycle of requirements in the agile framework.
In Scrum, the product to be developed is described by the backlog, a set of items — generally user stories, but not necessarily; backlog items are selected according to priorities for implementation in successive iterations — which Scrum calls sprints. According to , a user story in Scrum goes though the following states:
Created → Accepted → Estimated → Planned → Ongoing → Done
This is only the nominal path; this simplified view does not represent all the possible sink states, nor the loops that may occur: items that are “Created” but never “Accepted”, “Ongoing” stories that appear not to be feasible within constraints when one attempts to implement them, requirements that are found to be not so useful or not usable once developed, etc.
We add two superstates to highlight two phases: Maturing (the requirement or user story is being elaborated and analyzed), and Selected (it is actually implemented in the product). We also add states Issue and Dismissed to cope with refusal from stakeholders and defects not corrected in the current iteration. We obtain the following diagram:
We are not fully satisfied with the initial proposal presented in Figure 1. In this section, we will expose our reservations and propose various improvements.
Developments should sometimes start with immature requirements to produce a demonstration of the proposed feature and to collect feedback to validate or improve the initial requirements — and this is an essential principle of agile approaches. Quick development is, then, a particular requirement elicitation technique, and a requirement analysis and validation technique as well: the validated requirement is then the result of the development, rather than a precondition. It makes prototyping a more frequent practice than in the linear approaches [b].
In some cases, starting to develop something is even the best way to check ideas, to verify feasibility or to obtain better estimates. This is very much a part of the agile attitude, and was formalized in XP and SAFe  as spikes : development to gain knowledge. Thus, our agile lifecycle should sometimes allow a requirement to be “Planned”, then “Ongoing”, before being “Accepted” or “Estimated”.
Thus, our agile lifecycle should support “develop to specify” as well as “specify to develop”.
Customer requirements or user stories are statements of the problem to address, and development costs result from activities needed to develop the solution. Taking a shortcut from problem to solution, then from solution to activities, is certainly applicable to simple systems, but not to more complex ones in which architecture and generic components represent a large amount of the work to do, and that do not directly relate to user stories.
In a previous article , I explained why, at the end of an iteration, some requirements should not be marked as “done”, even after they have been verified successfully. Most examples I found of such requirements are non-functional requirements, i.e. quality requirements and constraints. SAFe calls them NFR and manages them in a dedicated place, as items that “may constrain zero, some or many backlog items” .  also mentions non-functional requirements (§2.6) and recommends conducting “End of Lifecycle Testing” (§3.5) before each release.
I prefer calling them “neverdone” requirements, because not all non-functional requirements fall into this category, and because I can think of some “neverdone” functional requirements.
In short, neverdone requirements are requirements that could easily be undone in a future iteration and must be verified before each delivery. While this may not be a concern in a pure waterfall approach, because all requirements are implemented together, then shipped together after a global, complete verification, it may become one for agile approaches with multiple development iterations and, more importantly, multiple deliveries following partial verifications.
Thus, we want our agile lifecycle to allow a special treatment for these neverdone requirements [c].
Our revised lifecycle must satisfy the following requirements:
The principle in Figure 2, below, is to consider requirements maturation and development as parallel, interacting processes, rather than sequential phases.
For readers not familiar with UML statemachines, the diagram can be read as follows: the state Considered runs three nested statemachines in parallel. The actual statemachine is the product of three statemachines: Maturity status × Implementation status × Lifecycle policy. A requirement in state Considered is simultaneously in one substate of each statemachine. For example, the initial state of a requirement, starting from the left, is (In Progress; Not Implemented; Undetermined).
Note that not all combinations of states are valid (see also Practical implementation in a requirements repository, below), but our intent is to illustrate several paths that developments can follow, particularly — but not only — in agile approaches. Let us review all states and substates in more detail…
The actual substates can be adapted to the requirements validation process of each organization or to the specific stakeholders of a project.
There too, the actual substates may need to be adapted to the implementation and verification process applicable within the organization, to follow the process imposed by the customer, and may depend on the technologies used to develop and verify a specific product or component.
Note that it is possible to go through all these states even when Maturity status is Proposed or Issue, when implementation is used as a technique to make poor requirements evolve. Eventually, the requirement is rephrased, fine-tuned, according to the findings and measurements made thanks to actual software. Nominally, an implemented requirement must be finalized and marked as Validated to be also Verified: a product can be perfectly compliant with irrelevant requirements, but it should lead to dismissal of the requirement.
Once compliance with a requirement has been successfully verified [e], if it was decided that the Backlog item Lifecycle policy applies to it, it can be marked as Done. It is no longer in the set of requirements to be implemented or verified (or backlog) — unlike the neverdone requirements.
The requirement was rejected, either when it was analyzed in state Proposed, or later, at implementation time, because of some issue that could not be addressed within constraints.
Figure 3 illustrates that this new representation captures two paths:
The reader will find more examples at the end of this article in Appendix 1 — Selected examples of the multiple paths that a requirement can follow.
As complex as it may appear at first sight, the statemachine in Figure 2 can easily be represented by three attributes that are modified independently by the requirements manager as their status evolves:
This will of course allow irrelevant combinations such as (In progress; Verified; x) or (x; Done; Neverdone) and illegal transitions, but requirements management is executed by human beings supposed to understand the meaning of each attribute — not by a compiler. These attributes are solely used to record the status decided by authorized people, not for automation; thus complete formalization and automated control is superfluous.
It is interesting to observe that the statemachine in Figure 2, though it was revised with the intent to support agile practices better, is perfectly suitable in a waterfall context:
Readers are invited to read Appendix 2 — What lifecycles do not show about agility for a development on the differences and similarities between waterfall and agile approaches from the perspective of requirements and their respective lifecycles.
We first examined the lifecycle for Scrum user stories and proposed to modify it : first, to support some key aspects of agility, particularly the “develop to specify” path; second, to introduce the concept of “neverdone” requirements, those that cannot be managed in the backlog. Last, we proposed a general lifecycle for requirements, applicable to backlog items as well as neverdone requirements, supporting both “specify to develop” and “develop to specify” paths.
1…3: The requirement follows the usual maturation process, and is eventually planned.
4, 5: Once Validated, it is implemented (with all other requirements selected for the release, in the case of a waterfall approach; with a few other requirements, in the case of an agile approach).
6, 7: After checking that the requirement is not a “neverdone” requirement, it is marked as done.
1…3: The requirement is first elaborated, but cannot be validated because of doubts regarding its relevance, its feasibility or its verifiability. While its Maturity status is Issue, it appears that trying to implement the requirement will be the best way to finalize and validate it, or to reject it. It is therefore planned, though it is (or rather, because it is) immature.
4, 5: The requirement is implemented in actual code, the code is verified against the requirement, which is eventually considered Validated — possibly after modifications to reflect results of the experiment.
6, 7: After checking that the requirement is not a “neverdone” requirement, it is marked as done.
1…5: This path is identical to the nominal path for a backlog item…
6: … except that once it is marked as “neverdone”, it cannot be marked as “done”. Thus, it will have to be checked again when the product is modified.
Ideally, traceability links establish a relationship between this requirement and stories in the backlog that may violate the requirement later. Similarly, traceability links to components will trigger this requirement’s verification when they are touched.
7: When a new iteration is executed, the requirement is put back to the state To be Verified. This may be because a specific story is implemented, because a specific component is modified (assuming this story or this component have been identified as risks to alter compliance with the requirement), or simply because this requirement is in the goals of non-regression testing.
8: It may happen that the requirement is no longer satisfied in this new release, because it was undone by the modifications made in this iteration.
9: After corrections in the code, the requirement is To be Verified, again.
A classic waterfall approach — not describing prototyping and neverdone requirements — could be represented as follows:
This description, largely inspired by , will not be discussed in detail; it is only provided to highlight some particular points. What is illustrated in Figure 4 is that we have two main, sequential phases [f]:
During phase I, all requirements are being worked on, and none is being implemented; their statements evolve to reach their “ideal” formulation and values, so as to meet customer expectations and to be realistic, given constraints of all kinds. In a waterfall approach, the end of this phase is signified by a project milestone authorizing developments to start.
During phase II, the requirements should ideally not change. When a requirement is added or modified in this phase, this is formalized by a change request, initiating a potentially costly impact analysis, possibly causing heavy rework. The intent of the waterfall approach is precisely to try to avoid that situation as much as possible [g]. A core principle of a waterfall approach is that most requirements should be identified and validated (frozen) at a given milestone, before allowing developments [h]. As suggested by the two words “ideally” and “most” in the preceding sentences, changes will in fact happen. We have not modelled here the management of change requests, which is handled separately, since a change request may concern a requirement in any state.
Compare Figure 4 with Figure 1. Lifecycle of a backlog item in Scrum: both look very similar, though one is meant to describe the lifecycle of an agile user story, and the other the lifecycle of a “classic” requirement in a waterfall approach. How can that be?
Though Figure 1 and Figure 4 look similar, the meaning of the two superstates is very different. In a waterfall approach:
On the contrary, in an agile approach as Scrum, presented in Figure 1,
Thus, one of the key differences between waterfall and agile approaches does not show in the lifecycle, because it relates to the management of sets of requirements, rather than individual requirements. The following two diagrams will illustrate this:
In a waterfall approach, as said before, all requirements are expected to be frozen before a certain project milestone (first red rectangle), which allows the beginning of development — and of the formal change management process. Ideally, no new requirement will be added, no requirement will be modified after the milestone is passed — and as the word “ideally” suggests, this will happen anyway, which is why a formal change management process is put in place. Eventually, all Verified requirements are shipped together (second red rectangle) [i].
In an agile approach, a few items from the product backlog are selected at each iteration — a sprint in Scrum. Only requirements implemented in the current iteration (the sprint backlog) are frozen, the product backlog continues to evolve until the end of the project. As a matter of fact, it is even intended to evolve as a result of past iterations. This is not seen as an exceptional situation to be avoided; on the contrary, agile principles favor the early collection of feedback on the demonstrator resulting from the first iterations.
In other words, in a waterfall approach repository, most requirements nominally move during the initial phase, but only a few are modified later as a result of issues; in an agile approach, they move individually, during the whole development: requirement maturation and product implementation are interleaved; requirements maturation goes on after development has started. There is no milestone to split the project in two phases, the “phase of changing requirements” and the “phase of development based on mostly stable requirements”; instead, the backlog progressively migrates from its initial state to its final state, as its items migrate individually, each in its own time, from Accepted to Done. After some iterations, Done requirements are taken from the backlog and integrated to a new release of the product (red ovals in Figure 6) which is shipped and deployed, as shown by the triangular shape of the "Shipped" (blue) area.
Another difference that does not show in the statemachines is the amount of work needed to reach a certain level of maturity for a requirement or backlog item in both approaches. The arrows representing transitions between states may look the same in Figure 4 and Figure 1, but they do not have the same weight!
For complex systems, writing a good requirement with sound statement and values may require a considerable amount of work: modelling, simulation, mockups, multiple reviews… Writing a user story will generally require less work, and for the kind of systems built with agile approaches, it is often acceptable, or even preferable, to start implementation with relatively immature requirements [j], and to use the resulting demonstrator to refine and improve the initially poor requirements. This can only be the case when the cost of developing the product (coding) and describing it (writing requirements) are of the same order of magnitude.