Applying IREB RE practices in an agile environment

Introduction

You probably recognize the situation, when you as a requirements professional are assigned to a new, challenging, agile project ^[a] and you have to make your way through the new agile “jungle”. As a Requirements Engineer (RE) or Business Analyst (BA), having acquired the CPRE-FL certification, you’ll probably ask yourself how you will do requirements engineering, due to the fact, that Scrum does not know the role of a RE or BA. Perhaps you’ll either become the Product Owner or be part of the Scrum Team (which consists of members with cross-functional know-how). Either way, the activities of requirements engineering will be executed in some way in an agile environment: that is handling requirements, often associated with user stories, eliciting needs from various stakeholders, documenting them accordingly, negotiating them and achieving acceptance and finally dealing with changes. As you can see, there is still a lot going on with requirements in agile projects. Sometimes, you may not recognize that a practice used is nothing other than the basic method recommended by the IREB CPRE-FL syllabus ^[1]: for example prioritisation becomes even more important and may be performed in a very similar way to traditional approaches (e.g. single-criterion classification or the Kano model), even if the result is represented as a sorted Product Backlog.

In this article, I will make some propositions which practices of the four major activities of requirements engineering (elicitation, documentation, validation, management) may be implemented in a Scrum environment – and how. So you as a professional BA or RE (for the sake of simplicity, I will use RE from now on) will get an idea, which of the practices recommended by the CPRE-FL syllabus are helpful in projects using the Scrum methodology, and which specific practices are currently missing. If you are not familiar with Scrum, I recommend reading through the Scrum Guide ^[2], to get an idea what Scrum consists of and which practices and artefacts are being used.

Elicitation

Every new product development starts with a vision clarification of the target state. The CPRE-FL syllabus lists techniques to distinguish between in and out of scope, like use case diagrams or data flow diagrams on level 0. In my experience, use case diagrams always help in defining the system boundaries and other involved systems. They are easily understandable and show the core functionality, which is very nice when discussing with stakeholders the key features and the context of the project. Because a use case diagram is easy to change, it is well aligned with the agile principle of simplicity. I prefer using use case diagrams for projects dealing with new requirements for an existing product.

Another good technique, though not contained in the syllabus, is a product vision board ^[3]. It also lists the target group, their needs and the key features of the product, together with the value they create. It both forces the team to keep the focus on the most important elements of the product and guides all development effort. It is a great beacon for new product development.

When it comes to elicitation, the management of stakeholders is still vital for a project. In Scrum the Product Owner is responsible for managing all the stakeholders who will influence the product to be developed. He also has to elicit their needs and features until he knows what is valuable and needed by the customers or future users. This information will be input to the Product Backlog. Scrum itself does not describe how this should happen, it only defines that the Product Owner is the only one who has the authority to make these changes. So the practices in the CPRE-FL syllabus like stakeholder management, sources of requirements, different elicitation techniques and applying the Kano model are also appropriate for a Product Owner. This is one area where a certified requirements engineer is perfectly suited to help with or do this job. One missing technique, which is gaining more and more popularity in the agile world, is the definition of personas ^[b]. A Persona is a fictional person who represents a cluster of users exhibiting similar behavioural patterns concerning the product or part of a product. To create such personas, an in-depth analysis of the targeted audience (meaning potential buyers or users) is necessary. Such analysis is much more complex than the stakeholder management described in the CPRE-FL syllabus, but once created you get a clearer picture of your different customers or users. These personas may also be used when writing user stories.

One important concept of Scrum is the value of the Product Backlog items, because the Product Owner has the goal to maximize the value of his product. To determine the value of each item, the Kano model may be one input factor. Other concepts or techniques not mentioned in the syllabus like customer benefit, potential revenue, risk analysis or validated learning will be evaluated and included in the value of a backlog item. In my experience it is very helpful to analyse the different dimensions and aspects of the backlog items to get a holistic view of the resulting priorities. There are plenty of additional prioritisation methods out there like bucket prioritisation, 100-point method, buy a feature or priority poker, but always keep in mind that you should try to make things easier, not more complex.

Documentation

When moving from traditional towards more agile projects, the way of specifying requirements is where I have seen most differences. It is not easy at the beginning to adhere to the agile principle of “working software” being valued more than “comprehensive documentation”. What this means is, that Scrum questions the value of documentation, hence only what has a value in the long run should be documented; everything else is waste. Maximizing work not done is key in Scrum. Therefore Scrum is often misused to argue that essential requirements documentation is not produced. This is not what the authors of the manifesto wanted to express. A common pattern in traditional project methodologies is the upfront documentation of all the requirements, prior to their implementation. Many studies show that big, upfront requirements documentation is often a source of misunderstandings and expensive to change. Because the cost of changing such requirements is relatively high, change is not a friend in these projects: but change is an inherent factor of software development projects. Unlike the statement in the syllabus “in RE it is necessary to document all important information” agile places greater emphasis on direct communication and tries to minimize formal documentation. That is where the widely used user stories come into play as a form of managing input for software development. Scrum does not prescribe what the items should look like in the backlog, it only tells you that these items shall have a few, mandatory attributes. A user story is formulated with a language template that looks like:

This language template answers the questions of who, how and why. Originally invented in XP, a user story is not meant to fully specify a requirement. It serves three major purposes (according to Ron Jeffries CCC): It is a physical card (Card), which is usually maintained on a board; it is a token to indicate that the Scrum Team has to talk about this requirement (Communication); and it has acceptance criteria written on the back (Confirmation). Because direct face-to-face communication is preferred in Scrum Teams (they are usually co-located) there is less need to document everything. Nevertheless, information to be documented will be produced during Product Backlog refinement sessions, also known as grooming. During this session, user stories are explained and explored by the team with the goal to understand the intent of the story. If helpful, they are extended with draft screen layouts, slim UML models on a whiteboard and so on. Such sketches are usually saved within a wiki, where it is available for everyone with access to it. In a wiki it is relatively easy to make changes and to enhance details on the fly. Again in Scrum the value of documentation is essential: however, only if a model has value and helps the team to progress towards the Sprint Goal, though, is the model created and documented. Otherwise it is seen as waste and therefore omitted.

If a story is seen as being too big to be implemented in a Sprint, it is sliced until it is small enough. When using user stories without any other form of documentation, it is easy to lose the overview about what functionality has been built in the product. What I suggest when working with user stories, is having a high level use case model to provide a clear picture of the “overall” story. Every user story that is written down is a small slice of the functionality of a use case. This approach is proposed by the concept of use case 2.0 of Ivar Jacobson ^[4]. However, teams often fall into the old ways of working when trying to specify every requirement with the same, detailed level of granularity. With Scrum, the time of the big upfront requirements documentation (fortunately) has gone. In Scrum the focus is more on a JITS (just-in-time specification). A major difference to the syllabus of the CPRE-FL is the fact that it is not a requirements specification document that serves as input for planning, architecture, test, and so forth, but the user stories maintained in the Product Backlog, which, by the way, is always unfinished (completed) until the last Sprint.

The syllabus describes certain quality criteria for requirements. Let’s see in which way these quality criteria can be applied to user stories:

• Agreed/Unambiguous: During backlog refinement, stories are discussed until fully understood. In the Sprint Planning, the team commits to the stories selected for the upcoming Sprint.
• Necessary: The Product Owner is responsible for the Product Backlog and only puts stories on the Product Backlog which do have a value and therefore are necessary.
• Consistent: Is more difficult to establish due to the fact that only backlog items in status “ready” are fine-grained and clarified. The stories on the lower side of the board are coarse-grain and perhaps not discussed in depth. To establish consistency, a story may help in seeing the whole picture and discovering any inconsistencies.
• Verifiable: The third C of CCC acronym is confirmation, which are acceptance criteria of the story. These acceptance criteria, together with some test data, will form the story’s acceptance tests. Additional criteria in the definition of done (DoD) may also support the verifiability of a story.
• Feasible: If the Development Team commits to a story when selecting it for their Sprint Backlog they are convinced that it may be realized, otherwise the forecast would not be reliable.
• Traceable: In the story template, the “…so that…” part is the trace to the overall goal of the project defined in the product vision. Other traces to higher level requirements like epics are visible on a story map (see figure 2).
• Complete: A user story is never a complete specification, only a hint that the Development Team has to talk to the Product Owner. The details of a story then emerge during the Sprint, when they are negotiated with the Product Owner. Due to the fact that a story has to be small enough to be implemented in a Sprint, it usually represents a small chunk of a bigger story. If seen in this context, the story is only complete together with the other stories fulfilling the higher level story (also known as epic, see figure 2).
• Understandable: Is implicitly given, if the Development Team selects the story for a Sprint. This indicates that they have understood the story, perhaps not in all details, but these will be clarified during the subsequent Sprint.

As recapitulation, most of these quality criteria are applied with the usage of user stories in a Scrum project, not always in an explicit way, but implicitly when the Scrum practices are adhered to.

So when you start your agile project, you have to think carefully about what, how and when you document requirements, depending on your organisation and other determining factors. It is very easy to skip a lot of formerly used documentation and then get into troubles when you have to be compliant with regulations (governmental or organisational).

Validation

In Scrum the need for formal validation of a requirement is not given because there are several steps of validation inherently included in Scrum:

• In a backlog refinement session, understanding of the requirement, including its acceptance criteria, is achieved. This is usually done with a walkthrough of the next most important Product Backlog items planned to be selected for the next Sprint.
• During the Sprint the Product Owner answers or clarifies any questions from the developers concerning the story details.
• At its completion, a story is accepted or rejected by the Product Owner through the results of the story acceptance tests, based on the acceptance criteria, and through applying the criteria defined in the DoD.
• In the Sprint Review meeting, various stakeholders review the demonstrated stories and give their feedback.
• Due to the fact that every Sprint implements particular requirements (Increment), the Product Owner (and stakeholders) do not only see if the requirement is understood correctly, but also see if the software works as it is intended to work.
• At least with every new Sprint the Product Owner gets a chance to correct former decisions about how a need should be implemented and to react to current market challenges.

In Scrum, requirements negotiation is an ongoing process during maintenance of the Product Backlog by the Product Owner. As long as there are items in the Product Backlog, the Product Owner always has to negotiate the importance of the item with his stakeholders, because the items are sorted according to their priority. Additionally, any Product Backlog item may be added or deleted by the Product Owner at any time during a Scrum project, which raises the need to re-negotiate the remaining items.

Compared to the techniques referenced by the CPRE-FL syllabus, the validation and negotiation techniques within agile projects are less formal but performed more frequently. Because the feedback loops are much shorter than in traditional projects, any misunderstandings will be detected while accepting implemented stories during the Sprint. Thus the learning curve for business and IT is much faster by examining not only requirements, but implemented software, ideally software which is deployed to end customers or users which will provide immediate feedback about the usefulness of the product.

Management

When starting a new project, I suggest having no cumbersome tool but only a pinboard or whiteboard as a light and flexible toolset. Thereby the focus is put on the content of the stories and not on the tool handling or customizing. Many Scrum projects start without having a dedicated tool to manage requirements. The easiest way is having a physical board on which all the user stories are pinned. In this situation, the requirements attributes are reduced to the description (“As a …”), acceptance criteria (could also be seen as part of the description), effort estimate, status (e.g. draft, ready for Sprint) and priority. Often seen in Scrum projects are story maps ^[5] where you see the traceability from a user story to the epic (or use case) it is derived from and the sequence in which stories are going to be implemented by bundling them to releases (see figure 2). If dedicated tools are used, other attributes like identifier, risk and requirements provider may be introduced too. But be aware that only attributes should be used which will have value for the team. This maxim is not only valid for agile projects, but is often forgotten in traditional projects.

In the previous chapters I already mentioned that the backlog items are always prioritized. This is a major aspect in Scrum, because one of the defining characteristics of the Product Backlog is that its elements are ranked (a Product Backlog should be DEEP: detailed, estimated, emergent and prioritised). The simplest way of doing such a prioritization is to position a Product Backlog item (user story) on the board on the right place: The higher on the Product Backlog the more important the story. This mechanism also eliminates the problem of other techniques like [high, middle, low] where often more than 80% are ranked as high, which makes a prioritization useless.

One difference to traditional projects is the use of the must, should, could, won’t (MoSCoW) prioritization. There is no difference in formulating the story, but the position within the Product Backlog indicates if this is a story of high value and needed soon (at the top of the list), or if it has less value and is less detailed (at the end of the list) and is thus less important (and may even not be implemented at all). Because the Product Owner decides on the priority of a Product Backlog item, he may de-prioritize a story at any time, unless the story is already in the Sprint Backlog. Such de-prioritisation is a pattern which I see often in my projects after a couple of Sprints. If the Product Owner sees visually on a board what is really necessary and what may be skipped, he gets a much better insight about the value of his product and what the different releases contribute to the overall value.

Because Product Backlog items may be changed at any time by the Product Owner, there is no formal change process within Scrum. In contrast to changes in the Product Backlog, changes on the Sprint Backlog are only allowed by mutual agreement between the Development Team and Product Owner. But changes in a Sprint Backlog should be the exceptional case. At the end of every Sprint, during the demo of the Increment, the stakeholders and the Scrum Team discuss changes or new inputs and lessons learned from the market. This feedback is taken into consideration without passing a formal change process. Because the Sprint length is limited to 30 days at maximum, the stakeholders and Product Owner get the opportunity to change the priorities with every Sprint and to give new inputs on a regular basis. This precludes the need for a formal change process as explained by the CPRE-FL syllabus. Thus reacting to change is much easier with Scrum than with other process frameworks. Additionally, if the Increment is released at the end of a Sprint, you get real feedback from real customers (internal or external) quickly, which is essential if you want to learn whether you are on the right track.

Summary

The methods and techniques recommended by the CPRE-FL syllabus are very helpful for people doing requirements engineering. It is a profound basis to build upon. Because Scrum itself is mainly a management framework, which does not provide the practices needed for the different disciplines of software development (requirements engineering, development, testing, etc.), most of the practices mentioned in the syllabus should also be applied in Scrum projects. The major differences I see are:

• Organizing and managing requirements through a Product Backlog as [“…the single source of requirements for any changes to be made to the product.”] as stated in the Scrum Guide. Often accompanied by a story map. As a major difference, the Product Backlog evolves throughout the whole project and is never finished, as long as the product is alive.
• The change process is less formal but more visible, because there is always a visible representation of the requirements associated with their priority in the Product Backlog. If you want to change a requirement, its impact is immediately visible when you add a card to your story map and you therefore have to move an already existing one further down or even purge it.
• No heavyweight, formal validation of requirements. This is replaced by applying constant inspection and adaption. Through such regular feedback loops any inconsistencies or errors in requirements will be detected early and will be corrected directly.
• Acceptance of requirements is much more visible and tangible within agile projects. With every Sprint you see whether a requirement has been understood and implemented as you intended it to be.

I advise to any CPRE always to invest and read further books, articles or blogs about agile requirements engineering and to participate in agile communities to exchange with other REs. Go ahead and discuss your current problems or challenges and search for feedback and solutions from people in the community, because in the meantime, many solutions and experience are available.

---

• [a] In this article I will assume, that Scrum as a methodology is used for agile projects. Scrum is currently the most used agile framework (56%) according to current studies (9th Annual State of Agile™ Survey, VersionOne, 2015)
• [b] Personas are a practices recommended by the IREB CPRE Advanced Level Elicitation & Consolidation syllabus

References and Literature

• [1] Syllabus CPRE – Foundation Level v2.2; 2015; IREB e.V.; www.ireb.org
• [2] The Scrum Guide. The Definitive Scrum Guide: The Rules of the Game; 2013; Ken Schwaber and Jeff Sutherland; www.scrumguides.org
• [3] Agiles Produktmanagement mit Scrum: Erfolgreich als Product Owner arbeiten; 2013; Roman Pichler; d.punkt Verlag
• [4] Use Case 2.0: The Guide to Succeeding with Use Cases; 2011; Ivar Jacobson, Ian Spence, Kurt Bittner
• [5] User Story Mapping: Discover the whole story, Build the right Product; 2014; Jeff Patton; O'Reilly Media