RE in Agile Projects: Survey Results

Eight requirements problems –software development issues relating to the topic of requirements – were identified in the literature, and together with 14 of the most common requirements elicitation and analysis methods were used as the basis for a survey of current RE practice within Agile projects. Based on completed questionnaires by respondents from across industry, the research showed that many of the identified requirements problems are indeed recognised as challenges within Agile projects. While some issues such as changing requirements and over-specification were shown to be well addressed in Agile approaches, other problems such as hidden stakeholder needs and missing, ambiguous or inaccurate requirements continue to present important challenges.

The research provided evidence that the use of many RE methods within Agile projects is already widespread and is not in practice seen to be in conflict with Agile principles. The use of some other methods, such as questionnaires, goal models and state machines, for example, appeared in the results to be less well established.

Finally the results were reviewed with an expert panel and nine guidelines for the application of RE methods in scaled Agile projects are presented here. These recommend an increased awareness of key requirement challenges and are supported by evidence gathered from current practice.

Aims and Objectives

Discussing requirements in the traditional terms of the analysis phase and specification document often leads to the entrenched views of the Agile versus waterfall debate. This study therefore first set out to establish a common language between the different approaches, before then gathering data from current practice as to which RE methods are currently in use within Agile projects. The specific research objectives were therefore:

1. Identify a set of core requirements problems (i.e. software development issues relating to requirements), independent of any particular development approach, which may provide a common ground between Agile and traditional approaches.
2. Identify a set of standard RE methods with potential for reuse within Agile.
3. Gather data from current practice in commerce and industry.
4. Formulate guidelines during an expert review phase.

Results of the Literature Search

From a survey of the literature eight of the most frequently discussed requirement problems were identified in fulfilment of objective 1:

1. Missing requirements
2. Hidden stakeholder needs
3. Inadequate or ambiguous requirements
4. Conflicting requirements
5. Lack of measurability or testability of requirements
6. Onward communication of requirements
7. Changing requirements
8. Over-specified requirements

Concerning RE methods, those in the categories of requirements elicitation and analysis were found to offer more potential for reuse in Agile projects than other RE topics such as requirements management, whose methods are typically closely linked with a particular development lifecycle. Fourteen RE methods were therefore identified:

Elicitation Methods:

• Workshops and/or interviews
• Storyboards and scenarios
• Mock-ups and prototypes
• Background studies and system archaeology
• Questionnaires or surveys
• Creativity techniques Brainstorming or brainstorming paradox

Analysis Methods:

• Context or problem diagrams
• Use cases and/or use case diagrams
• Conceptual data models (ERD's, class diagrams, etc.)
• Business process or activity modelling
• Sequence or event trace diagrams
• State models or machines
• Goals model or AND/OR trees
• Special analysis of non-functional requirements

Key Sources:

• Van Lamsweerde (2009) ^[7]
• Robertson & Robertson (2012) ^[5]
• Pressman (2005) ^[4]
• Pohl & Rupp (2011) ^[3]
• Faulk (1995) ^[1]
• Leffingwell (2010) ^[2]

  Research Method

A questionnaire was designed asking participants in Agile projects whether the above problems were recognised in practice and to what extent they were currently addressed. Respondents were also asked to report which, if any, of the identified methods were found to be useful in their Agile projects. The questionnaire was distributed electronically to a broad range of RE and Agile practitioners using relevant internet forums and communities. The results of the questionnaire were reviewed by an expert panel, with the ultimate goal of deriving guidelines as per the research aim.

Results of Primary Research

The survey was published in the IREB Magazine (Issue 2015-02, http://re-magazine.ireb.org/issues/2015-2-bridging-the-impossible/re-in-agile-projects-a-survey/), with links then posted in both Agile and Business Analysis Linked In forums. By the cut-off date in July a total of 43 completed responses had been received. The results for each question are displayed graphically below, with a brief analysis at the end of each section. Further interpretation of the results is provided in the Expert Review and Conclusion sections.

Section A: You and your project

Respondents were first asked to provide information about their role and type of Agile project:

Thus it can be seen that while respondents were predominantly from an RE background, a number of Agile specialists also participated. Scrum was seen to be clearly the most popular Agile method, though in a majority of projects one or more scaling factors (i.e. factors requiring a scaling of the pure Scrum approach as described, for example, in the Scrum Guide [Schwaber & Sutherland, 2011] ^[6]) were also found to be present.

Section B: Requirements Problems

The results appear to show that requirements problems are, broadly speaking, recognised within Agile projects. The extent to which these problems are addressed varied: respondents reported fewer errors as a result of issues such as changing requirements, communication of requirements to developers and over-specification, while other problems such as missing, ambiguous or inaccurate requirements and hidden stakeholder needs continued to present important challenges.

Section C1: RE Elicitation Methods

Particularly positive responses were received for workshops and interviews (C1), storyboards and scenarios (C2), and mockups and prototypes (C3). This is perhaps not surprising: intuitively, storyboards and scenarios match well to the Agile technique of user stories, while prototyping has a similar emphasis on an iterative, ‘working code’-based approach. Workshops and interviews can also be understood to be focusing on direct customer communication, though a distinction can be drawn here between the communication within an Agile team and a more systematic interviewing of the wider stakeholder community. The most negative responses were received to question C5 on questionnaires. One explanation for this might be that this technique is perceived as too slow and document-intensive for Agile.

Section C2: RE Analysis Methods

In general the results show that many anaylsis methods are used in Agile projects (49% of responses show agreement as opposed to 37% disagreement), though a consistent pattern for the general applicability of particular methods is hard to discern. Taking as a measure the number of responses expressing agreement (somewhat or strongly) as compared to the number expressing disagreement, the following analysis methods were most favoured:

• Context or problem diagrams (28 agree compared to 8 disagree – C7)
• Use cases (29/10 – C8)
• Conceptual data modelling (21/13 – C9)
• Business process modelling (24/13 – C10)
• Analysis of non-functional requirements (23/13 – C14)

More negative results were seen for:

• Sequence or event trace diagrams (16/22 – C11)
• State models or machines (17/20 – C12)
• Goal models (9/26 – C13)

Whether such results are because the methods are not useful in Agile projects, or whether there is simply a lack of knowledge or training, is not clear.

Expert Review

Based on the above results, the supporting background research and the experiences of the individuals involved, a review process was carried out with a small panel of RE and Agile experts. The results of this review – which added a degree of interpretation above and beyond the direct analysis of the research data – were formulated as nine guidelines for the use of RE within Agile projects:

1. Agile methods, and Scrum in particular, typically prescribe small, cross-functional teams e.g. 7-9 team members. In more complex, scaled project environments, however, Agile does not preclude the use of larger teams including specialist BA or REng roles.
2. Establishing real business needs and communicating these as software requirements to the developers remains as challenging a task within complex, Agile projects as it was previously in traditional approaches. Those responsible for this task within Agile projects should in particular be aware of – and take actions to avoid – the following requirement problems:
   
   
   • Missing requirements
   • Hidden needs
   • Inadequate or ambiguous requirements
   • Conflicting requirements
   • Lack of measurability or testability of requirements
3. Agile approaches do not preclude the use of additional software engineering methods; RE methods in particular can play an important role in complementing core Agile methods. Though exactly how (e.g. over what scope), and when (i.e. in the project lifecycle) they are applied may vary, many RE methods for the elicitation, analysis and modelling of requirements are already being used in Agile projects. These include:
   
   
   • Interviews and workshops
   • Storyboards and scenarios
   • Mockups and prototypes
   • Background studies and system archaeology
   • Creativity techniques e.g. brainstorming
   • Techniques for the analysis of non-functional requirements
   • Context diagrams
   • Use cases and use case models
   • Conceptual data models (ERD’s, class diagrams, etc.)
   • Process models (business process models, activity diagrams, etc.)
4. RE provides methods formalising the approach taken to identifying stakeholders and to the elicitation of requirements in workshops and interviews. A more systematic application of such techniques before the start of development can help to identify hidden needs in complex, Agile projects.
5. There is a perception that in Agile approaches user stories replace all other forms of requirements: this is incorrect and increases the chances that the resulting requirements are ambiguous or inaccurate.
6. RE modelling methods (e.g. of data, process or behaviour) support both completeness in the analysis as well as providing views of the problem that can be shared and corroborated amongst project members. Knowledge of these techniques, and their systematic application throughout development iterations, can help address the identified requirements problems in complex, Agile projects.
7. Special attention needs to be given to non-functional requirements in Agile projects, where the focus would otherwise be on users' functional stories. This starts with a thorough analysis of non-functional aspects of the system such as usability, reliability, performance and portability.
8. Context diagrams can play an important role in Agile projects by providing an overview of project scope beyond the confines of particular development iterations (or sprints).
9. Business process models can play an important role in Agile projects by providing a coherent view of end-to-end processes, thus helping to place individual features (ie. backlog items) in a broader project context.

Conclusions

The research has provided evidence that many of the software development issues that the discipline of RE seeks to address – termed here ‘requirements problems’ – continue to be recognised as challenges within Agile projects. While some problems such as changing requirements and over-specification were reported as less prevalent under Agile, others, including hidden stakeholder needs and missing, ambiguous or inaccurate requirements, continue to be found with regularity.

The research has also found evidence that the use of many RE methods within Agile projects is widespread and is not, in practice, seen to be in conflict with Agile principles. The use of some other methods, such as questionnaires, goal models and state machines, appeared in these results to be less well established.

Based on the quantity of data gathered and the lack of project context provided, it would be hard to draw definite conclusions as to exactly how and when specific RE methods should be applied in Agile projects. Nevertheless following a review of both the research results and the underlying research topics, nine general guidelines for the application of RE methods within Agile projects were formulated (see previous section).

Afterword

This research has examined the premise that RE and Agile may be combined within a hybrid approach, which we may call Agile RE. By considering RE and Agile as collections of individual methods and techniques, the evidence of this research suggests that pragmatic approaches can be found that usefully combine elements of both.

As a final thought, however, one might additionally ask: is RE perhaps something more than the sum of its individual parts? That RE and Agile were once perceived to be in conflict may not only due to the incorrect assumption that RE is strictly tied to a waterfall process model, but rather on the more fundamental difference between an engineered, top-down approach to system development and the ideas of emergence (emerging requirements, emerging architecture) that underpin Agile. To understand not only if a technique, such as data or process modelling, is applicable, but exactly how (over what scope, at what granularity?) and when it should be applied, further examination of these questions may be required.

Acknowledgements

Many thanks to Kim Lauenroth and Thorsten Weyer for their participation in the expert review process, and to all those who took the time to respond to the survey.

References and Literature

• [1] Faulk, S.R. (1995) 'Software requirements: a tutorial', Software Engineering – Tutorial Volume for The Open University.
• [2] Leffingwell, D. (2010) Agile software requirements: lean requirements practices for teams, programs, and the enterprise, Addison-Wesley Professional.
• [3] Pohl, K. and Rupp, C. (2011) Requirements Engineering Fundamentals: A Study Guide for the Certified Professional for Requirements Engineering Exam-Foundation Level-IREB compliant, Rocky Nook.
• [4] Pressman, R.S. (2005) Software engineering: a practitioner's approach, 6th Edition edn. Software Myths page 13, Palgrave Macmillan.
• [5] Robertson, S. and Robertson, J. (2012) Mastering the requirements process: getting requirements right, Addison-Wesley.
• [6] Schwaber, K. and Sutherland, J. (2011) 'The Scrum Guide', Scrum Alliance, [Online]. Available at scrum.org (Accessed 01/09/2015).
• [7] Van Lamsweerde, A. (ed) (2009) Requirements engineering: from system goals to UML models to software specifications, 1st edn. John Wiley & Sons.