Options:

Gareth Rogers

How agile can Requirements Engineers really be?

My experiences from the Telecoms industry.


Agile remains a disputed approach. Despite the bold sounding claims of many articles on its proven ability to increase productivity and reduce costs, some experienced practitioners continue to feel uncomfortable with the setting aside of established methods and techniques in favour of a leaner, more lightweight approach. Like it or not, however, Agile is already a reality in the industry and the question for requirements engineers is not so much if, but how, we are to adapt.

In this article I will focus on the consequences of Agile on the role of requirements engineer. In a number of discussions on the subject the view emerges that the core skills of a requirements engineer remain relevant in the Agile world, even if they are applied in different ways and at different points within the project lifecycle [1]. On the basis of my experience within the Telecoms industry, I would like to challenge this view and consider the hypothesis that elements of the Agile approach may in fact be contrary to core goals of requirements engineering.

Which Agile are we actually discussing?

The ideas behind Agile development originated from a number of sources [6] and have developed in a number of directions, from Extreme Programming and Kanban to scaled Scrum approaches such as that proposed in the Scaled Agile Framework [2]. What these approaches have in common are the principles of Agile, in which emphasis is placed on the early delivery of usable software and face-to-face communication is preferred over extensive documentation.

It is the ceremonies and vocabulary of Scrum, however, that have arguably become mostly closely associated with Agile software development approaches: daily stand-ups, product backlogs and development sprints increasingly belong to the standard lexicon of the IT industry. Worth noting, however, is that according to the Scrum Guide itself [8], Scrum is neither exclusively aimed at software development, nor is it a “process or a technique for building products; rather, it is a framework within which you can employ various processes and techniques”. This is a point I will return to later.

Agile in the Telecommunications Industry

Characteristic, though by no means unique, to software development in the telecommunications industry is the need to build solutions within the constraints of complex existing heterogeneous system landscapes. Features must be brought to a fast-evolving market quickly while interfaces with existing infrastructure and business systems are maintained and regulations complied with. The functional requirements of the principal project stakeholders are of course key – the issues which ultimately cause project overruns and catastrophic failures are often, however, those associated with non-functional concerns such as scalability, reliability and compliance with external interfaces.

The ceremonies of Scrum are typically welcomed in the telecommunications industry as a means to encourage open communication within development teams and to increase the overall level of engagement of team members. Product backlogs are often viewed as a more flexible alternative to lengthy requirement specification documents, while regular sprints are a way of keeping development teams continuously productive in parallel with other project disciplines.

Where it is harder to generalize, however, is on the impact of adopting an Agile approach on the roles which are not explicitly mentioned in Scrum, amongst others: Business Analyst and Requirements Engineer, System or Solution Architect and Project Manager. In the following section I recount experiences as a requirements engineer from two projects delivered, to a greater or less extent, according to Agile methodologies.

Tales of Agile – Project A

I was recently involved on behalf of a consultancy in a project to deliver a mobile messaging platform to a major operator. An Agile approach was adopted with the aims of shortening the time to the delivery of usable software, reducing overall costs and encouraging a positive collaboration amongst all parties.

One result which became clear quite quickly was that features delivered by early sprints were not fully testable and therefore not ready to be released in their own right. Elements of the user interface, for example, had little validity until the middleware on which the system depends had matured enough for its configuration needs to be clear. A second example was that the look and feel of the application could not be fully and consistently designed until hidden browser dependency requirements had been fully uncovered.

Some thought had of course been given to such dependencies during release planning. What emerged, however, was that the nature of these dependencies was more subtle than initially expected. The preparation of test environments and data for intermediate sprints began to look more complex and time-consuming than the development itself. At this point early testing was suspended, accompanied by some unwanted management attention.

A second outcome of the development process only became fully apparent later in the project lifecycle. As the completion of more sub-components allowed the execution of more integration tests, problems associated with non-functional aspects of system behaviour – notably scalability and reliability – became more evident. Normal enough, you might say, except that this was not so much the expected tuning of system configuration as the realisation that the non-functional requirements themselves had not been sufficiently specified, and once they were, major redesigns of system architecture were required. The end result was that original development and testing estimates almost doubled.

In the project review it was concluded that insufficient thought had been given both to the planning of individually testable deliverables as well as to the detailed specification of non-functional requirements (these requirements being identified as a prerequisite for a more thorough architectural design).

Were these issues then to be blamed on Agile, or perhaps rather to the fact that Agile had not been properly carried out? Certainly there had been shortcomings in the observation of Scrum guidelines: teams could not be fully co-located, due to near-shoring of development teams, while the commercial interests involved in the client/supplier relationship had not exactly encouraged open communication. Before addressing this question I would first like to examine the results of a second engagement.

Tales of Agile – Project B

This project concerned the adaptation of provisioning processes in compliance with directives from a regulatory body. In this case a hybrid Agile approach was used, in which Scrum teams operated at multiple levels across the organization. Epics representing high-level business objectives were first drawn up, these were then decomposed by a product owner into tasks, and these tasks in turn broken down to user stories by a top level Scrum team that included requirements engineers.

To cut rather a long story short, the following quote from a management report summarizes some key project outcomes:
“…unexpected details and new requirements [were identified] during the implementation, many supposedly independent requirements were, in fact, found to be dependent on one another; changes to processes and systems had unexpected impacts on other areas; the consequences of changes to complex processes were not fully predictable.”

In this case the observation of Scrum ceremonies had, in general, been successful, including support from a number of specialised Scrum experts. More problematic had been the integration of the project team with other parts of the organisation not yet so familiar with the Agile approach. Resourcing, budgeting and operational release planning, for example, were still carried out according to more traditional approaches and caused friction when faced with the relative flexibility of a sprint-based delivery.

Requirements engineering had been focused on the tasks. Thus while individual tasks were themselves reasonably well understood, dependencies amongst tasks were not always easily identifiable.

A Comparison of RE and Agile Methods

“The later in the development project a defect in the requirements is corrected, the higher are the costs associated with fixing it”. This statement is quoted from the IREB Study Guide [3] and similar statements can indeed be found in the introductory sections of most RE text books. Such statements are, we are told, backed up by empirical evidence (perhaps, though, gathered in the context of non-Agile projects!). Most of us, however, feel this to be intuitively true: it simply feels wrong to start building a solution before we have satisfied ourselves that the problem has been adequately understood.

Through context diagrams, structured requirements documentation, multiple modelling perspectives and requirements validation techniques, RE provides methods and techniques that ensure a complete and consistent problem description while avoiding premature design decisions [4]. This is the added value of past experience: we do not simply accept requirements from individual stakeholders at face value; instead we analyze the problem from different perspectives, and bring these analyses together again to check their consistency. We ask difficult questions, probe for gaps and risks, tease out areas of commonality, and work to find the inherent structure in the problem domain that will form a basis for - though without constraining the nature of - the solution design.

A goal, then, of RE is to avoid expensive mistakes in the implementation by a rigorous analysis of requirements. Agile turns this problem on its head: with Agile, having identified dynamic market conditions and the resulting requirement churn as the greater danger, we try to avoid the mistake of expensive, up-front planning and analysis in favour of delivering working software as quickly as possible.

The focus in Agile is neither on the completeness of the requirements description nor on consistency or structure amongst the various requirements. The emphasis instead is on assembling and refining user stories - requirements expressed in the language of the stakeholder - within one or more product backlogs. User stories should be deliverable in their own right as working software elements, allowing a prioritisation of the product backlog according to business needs at the start of each development sprint. In this way features are brought quickly to the market and tested not against expectations of business analysts in some distant project phase, but directly by the customers or users of the product.

The question remains, however, of how to arrive at this selection of independently deliverable, loosely-coupled elements (user stories) in the first place? In the next section I argue that this, in fact, is where the problem lies.

So what really went wrong?

In the projects described earlier individual features, as described in their respective user stories, were, in many cases, fully and correctly implemented. Whether these features combined to fully meet customer expectations, however, and in doing so form a scalable and sustainable architecture, was shown to depend on the strength of the underlying analysis over the whole of the problem domain. Where this analysis or the resulting architectural design were lacking, hidden dependencies amongst the individual features caused a level of rework that rapidly began to outweigh any advantages of early delivery.

Failings in the application of Agile principles, or in the observation of Scrum ceremonies and ground rules, were, in my view, secondary. Communications amongst project members could have been further optimised, and some problems potentially recognised earlier, but the idea that the looked for architecture would have emerged informally from this process was, under these project conditions, simply unrealistic.

Had the goal of the projects been to build completely new systems, with features that could be easily understood in terms of the user interface, the results may have been different. The complexities of the underlying architecture and dependencies amongst user stories and on existing components would likely have played a less significant role. If the projects had not been constrained, by external directives or contractual agreements, to meet particular objectives by certain deadlines, then the outcomes might also have been different. With products that gradually evolve to meet less clearly defined end-customer needs (one might think of a Google or other, innovative, web-based products), the idea of an emerging architecture is more conceivable. These conditions, however, are rarely found in the development of core business systems for telecom providers.

Conclusions

It is my experience, then, that in two projects run in according to Agile approaches, a lack of thorough requirements engineering contributed greatly to project overruns and quality issues. I have argued, that by emphasising early delivery over a thorough analysis of the problem domain, Agile principles worked against the rigorous application of tried and tested RE techniques. This in turn led, perhaps unsurprisingly, to a recurrence of precisely the sort of issues that RE was designed to solve.

That tensions exists between the demands of RE and Agile – indeed between those of software engineering in general and Agile – is not new. Hybrid Agile methodologies, such as SAFE [2] (itself a controversial topic amongst some Agile purists – see [10]!) and ASM [9], recognise that in scaling Agile for larger projects attention must be refocused on upfront domain modelling and architectural design activities. On the other hand some authors, such as Alistair Cockburn in the preface to [5], question the whole idea of engineering as a paradigm for software development, preferring to think in terms of a highly human-centric activity.

The debate, then, goes on. My observation is that in the hype that sometimes surrounds the uptake of Agile in an organisation the subtleties of this academic debate might well be lost. Agile is not understood as complementary to the necessary disciplines of RE – or, to return to the earlier description of Scrum, as a ‘framework within which you can employ various processes and techniques’ - but is instead thought simply to replace them. Even as we, as RE practitioners, educate and adapt ourselves to these new methods, it is important to be aware of this challenge and to be prepared to reassert the fundamental importance of the RE discipline.

References



Gareth Rogers

Gareth is an experienced business analyst and product owner who has been working in the telecoms and other industries for longer than he’d like to be specific about. He has been an active member of IREB for some years and is a co-author and examiner within the RE@Agile working group. He lives and works in Germany having successfully escaped from a small, slightly uncooperative island.