Rainer Grau

Mission Possible

Concept for the successful handling of integral NFRs in Scaled Agile Environments.

Initial situation

We just can't get rid of them in certain areas: non-functional requirements (NFR) - quality requirements (QR) according to IREB. Whether in waterfall environments or in agile development - some NFRs have to be met across the board, integrally and holistically. These include, for example, security requirements (ISO/IEC 27001), the General Data Protection Regulation (GDPR) of the EU, domain and company-specific compliance requirements (SOX, Sarbanes Oxley Act), but also other industry-specific compliance and regulation constraints such as FINMA, FDA, EN50155 and others.

The special thing about these NFRs is,

• that they must be completely fulfilled in order to be compliant. A compliance rate of 95% is not sufficient.
• that in order to fulfill a single one of these coarse-granular requirements, often an entire set of fine-granular detailed requirements must be fully met in terms of:
  o organizational process steps,
  o architectural building blocks,
  o development,
  o validations or certifications.

In the following, we will focus on the area of cyber security. However, the concept is equally applicable to all auditable requirements of the above-mentioned areas.

For example, the GDPR contains the NFR "Encrypt, pseudonymize, or anonymize personal data wherever possible". To realize this requirement in an enterprise, we need:

1. a concept for identity and access management (IAM),
2. the implementation of the concept both, organizationally and in the software systems, i.e., in the standard software used (e.g., SAP) and in the system architecture for in-house developments
3. a test and validation concept and its implementation, to prove conformity with the requirement.

To address all of this, many departments and teams are involved and need to coordinate their work. Typical stakeholders are the Compliance and Information Security Office (CISO), system architects, development, testing and, of course, operations.

In a classic process model, it was "easy" to fulfill the requirement and provide proof of conformance, at least from a process perspective. Just because there were clear quality gates in the development (rough concept, detailed concept, design, implementation, test, acceptance, operation). Whenever there was a transition from one quality gate to the next, there were validations defined by the CISO, whose successful execution was monitored and documented by the CISO. Whether the validations were "easy" to perform is another story. At least, the validation points and the scope of the validation at each of these points were clearly defined and the execution was verifiable.

With an agile approach, it is no longer so easy to ensure conformity. Let's take a large company as an example, which develops part of the software itself according to the Scaled Agile Framework SAFe. A team of developers (one of possibly 50 teams in the company) implements software for quotation and deploys a new version of the software every second week. To fully comply with the GDPR requirement to "Enrcrypt, pseudonymize, or anonymize personal data wherever possible", everything needs to be in place in each deployment, i.e., an IAM concept, a software architecture (possibly in the cloud) that meets part of the requirement, and an acceptance test suite that provides documented proof that any changes to the software still meet the GDPR requirement.

The exciting question is: How do we synchronize these activities, which take place at different levels of abstraction in the company, require different know-how in different teams, and take place at different times?

Is this a Mission Impossible?

The conceptual approach

No, it is not. There is a conceptual approach to successfully meet cross-cutting and integral NFR in a Scaled Agile environment without reverting to classic quality gates and centralized testing organizations.

This conceptual approach works with the following roles cooperating in an agile setup:

The CISO

• Is responsible for defining the list of NFRs relevant to the business at the top level of abstraction.
• Is responsible for working with business owners to define what subset of requirements need to be met for a development project.
• Defines together with the business and system architects, which part of an individual NFR is covered organizationally, by the system architecture (the development platform), or in development and defines the corresponding acceptance criteria for this.
• Trains specific roles in the organization (users, software architects, developers) on how to perform security-related activities.

The operating units

• Will be trained in applying organizational measures in operations (example: agencies or customer service in opt-in from customers) and apply these measures.

The software architects (also possible as a decentrally organized community of practice)

• Define together with the CISO which detailed requirements of an NFR are covered by the system architecture (e.g., IAM) and provide the corresponding software building blocks, for example, in the form of a development platform.
• Will be trained by the CISO so that they have the appropriate NFR expertise to define the software architecture accordingly.

The development teams

• Develop together with the CISO the subset of sub-requirements and the associated acceptance criteria for which the development team is responsible. Fortunately, often no action is required here at all, since the sub-requirements are already covered organizationally or by the software architecture.

Testing and validation (also possible as a decentrally organized community of practice)

• Definition of concrete acceptance tests to validate the NFR. These acceptance tests, which are mandatory before deployment in the productive environment, are (ideally) part of an (automated) test suite. They validate those sub-requirements of an NFR that are to be fulfilled by architecture and development - including the documentation required for proof.

There is a charming advantage with overarching and integral NFRs - they change only very slowly. Once a company has initially defined the NFR, broken down each individual NFR into sub-requirements and decided which sub-requirements are to be met organizationally and which are to be met by the architecture or in development, this interaction changes only very slowly over time. It is therefore possible to work optimally with templates and checklists, which, for example, become part of a Definition of Ready (DoR) or a Definition of Done (DoD). In addition, a test suite of reusable test cases for the validation of sub-requirements is created over time.

Exemplary implementation of the concept

Let's build this as an illustrative example (and highly simplified) for the GDPR NFR "Enrcrypt, pseudonymize, or anonymize personal data wherever possible":

1. CISO, together with HR and the system architects, define a role-based IAM concept, i.e., which role gets which rights to which objects in the company.
2. The IAM concept must be anchored organizationally, i.e., it must be determined which employee(s) will assume which role(s) - both in development and in operations. The employees may need to be trained.
3. The system architects decide which software building blocks or standard software are used to implement the IAM.
4. When software is developed in-house, development teams develop IAM access modules that are part of the software architecture and are used in development to guarantee IAM-compliant access to objects.
5. The system architects, together with the CISO, define acceptance criteria and derived logical test cases that validate compliance with specific sub-requirements of the NFR.
6. The development teams implement specific test cases according to the acceptance criteria and the logical test cases. These are on the one hand acceptance tests, which are included in a general regression-capable test suite, and on the other hand specific developer tests.
7. The DoR of a feature to be implemented includes validating whether the IAM concept defines the necessary roles and access rights and whether these are implemented in the IAM.
8. The DoD for deploying a team deliverable includes a checklist of the following items:
   • Only the IAM access modules of the system architecture are used.
   • The IAM-specific acceptance tests of the regression-enabled test suite have been successfully executed and the necessary documentation of the tests is available.
   • The IAM-specific testing for the developed deliverable has been successfully performed and the necessary documentation of the testing is available.
   • Additionally developed regressive tests of development have been added to the regressive test suite.
   • The users of the software are trained (if necessary).

The tension between formalism and agility

This sounds like an awful lot of upfront work and little agility. However, this is misleading - provided that the company has a certain maturity in its processes regarding agile development. This maturity includes:

• Successfully setting up DevOps teams with clear DevOps pipelines in defined environments.
• Establishing regression-enabled test suites across the environments (e.g., development, integration, acceptance, operations) that are also integrated with DevOps pipelines.
• Living a clear agile development process with DoR and DoD definitions at the different abstraction levels of development. In SAFe, these abstraction levels are for example portfolio epic, feature and story.
• A clear repository concept that defines in which repository the documented validation of the NFR tests is stored and made available.
• A clear version management for the building blocks of the development, which is linked to the repositories in order to be able to track which validations exist together with the software taken operationally.

Once this - quite demanding - state of maturity is present in the company, the heavy upfront work dissolves because, as already mentioned, the overarching and integral NFRs change only slowly. Thus, an IAM concept should be sufficiently stable at some point, as should the associated building blocks of a system architecture. The checklists in the DoR and DoD should be in place and known, and a regressive test suite to validate the sub-requirements to the NFR should be in place. Thus, the change is limited to the delta of the respective development in the context of further features.

A classic manager would possibly argue: "I would like to have the responsibility for the complete fulfillment of the NFR with one person!

In fact, this is impossible. A single person is not in a position to possess the comprehensive and necessary know-how to understand and question all aspects of a validation in a sufficiently qualified manner. Inevitably, this person delegates activities to other persons or organizational units. The above approach also delegates - but with the difference of a clear responsibility for a defined subset of the sub-requirements and a mechanism defined in the process to combine and align the individual parts to the desired outcome.

Responsibilities	Description	Aids, Process Elements
CISO (Compliance and Information Security Office)	Definition of the relevant Security & Compliance NFR	DoR / DoD Template phrasing of acceptance criteria for portfolio epics, features and stories
Portfolio	Abstraction level Epic / Capability Roles: Business Owner, Epic Owner, Solution Engineer Experts: CISO, Security Expert Acceptance Criteria per relevant NFR in the epic context	DoR Fulfilled, agreed, accepted, and documented
ART (Agile Release Train)	Abstraction level Feature Roles: Product Manager, System Architect, RTE Experts: Security Expert Acceptance Criteria as checklist per feature in the DoR	DoR Fulfilled, agreed, accepted, and documented
Team	Abstraction level Story Roles: Product Owner, SW Architect, Scrum Master Experts: Trained team member Acceptance Criteria as checklist per story as part of the DoD	DoD Fulfilled, tested, validated, and documented

Only an externally required certification or acceptance by official regulators limits the freedom to deploy incrementally, since in this case a defined and traceable scope of a development with analysis, design, testing and validation results must be submitted to the official body for acceptance. Of course, this means a waiting period in development between achieving the DoD of a functionality and operational deployment. Depending on the effort of the official procedure to be followed, it is necessary to consider in which increments such a development is designed.

Organizational constraints and requirements

The conceptual approach described here to fulfill cross-cutting and integral NFRs in an agile setup requires a certain maturity and self-discipline in the company. Certain elements of the approach must be adhered throughout. These are primarily:

• Usage of DoR and DoD
• Usage of defined repositories in analysis, design, testing and validation in development, as well as for the documentation of operational systems
• Cross-team application of the same procedures in analysis, design, testing, validation and documentation
• Self-discipline of all teams involved in development and operations

Self-discipline is a sensitive issue that requires appropriate communication and leadership within the company. In a larger company, which is subject to regulatory constraints, all employees must be aware that certain procedures are defined in a holistic and consistent manner and must be adhered to. This does not violate the agile mindset. The individual tasks are still decentralized. How an individual procedure is implemented, e.g., the system architecture or the test suite for validation, is a definition that can be made, for example, by a community of practice (equipped with decision-making rights for certain standards).

All in all, his approach minimizes the administrative overhead for development teams. The administrative tasks to be complied with by an agile team for an increment of software to be deployed are limited to compliance with certain procedures according to DoR or DoD in analysis, design, testing and validation.

 
More in-depth information on this topic, including a detailed concrete implementation example, can be found in the author's blog.

List of Abbreviations

ART	Agile Release Train
CISO	Compliance and Information Security Office
DoD	Deifinition of Done
DoR	Defintion of Ready
GDPR	General Data Protection Regulation
IAM	Identity and Access Management
IREB	International Requirements Engineering Board
NFR	Non-functional Requirement
SAFe	Scaled Agile Framework
QR	Quality Requirement