Martin Tate

IT Requirements when Buying, not Making

Effective specifications to select off-the-shelf software

Procuring off-the-shelf or COTS software is often the only feasible route to access new facilities, due to the reduced time, cost, risk and organisational impact [1] [2]. For some time, off-the-shelf solutions have dominated certain areas of IT; this effect is intensifying, partly reinforced by cloud solutions. A growing percentage of an organisation’s IT portfolio is COTS software; the dominance means many organisations are moving from application development to application assembly [3] [4]. Selection projects must therefore tackle complexity that has internal (organisational) and external (solution) dimensions.

As with most IT projects, capturing and documenting requirements is a foundation for success. However, with selection rather than construction – a project to buy, not make – you need requirements that are specifically engineered for this purpose. This is sometimes termed COTS-Aware Requirements Engineering (CARE) [5] and Procurement-Oriented Requirements Engineering (PORE) [6].

This article, as a practitioner guide, does not contain radically new techniques, but aims to direct your mind-set and the approach you should take if you are the requirements engineer, requirements analyst or business analyst on a software selection project. While it references many academic and practitioner articles, it is not a full survey of the literature.

Your approach must also reflect the realities of a project that involves collaboration between different organisations (yours and multiple suppliers) that have different financial imperatives, perspectives and reporting structures. This means understanding where the organisational and personal interests align or diverge, and adopting appropriate and pragmatic techniques to manage stakeholders, expectations, requirements, documentation and meetings [8] [9] [10].


The terms artefact, completeness, priority, requirements document, scenario, stakeholder, supplier, user and validation are used with the meaning defined in the IREB Glossary [11]. The terms component and software component are used as defined in the IREB glossary, but a large solution like an off-the-shelf software product (whether on-premise or cloud) might also be regarded as a system in its own right.

COTS software means a Commercial Off-The-Shelf software component, which may be a cloud service or a more traditional on-premise package.

A candidate is the combination of one solution (software product) as put forward by one prospective supplier – this means that two candidates might be the same reseller putting forward two products, or the same product from two different resellers.


Requirements engineering has addressed the topic of selecting COTS software since the late 1980s. Of course, many standard techniques will apply, including Requirements Elicitation and Requirements Documentation, especially using Natural Language and Requirements Validation and Negotiation [12] [13]. Many common requirements engineering objectives apply, including the fact the “requirements engineer is supposed [to] elicit requirements in a way that is both unbiased and neutral” [14]. There may also be a need for mediation techniques to resolve conflicts in requirements [15], especially if the project spans organisational silos and supply chain boundaries.

This article offers more specific and directed approaches to selecting off-the-shelf IT solutions by drawing on the author’s book [16] and experience as an IT practitioner specialising in selections.

Selection Method Context

There are many published methods for selecting off-the-shelf ie COTS software [8] [17] [18] [19] [20] [21] [22] [23] [29]. This article uses the Decision Evaluation Selection Method to set the context. After scoping and requirements capture, there will be phases of longlisting, RFI, detailed evaluation with scoring, demonstrations, reference sites, negotiation, contract and implementation. See Figure 1.

While this is not a full review of selection methods themselves, some of the main differences between the approach used here and some existing techniques for requirements-based COTS-selection are as follows.

Requirements Have Multiple Roles Throughout the Selection

The agreed requirements document will almost certainly be the most important deliverable created during your selection project.

Requirement capture has several ‘general’ uses throughout the project, such as to engage stakeholders and to help familiarise all contributors with organisational processes.

Moreover, requirements specifically feed multiple stages of the evaluation, selection and procurement method. Figure 1 shows how the requirements, once elicited, feed stages further down the selection process. In some cases – see D and H – the main requirements document is provided to suppliers in its original form. In others – such as C and G – a sub-set of requirements is extracted and repurposed into documentation with a specific evaluation objective.

Figure 1: Overview of the selection approach showing requirements flows

A. Scope sets the overall context that affects all requirements. The scoping exercise will not only feed in critical requirements, but will also shape some of the weights for requirement importance or value.

B. The requirements statements provide the basis for the end-of-phase weighting exercise, which will distinguish between the must-have and nice-to-have requirements, to record the organisational value of each requirement.

C. The requirements are repurposed to derive your preliminary shortlisting questionnaire (the request for information, or RFI). One of the initial criteria for extracting the sub-set of requirements is to choose those requirements that have higher weights for importance.

D. The full requirements statement informs 3-4 shortlisted candidate IT suppliers of your organisational need. Candidates respond to the main part of the requirements document (the ‘scored requirements’) during detailed evaluation meetings.

E. The requirements form the basis for scoring shortlisted candidates after detailed evaluation.

F. Requirements feed your software gap analysis, because they create the yardstick that allows you to spot the requirements that are not met. (Note this gap analysis is subsequently one feed into both your questions for reference sites and your preparation for negotiations, but this extra use of requirements is indirect so it is not explicitly shown in Figure 1.)

G. Requirements feed your requested outline for the demonstrations (again, particularly those requirements that have higher weights for importance).

H. The full requirements document is one of three critical working documents that become an attachment to your contract with the successful candidate.

Managing Expectations During Requirements Capture

It is important to manage user expectations during requirements definition interviews or workshops.

Expectations About Meeting Requirements

Many software development projects assume all requirements will be met – even though not all requirements will translate into benefits [26]. With an off-the-shelf selection, the reverse is true – you assume you will never find an off-the-shelf product that fits perfectly. By definition, some requirements will never be met. In practise, most users recognise this is the case, but it is best to begin all requirements capture meetings with this warning.

When the requirement is recorded, there can be no guarantee that the requirement will be met at procurement. This is because the requirement may prove to be technically or economically unfeasible: the solution with best overall fit may lack support for the requirement; the desired facility may only appear in an extra module that is not cost-justified.

However, sufficiency beats perfection. Managing expectations at less than 100% fit is important to making progress. If users insist on a solution to meet every need, they will probably force a ‘shortlist of zero’ with all candidates disqualified. If their organisation lacks the appetite, time, funds or skills to build, they will miss out altogether on new software.

Requirements Cannot Be Ignored

At the other extreme, some organisations have obviously believed it was not worthwhile to consider their own requirements because the project is selecting a pre-existing artefact. Because the software already exists and you probably cannot change its function, limitations, source code, documentation or schedule, they may think there is no point in defining requirements before studying candidates. This approach to evaluation is sometimes called I’ll Know It When I See It or IKIWISI [27].

However, while you might not be able to immediately change the content of a candidate software product, your requirements can (and should) shape the set of facilities you adopt during your selection. The point may be obvious but is sadly often ignored – requirements are essential if you are to find the candidate with the closest fit to your needs.

Project Philosophy Concerning Customisation

It is also important to have a clear view during the project about customisation or tailoring – paying for program modifications to adapt the off-the-shelf software to the specific context of use.

This is tempting, and some software suppliers push the idea – it flatters the customer to say they deserve special software, it increases customer lock-in and their software becomes a Trojan Horse to make the real money on services for the whole life of the installation.

However, customisation should be resisted. In some areas such as financial accounting, off-the-shelf software has been available for over 40 years, so organisations may be using their third generation solution. I find organisations that customised their older software tend to be less likely to tailor now, because they have learned their lesson. For organisations new to COTS software, tailoring as a first resort rather than a last resort is a common approach (and commonly regretted later).

I have met project managers who had multi-million budgets solely to ‘return a product to vanilla’ and I’ve been asked to help companies recover from excessively-modified software that was only one year old. In short, tailoring off-the-shelf software often indicates ineffective change management; it is a risky, costly, inefficient way to make a product fit; it can cut you off from supporting products and future releases; recruiting new staff with experience of the standard software will not be a benefit if they don’t recognise it. Also note that some software suppliers now refuse to entertain customisation and will only implement the standard product. Rather than customise the code base, far better to adopt alternative approaches such as fully exploiting the configuration options (sophisticated settings) or supplemental solutions [16].

Team Communications

Managing user expectations during requirements definition is only one dimension of managing change during the organisational stress of IT-enabled transformations. Stakeholder management and communication amongst all team members are important throughout [28]. While it is a much larger topic than can be covered here, communications are particularly important when the project policies may at first seem shocking to the users, such as the very need to define requirements, or the insistence on barring customisation until the dust has settled on implementation.

Organising Requirements (Cataloguing)

People at requirements definition interviews and workshops rarely present you with ‘pure’ IT requirements. You are presented with a mixture of information about the organisation, its objectives, processes and flaws, and with human aspects, such as power structures. Moreover, when attendees cover requirements, they cover items at front of mind – they rarely give you a systematic trip through the organisation’s workflow. It is usual before documenting requirements to analyse, organise, classify and simply to think about them.

Accordingly, during the meetings and certainly before you document the requirements, you should organise your notes. If you have different notes sheets or online pages for different categories of requirements, you can capture some of them directly onto the relevant sheet during the interview or workshop. Examples include information on the current (incumbent) systems, glossary entries or business volumes.

You may maintain a separate section for project notes – this is project learning that you don’t want to lose about the organisation, but it is not a specific IT requirement that will make one candidate more suitable during selection. For instance, an interviewee may report that the organisation repeatedly under-estimates the importance of training. This is significant information to capture for later in the project, but is not a criterion for evaluating solutions. (Contrast this with factors that are criteria because they may reveal one candidate to be superior to the others – for instance that the supplier is able to offer training, or that there is an ‘ecosystem’ of trainers and training material, or that the solution is known for low training needs.) However, not allowing enough budget for training is an internal characteristic of the prospective customer, rather than one of the criteria for evaluating candidate suppliers. The risk statement may go into an appendix at the back of your requirement documentation, for internal circulation rather than release to suppliers.

You should reflect on your notes and ‘consolidate’ requirements where applicable. When cataloguing for a selection, it is especially important to group requirements into coherent subjects or business topics such as Finance, Scheduling or Documentation [16]. This will be important to allow specialism: subject matter experts in your organisation can review ‘their’ part of a wider requirement statement that cuts across organisational silos; internal specialists can attend meetings by exception, in order to weight requirements or to score candidates; the suppliers can put forward the relevant module expert to respond to a block of requirements.

You often get ‘fragments’ of the same requirement from different interviewees, and need to fit the jigsaw pieces together. The requirements need to be treated thematically because the processing that you purchase off-the-shelf will never be related directly to one interviewee or, indeed, department.

Of course, you may have conflicting requirements that need negotiation, mediation and reconciliation [15].

Adding Standard Requirements

There is also considerable scope for the requirements engineer to put forward requirements that have not been volunteered by business representatives. These may cover wider or specialist topics: technical issues, such as the preferred database technology; IT management dimensions, such as the mechanism for releasing updates; commercial dimensions, such as contractual stipulations; human and organisational change aspects, such as training; service dimensions, such as the types of consultancy available.

As part of compiling requirements, the requirements engineer can (and indeed should) add these ‘standard’ requirements – even though they were not explicitly reported during the requirements definition meetings. This is not prescribing the need to the user. If the requirement is inappropriate, it will be removed during the weighting meeting – see the section Weighting Requirements for Importance below.

Because the end-to-end requirements process will eliminate inappropriate requirements, this means it is safer to add an ‘unnecessary’ requirement than to risk something significant being missed. Often, requirements are not reported because of two extremes – at one end they are so ‘obvious’ that they are invisible to the organisation, or it is regarded as not even necessary to voice them. At the other extreme, the requirement is so sophisticated that is over the horizon for the current thinking.

This process of injecting requirements into the set means the requirements engineer can legitimately contribute subject matter: from other projects; from research into best practice; from the organisation’s library of standard requirements; from reverse engineering (studying software products to work backwards from features to potential requirements).

Later, this pool of requirements will be inspected by user representatives during the weighting meeting.

The Requirements Documentation

Remember that the fundamental objective is a requirement that can be scored later, meaning that (during the detailed evaluation stage) your evaluation team will examine three or four candidates against that requirement and (in a later, separate, scoring meeting) will allocate points for how well each solution meets each requirement. Each requirement needs to be expressed clearly enough that it exposes any significant differences between candidate solutions.

How you document your requirements is a type of decision-making. Decisions can be described as choosing consequences, so, as a practitioner, you should be aware of the consequences of certain styles of documentation, to avoid unintended consequences. Some of your decisions may need to be pragmatic and to actually disregard some of the guidelines of formal specification or quality methods, because of their impact upon the reader – a sales or technical representative at a commercial supplier.

Requirements must represent a blend of software facilities and supplier services. There will be a mix of functional requirements for software processing, plus quality requirements and constraints (such as the availability of documentation, the service desk hours, or commercial factors like the license price).

It is important to write at the correct length and level of detail. High-level, imprecise mission statements do not form an effective yardstick to measure software products or supplier capabilities. Equally, avoid large, multi-paragraph requirements that contain several detailed statements of need, even if they are related. One requirement covering a single page (of European A4 or US Letter) is probably too large for a software selection – solid real world experience indicates a minimum of 2–4 requirements per page is the appropriate level of granularity.

You should avoid overlap or redundancy. This means you should avoid measuring the same attribute multiple times. Also, avoid correlated or co-dependent variables, so that each requirement should ideally aim to test a different aspect of the software or supplier [29].

The Impact of Language on the Users and Suppliers

It is best if the same document is agreed by the user community and eventually provided without repurposing when you have the shortlist of 3-4 candidate IT suppliers. If you can engineer the document to make sense to both stakeholder groups (users and suppliers) this saves significant time and effort – and risk of translation error.

For a selection project, you are more likely to use natural language rather than an artificial specification language. You need to choose a format. The most important aspect of a specification format is that it is credible and engages the audience, so use what works.

You might write in a detailed requirements format, or in a shorter, more outcome-orientated scenarios format [30] [31]. Although user stories are associated with Agile and therefore software development projects, your organisation may be familiar with the format from other projects. If user stories are too high level to later probe the differences between candidates, they might be a precursor to the specification or (preferably) made more specific, for instance by adding test notes or acceptance criteria. These would be clearly associated with each story, without disrupting the main story flow.

Figure 2 illustrates the same requirement expressed within three different formats – one longer requirements definition format, one scenario and one user story.

Figure 2: One requirement in requirements definition, scenario & user story formats

The Roles of Language and Attributes

One specific aspect of wording is important. IREB guidelines suggest two alternative ways of fixing supplier liability for delivery (sometimes called legal obligation). Inline means the “fixing of liability by using the verbs ‘shall’, ‘should’, ‘will’, ‘may’ can be made in the text of the requirement. If the liabilities change, then the requirements change too. The use of attributes is another possibility for documenting the liabilities of requirements.” [32]

For the following reasons, it is much better to use the second approach – attributes, see the section Weighting Requirements for Importance below.

Ten Desired Characteristics of a Requirement for an Off-the-Shelf Selection

There are several general characteristics that are desired in any requirements statement – you always expect it to reflect all stakeholder groups, to be understandable, unambiguous, concise, traceable, consistent, quantified, value-driven, neutral/impartial and complete [14] [33] [34] [35] [36]. In addition, some specific characteristic are important when the requirements will be the input to selecting off-the-shelf solutions [16].

  1. Non-prescriptive: concentrates on business outcomes, without mandating the IT processing method, location, medium (such as paper) or person involved (except to set context). This is sometimes referred to as design-independent, as avoiding premature design decisions or avoiding architecture in requirement statements. Describe what the software needs to do, rather than how it should do it.
  2. Collected: brings together similar requirements from different conversations or project documents, from different times and, potentially, different requirements engineers.
  3. Thematic: avoids silos, is non-partisan, reflects systematic thinking.
  4. Unique: avoids duplication to prevent double counting during scoring (duplicated rewards or penalties for the same software processing or supplier attribute).
  5. Sequenced: placed in a logical order in the document flow, preferably within requirement ‘families’ or categories for a coherent response by module specialists at suppliers.
  6. Standalone: without concomitant peers that automatically score high/low in sympathy, so each requirement ideally measures a different dimension of the candidates. This is sometimes referred to as avoiding redundancy.
  7. Measurable: articulated with a contractual commitment to delivery in mind. This is sometimes referred to as verifiable.
  8. Shrewd: avoids vague (arguably naïve) terms such as friendly, flexible, fast, instant, immediate or ‘etc’. Because these are not enforceable, they are a poor basis for an evaluation or a contractual agreement.
  9. Positive: states future need, but avoids dogmatic vocabulary to better ‘manage the talent’ and to motivate suppliers.
  10. Referenced: such as by a number to tie back to the RFI, the detailed scoring matrix and the demonstration outline – manageably fine-grained, with at least 2–4 discrete requirements on a page.

Validate, Agree and Refine Requirements

As in any requirements capture, the requirements need to be validated and negotiated. Remember yourself – and remind users – that not all requirements can be met.

In practise, validating requirements – especially with a large consultation programme – means circulating drafts of the requirements document for comment. This allows you to consult additional people who could not attend the meetings. The fact that some document commentators are reading the requirements ‘cold’ is another reason for a natural language specification.

You may encourage a team to study the requirements document (or ‘their bit’ of it) at a more social review during a meeting, with an appointed scribe to send back comments.

You might invite people to send back a document with tracked changes.

It is vitally important to organisational change management that you reflect the comments you receive in the next version [37]. Often, the consultation itself is more important to project success than the ‘concrete’ requirements derived.


Requirements are not all equal - different requirements have different value to your organisation. Weighting requirements can be considered one part of validation. Weighting means deciding which requirements are must-have and which are nice-to-have – some projects use the MoSCoW approach [38]. While such terms are common, the best approach for a selection project is mathematic, as follows.

Note that the term Weight is equivalent to Priority in the RE Glossary. However, the term weight works better when referring to the number that indicates importance. It will be used on the Weighted Attribute Matrix for scoring candidates, where the weight for importance is multiplied by the score for fit [16]. During your selection project, you should avoid the term Priority in case it suggests a premature indication of which facilities in the new software will be implemented first – when writing the requirements document, you don’t know the facilities and constraints of the successful solution because you have not chosen it yet. Therefore, you cannot predict (and should not pre-empt) the sequence of implementing features. Priority in the sense of urgency will become relevant later – when planning the implementation of the successful candidate.

The two approaches to determining how important each requirement is can be summarised as ‘expert meeting’ or ‘mechanical’.

Importance of the ‘Weight’ Attribute

The weight is arguably the most importance requirement attribute in a selection project. It has multiple roles.

Exploiting the Requirements During the Subsequent Phases of Evaluation, Selection, Negotiation and Contracting

Requirements are so important that some repetition is warranted. Figure 1 has already shown the flows of requirements. It is worth repeating the subsequent or ‘downstream’ uses of requirements from the perspective of the document that ‘receives’ the requirements. Remember that directly and indirectly, requirements will feed evaluation, selection and decision-making at every subsequent phase. This is because software products are so complex you must avoid comparing them to each other. You must compare all the candidates to the same yardstick. At every major decision-making stage gate, the basis of decision is the result of comparing candidates to your requirements – or criteria, questions or prompts derived from these requirements [16].


For many organisations and types of software, off-the-shelf is the only practical option. For a selection project, you need requirements and this means you need requirements documentation. This must be agreed and weighted internally and the same detailed document should be released to the candidate suppliers on the shortlist. The language you use is important, as you need to set a collaborative tone for the project. Your language also heavily shapes the supplier responses. The requirements, as originally recorded or as inputs to other documentation, consistently provide the yardstick against which to measure candidates during evaluation, demonstration, reference sites, negotiation and contracting.

References and Literature

Martin Tate

Martin Tate is an IT consultant who has now personally run or rescued 52 projects to select off-the-shelf solutions, interviewed over 725 people to capture the requirements for selections and appraised over 1,000 candidate solutions. A ‘poacher turned gamekeeper’, he formerly worked for an IT provider and was trained to sell software. BCS, The Chartered Institute for IT, made him a Chartered Fellow for eminence in evaluating software and recently published his book, a practitioner guide to procuring off-the-shelf IT.