The paper is structured as follows: First, Chapter 2 gives an overview of the views and opinions expressed in scientific literature on the topic „requirements engineering and domain knowledge”, summarizing both the advantages and disadvantages discussed in the source material. The third chapter then proceeds to challenge these advantages and disadvantages by means of a survey that was conducted within the author’s company. The goal at this point is to compare and contrast the results gleaned from the scientific literature with the experience and opinions of requirements engineering practitioners. Chapter 4 then focuses on the author’s own experience by examining his roster of projects handled over the last couple of years and analyzing whether or not he was able to benefit from his domain knowledge. Finally, Chapter 5 draws a conclusion from the lessons learned.
2 Literature research
2.1 Research design
The search was carried out on March 7, 2019 by using both Google search and the search portal DigiBib [2] with the same key words [3]. After reviewing the results in order to find relevant articles, the sources of the articles were also examined, as well as articles citing the results [4]. In the end, eight articles were found which deal – to differing degrees – with the topic to be discussed. Since the articles displayed some redundancy in their findings, only the two most significant studies will be presented below.
2.2 Research results
The first relevant articles that was identified was written by Berry (Berry 1995). He discusses an observation from the year 1979 (first described in a paper published in 1983), in which he found out that it was precisely the lack of application domain knowledge that led to the success of the project. In his view, this can be explained by the understanding that there are unstated assumptions among domain experts that – due to being unstated - are not noticed by anyone. Moreover he postulates that no two people will have the same set of assumptions and that as a result, conflicting assumptions may be overlooked. He found out through his observation that a person without any knowledge of the respective field will ask exactly the right questions to identify these unstated assumptions. If this is done within the requirements engineering phase, there is a chance to identify inconsistencies at an early stage of the project and thus avoid the costs of a late correction. After verifying his observation multiple times over the course of several years, Berry now believes that a requirements engineering team should ideally consist of both people with domain expertise (“domain experts”) and people with no domain knowledge (“smart ignoramus”).
As Berry later discovered (Berry 2002, p. 83), this observation was already made previously in 1969 by Burkinshaw who said: “Get some intelligent ignoramus to read through your documentation and try the system; he will find many ‘holes’ where essential information has been omitted. Unfortunately, intelligent people don’t stay ignorant too long, so ignorance becomes a rather precious resource.” (Buxton and Randell 1970, p. 18)
Furthermore, Berry discovered (together with Niknafs) another advantage of involving people without domain knowledge (Niknafs and Berry 2013, p. 282): Their outside view allows for additional creative thoughts which can lead to the identification of additional useful requirements, especially for product innovations.
The second relevant paper – which due to its high relevance to the underlying topic should be given more space – was published by Kenzi et al. (Kenzi et al. 2010). It describes a study that identifies positive and negative effects of domain knowledge on elicitation techniques in requirements engineering. The study was conducted with students in a graduating class who, on their own or in groups, worked on real software development projects (including requirements engineering activities). The students were asked to describe their experiences with different elicitation techniques. In addition, they regularly had to complete questionnaires that asked for the effect of existing or non-existent domain knowledge on the individual elicitation techniques. In addition, the students were interviewed by the researchers and observations were recorded during class discussions to identify effects.
As part of the evaluation, the identified effects were then categorized and it was determined how frequently the individual categories were used. In addition, in this step a classification was made as to whether the effect was described by a student who had domain knowledge with regard to the underlying process or not.
The results obtained in the study are shown in the tables 1 to 4. Table 1 first shows the advantages of domain knowledge in the survey technique “interview”. By far the greatest advantage, with a total of 39 mentions, is the effect that domain knowledge helps the requirements engineer to ask focused questions during the interview. Looking at the two groups, it can be seen that the students without domain knowledge have described this positive effect a little less frequently than the overall second-placed effect, but the tendency is similar.
| Perceived advantage | Statements of students with domain knowledge | Statements of students without domain knowledge | Total |
|---|---|---|---|
| Helps to phrase focused questions | 31 | 8 | 39 |
| Provides a common language with the client | 5 | 10 | 15 |
| Helps to know what to focus on | 13 | 0 | 13 |
| Helps to understand the client | 6 | 2 | 8 |
| Helps to predict the client’s answers | 2 | 5 | 7 |
| Saves time learning the basics | 3 | 3 | 6 |
| Helps to cover all the relevant issues | 3 | 2 | 5 |
| Enables to provide information to other team members | 1 | 3 | 4 |
| Helps to know what to improve / preserve | 3 | 0 | 3 |
| Helps to direct the interview | 0 | 2 | 2 |
According to Kenzi et al. the “findings indicate that analysts without domain knowledge are more concerned about how to communicate with the interviewee, gain mutual understanding and direct the interview, and believe that this is where domain knowledge can help.” (Kenzi et al. 2010, p. 8) In contrast, participants with domain knowledge saw benefits in planning and conducting focused interviews, as well as actively assisting in making improvements to meet the stakeholder’s needs.
Table 2 provides an overview of the advantages that were identified regarding the elicitation technique “observation”. Only the advantage “Helps to know what to focus on” seems to play a role. It can also be observed that the majority of the assessment here comes from students without domain knowledge, whereas in observing the technique “interview” it came exclusively from students with domain knowledge.
| Perceived advantage | Statements of students with domain knowledge | Statements of students without domain knowledge | Total |
|---|---|---|---|
| Provides a common language with the client | 0 | 1 | 1 |
| Helps to know what to focus on | 2 | 14 | 16 |
| Saves time learning the basics | 1 | 0 | 1 |
This could be explained as follows (cp. Kenzi et al. 2010, p. 8): An observation conveys many details, so that existing domain knowledge could help to separate useful details from less useful details. On the other hand, for the students without domain knowledge, it seems more important in the context of the interview to find a common language with the stakeholder.
The next table shows the advantages that were identified regarding the elicitation technique “questionnaire”. There are parallels to the “interview” technique, which is presumably due to the fact that the preparation of an interview and the preparation of a questionnaire have similar steps (cp. Kenzi et al., p. 9f).
| Perceived advantage | Statements of students with domain knowledge | Statements of students without domain knowledge | Total |
|---|---|---|---|
| Helps to phrase focused questions | 4 | 1 | 5 |
| Helps to phrase understandable questions | 1 | 0 | 1 |
| Helps to know what to focus on | 1 | 0 | 1 |
| Helps to cover all the relevant issues | 2 | 0 | 2 |
Finally, disadvantages were also identified, albeit to a lesser extent than the benefits. The biggest disadvantage is the “Fixed point of view”, which was also discussed by Wiley (Wiley 1998, p. 716) as well as Niknafs and Berry (Niknafs and Berry 2013, p. 282). This might be a situation where someone considers a matter and thereby unconsciously focuses on aspects that correspond to his existing knowledge. This could lead to overlooking other facts.
As studies have shown, this effect regularly occurs in people with domain knowledge (cp. Wiley 1998, p. 716ff).
| Perceived advantage | Statements of students with domain knowledge | Statements of students without domain knowledge | Total |
|---|---|---|---|
| Fixed point of view | 5 | 2 | 7 |
| Missing information due to perceiving questions as trivial | 3 | 0 | 3 |
| Contradictions in the points of view | 3 | 0 | 3 |
2.3 Research summary
Research of relevant literature has shown that the topic has been, and still is, the subject of several (scientific) studies. The result is that domain knowledge can both benefit and hinder the requirements engineer.
The advantages and disadvantages listed by each author can all be found in the study done by Kenzi et al. (Kenzi et al. 2010). The advantages across the various investigation techniques and requirements engineering activities include the following points:
- Domain knowledge helps to phrase focused questions.
- Domain knowledge helps to phrase understandable questions.
- Domain knowledge helps to know what to focus on.
- Domain knowledge helps to cover all the relevant issues.
- Domain knowledge provides a common language with the client / stakeholder.
- Domain knowledge saves time learning the basics.
- Domain knowledge helps to understand the client / stakeholder.
- Domain knowledge helps to predict the client’s / stakeholder’s answers.
- Domain knowledge enables to provide information to other team members.
- Domain knowledge helps to know what to improve / preserve.
- Domain knowledge helps to direct an interview.
The disadvantages include in particular the following points:
- Domain knowledge leads to a fixed point of view.
- Domain knowledge leads to missing information due to perceiving questions as trivial or falling for tacit assumptions.
- Domain knowledge leads to contradictions in the points of view.
In the opinion of the author, the fact that the studies and experiments were conducted mainly with students warrants a critical examination. It is doubtful whether these students have sufficient experience with requirements engineering on the one hand and project implementations with and without domain knowledge on the other hand for the studies to yield reliable results. Therefore in Chapter 3 the next step will be to gather and contrast results from practitioners with real-world experience.
3 Survey
3.1 Survey design
The questionnaire for the practitioners was structured as follows: As an introductory question about the mindset, participants were asked whether or not they had already thought about the connection between requirements engineering and domain knowledge prior to the survey. Subsequently, in question 2, a generic query is made as to whether the participants consider domain knowledge a benefit, or rather an obstacle, to requirements engineers.
The basic premise of the survey was to compare and contrast the advantages and disadvantages identified in the previous chapter to results gathered from experienced professionals. Therefore, these advantages and disadvantages were included in the questionnaire (questions 5 to 8). In order to render the results comparable to the study by Kenzi et al. (Kenzi et al. 2010) and furthermore to be able to compare the findings per elicitation technique, the advantages were subdivided into the three elicitation techniques - “interview”, “observation” and “questionnaire” - similar to the study by Kenzi et al.
In order to record other possible advantages or disadvantages besides the aspects described in the scientific literature, corresponding free-text fields were added (questions 2 and 3). These were placed before the closed questions about the advantages/disadvantages to ensure that the participants’ free-text answers are not influenced by the closed questions. In addition to the later question about the certification levels, and unlike all other questions, answering these two questions was optional, since previous experience shows that the drop-out rate increases when free-text fields are mandatory.
In order to understand the practical experience of the participants in requirements engineering and therefore to be able to compare different levels of practice, the next set of questions (questions 9 to 11) asked about the knowledge level, the weekly percentage of requirements engineering work and any certifications. In addition the participants were asked whether or not they hold a leadership position (question 12).
Finally, question 13 enquired about the division to which the participant belongs, taking into account that there are different project cultures and process models in the individual areas (waterfall vs. scrum). It might be interesting to compare the results while differentiating between areas.
The resulting questionnaire was validated in a pre-test. For the pre-test, two different colleagues were asked to complete the survey: [5]
- A colleague with basic knowledge in requirements engineering and expert knowledge in questionnaire design. The aim here was to rule out methodological errors.
- A colleague without knowledge in requirements engineering. The aim here was to check the questions for comprehensibility and missing details.
The feedback was implemented accordingly.
The survey was conducted from March 14, 2019 to March 22, 2019. On March 14, 2019 all participants received an invitation via e-mail. On March 21, 2019 all participants who had not filled in the questionnaire until that time (or not completely) received a reminder via e-mail.
Invited to participate were all colleagues in the author's company who either completed an IREB training course held by the author [6] or subscribed to the author's requirements engineering Blog. In total 177 persons received the questionnairen [7].
3.2 Survey results
Of the 177 invited persons, 94 participated in the survey. Of these, 77 people (and thus 43.50 percent) completed the survey, while 17 people abandoned it before answering all questions. The analysis below incorporates only the results of the 77 completed questionnaires in order to prevent distortions in the interpretation of the answers.
Surprisingly, a majority of 61.04 percent had already thought about the effect of domain knowledge on requirements engineering before taking this survey (see Table 5). This can be taken as an indication that the participants have sufficient experience in requirements engineering.
| Option | Answers | |
|---|---|---|
| N | % | |
| Yes | 47 | 61.04 |
| No | 30 | 38.96 |
The prevailing opinion is that domain knowledge is a benefit rather than an obstacle for a requirements engineer: 92.21 percent of the participants find domain knowledge rather or very conducive, whereas only 7.79 percent find domain knowledge rather or very hindering (see Table 6).
| Option | Answers | |
|---|---|---|
| N | % | |
| Very hindering | 1 | 1.30 |
| Rather hindering | 5 | 6.49 |
| Rather conducive | 40 | 51.95 |
| Very conducive | 31 | 40.26 |
In order to adequately evaluate the advantages and disadvantages mentioned in the free-text fields, these were categorized by topic. In the next step, it was checked how often the points identified in the literature search were mentioned. Finally, any additional advantages or disadvantages mentioned beyond these clusters were also taken into account.
The three most common advantages that matched the literature findings are „Domain knowledge helps to cover all the relevant issues”, „Domain knowledge saves time learning the basics” and „Domain knowledge provides a common language with the client / stakeholder”. This contrasts with the Kenzi study (Kenzi et al. 2010), where these reasons played a minor role (see Tables 2 to 4) and „Domain knowledge helps to know what to focus on” was a more commonly mentioned advantage. This could be an indication – disregarding the different methodological approaches – of different priorities regarding the possible advantages of possessing domain knowledge between less professionally experienced requirements engineers and those with a higher level of experience.
| No. | Statement | Mentioned in literature | Answers |
|---|---|---|---|
| 3-1 | Domain knowledge helps to cover all the relevant issues | yes | 21 |
| 3-2 | Domain knowledge saves time learning the basics | yes | 16 |
| 3-3 | Domain knowledge provides a common language with the client / stakeholder | yes | 10 |
| 3-4 | Domain knowledge helps to understand the client / stakeholder | yes | 9 |
| 3-5 | Domain knowledge helps to phrase focused questions | yes | 6 |
| 3-6 | Domain knowledge helps to know what to improve / preserve | yes | 3 |
| 3-7 | Domain knowledge helps to know what to focus on | yes | 2 |
| 3-8 | Domain knowledge enables to provide information to other team members | yes | 2 |
| 3-9 | Domain knowledge helps to phrase understandable questions | yes | 0 |
| 3-10 | Domain knowledge helps to predict the client’s / stakeholder’s answers | yes | 0 |
| 3-11 | Domain knowledge helps to direct an interview | yes | 0 |
| 3-12 | Domain knowledge helps to consult the client / stakeholder | no | 18 |
| 3-13 | Domain knowledge leads to a higher quality of requirements | no | 10 |
| 3-14 | Domain knowledge enables to assess any impacts of the requirements | no | 9 |
In addition to the advantages already mentioned in the literature (statements 3-1 to 3-11) three other advantages were mentioned (3-12 to 3-14). The survey participants assume that a requirements engineer with domain knowledge can advise a stakeholder and challenge their requirements with a higher level of quality. In addition, participants believe that a requirements engineer, based on their domain knowledge, can better assess any impacts or risks associated with the implementation of the requirements. Lastly, they also believe domain knowledge generates a higher quality of requirements (especially with regard to precise phrasing).
The listed disadvantages matched the results of the Kenzi study more closely. In both studies, for example, the point “Domain knowledge leads to a fixed point of view” is the most commonly mentioned disadvantage. However, the new study also uncovers disadvantages that have not been mentioned before in the literature, like the two options “Domain knowledge carries the risk of influencing requirements” and “Domain knowledge runs the risk of anticipating solutions” (statements 4-4 and 4-5). Statements that were counted into the first point were for example: “There is a risk that the requirements engineer will formulate and shape more suggestive questions instead of keeping neutrality to the requirements and only identifying the needs of the customer” and “The requirements engineer may no longer be in his role of accepting the requirements in a neutral and value-free manner”.
| No. | Statement | Mentioned in literature | Answers |
|---|---|---|---|
| 4-1 | Domain knowledge leads to a fixed point of view | yes | 20 |
| 4-2 | Domain knowledge leads to missing information due to perceiving questions as trivial or falling for tacit assumptions | yes | 15 |
| 4-3 | Domain knowledge leads to contradictions in the points of view | yes | 2 |
| 4-4 | Domain knowledge carries the risk of influencing requirements | no | 18 |
In order to be able to see and compare questions 5 to 8 directly from the point of view of individual groups of participants, the results of questions 9 to 13 are presented first. As shown in Table 9, there was no participant who rated his requirements engineering knowledge as „very low“. This was not surprising since – as described in Section 3.1 – the participants either completed a requirements engineering training or at least subscribed to a requirements engineering blog. The majority described their requirements engineering knowledge as “rather high”, and only one participant stated that he had a very high level of knowledge in requirements engineering (although two participants are certified in the Advanced Level, as Table 11 shows).
| Option | Answers | |
|---|---|---|
| N | % | |
| Very low | 0 | 0.00 |
| Rather low | 30 | 38.96 |
| Rather high | 46 | 59.74 |
| Very high | 1 | 1.30 |
The percentage of requirements engineering activities in their daily work is rather low with an average of 27.92 percent and a median of 20 percent, when assuming that this survey is aimed at requirements engineering professionals. However, in this company it is rare for any role to exclusively cover requirements engineering. Instead, project managers and product developers usually handle requirements engineering tasks in projects in addition to their other tasks.
| Option | Answers | |
|---|---|---|
| N | % | |
| 0 % | 4 | 5.19 |
| 10 % | 20 | 25.97 |
| 20 % | 18 | 23.38 |
| 30 % | 13 | 16.88 |
| 40 % | 5 | 6.49 |
| 50 % | 9 | 11.69 |
| 60 % | 3 | 3.90 |
| 70 % | 3 | 3.90 |
| 80 % | 2 | 2.60 |
| 90 % | 0 | 0.00 |
| 100 % | 0 | 0.00 |
As expected, due to the choice of participants, most of the participants have a requirements engineering certification (see Table 11). As determined through a survey software evaluation, all participants with the RE@Agile Primer also have the Foundation Level (the participants with the Advanced Level have always completed the Foundation level by definition). As a result, 74.03 percent of participants have at least one certification and 24.97 percent have no certification.
| Option | Answers | |
|---|---|---|
| N | % (of the 77 participants) | |
| IREB Foundation Level | 57 | 74.03 |
| IREB RE@Agile Primer | 15 | 19.48 |
| IREB Advanced Level: Elicitation | 0 | 0.00 |
| IREB Advanced Level: Requirements Modeling | 1 | 1.30 |
| IREB Advanced Level: Requirements Management | 1 | 1.30 |
| IREB Advanced Level: RE@Agile | 0 | 0.00 |
| Others | 0 | 0.00 |
14.29 percent of the participants are executives; the majority - 85.71 percent - of the participants do not hold a leadership role (see Table 12). Again this is not surprising, as requirements engineering is a very operational activity (although good requirements engineering obviously requires a smart strategy).
| Option | Answers | |
|---|---|---|
| N | % | |
| Yes | 11 | 14.29 |
| No | 66 | 85.71 |
As shown in Table 13, the majority of the participants are in product segment “A” (70.13 percent). Only 9.09 percent work in product segment “B” and 15.58 percent of the participants in product segment “C”. 5.19 percent are employed in the parent service company („cross”).
| Option | Answers | |
|---|---|---|
| N | % | |
| A | 54 | 70.13 |
| B | 7 | 9.09 |
| C | 12 | 15.58 |
| cross | 4 | 5.19 |
To be able to compare different groups of participants, at least 30 participants per group are required. Otherwise, no statistical significance can be calculated. Thus, the results of the following questions cannot be used:
- Certifications: There are only 20 participants without a certification.
- Executives: There are only 11 executives.
- Product segment: Only 7 participants belong to product segment „B” and only 12 participants belong to product segment „C”.
Furthermore, the analysis has shown that there are no differences in working hours when dividing the participants into two groups (a group that has less than the average 27.92 percent work share and a group that has more than the average 27.92 percent work share). Statistical significance can be identified only with regard to the domain knowledge questions as shown below.
| No. | Statement | I totally don’t agree | I rather don’t agree | I rather agree | I totally agree | ||||
|---|---|---|---|---|---|---|---|---|---|
| N | % | N | % | N | % | N | % | ||
| 51 | Domain knowledge leads to a fixed point of view | 4 | 5.19 | 17 | 22.08 | 41 | 53.25 | 15 | 19.48 |
| 52 | Domain knowledge leads to missing information due to perceiving questions as trivial or falling for tacit assumptions | 3 | 3.90 | 17 | 22.08 | 36 | 46.75 | 21 | 27.27 |
| 53 | Domain knowledge leads to contradictions in the points of view | 4 | 5.19 | 11 | 14.29 | 41 | 53.25 | 21 | 27.27 |
| 54 | Whether domain knowledge is more conducive or rather hindering depends on the domain (for example, it could be conducive to a more formal domain such as taxation and hindering in a less formal domain such as product development) | 14 | 18.18 | 24 | 31.17 | 27 | 35.06 | 12 | 15.58 |
Table 14 shows the disadvantages identified in the literature (statements 5-1 to 5-3). About half of the participants opted for “I rather agree”. Together with the results of “I totally agree” (19.48 to 27.27 percent), a majority of the practitioners confirmed that these disadvantages exist, but, unlike the Kenzi study (Kenzi et al. 2010, p. 9) and the results of the free-text questions presented above (see Table 8), statement 5-1 is not ranked first if you add up the values of “I rather agree” and “I totally agree”.
A fourth statement was added to the statements found in the literature to reflect the fact that, in the literature, different requirements engineering activities and different requirements engineering techniques were investigated with regard to possible effects of domain knowledge. However, it was not determined whether the impact depends on the respective domain.
In contrast to the prior statements the answers for this section are somewhat more evenly distributed with a focus on the two middle options, and show no clear consensus. A hypothesis at this point would be that the participants answered from the point of view of their respective domain (and since the domains are heterogeneous, the answers are, too).
| No. | Statement | I totally don’t agree | I rather don’t agree | I rather agree | I totally agree | ||||
|---|---|---|---|---|---|---|---|---|---|
| N | % | N | % | N | % | N | % | ||
| 6-1 | Domain knowledge helps to phrase focused questions | 0 | 0.00 | 5 | 6.49 | 44 | 57.14 | 28 | 36.36 |
| 6-2 | Domain knowledge helps to phrase understandable questions | 1 | 1.30 | 14 | 18.18 | 40 | 51.95 | 22 | 28.57 |
| 6-3 | Domain knowledge helps to know what to focus on | 2 | 2.60 | 17 | 22.08 | 33 | 42.86 | 25 | 32.47 |
| 6-4 | Domain knowledge helps to cover all the relevant issues | 3 | 3.90 | 23 | 29.87 | 32 | 41.56 | 19 | 24.68 |
Table 15 now shows the alleged advantages of domain knowledge in the elicitation technique „questionnaire”. The majority has marked „I rather agree“. It turns out that the statement „Domain knowledge helps to phrase focused questions”, with 57.14 percent at „I rather agree“ and 36.36 percent at „I totally agree“, is perceived as the biggest advantage. This is in line with the results of the Kenzi study (Kenzi et al. 2010, p. 8), where this statement was also seen as the greatest benefit for the questionnaire elicitation technique.
| Option regarding requirements engineering knowledge | I totally don’t agree | I rather don’t agree | I rather agree | I totally agree | Total | ||||
|---|---|---|---|---|---|---|---|---|---|
| N | % | N | % | N | % | N | % | ||
| Very low | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 |
| Rather low | 0 | 0.00 | 6 | 20.00 | 12 | 40.00 | 12 | 40.00 | 30 |
| Rather high | 3 | 6.52 | 17 | 36.96 | 19 | 41.30 | 7 | 15.22 | 46 |
| Very high | 0 | 0.00 | 0 | 0.00 | 1 | 100.00 | 0 | 0.00 | 1 |
Furthermore, when evaluating statement 6-4 based on requirements engineering knowledge, there is a significant difference for “I totally agree” (see Table 16) showing that requirements engineers who have a less comprehensive methodological knowledge (regarding questionnaires) may feel safer with domain knowledge to ensure they cover all relevant issues. Conversely, experienced requirements engineers rely on their methodological knowledge and therefore appreciate the value of domain knowledge in this context less.
Table 17 shows the results regarding the advantages for the elicitation technique “observation”. In the study by Kenzi, the advantage “Helps to know what to focus on” was mentioned most frequently by far (Kenzi et al. 2010, p. 8). This is not confirmed by the survey results, since “Domain knowledge saves time learning the basics” receives a much higher degree of approval.
| No. | Statement | I totally don’t agree | I rather don’t agree | I rather agree | I totally agree | ||||
|---|---|---|---|---|---|---|---|---|---|
| N | % | N | % | N | % | N | % | ||
| 7-1 | Domain knowledge provides a common language with the client / stakeholder | 1 | 1.30 | 13 | 16.88 | 35 | 45.45 | 28 | 36.36 |
| 7-2 | Domain knowledge helps to know what to focus on | 1 | 1.30 | 22 | 28.57 | 26 | 46.75 | 18 | 23.38 |
| 7-3 | Domain knowledge saves time learning the basics | 0 | 1.30 | 4 | 5.19 | 34 | 44.16 | 28 | 49.35 |
Finally, Table 18 presents the results for the elicitation technique “interview”. Here too, the practitioners mostly agree with the advantages, except for the statement “Domain knowledge helps to predict the client’s / stakeholder’s answers”, for which the majority does not agree. This is in line with the Kenzi study (Kenzi et al. 2010, p. 7) and the analysis of free-text fields (see Table 7).
| No. | Statement | I totally don’t agree | I rather don’t agree | I rather agree | I totally agree | ||||
|---|---|---|---|---|---|---|---|---|---|
| N | % | N | % | N | % | N | % | ||
| 8-1 | Domain knowledge helps to phrase focused questions | 0 | 0.00 | 10 | 12.99 | 40 | 51.95 | 27 | 35.06 |
| 8-2 | Domain knowledge provides a common language with the client / stakeholder | 0 | 0.00 | 1 | 14.29 | 40 | 51.95 | 26 | 33.77 |
| 8-3 | Domain knowledge helps to know what to focus on | 0 | 0.00 | 17 | 22.08 | 42 | 54.55 | 18 | 23.38 |
| 8-4 | Domain knowledge helps to understand the client / stakeholder | 3 | 3.90 | 7 | 9.09 | 34 | 44.16 | 33 | 42.86 |
| 8-5 | Domain knowledge helps to predict the client’s / stakeholder’s answers | 8 | 10.39 | 34 | 44.16 | 25 | 32.47 | 10 | 12.99 |
| 8-6 | Domain knowledge saves time learning the basics | 0 | 0.00 | 6 | 7.79 | 36 | 46.75 | 35 | 45.45 |
| 8-7 | Domain knowledge helps to cover all the relevant issues | 4 | 5.19 | 2 | 28.57 | 41 | 53.25 | 10 | 12.99 |
| 8-8 | Domain knowledge enables to provide information to other team members | 8 | 10.39 | 2 | 28.57 | 32 | 41.56 | 15 | 19.48 |
| 8-9 | Domain knowledge helps to know what to improve / preserve | 2 | 2.60 | 11 | 14.29 | 44 | 57.14 | 20 | 25.97 |
| 8-10 | Domain knowledge helps to direct the interview | 10 | 12.99 | 19 | 24.68 | 31 | 40.26 | 17 | 22.08 |
Another significant difference is found by evaluating the statements according to requirements engineering knowledge: For statement 8-3, significantly more participants with less requirements engineering experience fully agree with the statement “Domain knowledge helps to know what to focus on” than do participants with more requirements engineering experience. Again, they may assume that domain knowledge can compensate for a lack of methodological experience.
| Option regarding requirements engineering knowledge | I totally don’t agree | I rather don’t agree | I rather agree | I totally agree | Total | ||||
|---|---|---|---|---|---|---|---|---|---|
| N | % | N | % | N | % | N | % | ||
| Very low | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 |
| Rather low | 0 | 0.00 | 6 | 20.00 | 13 | 43.33 | 11 | 36.67 | 30 |
| Rather high | 0 | 0.00 | 11 | 23.91 | 28 | 60.87 | 7 | 15.22 | 46 |
| Very high | 0 | 0.00 | 0 | 0.00 | 1 | 100.00 | 0 | 0.00 | 1 |
3.3 Survey summary
Of course, the results of this survey have to be evaluated thoroughly, as the participants answered from their subjective perception. Conclusions should be drawn with care. Consequently, the statements in chapter 3.2, as well as the statements in this summary, cannot be generalized, but rather provide a snapshot.
The survey initially concluded that from the practitioner's point of view the advantages and disadvantages described in the literature are valid, generally speaking, but that there are additional advantages and disadvantages: [8]
- Domain knowledge helps to consult the client / stakeholder.
- Domain knowledge leads to a higher quality of requirements.
- Domain knowledge enables assessment of any impacts of the requirements.
- Domain knowledge carries the risk of influencing requirements.
- Domain knowledge runs the risk of anticipating solutions.
Further, the evidence indicated that requirements engineers rely on being able to compensate for lack of method knowledge with domain knowledge. These findings will be considered in the next chapter which deals with a project analysis.
4 Project analysis
4.1 Project analysis design
In addition to the literature research and the survey among practitioners, particular project experiences will be discussed in this chapter. Likewise, and even more so than in previous chapters, the project analysis is subjective. It is therefore only a personal experience report by the author with the aim of presenting his own view of the facts.
The projects chosen are tax-related, particularly dealing with changes in Value Added Tax (VAT). Over the course of time the author has acquired significant domain knowledge in this context. Passively, by receiving specialist input from the stakeholders, as well as actively through researching the subject matter since – as mentioned in Chapter 1 – the requirements engineer is tasked with familiarizing himself with the application domain of the stakeholders. Since tax projects usually contain legal requirements, domain knowledge was acquired by analyzing relevant legal texts and comments published online (e.g. by tax consultants).
Starting out with no specific knowledge of VAT at all, the author acquired an increasing level of domain knowledge over the course of each project, thus building up expertise in the subject matter. Therefore this category of projects offers an excellent basis for analyzing the impact of domain knowledge on the project outcome.
4.2 Project analysis
Project 1 dealt with the implementation of a so-called tax engine, a tool to facilitate the VAT calculation for the US market. In the US market there is no uniform tax rate. Instead, the tax rate depends on the respective state (sometimes also on the county) and the product type. For online purchases the location of the buyer is also significant. In order to choose the correct tax rate out of this myriad of possible options, an external tool was purchased (the “tax engine”). The project goal was to integrate this tax engine into the company’s technological infrastructure.
In this project the author set up workshops to gather and define the requirements. These workshops were attended by representatives of the departments involved (Taxes, Accounting, Accounts Receivables, Customer Care etc.) as well as external consultants. In addition to the variety of tax rates described above, the author was able to acquire a wide range of domain knowledge over the course of the project. One example is the fact (unlike Germany) that non-profit organizations in the USA can be exempted from VAT, which led to the design of a corresponding process with the necessary detailed requirements.
In retrospect it is the author's assessment that pre-existing domain knowledge would have saved a lot of time while preparing the workshops. In addition, issues such as tax exemption could have been addressed more specifically and analyzed in more detail. Without pre-existing domain knowledge, the issue of tax exemption was discussed as a side note initially and then had to be put on the agenda for an additional workshop. Further effects were not noted. On the other hand, this means that a requirements engineer with sufficient methodological knowledge could have fully elicited the requirements even without any domain knowledge, just with a higher expenditure of time.
The second project selected for analysis was the implementation of a legal amendment in Spain. Until this change, VAT reporting in Spain was carried out on a quarterly and consolidated basis. This means that only the sum total of VAT issued and the sum total of VAT received were reported to the Spanish tax authorities. The legal amendment obliged companies with an annual turnover of EUR 6 million or more to report VAT on a daily basis and on an invoice level. Each individual incoming invoice and each individual outgoing invoice now needs to be reported electronically to the Spanish tax authorities. This mandatory reporting must include information about the business partner, their tax number, the net invoice amount, the amount of VAT, etc.
While the focus of this project was different to the project scope described above, the domain knowledge acquired in the previous project was nevertheless helpful, since the author was already familiar with many technical terms (e.g. reverse charge or nexus). In this case the domain knowledge led to time saved learning the basics, as well as allowing more specific questions to be asked to elicit requirements. For example, the author asked the question whether there were circumstances in Spain similar to the tax exemptions in the US market. It turned out that this is indeed the case (business partners on the Canary Islands do not have to be considered, since the Canary Islands represent a special tax region in Spain). The specific VAT domain knowledge was therefore beneficial for the author in his role as requirements engineer.
The next projects were dealing with the same issue: The author's company offers its digital products and services worldwide, although not every country has a physical business presence. There are, for example, no branch offices in Switzerland, Turkey, Russia and Canada. That means customers in these countries are in fact buying their products and services from abroad. Previously, there was no legal requirement to add VAT for such products or services, which is why net invoices were issued. However, the affected countries have now made the necessary legal changes to ensure that future invoices will have to include VAT for services bought online from abroad. The project goals were therefore to implement the calculation, collection and payment of VAT in these countries.
Switzerland was considered first. The tax department wrote an initial draft of the relevant requirements and provided this business concept to the requirements engineer as a basis for further analysis and elicitation.
After reviewing the initial requirements, the author did extensive research on the Swiss Value Added Tax Act. He noticed that the Swiss VAT applies not only in Switzerland, but also in the neighboring country of Liechtenstein and in the German municipality of Büsingen, which is completely surrounded by Swiss territory. Taking this into account, the core requirement was no longer: “The system must take into account a VAT rate of 7.7 percent for customers seated in Switzerland as of 01.01.2018", but “The system must take into account a VAT rate of 7.7 percent for customers seated in Switzerland, Liechtenstein or Büsingen as of 01.01.2018”. Büsingen was of particular importance at this point, since the European billing system had previously only been able to reflect different VAT rates per country, not on the level of individual municipalities. This required a modification of the logic within the existing billing system causing a significant increase in the time and effort required to now ensure that in Büsingen the Swiss VAT is calculated at 7.7 percent and not at the German VAT rate of 19 percent. If the author had relied on the requirements provided by the specialist department without acquiring additional domain knowledge of his own, the incomplete requirement would only have become apparent at a much later point in time (for example during a tax audit, with all the potential consequences). Therefore in this case domain knowledge helped cover all relevant use cases.
The next project dealt with the introduction of VAT for Turkey. This time the determination, examination and coordination of the requirements was completed more quickly than for Switzerland since the Swiss requirements could simply be reused with a different tax rate. In addition, there was the special feature that only private customers would have to receive VAT invoices in future. Corporate customers will continue to receive net invoices. This caused additional efforts since the billing system had to be modified to separate between private and corporate customers.
For Russia, as the third affected country, both broad domain knowledge and a requirements basis with a high degree of maturity were already available. Although the author initially only received a very superficial requirements document from the business side, he was able to work out all the necessary details for implementation by asking specific questions, such as, for example:
- Do only private customers have to be considered or all customers?
- Are there any special regions to be considered?
- Are there any tax exemptions?
- Must the VAT declaration be made in the currency of the invoice (US dollar) or in the national currency (Russian ruble)?
- Do “zero declarations” have to be made (if there were no sales in the reporting period)?
- How to deal with invoices for which the performance period starts before the VAT introduction reference date but ends after the reference date?
- What is the deadline for submitting VAT returns?
The ability to ask these questions reflects the domain knowledge that the requirements engineer had acquired over the course of the preceding steps. Naturally, these questions were then also applied for the implementation in Canada. Thus, the author was able to determine at an early stage that tax exemptions exist in Canada as well (for companies only dealing with B2B). Defining and implementing this process and its requirements for the Canadian market was associated with high costs: First, existing customers had to be asked whether they were tax-exempt. If so, they needed to provide their tax ID. This tax ID had to be stored in the backend systems and printed on every future customer invoice. At the same time, a VAT rate of 0 percent had to be calculated for these customers. Secondly, the sales frontend needed to be modified to allow all new customers the option to list tax exemption and their tax ID during the order process. As a result, domain knowledge in this case not only helped again to ask focused questions but also to avoid a dramatic rise in costs later on in the project.
Only one notable bug occurred during the VAT implementations in Switzerland, Turkey, Russia and Canada that was due to inadequate requirements. In Russia, the tax department decided during the design phase that 18 percent VAT had to be taken into account. In the period following the design phase, however, Russia raised their VAT rate to 20 percent. This VAT change was initially overlooked, so that invoices sent after the deadline included an incorrect VAT rate. Due to the technical and organizational complexity in the affected company, a total of 13 teams had to be involved to correct this bug – even if it seemed insignificant at first. Due to the complexity, a follow-up project had to be initiated to implement the correct tax rate and correct the faulty invoices. This shows that even broad domain knowledge cannot fully prevent serious errors or omissions in the requirements.
The effect of domain knowledge is also illustrated in the following table. This table shows how often the requirements changed on average during the design phase vs. later on in the project. These figures are available for these four countries, since the requirements engineering software “Polarion” was used to document the requirements in these projects, whereas for the USA and Spain the documentation was only available in Microsoft Word. Due to the database storage of requirements (and all requirements versions), it is now possible to automatically determine rates of change in the requirements.
| Country | Rate of change during design phase | Rate of change after design phase |
|---|---|---|
| Switzerland | 7.9 | 4.5 |
| Turkey | 5.1 | 0.7 |
| Russia | 4.7 | 0.0 |
| Canada | 5.4 | 0.0 |
The figures show that the introduction of VAT in Switzerland underwent many changes both during the design phase as well as after completion of the design phase. In the following countries the change rate stabilized at around 5 changes per requirement during the design phase, with only a few, if any, adjustments necessary later on.
4.3 Project analysis summary
In the given projects, domain knowledge helped the author to save time by identifying all relevant circumstances early on. Domain knowledge also helped to ensure completeness and correctness, reducing the need for changes after the design phase had been completed. It is interesting to note that domain knowledge in these cases enabled the author to identify exceptions to the main scenario that were overlooked by the stakeholders. However, these exceptions are often the factors that add significant cost and complexity to an implementation project, especially if they are identified too late.
In the context of these projects, the author found no indication that domain knowledge hindered him in any way. However, this does not mean that it always has to be like this, because no generalization can follow from an example.
5 Conclusion and recommendation
This essay examined the question whether domain knowledge hinders a requirements engineer or not. For this purpose, three different perspectives were chosen (literature research, survey, project experience). In summary, it can be said that domain knowledge provides the requirements engineer with many advantages, but can also cause certain disadvantages.
One considerable disadvantage of acquiring domain knowledge can be the development of a fixed point of view. The discrepancy between the scientific literature and survey results and the author’s practical experience shows that the influence of domain knowledge on the requirements engineer also depends very much on the methodological expertise. This is confirmed both by scientific literature and by the survey results (see in particular statement 4-1 in Table 8). This insight is especially important because there is currently a discussion about establishing the role of a digital designer, who would then also perform requirements engineering tasks (cf. Schulz 2018). In this role a fixed point of view or “silo mentality” would lead to potentially disastrous consequences.
Furthermore, another possible disadvantage is that domain knowledge may cause a requirements engineer to overlook tacit assumptions. This view is supported by Berry in particular (cf. e.g. Berry 2002) and was confirmed by the practitioners. Here, too, it is interesting that the point does not coincide with the author's experience. Perhaps this is due to the specific domains primarily dealt with by the author or perhaps due to the methodological approach. It would therefore be interesting to investigate this point in more detail in future analysis.
The influence of domain knowledge on the requirements engineer may be very dependent on their methodological expertise: this could explain the discrepancy between the scientific literature and survey results and the author’s practical experience. Thus, it could be concluded that the benefits of domain knowledge turn out to be less significant if the requirements engineer is more experienced (see tables 16 and 19). This corresponds to the author’s experience (the analysis shows that even without specific domain knowledge projects were still completed with a high level of quality).
In summary, it can be concluded that there is no universal ideal approach. If you really want to be sure that you can reap all the benefits, then you have to choose the approach suggested by Berry and use both a requirements engineer with, as well as one without, domain knowledge (“domain expert” and “smart ignoramus”). This would be the ideal solution; however, the personnel situation will rarely allow this approach. Therefore each case will require analysis to determine whether it would be most beneficial to use a requirements engineer with or without domain knowledge.
Table 21 (see below) provides a tool for this analysis with a recommendation for different situations based on the author’s experience and the findings of this essay. A “+” means that in this case the approach is recommended. Conversely, a "-" means that it is not recommended. A “0” means that the approach would offer neither benefits nor disadvantages and can therefore be applied (and thus other potential constraints need to be factored in to reach a decision).
Ultimately, however, in a real-life situation a project can only be staffed completely or partially with requirements engineers without domain knowledge if these are available. It is not possible or desirable to prevent requirements engineers from acquiring a wide range of domain knowledge over the course of their professional lives. Thus, a recommendation is also needed in case no requirements engineers without domain knowledge are available. Based on the author's personal experience, the benefits of the smart ignoramus can also be accomplished by applying methodological knowledge. Consequently, if it’s not feasible to find a requirements engineer without domain knowledge, an available requirements engineer with strong methodological knowledge can provide the same benefits or avoid the pitfalls (only needing more time). This can be ensured in the long term by making sure that requirements engineers receive solid basic (and ongoing) training.
In conclusion, it can be said that the findings of this study are of course "ceteris paribus". There are many other aspects that influence requirements engineering (knowledge and skills to apply methods, techniques and tools of requirements engineering, soft skills, quality of project management, management support etc.).
It has also been shown that there are a number of other interesting starting points for future research. These include, for example, the questions of who benefits from domain knowledge under which circumstances, how to use it intelligently, and what potential risks exist when relying on domain knowledge as a requirements engineer.
| Criteria / option | RE with knowledge | RE without knowledge | Explanation |
|---|---|---|---|
| Project with time criticality | + | - | In time-critical projects, domain knowledge helps to implement the project more quickly and thus to maintain the chance to meet the target date. Further, domain knowledge will help to know what to focus on. Without domain knowledge, time must first be invested in familiarization. |
| Project without time criticality | 0 | + | If the project is not time-critical, then the advantage should be used that a requirements engineer without domain knowledge will question many facts. |
| Project with creative scope | - | + | A project with a creative scope can benefit from the open-mindedness of a smart ignoramus. |
| Project with formal scope | + | 0 | A project with e.g. a legal or compliance scope that involves implementing guidelines and specifications is more likely to benefit from domain experts who can discuss the requirements on a par with stakeholders. |
| Project with high communication needs | + | - | A project with strong communication needs can benefit from a requirements engineer with domain knowledge, as these are communication brokers. |
| Project with low communication needs | 0 | 0 | In this case there are no advantages or disadvantages to either approach. |
| Project with a high need for unbiased thinking | - | + | Requirements engineers with domain knowledge tend to influence requirements and anticipate solutions. |
| Project with no high need for unbiased thinking | 0 | 0 | In this case there are no advantages or disadvantages to either approach. |
| Stakeholders with a low level of expertise | + | - | As the analysis of the author’s personal projects has shown, a business experienced requirements engineer can help to compensate for a lack of expertise among stakeholders. |
| Stakeholders with a high level of expertise | - | + | Stakeholders with a high level of expertise require less support from the requirements engineer. However, they tend to make tacit assumptions. Therefore requirements engineers without domain knowledge are recommended. |
| Stakeholders with an affinity for tacit or conflicting assumptions | - | + | If the stakeholders involved have an affinity for tacit or conflicting assumptions, then a requirements engineer without domain knowledge is more suitable. |
| Stakeholders without an affinity for tacit or conflicting assumptions | 0 | 0 | In this case there are no advantages or disadvantages to either approach. |
| Requirements engineer has high methodological skills | + | 0 | A requirements engineer with high methodical knowledge can provide the same benefits as a requirements engineer without domain knowledge. |
| Requirements engineer has low methodological skills | 0 | 0 | In this case there are no advantages or disadvantages to either approach. |
6 References
Berry, Daniel M. (1995): The importance of ignorance in Requirements Engineering, in: Journal of Systems and Software, volume 28, no. 1, p. 179-184.
Berry, Daniel M. (2002): The importance of ignorance in Requirements Engineering: An earlier Sighting and a Revisitation, in: Journal of Systems and Software, volume 60, no. 1, p. 83-85.
Buxton, John N. / Randell, Brian (1970): Software Engineering Techniques: Report of a conference sponsored by the NATO Science Committee, Rome, Italy, October 27-31, 1969, homepages.cs.ncl.ac.uk/brian.randell/NATO/nato1969.PDF. [07/03/2019]
Kenzi, Keren / Soffer, Pnina / Hadar, Irit (2010): The role of domain knowledge in requirements elicitation: an exploratory study, in: In: Proceedings of the Fifth Mediterranean Conference on Information Systems (MCIS), Tel Aviv, Israel, September, 12-14, 2010, paper 48.
Niknafs, Ali / Berry, Daniel M. (2013): An industrial case study of the impact of domain ignorance on the effectiveness of requirements idea generation during requirements elicitation, in: Proceedings of the 21st IEEE International Requirements Engineering Conference (RE), p. 279-283.
Pohl, Klaus / Rupp, Chris (2015): Requirements Engineering Fundamentals – A Study Guide for the Certified Professional for Requirements Engineering Exam Foundation Level / IREB compliant, 2nd edition, Santa Barbara: Rocky Nook.
Rupp, Chris (2014): Requirements-Engineering und -Management – Aus der Praxis von klassisch bis agil, 6th, updated and expanded edition, München: Hanser.
Schulz, Christopher (2018): Wir brauchen eine Gestaltungsprofession für die Digitalisierung – Dr. Kim Lauenroth im Interview, [online] https://www.mosaiic.com/interview_dr_kim_lauenroth/ [28/03/2019]
Wiley, Jennifer (1998): Expertise as mental set: The effects of domain knowledge in creative problem solving, in: Memory & Cognition, no. 26(4), p. 716-730.
Footnotes
- [1] International Requirements Engineering Board
- [2] The DigiBib includes library catalogs and literature databases from all over the world, such as Elsevier, Springer and Wiley Interscience.
- [3] The following key words were used: „requirements engineering domain knowledge“, „anforderungsmanagement domänenwissen“, „domain knowledge useful for requirements engineering“, „hilft domänenwissen im anforderungsmanagement“.
- [4] The citing essays were found with the help of Google Scholar.
- [5] Unintentionally, Berry's approach has been used (a team consisting of one domain expert and smart ignoramus, see Berry 1995).
- [6] Foundation Level and / or RE@Agile Primer.
- [7] The questionnaire was created in the tool SurveyMonkey.
- [8] It may also be that the literature research was not thorough enough to discover all aspects.
Till-J. Faßold is Expert Strategic Planning at 1&1, Germany. In this role – besides other responsibilities – he coaches professionals from various departments and is responsible for the requirements engineering processes. Till-J. Faßold hold a master’s degree in information management from the University of Koblenz and is certified to the IREB CPRE Expert Level.