Constraining Discussions in Requirements Engineering via Models

(File Last Modified Wed, May 29, 2002.)

• MBRE01 Review1
• Significance
• Originality
• Quality
• Clarity
• MBRE01 Review2
• Significance
• Originality
• Quality
• Clarity
• General comments for the authors

MBRE01 Review1

Significance

How important is the work reported? Does it attack an important/difficult problem or a peripheral/simple one? Does the approach offer an advance in the state of the art?

This is clearly significant work as it helps to solve an important issue in requirements engineering - how to handle the large number of interacting requiremtnts, and how to identify leverage points in such a set of requirements.

Originality

Has this or similar work been previously reported? Are the problems and approaches completely new? Is this a novel combination of familiar techniques? Does the paper discuss relevant research, or is it reinventing the wheel using new terminology?

This seems to be an original application of machine learning techniques from the AI field to the field of software requirements.

Quality

Is the paper technically sound? Does it carefully evaluate the strengths and limitations of its contribution? some dimensions for evaluation include generality, empirical behavior, theoretical analysis, and psychological validity.

The paper seems to be based on a sound theoretical basis. The collection of data and its analysis seems appropriate. I have just a couple of questions about the technical aspects of this work. On page 5, the term ``best Treatment'' is defined. Is it true that there will always be a best? Does it matter it there is not? On page 4, the paper seems to imply that the assumption of a ``partial ordering'' implies that there will be a best. There can clearly be a partial ordering with no best, even in a finite set.

Clarity

Does the paper describe the methods in sufficient detail for readers to replicate the work? Does it describe inputs, outputs, and the basic algorithms employed? Is the paper well organized and well written?

To a reader who is not deeply involved in this field, the connection between your examples and a set of requirements may not be entirely clear. Perhaps a statement about how these examples are analogous to software requirements would be helpful.

There were several times in explaining the experiments that you said that ``TAR2's predictions proved accurate.'' Those claims are not entirely clear. The figures were useful, although their layout was sometimes distracting to the text. Place them a little closer to their reference, and don't place two together.

What is the meaning of ``$X'' on page 5, column 1?

MBRE01 Review2

Significance

How important is the work reported? Does it attack an important/difficult problem or a peripheral/simple one? Does the approach offer an advance in the state of the art?

If I understood the point of the paper correctly, it reduces the state space of models building a qualitative model and then it use TAR2 to learn a new (constrained) model for the requirements. The approach is very interesting and the presentation is elegant and fine, but I believe that is neccesary to give more experimental results to evaluate the technique.

The case of application is very simple.

Originality

Has this or similar work been previously reported? Are the problems and approaches completely new? Is this a novel combination of familiar techniques? Does the paper discuss relevant research, or is it reinventing the wheel using new terminology?

Quality

Is the paper technically sound? Does it carefully evaluate the strengths and limitations of its contribution? Some dimensions for evaluation include generality, empirical behavior, theoretical analysis, and psychological validity.

The paper dont give any estimation of performance in the case of realistic models.

Clarity

Does the paper describe the methods in sufficient detail for readers to replicate the work? Does it describe inputs, outputs, and the basic algorithms employed? Is the paper well organized and well written?

In many parts (mainly, the TAR2 section and discussion of experiments), the paper can be hard to fellow by not expert readers. I believe that it should be better to give a intuitive example to explain the general idea of the technique.

General comments for the authors

What changes should be made?

In general, improve the description and outputs of TAR2 tool.