Agents in a Wild World

(File Last Modified Wed, May 29, 2002.)

• Reviews from the editors
• Review 1
• Review 2
• Review 3
• Note:truth maintenance system (TMS)

Reviews from the editors

Review 1

• A good paper. However, the abstract itself wouldn't entice me to read the paper or know what the paper was even about - this should be re-written.
• Apart from that the paper reads well - my major criticism is that the authors do not reference any mainstream agent papers. For this to work as a chapter in an agent book, the authors need to make more of an attempt to say how the work compares or builds on existing work in the agent field - I have tried reading the conclusions several times but don't feel that (same as the abstract) the authors do themselves justice. As someone who uses new theoretical results to build computational specifications of practical implementations I want to know how these results impact on the design of agents in general.
• More specifically, there should be no references to work in preparation or just submitted - if there's a reference to some work then (at the very least) there should be a web address where it can be attained.

Review 2

• I would say that this paper is written for a number of experts, many technicalities are not explained. The Example of Figure 1.2 helps, but it is not consistently continued. For instance, how do the types op valuables of Fig 1.1 re-occur in this example?
• From section 1.3 on, there is a lot of analysis in terms of probabilities. I did not see how these enter the picture, and how they are related to the example.
• What kind of theories you are allowing? Can nogoods appear only in assumptions? How general are these theories? Or would we also allow the theory: q if c, r if d, and q and r being contradictory? I guess it would not fit in the definition of funnels and the theory about them.
• Altough the claims 1 and 2 are clearly stated, they seem to be too informally formulated to prove them.
• The many figures and tables in the paper were not understandeble to me, and I would suggest that the authors add more explanation between the various equations.

Review 3

Content: The authors describe a model of a situated agent in which some variables are subject to environmental and random forces, outside the control of the agent. They make two general assertions which they then claim to justify in the paper:

• That, on average, it is possible for an agent to learn effective and cheap actions, despite the so-called wild variables
• That it is possible to redesign particular devices in order to increase their immunity from wild variables.

My first comment is that this appears to be a paper about Truth Maintenance Systems (TMS) in engineering control theory, not about the design of agents or of multi-agent systems. The authors do, it is true, use the word ``agent'' several times (especially on the first two pages), but they do not show how the paper's content relates to the design of agents. It is possible that the work here DOES bear on the problems of agent design, and that the authors have an entirely new approach to this. If so, they should demonstrate this, and should do so with reference to the literature on agent design. For example, the paper cites 25 publications, none from the literature on agent design or MAS.

As I note above, the authors make two assertions and then claim to demonstrate them in the paper. I am not convinced by their demonstrations. Despite the generality of their claims, their justifications are particular, hedged with assumptions or based on limited empirical studies. I believe the two claims would be more convincing if they were not expressed so generally.

At several places, Claim 1 is interpretated as being equivalent to some other statement (e.g. at lines 155 & 213). The inference steps which link the main claim to these other statements are not given formally or completely, and I am not convinced of these purported equivalences. It also does not help the authors' case that these subsidiary claims are buried deep in the text. I think part of the problem here is that the authors are not formal enough, so that implicit assumptions are not noticed. If they were to represent their claims in more formal language the hidden assumptions they are making would become evident. The authors could, I think, also achieve this if they were to first state the specific (not general) claims which their arguments do support and work upwards from there, rather than downwards (from general to specific) as they have done.

About samples, ``sample'', by definition is a proper subset of some population; if the sample is equal to the entire population it is no longer a sample. Once the ``sample'' is the entire population, the statistical theory of sampling usually changes qualitatively. (Because statistical sampling theory is based on asympotic arguments, this qualitative difference typically arises when some parameter actually achieves its asymptotic value, e.g. log(x) as x --> 0 from above. Observe that log(x) is qualitatively different to log(0), even when x is extremely small.)

I think part of the problem here is that it is never made clear what precisely is the population being sampled in the Random Worlds Search on page 6: Is it a population of paths; of arcs within paths; of end-states of paths; or of possible worlds; or of something else? The meaning seems to vary through the paper. Perhaps this is obvious to people in the TMS or control theory communities; I don't believe it is obvious to people in the MAS community.

The mathematical notation needs clarification in places. For instance, the authors use the symbols ``c'' and ``d'' to denote probabilities of paths on page 8. (In fact, these are conditional probabiliites). Later, on page 9, they refer to pathways with these symbols. The latter references should be to ``the pathway whose probability is c'', not ``the pathway c'', etc. But I'm not sure if this is what the authors mean. The use of ``andp'', ``in'', etc (pages 10-11) as symbols for integer values I find clumsy. In any case, if the notation is difficult to read, it is hard to determine what the authors are saying.

The graph on page 5 appears to be a Bayesnet, and the authors, by assigning probabilities to paths, treat it as one. It would be helpful to have a statment saying it is (or isn't) a Bayesnet, and why the authors have (or haven't) drawn on the work in AI for its analysis.

On page 9 (line 234) is the claim: ``That is, under the assumptions of this uniform case, as the wide funnel gets wider, it becomes less and less likely that it will be used.'' This is counter-intuitive. It would be helpful to have a paragraph (of text, not of math) explaining why it is the case, and why our intuitions are wrong. Or is this an artefact of the various assumptions used, and thus not true more generally? (e.g. have the authors used a non-uniform distribution? See next comment.) In either case an explanation would be helpful.

The authors' definition of ``uniform distribution'' evident from the top of page 10 is not the standard definition used in statistics. The ``uniformity'' of a uniform distribution in statistics arises from the equal distirbution of probabilities across a finite set of events, not simply that these probabilities sum to unity. (This may be a serious mis-understanding of the meaning of ``uniform distribution'' or it may simply be an instance of something that needs to be clarified at the top of p. 10.

There are also some editing problems that need to be corrected. There are some words missing, some mis-spellings, ungrammatical and ambiguous constructions, and a few meaningless sentences (e.g. at line 87, which looks like a ``cut & paste'' effort that went awry), missing blank lines and changes in font sizes (e.g. page 12). A final editing should clear these up.

Note:truth maintenance system (TMS)

A truth maintenance system is a mechanism whereby a knowledge based system can keep reasonably consistent and truthful as its knowledge changes. For example, if facts have been added to the KB through inference based on a set of premises, and one of the premises is later removed from the KB, any conclusion that depends on that premise should also be removed from the KB. The Cyc? TMS relies on the fact that each assertion has its support recorded in the data structure.