Search Results

One of the main goals of this research was to develop a computational model that could be implemented and tested. Testing could serve at least two purposes: (1) Demonstrate the viability of the Missing Axiom Theory for dialog processing; and (2) Determine the ways that varying levels of dialog control influence the interaction between user and computer. Consequently, an experiment involving use of the system was constructed to test the effects of different levels of dialog control. The format and results of this experiment are reported in this chapter. The following hypotheses are proposed as performance differences by users as they gain experience and have the initiative. • Task completion time will decrease. • The number of utterances per dialog will decrease. • The percentage of “non-trivial” utterances will increase (a nontrivial utterance is any utterance longer than one word). • The average length of a non-trivial utterance will increase. • The rate of speech (number of utterances per minute) will decrease. These hypotheses are consistent with the intuition that as the user has more initiative, the user will put more thought into the process, reducing the rate of interaction. In addition, it is expected that when the user has more initiative, there would be an attempt to convey more detailed information in each non-trivial utterance. Finally, it is also believed that increased user initiative will be more helpful when the user gains experience and has more knowledge about performing the task independent of computer guidance. Two graduate students in computer science volunteered to use the system. Each subject received about 75 minutes of training on the speech recognizer with the 125 word vocabulary. The subjects then participated in three sessions on differing days. Each session consisted of four different problems where each problem consisted of a single missing wire. The results from these subjects tended to support our hypotheses. However, the experimental control for this testing was not well-defined. The two subjects are involved in AI and NL research and consequently have strong preconceptions about NL systems and what constitutes “proper” behavior toward such systems.

The most sophisticated and efficient means of communication between humans is spoken natural language (NL). It is a rare circumstance when two people choose to communicate via another means when spoken natural language is possible. Ochsman and Chapanis [OC74] conducted a study involving two person teams solving various problems using restricted means of communication such as typewriting and video, typewriting only, handwriting and video, voice and video, voice only, etc. Their conclusion included the following statement. . . . The single most important decision in the design of a telecommunications link should center around the inclusion of a voice channel. In the solution of factual real-world problems, little else seems to make a demonstrable difference . . . Thus, it would seem desirable to develop computer systems that can also communicate with humans via spoken natural language dialog. Furthermore, recent reports from the research community in speech recognition [Adv93] indicate that accuracy levels in speaker-independent continuous speech recognition have reached a threshold where practical applications of spoken natural language are viable. This book addresses the dialog issues that must be resolved in building effective spoken natural language dialog systems—systems where both the human and computer interact via spoken natural language. We present an architecture for dialog processing for which an implementation in the equipment repair domain has been constructed that exhibits a number of behaviors required for efficient human-machine dialog. These behaviors include the following. • Problem solving to achieve a target goal. • The ability to carry out subdialogs to achieve appropriate subgoals and to pass control arbitrarily from one subdialog to another. • The use of a user model to enable useful verbal exchanges and to inhibit unnecessary ones. • The ability to use context dependent expectations to correct speech recognition and track user movement to new subdialogs. • The ability to vary the task/dialog initiative from strongly computer controlled to strongly user controlled or somewhere in between.

This book has presented a computational model for integrated dialog processing. The primary contributions of this research follow. • A mechanism (the Missing Axiom Theory) for integrating subtheories that each address an independently studied subproblem of dialog processing (i.e. interactive task processing, the role of language, user modeling, and exploiting dialog expectation for contextual interpretation and plan recognition). • A computational theory for variable initiative behavior that enables a system to vary its responses at any given moment according to its level of initiative. • Detailed experimental results from the usage of a spoken natural language dialog system that illustrate the viability of the theory and identify behavioral differences of users as a function of their experience and initiative level. This chapter provides a concluding critique, which identifies areas of ongoing work and offers some advice for readers interested in developing their own spoken natural language dialog systems. This section describes important issues we did not successfully address in this research because either (1) we studied the problem but do not as yet have a satisfactory answer; or (2) it was not necessary to investigate the problem for the current system. Regardless of the reason, incorporating solutions to these problems is needed to strengthen the overall model. In section 4.7.3 we have already discussed the difficulties in determining when and how to change the level of initiative during a dialog as well as the problems in maintaining coherence when such a change occurs. Ongoing work in this area is being conducted by Guinn [Gui93]. His model for setting the initiative is based on the idea of “evaluating which participant is better capable of directing the solution of a goal by an examination of the user models of the two participants.” He provides a formula for estimating the competence of a dialog participant based on a probabilistic model of the participant’s knowledge about the domain. Using this formula, Guinn has conducted extensive experimental simulations testing four different methods of selecting initiative.