Next: Repeated questions across evaluators
Up: Baseline model performance
Previous: Results
The evaluation results demonstrate that ISAAC is reading
and comprehending these three stories
at a level indistinguishable from humans, based on independent
judges. This was the overall result I expected--ISAAC
is a capable reader.
Three points are worth mentioning. First,
ISAAC lacked the crucial concepts it needed to comprehend
the stories before reading them. Second, the initial
evaluation
performed indicates that
ISAAC is reading
and comprehending the three stories at the same level
on these comprehension questions as
the human participants, as judged by the
independent evaluators. Third, I am able
to examine ISAAC's memory after the reading episodes
and examine the concepts which were created
during the course of reading--in each case,
the model successfully creates concepts which
allow it to comprehend the stories without creating
unnecessary concepts or bizarre concepts.
Kenneth Moorman
11/4/1997