An interface for targeted collection of common sense knowledge using a mixture model

Authors:

Robert Speer	MIT CSAIL, Cambridge, MA, USA
Jayant Krishnamurthy	MIT CSAIL, Cambridge, MA, USA
Catherine Havasi	Brandeis University, Waltham, MA, USA
Dustin Smith	MIT Media Lab, Cambridge, MA, USA
Henry Lieberman	MIT Media Lab, Cambridge, MA, USA
Kenneth Arnold	MIT Media Lab, Cambridge, MA, USA

Summary:
This paper discusses a common sense knowledge gathering system constructed by the authors which is perceived as a 20 Questions type game while at the same time gathering information from its users. Below is an example of their program running:

Figure 1. Open Mind learns facts about the concept “microwave oven” from a session of 20 Questions.

The reason behind using a '20 Questions' game to collect this common sense knowledge is based on studies showing that users are not willing to freely contribute information unless they can be enticed somehow such as by entertainment. There have been some previous common sense acquisition games including Peekaboom and the ESP Game, both of which pitted two users against each other in an attempt to label images with the same description. The ESP Game focused primarily on generic labels for images while Peekaboom focused on particular components of images. There have also been a couple that work with matching phrases and words that generally correspond or describe each other, such as 'horse' and 'it has a hoof'. A couple examples of these games are Verbosity and Common Consensus.

The model that the authors used to collect common sense knowledge was built on a concept/relation representation similar to that of ConceptNet's data model. With these models they could determine certain 'features' which could simplify the algorithm in their 20 Questions game, where a features is described as 'A feature is a pairing of a concept with a relation which forms a complete assertion when combined with a concept.'. Through these features the authors were able to graphically show the AnalogySpace of these concepts and relations in a clustering model.

The authors also went into great detail in demonstrating equations and algorithms behind their common sense acquisition models. Below is another example of their game running:

Figure 3. Using the 20 Questions interface to develop a concept.

Later on in the paper they started to discuss some of the interface design objectives for their system. The primary goals are listed below:

• Feedback, in this case the authors want a system that will provide what the computer is currently thinking so that the user can see how their responses are directly affecting the computers deterministic approach.
• User Enjoyment, they just want the interface to be as enjoyable as possible to keep users interested in playing.
  
• Minimalism, the game shouldn't be a stand alone or stand out in any situation, but should be there when needed and run seamlessly with the website.
  
• Effortless Acquisition, in this case they don't want users to feel that they have to work at providing information, but instead it should appear 'effortless'.

A user study was done in which the authors presented an online comparison between the current manual OpenMind interface and the new designed 20 Questions. Users operated each system and afterward were asked a sample of questions to determine their enjoyment and the effectiveness of each system. The results of the study showed that the 20 Questions system out-performed the current OpenMind one on such fields as “I would use this system
again”, “I enjoyed this activity”, and “The system was adapting as I entered new information”. Apart from that the 20 Questions system took considerably less time to complete as seen in Figure 8.

Figure 7. The mean and SEM of the survey responses, grouped by test
condition.

Figure 8. Themean and SEMof the elapsed time to complete each task.

The results of their study and the conclusions they drew were that with this interface users will be more willing to contribute data, and that this will lead to better knowledge acquisition.

Discussion:
The overall point of this paper was that the authors designed a new interface for data acquisition to replace the current OpenMind one, and that their new system is based off of the '20 Questions' game. Despite this relatively simple point, they somehow found a way to express this through 10 pages. I did like that they modeled their system after a game, because it is quite normal to expect people to not want to contribute unless they can get something out of it, in this case some mild entertainment. Overall I did not find this paper interesting, but perhaps it has uses I can't foresee.