Overview:

Johnson-Laird gives an overview an account of mental models that originally is derived from Kenneth Craik. Craik’s use of models was originally directed towards an account of explanation. The review Johnson-Laird gives is to find a mechanism for formalizing meaning in language that explains cognition. The formulation is strongly tied in the notion of computation, and models are represented as computationally formalizable. This puts Johnson-Laird at odds with proponents of embodiment, but his theory nonetheless gives a formal strategy for forming and understanding mental models.

Notes:

The prologue introduces a set of questions which is good for characterizing the investigation. Here are a couple of them (p. ix):

• Why is it that we cannot think everything at once but are forced to have one thought after another? Our memories exist together, yet we cannot call them to mind all at once, but only one at a time.
• Why are there silences when we think aloud? Aren’t we thinking at those moments, or are we unable to put our thoughts into words? It seems unlikely that thoughts should be grossly intermittent, so what barrier prevents them from being articulated?
• What happens when we understand a sentence? We are aware of understanding it, and are more aware of having failed to do so. Why can’t we follow the mental processes of comprehension as we can follow the action of tying a shoelace?

The concept of mental models derives from Craik. Johnson-Laird notes Weizenbaum’s ELIZA, but claims that it is not a simulation, but rather a dissimulation. It does not have a process of thought, but conjures thought instead. This distinction raises the contrasting idea that ELIZA matches and responds to the interactor’s model, rather than having a model of its own.

The Nature of Explanation

Most theories of cognition consist of description, and lack are not formal (in the sense of algorithmic). Johnson-Laird asks what the criteria is for a definition of cognition. This criteria, he explains, should describe theory in the form of an effective procedure. Theory must be in the form of an algorithm. This should not be a limitation in what exists in the world, but rather, what constitutes a theory that describes the world.

Later in the chapter, there is an extensive discussion of Turing machines, and explaining their universality. He is very impressed by and fascinated with the capacity for Turing machines to do any computation, and furthermore represent each other. If theories are algorithms, then they must be computable. That assertion is the claim of functionalism, to which Johnson-Laird ascribes.

The Doctrine of Mental Logic

Models are intended to replace the doctrine of mental logic, which is the propositional model of cognition. Propositional logic is a fallacious as a model for cognition because of the many logical mistakes that people make on a daily baisis. If our brains worked according to mental propositional logic, then we would be able to more readily correctly answer certain logical problems, which is clearly not the case. Johnson-Laird is not attempting to argue against logic, but rather, that there are multiple kinds of logic.

The logical problem demonstrated is case where the subjects are shown a set of cards with the symbols: [E, K, 4, 7], and told that every card has a number on one side, and a letter on the other. The subject given the generalization, “If a card has a vowel on one side then it has an even number on the other side.” The subject is then asked what cards to turn over to find out whether the generalization is true or false. (p. 30)

These simple logic problems are strongly affected by context. Context affects inference. Changing context in logical problems leads to variable results in whether people can solve the problem correctly. Certain formulations of equivalent problems are frequently solved correctly, while other formulations are frequently solved incorrectly. Familiarity generally helps performance. This argument surfaces when making the connection to embodiment and associative reasoning.

The conclusion of this section presents 6 bullet points (p. 39):

1. People make fallacious inferences.
2. Which logic is found in the mind?
3. How is logic formulated in the mind?
4. How does logic arise in the mind? (development)
5. Deduction depends on the content of the premises. When an individual is familiar with (or has a model of) a situation, they are more likely to reason about it correctly.
6. “People follow extra-logical heuristics in making inferences. They appear to be guided by the principle of maintaining the semantic content of the premises but expressing it with greater linguistic economy.” That is, when presented with propositions p and not-p or q, they are likely to conclude q, instead of p and q.

Theories of the Syllogism

Propositional logic is psychologically flawed. A more accurate logic occurs in syllogisms. Syllogisms are first order declarations: All X are Y, or some X are Y, no X are Y, etc. Johson-Laird puts forth several goals for a theory of reasoning (p. 65-66), and will later deduce that syllogisms satisfy these goals.

1. “A descriptively adequate theory must account for the evaluation of conclusions, the relative difficulty of different inferences, and the systematic errors and biases that occur in drawing spontaneous conclusions.”
2. “The theory should explain the differences in inferential ability from one individual to another.”
3. “The theory should be extensible in a natural way to related varieties of inference rather than apply solely to a narrow class of deductions.”
4. “The theory should explain how children acquire the ability to make valid inferences.”
5. “The theory must allow that people are capable of making valid inferences, that is, they are potentially rational.”
6. “The theory should shed some light on why formal logic was invented and how it was developed.”
7. “The theory should ideally have practical applications to the teaching of reasoning skills.”

How to Reason Syllogistically

In giving a description for how people might reason using syllogisms, Johnson-Laird gives an example of how syllogisms might be visualized by an individual. The syllogism is of the form, “All the artists are beekeepers, and all the beekeepers are chemists.” A way to visualise syllogisms without using a Euler circle or Venn diagram is to imagine a tableau of actors who play the parts of artists, beekeepers, and chemists. Thus, there would be artist-beekeeper-chemists, beekeeper-chemists, and a lone chemist. (p. 94) The metaphor of the tableau is useful for representing mental representations of the situation, but more telling is the use of the troupe of actors who enact these roles. This representation covertly emphasizes the cultural and embodied manner by which the syllogism is understood.

Going a step further, though. Johnson-Laird produces an algorithm for how to reason syllogistically. However, syllogistic logic is still not a complete representation of the logic that humans follow when reasoning, because we still make reasoning mistakes in complex syllogistic problems, for example: “Some B are A, no C are B” yields incorrect conclusions in almost all cases. (p. 74)

Inference and Mental Models

The key to this chapter is how to reason without rules of inference. Both propositional and syllogistic logic define rules for drawing inferences, but they do not line up to natural everyday reason. Mental models are introduced with relational expressions. These may all take the form of predicates or relational expressions. Relations are a bit heavier than ordinary propositions, but still work on the same level. At this point, all mental models are of the form of tableaus.

With the focus on tableaus, mental models can be understood as devices for association, and defining relationships. Both of these can be addressed by non-symbolic and embodied means (Lakoff and Johnson), so even though Johsnon-Laird’s formulation is intended to be computational in nature, it can be more than that.

There are some final bullet points regarding mental models:

1. The theory embraces both implicit and explicit inferences. This means that they should be able to represent all arguments.
2. Children can learn to reason before understanding rules of inference, because reason is possible without logic.
3. The theory is compatible with the fact that people can use logic.
4. It is also compatible with the historical origin of logic.

Images, Propositions, and Models

There is a conflict over how images fit into cognition and psychology. The two sides are the ‘imagists’ (Paivio, Shepard, and Kosslyn) and ‘propositionalists’ (Baylor, Pylyshyn, Palmer). Johnson-Laird argues for the encoding of images in the mind, and goes for a functional account of mental processing. This does liken the mind to a computer: it can procedurally transform images into systems of symbols.

Johnson-Laird describes the relationship between mental models and propositions. “The crucial problem for the mental language is the nature of its semantics. Propositions can refer to the world. Human beings, of course, do not apprehend the world directly; they posess an internal representation of it, because perception is the construction of a model of the world.” Thus, the mental model operates between the individual’s logic and the world itself. Any propositions in that individual’s mind must act on the model, rather than on the world directly. Models work via analogy, and images are views of models.

Meaning in Model-Theoretic Semantics

This section describes how meaning is constructed and composed (in the sense of built compositionally) in model theory. Johnson-Laird references Tarski here, in terms of understanding truth values. A big bit of this is still in terms of truth vs falsehoods. The discussion raises the issue of worlds, and how models connote not only existing meaning, but a set of potential configurations that are enabled by that model. The world of meaning enabled by a model is called its extension.

There is some discussion of Montague grammar, which is an attempted formalization of natural language. This segues into a model-based formulation of meaning, which derives neatly from mathematical logic. The following passage is dearly familiar to the theory of models in logic. “The power of model-theoretic semantics resides in its explicit and rigorous approach to the composition of meanings. It provides a theory of semantic properties and relations, e.g., a set of premises entails a conclusion if and only if the conclusion is true in every model in which the premises is true.” (p. 180)

What Is Meaning?

The discussion of meaning traverses from psychology to word meanings. THere is a great deal of philosophy and squabbling over where meanings come from, or what concepts like “water” or “jade” are, intrinsically. This entire discussion neglects the use of practice, where words and other signs may hold different meanings to different observers, under different circumstances. The importance of language meaning is critical in this treatment of mental models, because the models are based on language.

The conflict in this is between meaning Psychologism and Realism, which respectively attest that meaning is in the mind or outside of the mind. Johnson-Laird is attempting to find a middle ground in this, and looks to an encoding of meaning that allows for intersections and vagueness. However, fuzzy logic exposes the same problem. Propositions, even with values of confidence, are divorced from a knower. For language to work, the knower must have a context and a state of mind. The relative values of “tallness” (given in his example on p. 200) are only meaningful in context.

To address the question of meaning, the psychological perspective asserts that meaning is wholly in the mind, whereas the realistic perspective asserts that meaning is wholly outside of it. Johnson-Laird seems to claim that meaning works within a model, which is grounded in language, which has elements that are both inside and outside the mind. There is an added dimension of culture, though which is extremely relevant. Meanings (and models) are shared between individuals in a culture, so meaning exists beyond the individual, but also beyond the literalism of language. I would argue that it is instead a consensus. This position is not incompatible with models, but requires a reppropriation of Johnson-Laird’s use of models.

The Psychology of Meaning

Models are procedural structures that may be adjusted over time or through discourse according to some rules. There is a set of bullet points describing these:

1. “The processes by which fictitious discourse is understood are not essentially different from those that occur with true assertions.” Thus we use the same logic for processing information into models, even if we know the information is fictional or false.
2. “In understanding a discourse, you construct a single model of it.”
3. “The interpretation of discourse depends on both the model and the processes that construct, extend, and evaluate it.” The model for discourse can vary over time.
4. “The functions that construct, extend, evaluate, and revise mental models, unlike the interpretation functions of model-theoretic semantics, cannot be treated in an abstract way.” There must be some formal algorithms for changing mental models.
5. “A discourse is true if it has at least one mental model that satisfies its truth conditions that can be embedded in a model corresponding to the world.”

~~Intermezzo~~

The next couple of chapters deal with the understanding of grammar and the parsing of language into propositional expressions. There is a great deal of noun-phrase, verb-phrase stuff. The analysis of grammar is heavily extended from Chomsky.

The Coherence of Discourse

Johnson-Laird gives a surprising interjection regarding story grammars. This makes some sense given the focus in the preceeding chapters on the relationship between language grammar and models. The challenge to story grammars can be seen as a critique of a particular kind of structuralism. Earlier pages compare blocks of text that form coherent paragraphs versus those that do not. Coherency relates to consistency and discourse history, which is a type of context. Models have the formal power to use this context in a way that grammar lacks.

The Nature of Mental Models

Some properties of mental models:

1. Computability. Mental models are computable, and so are the tools for manipulating them.
2. Finitism. A mental model must be finite, and cannot directly represent an infinite domain.
3. Constructivism. A model is constructed from symbolic tokens and structurally composed.

A typology/heirarchy of models:

1. Relational. This is a finite set of tokens representing entities, a finite set of properties, and a finite set of relations connecting entities to properties.
2. Spatial. This is a relational model where the relations are spatial.
3. Temporal. A temporal model consists of frames of spatial models, that occur in a temporal order.
4. Kinematic. This is a temporal model that is psychologically continuous, there are no temporal discontinuities.
5. Dynamic. A kinematic model which relates causal relations between frames.
6. Image. The image is a viewer-centric representation of a spatial or kinematic model.

It seems to me that this formulation reverts to computational models, and begins to become severely detached from underlying psychology.

Consciousness and Computation

The final chapter works to give a formal and procedural account for consciousness. Essentially, consciousness is already computational, when understood as processing of mental models. An excuse is given here, that while cognition may be computational, other human traits, such as spirituality, morality, and imagination cannot be modeled and will “remain forever inexplicable.” This is a cop out. Johnson-Laird cannot introduce a hulking device for representing psychology and then blow off its application to other psychological traits.

There are significant critiques to be had with the computational formulation of mental models. I would argue that the computational imposition is severely flawed, but models remain invaluable as a tool for understanding cognition. The use of modeling is especially important in the representations of spirituality (cultural beliefs), morality, and imagination.

So, one thing that has emerged in recent thought experiments about simulation of characters, is that the plan based model of behavior is really problematic. A lot of times, most of the time, characters do not have plans. I would argue that furthermore, people don’t have plans, and that a plan is something that is generally interpreted from everyday behavior, rather than something at the root of it. AI is big on planning. Most AI research on simulated characters focuses strongly on character plans. What is even more ridiculous about this is that character plans are always rational.

I want to look at an alternative model to planning, which is situation. Then, I want to look at something embedded deeper in plans, which is the notion of goals. I would take the critique of planning even further and extend that to the notion of goals. I believe that much of character (and human in general) behavior is goal oriented, a lot of behavior is without goals, and simply reactive. Emotional response tends to be without goals, as does casual conversation. The notion of goal also fails to account for the element of motivation. A condition might be a character’s goal, but it belies the driving motivation behind that goal. To account for this, I will introduce a subtle variant, which is purpose.

Plans are fallacious as a general model of behavior, because they impose a heirarchical structure on thought, and fail to account for human versatility. Furthermore, plans also have tremendous difficulty modeling very straightforward situations, especially as relates to communication and interaction. Plans to not account for a great deal of contextual or situated behavior, the importance of which has been stressed in recent work in cognitive science.

An alternative to planning incorporates elements of plans into an agent’s state. The matter no longer becomes one of top-down organization, but of bottom-up emergent behavior. Characters and people are not entirely reactive, but our behaviors and modes of action are largely context dependent. This is especially the case in terms of social interaction. There is a particular code of conduct that an agent should abide by while in a meeting, as opposed to at dinner, or walking down the street. The situated nature of behavior removes the demand for planning to use a single root goal that informs all other behaviors. Instead, in my situated model, the task specified by a plan becomes part of the agent’s state. Metaphorically, the difference is instead of action happening at the character’s mind, it originates in the character’s identity. A character who is a student has a goal, “graduate,” but this is a long term goal, and is not considered at every decision, but is something that is a part of the character’s being. Similarly, “going to the grocery store,” is a similar state, which informs later actions, but does not prevent the character from stopping for coffee, or having a conversation.

Planning has a deeper anchor in the notion of goals. The first mistake is to assume that goals are the driving force behind all behavior. This is an outright falsehood. Many times, people, and characters in particular, will act at a purely automatic or emotional level, responsively. Goals implicitly endorse the rationality of human behavior, because without rationality, goals could never be met, and without goals, rationality would be meaningless. No character is ever fully rational, though. People might act against their own goals, not even knowing that this action is harmful. What motivates the action, then? Good examples of this are situations when a character is in an explicitly “irrational” state, such as intoxication or being “overwhelmed” with emotion. However, it is hard to imagine that anyone is ever truly rational at any other time. Potential responses to this from the AI perspective are to devise different standards of rationality.

Changing the standard of rationality is a step in the right direction, but it does not account some other situations, particularly, the relative ease at which people respond to emotions or have conversations. I doubt what is taking place in these situations is rapid revision of goals and intentions, but rather some behaviors and states are induced naturally by circumstance, without the character ever needing to formulate a goal explicitly.

While two characters may have the same goal at a given moment, they may not have the same purpose, and the difference in purpose will tell a great deal about how the character’s actions may be executed. Consider, for instance, the airplane safety checklist executed by a pilot before takeoff. The immediate goal of the pilot’s actions is certainly to correctly do the check and respond appropriately. However, the deeper implications of that goal are less clear. Who is the pilot performing the check for? Is it because of genuine concern about safety? Is it to correctly satisfy the safety check ritual for the purpose of regulations? A lot of different theories could explain the form of the pilot’s actions: ritual, performance, practice, directed action, etc. However, the immediate goal is the same, but the purpose, the contextual goal, may be different. Purpose transcends goal, involves meaning, and dismantles the discrete, abstract nature that comes with goals. Purpose is intrinsically situated and linked to identity rather than symbolic mind alone.

This is a collection of essays in honor of Philip Johnson-Laird, one of the founding figures in mental models. These essays represent application of his theory to several particular domains.

George Miller: Contextuality

This essay is about handling words and stituations with multiple meanings. The process of figuring out these meanings is contextualization, described as a basic cognitive process. This is closely related to Goffman’s frame analysis. The goal of contextualization is to resolve ambiguity that is heavily present in interpretation of everyday language and knowledge.

Computational linguistics is a tricky area in cognitive science and computation. It is deeply affected by the issue of context. Miller’s analysis focuses on linguistics exclusively (mirroring Johnson-Laird), as opposed to other sorts of ambiguous circumstances. Computational linguistics involves processing language and attempting to identify and process the correct word meanings from that language. Miller mentions Bar-Hillel (1960), who finds that this sort of language processing can identify correct meanings about 80% of the time. He estimates that this last bit could never be achieved without significant advances in AI.

Expert systems, which are the general approach for working with specialized knowledge, limit the domain of word meanings to a significant degree, but this still does not absolve the “Curse of Bar-Hillel.” Miller theorizes that context identification is the key to unlocking this last bit of meaning.

An aside to note is that Miller is a collaborator on WordNet.

Alan Garnham: The Other Side of Mental Models: Theories of Language Comprehension

This essay looks at language comprehension by examining issues of reference and inference. One key element to inference and communication is instantiation. Where an abstract idea is replaced by a more concrete (or other known) one. However, Garnham is concerned with the communication of abstracts, and notes that we communicate information about abstracts without instantiation.

Propositional relations are a strategy used frequently in AI for world modeling, and relate to information as discrete facts. Garnham gives an example which uses locational prepositions, things of the form: “The lamp is in front of the candle,” etcetera. In terms of these locational structure here, it seems dubious. There is a suggestion that mental models use a more analog depiction and representation of spatial relations.

It is true that we do use abstracts in communication, but models of communication that have emerged from Vygotsky indicate that communication emerges in development when social interaction transforms from something embodied and physical to something symbolic. If we follow Lakoff and Johson, then relations are all metaphorical and based ultimately in the body.

Paolo Legrenzi and Vittorio Girotto: Mental Models in Reasoning and Decision-making Processes

This essay discusses decision making according to psychological studies, and explained in terms of mental models. The interesting thing here is that totally rational decision making is not present, rather, decision making is based on the matter of focusing. This sounds a lot like priming and activation (related to neural networks). Models illustrate the construction of ideas, but neglect to factor how the focusing works intrinsically.

David Green: Models, Arguments, and Decisions

Green builds a theory of decisions (as derived from Craik, 1943) based on argument. Argument is done through warrants, which are bits of relevant information.This work is built from Toulmin’s scheme. Further, the goal here is to analyze argument through mental models. The interplay between observation and model mirrors warrant and argument. Warrants also relate to beliefs, which may be connectable to the belief, desire, and intention scheme in AI. We can also apply warrants to causal models.

The final conclusion in this section is that there is an interplay between argument and simulation, as well as decisions and commitment.

Keith Oatley: Emotions, Rationality, and Informal Reasoning.

Oatley’s focus here is on informal reasoning as it relates to emotions. Informal is opposed to logical or day-to-day. Oatley argues that emotion is critical to this sort of everyday conventional reasoning.

He opens with an analysis of Aristotle’s rhetoric, which discusses two types of reason. There is absolute mathematical reasoning, and also persuaded reasoning, where there is no demonstrable truth. Persuasion instead aims to achieve the best truth possible. The interesting example with this is that Aristotle’s writing is based on the sort of rhetoric used in his day, where law is extremely dependent on performance and dramatic emotional appeal.

Oatley makes some notes on Aristotle’s Rhetoric: The first is that persuasion applies to the imperfectly knowable, and the field of the imperfectly knowable is huge. AI, on the other hand, seeks to only understand that which is perfectly knowable, or that which can be logically concluded or deduced. The second point is that Aristotle explains emotion as a tool of judgement, as opposed to something that is bestial or irrational. Furthermore, there are three impediments to making rational decisions:

1. Limited knowledge and resources. Our mental models are incomplete to fully predict the effects of our actions.
2. Multiple goals. Multiple goals cannot always all be satisfied rationally.
3. Distributed agency. Actions are performed in relation to others, planning must occur among multiple agents, where the problem of limited knowledge becomes especially difficult.

Rationality depends on environment and context. Emotion is used as a form of feedback for goals. Oatley describes emotion as a heuristic function for potential actions.

Oatley makes a connection to Vygotsky and Hutchins. Emotions play a role in the distribution and extension of cognition. There is a connection between Aristotle and the Roman historian Quintillian, who documents the practice of law in the Roman court. This is an argument for the theatricality of reason, relating to the ideas of performance. The performance of law is an enactment and exaggeration of events. The social nature of the audience is essential.

Oatley follows this with the analysis of two experiments, where individuals change behavior based on emotional priming. Emotional induction proved to be immensely relevant in both examples. One of which consisted of examining decision making in judgement of evidence of a trial (after having viewed a happy or a sad film clip), and the other examined forward or reverse reasoning (after reading an angry or sad short story). The experimental corrolation was immensely strong in both examples.

Ciuliano Geminiani, Antonella Carassa, Bruno Bara: Causality by Contact

This essay is about the role of causality in reasoning. Causality is related to the construction of scientific models, but also is relevant from the perspective of narrative. Using causality implies the use of simulation mentally. Causality has an evolutionary basis that is associative (for instance, a rat who smells a type of food on a dead rat will not eat that type of food). This associative logic is also imaginably present in humans, but humans also do use causal reasoning, which comes with the demand for knowing why something occurs. This connects well to Vygotsky and development. The why relates to the narrative/linguistic model of thought.

An interesting note: In developmental study, causality is dependent on contact. Touching is necessary for causality to be interpreted by infants. Gradually, though, causality becomes analogically based. Causal models are a subset of dynamic models. The authors give a funny example of two narrative segments: “Cleopatra was bitten by an asp, Cleopatra died” versus “Cleopatra was bitten by an asp, a tourniquet was applied to her arm, Cleopatra was saved.” This example is a little strange, but is used to understand how people might model what happens to the poison. Mental imagery and metaphors are especially important: poison is a particle, poison is like paint, etc. The important thing to note here is that the example is fundamentally a narrative one.

To understand how models are formed and used, the authors give a three part theory for development of causal models: Construction, comparison, falsification. The construction phase involves taking the components (as a pre-model) and understanding them quantitatively. This is literally formulated as collecting symbols and describing them qualitatively. Next, qualities are quantified, fixing values and times. Finally, the model is simulated dynamically at a sub-cognitive level. The sub-cognitive simulation involves 1) activation of implicit knowledge, 2) generation of instantaneous changes in quantities according to the simulation, and 3) simulation of the temporal evolution of the model.

At the comparison phase, the effects of the mental model with the base model are compared. In this context, the base model is imaginably the observed phenomenon, which is the original story. This comparison intiates revisitations and inferences. Finally, in the falsification phase, plausibility and counterexamples are considered. This sort of analysis derives from Qualitative Process Theory (Forbus 1984), which seems like a good place to check the connection between narrative and models.

This approach is useful in looking at models of fiction as pertains to adaptation, especially in terms of emotional value and responses.

This was written as a problem formulation for one of my courses with Nancy Nersessian. I thought it was handy and relevant, so decided to put it up.

To open, let us consider an argument made by Keith Oatley (in “The Science of Fiction,” New Scientist, 2008): “Fiction is a simulation that runs on the software of our minds.” Oatley’s argument is based on studies of readers of fiction who demonstrate better emotional interpretation skills after reading fiction. The interpretation that he gives is that readers mentally predict the emotions of the fictional characters, and can compare the prediction with the result that actually plays out. This can be seen as either modeling and conceptual change, or a sort of psychological activation of emotional parts of the brain. It is hard to tell for certain, but let this contextualize what follows:

My personal research is in the adaptation of fiction though simulation of the fictional characters and worlds. When the characters themselves are embedded in an electronic simulation, the problem immediately rises of how to develop emotional responses of the characters. Previous work in simulating characters (Perlin, Crawford, and many others) have given varying approaches to modeling characters and communicating states. Game studies (from which my work is emerging) has borrowed significantly from the theories of cinema and theatre for how to represent appealing and believable characters. Much of what has emerged from this is centered on the issue of providing more visible, clearer characters. A great deal of attention is paid to faces (and the movement from text, to image, to animation), which are, admittedly important. Everything boils down to dramatic presentation, which comes from drama, which may be embodied and visceral, but is not an interactive medium.

A simulated fictional world, in which the player is a participant, must look beyond interaction, though. The player must be able to communicate with other agents, for the fictional experience to be enacted. Many times in games, players are in opposition to the game world and its designers, a situation that does not satisfy the potential for emotional relevance. Despite the significant advances in character depiction in many contemporary games, this issue remains a problem. Bonnie Nardi does not give a solution, but provides a way to understand the failure of graphical technology in developing emotional experiences.

In Bonnie Nardi’s ethnography, she studies the methods of communications of business employees, investigating specifically how they use mediated communication. She finds that there are three dimensions to communication: affinity, commitment, and attention. These are explained to make sense within the business model, but are all grounded in sociology and anthropology. Expressions of each of these elements can be seen in every form of communication, and they seem almost tribal. Clearly, there is some deep importance to these channels. Individuals make use of the affordances of mediated communication to work with these elements, but the elements are still deeply embodied. The reliance on information bandwidth is insufficient to operate on these channels.

With Nardi’s Beyond Bandwidth in mind, developments in technology in games are clearly matters of attempting to increase bandwidth. For an interactive experience with fictional characters to succeed, communication must be necessary, and this depends on the relationship of the player to other characters. This does explain why emotional experiences in games work independently of technology. Nardi’s dimensions are a starting point, and open a suite of complex issues: Mediated communication has affordances for the elements of communication, but what are those affordances in games and simulation? How can affinity, commitment, and attention be represented in an artificial world?

This paper is on computer mediated communication. The abstract presents the paper as a critique of some standard approaches, which emphasize the role of bandwidth as a means for understanding different different means of mediated communication. Instead, the subject should be the relationship between the communicators, and the focus on how the mediated communication affects that relationship. Nardi proposes a model that uses three fields of connection: affinity, commitment, and attention. These fields form the dimensions of a communication space, wherein the values of each change and evolve over the course of communication.

Nardi reviews some existing theory on computer mediated communication. These are nuanced, but all fall under the category of exploring bandwidth: Media richness theory (Daft and Lengel 1984), Social presence theory (Short et al. 1976), and others. All these are about understanding communication in context of more objective information about the communicators.

Nardi’s analysis instead looks at an ethnography of instant messaging in the workplace. She finds that what is important is not the bandwidth and or richness of information, but rather communication is about the feelings of connection and the sense of openness in interaction. These feelings were established by bodily interactions and “informal discourse of low information content.”

In a study of instant messaging, Nardi finds that well regarded executives perform quick and informal “purposeless” communication with colleagues through IM. Nonetheless, this informal communication does serve a function of staying connected. Activities of connection occur through computer mediated communication, but simply through different channels than in regular conversation (which has a lot of embodied and visual signals). This idea is interesting because it puts a mediated spin on the sociology of interaction.

In reviewing social presence theory, Nardi introduces a number of concepts from sociology. Particularly relevant to mediated communication is the idea of symbolic interaction, which lends the insight that different media may be chosen to communicate different things based on the social role of the medium. Interaction relies on cueing, and different media have varying affordances for cues.

Nardi’s analysis of communication is broken down into three dimensions: Affinity, commitment, and attention.

Affinity is a feeling of connection, and the degree of openness in interacting with another. Affinity may be derived from social bonding activities. The examples of social al bonding that Nardi provides are: Touch, eating and drinking, sharing experience in a common space, informal conversation. Bonding is heavily embodied. These experiences are also tightly connected to not only sociological traditions, but also anthropological ones.

As an aside, the idea of bonding might be an interesting thing to use to develop emotional significance with artificial agents. A bit of work has been done looking into the emotional appeal of games and game characters (which is especially significant in the cases where tragedy strikes), and this might be useful in other cases. It may also be a good frame to analyze the Sims.

The expression of commitment is important. Nardi explores some interviews and finds that commitment is very much about establishing a bodily presence. What is important is making some sort of visible expression to indicate just how committed the actor is. Commitment is about performance and ritual. In some cases, it is related to expenditure, but bodily presence is especially valuable. For example, flying out a long distance to meet clients for a day. Commitment can probably be compared to issues of investment and personal sacrifice. Again, this is a common practice that has deep roots in anthropology.

Procuring attention is about capturing the focus of a subject. In personal interaction, attention has a lot to do with eye contact. The gaze is another heavily embodied element of communication, and cannot adapt well to the bandwith model of communication. Attention is also about conveying availability for interaction.

One curious thing about Nardi’s analysis is that her experimental subjects are all modern professionals, but each of the elements of study are overwhelmingly anchored in anthropology. The effects of the examples of highly paid business executives seem right at home next to the effects of tribal rituals.

Using these three elements as essential parts of communication, Nardi explains that some of these elements can be carried over into mediated communication, but their operation is different. Mediated communication enables the elements of contact, but in a subtle and definitively weaker manner than in full face-to-face communication. Nardi’s point can be seen that improved bandwidth can not improve communication. Rather, communication might be improved by focus on the elements of connection and relationships.

Opens by outlining the general failure of theoretical psychology to apply to HCI. HCI is generally developed from practical knowledge. The problem is especially related to the psychological theory of information processing, deriving from cognitive science. Kuutti describes a movement in HCI that reacts against the symbolic/Cartesian model.

He examines Liam Bannon, who presented a critique of Human Factors (as likening humans to components to factor into a human-machine system), and further criticizes HCI as exploring only inexperienced users. This critique relates back to the idea of everyday life and practice. Experienced users make use of emergent practices that may augment or subvert the original model.

HCI is composed of three levels. The first is oriented toward ergonomics, perception and motor skills. The second is concerned with information processing and conceptual psychology. The last is an emerging level that addresses deeper complexity, and is the target for applying activity theory.

As background, activity theory originates in Kant and Hegel. This German philosophy developed against objective empiricism, the idea that meaning is external and may be discovered through experience, and replaced it with constructivism, where meaning is actively constructed. This idea was later adopted by Marx and Engels who applied construction and activity to political ends. Activity was finally developed by Vygotsky and his school of psychology. Kuutti suggests that Mead’s symbolic interaction (which led to Goffman) also followed a similar vein.

An activity is the basic unit of analysis. The idea is to define an activity as a subset of actions that has a minimal meaningful context. Activities are mediated, and are also permeable and flexible structures. It also provides an interesting approach to tool use. Tools are media that enable activities, but they limit the user in things that do not belong to that activity. This is especially important if we consider tools or mediators to be more than physical objects. Mediation can be through environment or other contexts. In this sense, the limiting nature of tools reflects pattern matching in phenomenology as well as the notion of keying in sociology.

Activity also works with the idea of a community formed around activities and the mediating object. This establishes the notion of a collaborative activity, where members may have different roles. Kuutti calls the division between these roles a division of labor, but I would argue that activity is not dependent on labor.

Kuutti does relate activity to planning, but as a step that is about modeling, not outlining the action. The initial phase of an activity is orientation, during which the agent models the world into consciousness. The execution of an activity is composed of “fluent” engagement with the world, which is composed of operations. This model allows for a hierarchical model of activity that derives from learning and practice. For a novice learning how to drive a car, the process of changing the gearbox is a highly conscious activity that requires a lot of attention to the individual elements of the task, but later on, these operations become fluent elements and moving the gearbox is merely an action in the larger activity of driving. Kuuti does not examine this hierarchy, though, and seems to assert that activity only happens at this higher level.

Activity relates to HCI by looking at how people go about using computers for activities. This exposes some of the ways in which traditional information processing psychology has failed to help HCI. Notably, it addresses the issues of complexity and use and practice.

Notes:

The authors set out to bridge what they consider to be a false division between intelligence and technology. The idea of technology is something that is artificial, and usually electronic. Intelligence is seen as something that enables reason and planning, and is a biological property of individuals. The authors’ notion of technology is a tool-mediated social practice, which enables a very different model of intelligence. In this context, intelligence is “a process of adaptation to, and transformation of, the conditions of life.” (p. 2)

Artifacts are both ideal and material. This idea derives from Dewey, and is supported by constructivist philosophy. This unity is also something that exists in cognition: material and symbolic are tied together in thought, departing from Cartesian dualism. The authors outline levels of artifacts as identified by Wartofsky, which move artifacts from primarily material to primarily symbolic.

The authors continue by looking at how artifacts augment cognition, and some of which are explicitly psychological tools. This notion comes from Vygotsky, but the idea that tools are used to augment understanding extend back from Francis Bacon. The cognitive prosthetic continues through Norman (and also through other figures in early computer science, notably Vannevar Bush and Norbert Weiner). Norman’s principles are bulletted here: (p. 6)

• A representation is a set of symbols that substitutes for the real event.
• Once we have ideas represented by representations, the physical world is no longer relevant.
• Representations are abstractions so good representations are those which abstract the essential elements of the event.
• The critical trick is to get the abstractions right, to represent the important aspects and not the unimportant. This allows everyone to concentrate upon the essentials without distraction from irrelevancies.
• Representations are important because they allow us to work with events and things absent in space and time, or for that matter, events and things that never existed — imaginary objects and concepts.
• A person is a system with an active, internal representation.

This is an interesting approach with which to analyze representation. Norman uses this but focuses exclusively on the ideal or conceptual level and denies the material element in the cognitive. Further, the authors criticize Norman’s neglecting of the environment and broader social context surrounding the use of artifacts.

The negotiation between intelligence and technology leads to a reconsideration of the role of culture in cognition. When intelligence and culture are bridged, artifacts become material and conceptual aids to cognition. The idea of culture as a large pool of knowledge is heavily challenged. The authors cite Geertz as circumventing the ideal/material dichotomy, using semiotics as a means to embed culture in material artifacts.

The authors continue exploring Geertz, specifically his theory that the nervous system requires culture in order for human development. This argument is followed by Quartz and Sejnowski, who encourage the neurological aspect of culture, and the extension of intelligence and cognition into the environment.

To bridge the connection between technology and intelligence through culture, the authors give the example of the use of the Abacus in Japan. The tool was delivered culturally and developed a cultural role and function. Additionally, there is a cultural intelligence associated with it that becomes evident in highly skilled abacus users.

It seems like every day I’m finding more and more reading material for my specials list. I need 150 works, and every time I take one out, I find two more that I really want to put in.

I think what is really taking place here is that I want to write not one, but two dissertations. The first covers narrative, fiction, adaptation, and cognitive models, and simulation at a conceptual level. The other is really an AI or CS thesis which focuses on AI, situated cognition, simulated and believable agents, and simulation as a methodology. Considering this, what may wind up happening is that I’d need to choose one or the other, or writing some magnificent and terrifying two-part monster. Either way, I am liable to spend the rest of my life in graduate school. Interesting things will happen, though. Sometimes it is hard to be patient.

Overview:

Lev Vygotsky is one of the most unusually influential figures in modern cognitive science. He is unusual in that he was, in his day, a controversial figure within his native Soviet Russia, and because of this fact, his ideas did not become popular in the west until about thirty years after his death. When his writing did become circulated in the west, it shone on many subjects from cognitive science to developmental psychology. Vygotsky is also remarkably ahead of his time in critiquing both rationalism and behaviorism, arguing instead for a remarkably nuanced take on development and cognition, wherein cultural, social, and embodied contexts are necessary for proper study of learning.

Mind in Society criticizes existing psychological methods, and presents an argument for looking at psychology from a cultural and social perspective. At the heart of his examination is the idea of the formation of symbols, which occurs as a social function. The process of learning symbols is called internalization, and it involves the internalization of signs, but this is matched with an externalization of meaning. Essentially, the emergence of symbols occurs simultaneously with the extension of cognition into the environment. Vygotsky’s analysis gives light on how to treat the symbol-embodiment problem with artificial agents.

Notes:

Western psychology was heavily derived from Descartes until Darwin. Darwin’s influence likened humans to animals (which were always cast as below human in Cartesian reasoning), and triggered the behaviorist movement. Gestalt psychology came out of or alongside that. Vygotsky was a scholar of the Wundt school, which also came from the behaviorists, but argued for an introspective method (as opposed to the behaviorists who were much more external). Both behaviorism and the Wundt school argued for a stimulus-response methodology, which has remained influential in modern psychology.

Vygotsky aims to develop a comprehensive theory of psychology, that can reason about higher level mental functions, as opposed to the behaviorism, which is specifically oriented towards lower level functions. He notes that culture is important to psychology, and looks toward development as a methodology. Development, though, is more than just the process of maturation, but a complex suite of events that includes maturation and learning.

Tool and Symbol in Child Development:

Vygotsky seems to be arguing that the behavioral model is insufficient to explain ongoing developmental processes. Specifically, early development makes use of “pracitcal intelligence,” which makes use of the environment, tools, and by extension, language to serve as aids. These things are all instrumental and work to augment practical intelligence.

Childrens’ speech is used as a constant narration that operates in parallel with activity. This is (I think) the sort of egocentric/autistic speec described in early development. Speech is instrumental in reasoning and modeling the world and behavior. What is notable here is that this speech is used instrumentally to forma sort of narrative underpinning of the world, and cements the strength of the linguistic model of consciousness.

Planning, as a component of thought, originates in inner/social speech preceeding an action. Speech is also social, and interaction with others is necessary for the interaction with objects. Development of planning is socially dependent. This is a great ground to critique the models of planning found in symbolic AI.

The Development of Perception and Attention

Visual perception is limited in animals (even in apes). The key element to human perception is the ability to transform visual perception into language. The idea is that visual information is transformed into signs, via language. Thus, language is necessary for the process of siginification. Attention is a mechanism for controlling and directing perception and awareness.

Mastery of Memory and Thinking

Sign usage is a mediated form of thought. Mediation is also a very gradual process to incorporate into thinking. “We have found that sign operations appear as a result of a complex and prolonged process subject to all the basic laws of psychological evolution. This means that sign-using activity in children is neither simply invented nor passed down by adults; rather it arises from something that is originally not a sign operation and becomes one only after a series of qualitative transformations.” (p. 46)

There is a complex relationship between memory and thought. In early childhood, thinking means remembering. This references the heavy associative nature of thinking, but later, individuals are more “logicalized”, that is, information is associated through systems of signs, so remembering is more mediated/augmented. The function of memory extends out into the environment. Individuals use environmental cues to trigger associative memories and contextualize thought.

Internalization of Higher Psychological Functions

Tools and signs are both mediating. They provide a level of indirection in everyday interactions. However, tools are externally oriented and symbols are internally oriented. Develpment seeks to internalize interpersonal processes into intrapersonal ones. The child’s interaction with others becomes a way to think about the world internally. This is internalization of signs, but it comes paired with an externalization and extension of cognition into the environment.

Problems of Method

Vygotsky is rejecting the stimulus-response method, originally developed by behaviorism, from higher psychology. He claims that is simply inadequate for addressing higher functions. He notes that it is unidirectional and reactive (after Engels). This suggests that there can be complex interactions with the environment in cognition. Vygotsky’s goal is to instead look at processes and not objects, and instead wants the method to focus on development as a general tool for understanding.

Interaction Between Learning and Development

There is a complex relationship between learning and development. There are several competing theories on how the two relate: One is that the two are totally independent (Piaget), the second is that the two are equivalent (James), and the last is a combination of the first two, that the two processes influence each other (Koffka). Development here is the natural process of maturation, while learning is socially based gaining of knowledge. Vygotsky’s conclusion to this is that contrary to intuition, development follows learning.

The argument is made that learning is partly a social process, and that it is socially supported. Do not look at the child alone, but rather look at the child in the social setting, with others and the environment as support. “Over a decade even the profoundest thinkers never questioned the assumption; they never entertained the notion that what children can do with the assistance of others might be in some sense even more indicative of their mental development than what they can do alone.” This is the Zone of Proximal Development. This idea challenges the notion of solitary performance that is still used in evaluation and test-taking to this day.

Imitation relates to the internalization of cultural/social practices and values. For example, playing house, cowboy and indian, other sorts of games. Imitation also serves as a basis for metaphor and supports the neural basis for the establishment of meaning.

The Role of Play in Development

Play creates an imaginary situation, and seems to emerge when the child experiences unrealizable tendencies. Play satisfies some unrealizable desires. It requires rules to constrain its imaginary world. “Just as we were able to sho at the beginning that every imaginary situation contains rules in a concealed form, we have also demonstrated the reverse–that every game with rules contains an imaginary situation in concealed form. The development from games with an overt imaginary situation and covert rules to games with overt rules and a covert imaginary situation outlines the evolution of childrens’ play.” (p. 95-96)

Play and symobls depend on symbolic abstraction. An early child cannot differentiate visual truth from meaning. Later, meaning can be separated, lies told, and objects imagined. When a child forms the capacity to internalize symbols from the environment, he also gains the ability to project those symbols onto objects. Thus, the wooden stick can become a horse. This is the same process that is used to imbue meaning metaphorically, and can be extended beyond the realm of play and games, but it is interesting as a point of origin.

Overview:

This is a collection that is primarily about developing formalizations of communication. The essays are primarily by AI scholars as well as a few notable linguists and philosophers. The entire suite of essays though does presuppose the idea that goals and plans are intrinsic to cognition and communication. There are a number of logical formalizations of communication and intention, but each of these requires a propositional model to account for knowledge in the world. This requirement is a failing from my purposes, but the approach and methods of formalization seem like fertile ground.

What I am trying to get out of this reading is a way to model knowledge and communication that is embodied (at least to the agents), situated, relational, social, and performative.

Notes:

Philip Cohen, Jerry Morgan, and Martha Pollack: Introduction

On the very first page, the editors describe a great example of communication, that is very hard to model.

Let us begin with an example. With the Wednesday advertising supplement in hand, a supermarket patron approaches the butcher and asks “where are the chuck steaks you advertised for 88 cents per pound?” to which the butcher replies, “How many do you want?” (p. 1)

This example is difficult because there is a lot of things implicit in the communication, and while the literal meanings of the responses are barely connected, the response is perfectly natural and normal. The editors follow this up with a little analysis, but I disagree with their conclusions. They suggest that this is about communication of intentions and realizing what the other agent wants and acting to satisfy this need. While I agree that the butcher in the example must know at some level what the customer wants, I think that is a poor candidate for what is going on in his head.

I would say that this is better explained by sociological methods, especially the notion of scripts. This is an instance of the “shopkeeper” script. A person walking by is attributed the role of customer, and the butcher is the shopkeeper. This attribution of roles is enforced by the environment and situation, but also more importantly by ingrained cultural experience. Everyone knows what shopkeepers and customers do. Shopkeepers and customers know what to do, they do not need to analyze each others’ intentions, but simply perform their roles.

The authors outline six questions which are to be addressed by the rest of the book:

1. Meaning: What is meaning? Is there a notion of meaning that is appropriate for all expressions?
2. Composability: How are meanings of complex expressions composed of the meanings of their parts?
3. Action: How does speech perform actions beyond the mere act of saying?
4. Indirectness: How can a sentence convey more information than the literal meaning?
5. Discourse compositionality: How does the interpretation of discourse reveal more than the sum of the meanings of the individual sentences?
6. Communication: What is communication? How are linguistic and nonlinguistic communication related? How is communication related to meaning?

These questions are important considerations for any theory of communication and would be good to address for future endeavors.

Michael Bratman: What Is Intention?

The goal here is to define and understand intention. THis concept is something that ties together mind and action. The idea is that intention is a sort of intermediary buffer between thought and action. Before perfoming actions, one might form intentions to perform higher goals, and this would direct the performance of the actions.

A challenge comes with distance, and this is explained through the rhetoric of planning. The problem is explained with a nice example: “Suppose I intend today to drive over the Golden Gate bridge tomorrow. My intention today does not reach its ghostly hand over time and control my action tomorrow; that would be action at a distance. But my intention must somehow influence my later action; otherwise, why bother today to form an intention about tomorrow?” (p. 16) Bratman poses a trilemma of issues posed by intention at a distance:

1. Distant intentions are metaphysically objectionable, because they involve action at a distance.
2. Distant intentions are rationally objectionable, because they are irrevocable.
3. Distant intentions are a waste of time.

Bratman’s solution to this mess is the idea of planning. One has a hierarchy of intentions (or goals), and this hierarchy may be revised in context of changes in state and information. I think that a similar, but intrinsically different conclusion can be drawn. Instead of plans as mental constructions, intentions become part of an intrinsic state, essentially, intentions become roles. This is the sort of approach specified by Clancey et al in Cognition and Multi-Agent Interaction.

This idea gets hinted at some more later on: Bratman relates planning to action. ” (p. 19) I have a plan to A only if it is true that I plan to A.” This distinction is subtle, and it could also be used to relate planning to identity. For example: the plan, “I am planning to graduate” relates the identity, “I am a student.” This idea is touched on when Bratman connects intention to the idea of a “pro-attitude.” The function of the attitude can be examined as “I am someone who intents to A,” or, alternately, “I am conducting myself as to A.” This can be used to bridge intentions and roles.

Bratman outlines some issues that connect intentions to beliefs. With relation to each other and general knowledge, intentions must be consistent, coherent, constrained, admissible, and stable, etc. All of these hinge on the matter of beliefs and knowledge. The constraints and rules for determining intentions resemble the way that Soar considers operators.

At the conclusion of Bratman’s paper, he is trying to address the problem of the “package deal” when actions toward intended goals can have unintended effects. He uses an example that is rather disconcerting, though. The example is a wartime situation involving two bombers who may or may not bomb a school full of children (actually, it is whether to bomb a munitions factory that is next to a school full of children which would suffer collateral damage). This example is intended to illustrate the complexities of decision making, but wholly leaves out the charged emotional element. It assumes a totally rational process, and in this case, curiously, the rational agent decides to bomb the children. This is a key example of the danger of the emphasis on rational planning.

Philip Cohen and Hector Levesque: Persistence, Intention, and Commitment

The goal of this paper is about the “rational balance” of beliefs, intentions, and actions. The focus is on an AI controlled (or modeled) rational agent in a wold with other agents. While the formulation here has what I would consider to be an undue reliance on the dogma of rationality, the authors do emphasize a social element to action, and their formalization requires the existince of other agents to work.

The authors define a seven-point theory of intention. In this framework, an intention is essentially a persistent goal.

1. Intentions pose problems for an agent. The agent must determine a way to achieve them.
2. Intentions provide a “screen of admissibility” for adopting other intentions.
3. Agents “track” the success of their attmepts to achieve their intentions.
4. To intend p: The agent must believe p is possible.
5. To intend p: The agent does not believe he will not bring about p.
6. To intend p: Under certain conditions, the agent believes he will bring about p.
7. Agents need not intend all the expected side effects of their intentions.

Outlined here is a thorough and rigorous model of a logical formalization of action, beliefs, and intentions. The manner of expression is through propositional predicates. For example: (GOAL x p), (BEL x p), (HAPPENS a), etc. Over the course of the paper, more types of propositions and logical constructs are added. The formalization here is concerned with a precise logical modeling of the world. A problem, though, is that it can be used to describe agent models (the space of beliefs held by the agent), but all of these are absolute and literal. They are independent of perception or context.

One element in this analysis is the aim to formalize the rules of intentions described by the authors. This is successful, but it exposes the flaws and weaknesses in the original model. Here, issues such as procrastination and dedication are explained by complex structuring of beliefs and intentions with respect to time. Instead, I think that procrastination and dedication are based much more strongly in situation, personality, and emotion. The awkwardness and complexity of modeling some supposedly simple emergent qualities of behavior suggests that there is a failure to consider something important within the original model.

Martha Pollack: Plans as Complex Mental Attitudes

Pollack aims in this to present an alternative approach to planning, differing from a number of original models, specifically STRIPS and NOAH, which derive from Allen. Planning frameworks, which depend on graphs, heirarchy, and decomposition flounder because of several reasons. The one that Pollack is focused on is the human disconnect. It must be possible for a planning framework to handle invalid plans, ones that would be successful given the agent’s beliefs, but cannot occur because some of those beliefs might be false.

An element here is that, based on observations, plans may be inferred and analyzed. This inference is an important topic that could (and will) see more attention. Pollack is using an example of communication wherein one agents is trying to reach a friend: A: “I want to talk to Kathy, so I need to find out the phone number for St. Eligius.” S: “St. Eligius closed last month. Kathy was at Boston General, but she’s already been discharged. You can call her at home. Her number is 555-1238.” The nature of this communication is tricky, and I would argue that it is socially defined, but it would be hard to say that it is part of a formal script. I would agrue that it is based on experience, convention, and practice, but these are hard to formalize.

To deal with these situation, Pollack describes the idea of an explanatory plan, or an “eplan”, which gives a solution to the inference problem, but is vague. Other than by direct inquiry, how does one recognize a statement or question as an explanatory plan?

Henry Kautz: A Circumscriptive Theory of Plan Recognition

This essay investigates plan recognition. The approach here depends on keyhole recognition, which assumes more complex knowledge. The analysis here is done by structruing a logical formalized representation of observed events. Observation and analysis of plans must work according to several methods, among which are entailment (deductive) and closure (inductive). These rules define how plans may be infered from observed actions.

Kautz moves into a formal representation of events and plans. Plans are broken down into components, agents (participants), constraints (temporal and equality), and preconditions. What follows are then a few theorems on how models relate to events and fact determination. Communication is expressed through this logic as a collection of predicates: indirect requests, direct requests, inquiries, etcetera. Kautz then concludes with an algorithm for the recognition of plans and intentions.

This thought is really perplexing, because it subtly suggests that human intelligence consists of performing this algorithm.

An Aside

Much of what is going on here assumes a sense of objective truth. It assumes that minds and communication occur in pure, abstract (or purely representative) methods, ignoring cultural or social context and influence.

C. Raymond Perrault: An Application of Default Logic to Speech Act Theory

This section is on modeling and understanding speech acts. There is a good review of types of speech acts derived from Austin (1962), of the types of speech acts: locutionary, illocutionary, perlocutionary. It is interesting that the author acknowledges the performative element here. Illocutionary acts are rich with variable meanings, for instance, irony, sarcasm, lies, etc. The focus in this essay is on the application of logic to illocutionary acts, and understanding how they succeed or fail.

The logical structure used is derived from predicate elements : Kxp, Bxp, Gxp, for knowledge, belief, goals. The structure outlined enables complex formalizations of reflexive social knowledge and goals. An important element of communication here is the cycling of knowledge. For instance BxByBxByp. This analysis allows for an easy formalization of certain structures: “I want x to know that I want p.” However the recursive cycles are very awkward. What follows from here is leading to a non-monotonic logic where beliefs can change with changes in knowledge.

One of the issues and challenges with this model is that it bases all speech as propositional. If the performative element of speech is analyzed, it introduces complexity and challenges. Speech operates with scripts (that may be parameterized) and rituals. Speech may also have non propositional functions, like “annoy.”

Daniel Vanderveken: On the Unification of Speech Act Theory and Formal Semantics

This essay seeks to connect the theory of illocutionary acts (from Searle and Austin) to truth semantics (Frege and Tarski). The method is to separate speech acts into several types: declarative (make assertions), imperative (give directives), interrogative (ask questions/find knowledge), exclamatory (express state), optative (express wishes). I would argue that this is the right idea, breaking speech down into its functional elements, but speech acts in conversations really can have a lot more than single functions embedded into them.

Vanderveken notes that illocutionary acts are more than propositional content, but also contain other sorts of qualities. He explains these as being a set of qualifiers (forces) on the statements. These relate to the performative aspect of speech, but he does not go as far as suggesting that qualifiers might convey other information in of themselves.

An important thing to note in this discussion of general semantics is that it assumes literal meaning. This is especially interesting in the context of Lakoff, as well as Rumelhart, who both challenge the idea of literal meaning.

Philip Cohen and Hector Levesque: Rational Interaction as the Basis for Communication

The idea here is to examine illocutionary acts as instances of actions. This is aimed to be a reaction against some existing theories. The unity of speech and action has a strong psychological precedent, especially from Vygotsky. However, this is extended from the rational action framework in the earlier chapter, whcih fails to account for other contextual elements.

The authors describe a world model that uses the propositional approach (looking at cleaning floors and opening doors). The formulation of statement predicates is interesting for further use, but seems incomplete in light of some of the underlying problems, for instance, the dependence on literal statements. Abstracting to an action model can cupercede this. I think the speech actions should be more emotionally/socially oriented. For instance, support, praise, etc.

This theory and its complex logical formulation of a helpful agent is fascinating but hugely complex (p. 243). It seems like this is overboard in explaining the simple social logic of requests. A much better, simpler, strategy would be to work from another model of interaction based on sociology or cultural anthropology.

John Searle: Collective Intentions and Actions

Searle forms a complex analysis of intention, as relates to plans and sub-plans. Intention propagates to sub-tasks and gets messy. It seems like this fails when compared with embodied interaction and knowledge. (Recall Hubert Dreyfus on phenomenology). The collective analysis here is atomic. It does not relate to extended or proximal cogition, or social practice. The world being presented here is one in which every human agent is alienated from context and must decipher precisely every intention and goal before communication or action. While Searle is a critic of AI, he ascribes here the disembodied nature of reason held by AI scholars.

Herbert Clark and Deanna Wilkes-Gibbs: Referring as a Collaborative Process

This, the last chapter in the volume, makes an interesting and very deep analysis of conversation and reference that picks up on many ideas neglected by the other authors in the book.

The essay looks at conversation as collaborative, in which meaning is co-constructed. A comparison is immediately made to sociology. The authors note how conversation is a very flexible, interactive thing, where ambiguity is made and then clarified or contextually understood. This also, notably, is very tied to the environment, not abstract propositions, but world referents.

The authors describe four assumptions of literary models that are destroyed in real conversation. These are tacit idealizations that ascribe to communication formal elements of written speech which is simply absent.

The literary model makes these tacit idealizations. (1) The reference is a proper noun (for instance, Napoleon, King George), a definite description (this year, the man with the moustache), or a pronoun (he, this, they). (2) The speaker uses the noun phrase intending to the addresse to be able to identify the referent uniquely against their common ground. (3) The speaker satisfies her intention simply by the issuing of that proper noun phrase. And (4) the course of the process is controlled by the speaker alone.

The authors reference Goffman (!) and claim that understanding is developed interactively. Communicated statements are followed with “continuers.” These enable conversation continue, and confirm that meaning is shared. This is based on practice and performance.

What is also neat about this analysis is the fact that the discourse in the experiement is entirely metaphorical. The experiment consists of having the subjects be given a grid of tangrams (making up arbitrary, but evocative shapes) and giving one subject a list of tangrams to instruct the other to recognize. The shapes are being refered to by imagery, analogy, and association. All of these are embodied and none are propositional (at least not in the sense of the other essays).

Recognition, and general communication, is a process. It involves various potential reactions, which have active knowledge changing, belief changing, and linguistic elements. These examples are: acceptance, rejection, postponement. Most notably is that each of these responses may be considered an action.