I wrote down some notes from a recent meeting for Janet Murray’s narrative project studio, SNAPS. I think there is a web link for this somewhere, but I can’t seem to find it. During the meeting, we discussed several cases of narratives that are especially interesting from the perspective of digital media. Much of the discussion led to some reassurance of the idea that narratives are really about underlying models. Additionally, there was discussion of the idea of the essence of narrative, or the quality of “storyness” that makes narrative satisfying.

The primary conclusions that I drew from the meeting where this: Narratives work to describe a story world, and underneath the story world is a model that drives the causal logic of events in the world. The model in these cases is generally not a simple reduction or maxim, but rather a system that allows everything to ultimately make sense. However, the reader or viewer is exposed to the model not by outright declaration, but by witnessing it unfold and comparing it against predictions or expectations. This is part of appeal when Keith Oatley says that narrative is really just a simulation that runs in our minds. It is not just a simulation, but a predictive effort.

I think that stories are most engaging when the reader/viewer is trying to put the pieces together and things to not add up until the end. This end is usually a catharsis or a dramatic revelation or a climax or some ultimate resolving moment. I think what is happening at these moments is that the underlying model finally snaps into place, and its implications become more fully known. There are many approaches to looking at how a climax works, from an aesthetic perspective. Aristotle is an obvious example. There is an emotional, embodied, or vicarious response that might occur. These perspectives are important, but looking at it from the perspective of models brings in a funny cognitive dimension to an emotional experience. I would say that no one of these elements is superior to the other, but they each address the matter of the reader/viewer experience in a different way.

In many cases, when narratives are adapted to digital media, there is an approach which aims to strongly preserve the form of the original work. Some approaches will express the work in a manner that is navigable either spatially or temporally, and from different perspectives. This exposes the story world much more, which can communicate the model, but it does not allow the user to reach down into the model and see what it is about. To do so may not be possible or even desirable in some cases, but I think it would be ideal in others.

I am going to give several examples which were discussed in the meeting and relate some of the above aspects about these particularly.

The Norman Conquests: The Norman Conquests is a play by the British playwright Alan Ayckbourn. This is sort of a triptych of three different plays, having the same cast of characters, and taking place in different parts of a house during the same weekend in diegetic time. The curious bit is that when one character goes off stage from one play, they will probably make an entrance onto another. Each play is meant to stand on its own, but the user experience is enhanced by seeing all three of them, because the elements of the “underlying story” are understood more fully when seen from a complete perspective. Essentially, each play is a view of the same scenario that is taking place. What is happening across all three plays could be described as a story, but might be better described as a story world. A story takes a world and describes a representation of that through narrative. If one were to construct a story out of the entire Norman Conquests, it would be a new construction, as opposed to something that was already there.

Two Towns of Jasper: Two Towns of Jasper is a documentary about the horrifying racially motivated murder of James Byrd in Jasper, Texas in 1998. The documentary is unique in that, to explore the depth of racial division within the town, the filmmakers (one black, one white) collaborated by separating into segregated teams and following the trials of the murderers and interviewing the white community and black community separately. The filming is unique in its use of methodology to illustrate very precisely the depth of racial division. The resulting documentary forms a consistent narrative, but has a disjointed feel. There was an IDT project to make an interactive version of the documentary, which would allow the viewer to follow different characters, and navigate the places in the town spatially. This approach also provides a way to navigate the world as the events around the trial unfold. The essence of the film is about division. Division is the foundation of the model that controls this world.

Bertolt Brecht: Brecht is most well known for the Threepenny Opera his only commercial success, but this was a play he considered his worst failure, as it failed to communicate his message. Brecht was a Marxist playwright who worked to spread the model of Marxism through his plays in Germany before the Nazis rose to power. His plays were in the tradition of modernist theatre, and often explicitly denied the audience a clear climax or resolution, but attempted to instead communicate the idea that the social world taking place in his plays was really the same as the world of Wiemar Germany. The world of Brecht is filled oppression and squalor, and the working people subjugated by the bourgeoise. What is interesting about these plays (I don’t have a specific one to refer to) is that they do not provide a climax in the context of the narrative itself. Instead, the climax is rather when the epiphany is made by the audience that the world of the play is no different from the world of the audience, and they are incited to rebel. What is also the case is that the plays are instantiations of the model of Marxism. Greg Costikyan has written a fascinating description of a role-playing game based on the ideology and aesthetics of Brecht, called Bestial Acts.

Speculations About Jacob: Another example of German literature comes from Uwe Johnson: a modernist and experimental novel called Speculations About Jacob (I can’t find a Wikipedia article, sadly). The book is about the character Jacob, who is suddenly killed in the beginning by a train. It is set in East Germany, and is filled with the sort of fractured portrayals that echo the divided nature of the country. The book paints a spotty and incomplete picture of Jacob, his life and surroundings. The eponymous speculations are what led to his death and why. The narrative approach of using incomplete information to convey a story world is not unique, but helps convey a model of a world that is made only partially visible or knowable. Interestingly, the book also transitions suddenly from one mode of narrative to another: a character’s thoughts may suddenly turn into a conversation. Not only is explicit information missing, but pieces of the connective logic or framework are absent as well, leading to ambiguity. The reading also denies a cohesive resolution. Instead, it is an essentially open work (in the Umberto Eco sense), leaving the readers to finish the construction of the world or model themselves.

One of the problems of being a PhD student with an extremely broad and vaguely defined field is that it is hard to introduce yourself. Someone will ask me “what do you do? What are you studying?” and then have to wait through the pause and look of anguish on my face as I try to figure out the best way of explaining my work to the individual in question. No more! From now on, I will respond, boldly, immediately, with the following phrase “simulating fictional worlds”.

A long explanation is that I am interested in simulation, mental models, AI, games, and communication and expression through software. Computers can represent systems formally and simulate them, and this is a way to communicate ideas and models of how the world works: you build a model and simulate it. Some cognitive scientists might argue that meaning is inextricably bound to models, and what better way to share meaning than through demonstration? A lot of the work that I do from day to day is also tied in educational software, which operates to expose and communicate and teach very specific models for approaching a domain. However, I also think that games are a powerful tool for the demonstration and enactment of models. Games expose (or conceal) meaning through representing systems, and can do so intricately and playfully. The interest in fiction comes from a desire to expand the simulative capacity of games, and open up social worlds in addition to the worlds demonstrated by most games nowadays.

The expression “simulating fictional worlds” gets under that mess directly and quickly. The surface meaning is relatively easy to parse or explain, but if someone wants to know why I am interested in this stuff, then I can get into the background.

Overview

Fine’s book is one of the first seminal studies of the culture of roleplaying games. The work is conducted as an ethnography, and was probably the original study to examine roleplaying as a legitimate culture. The content of the investigation explores the social structure, the creation of meaning, the frames of interaction, and the types of people who enjoy these games. The study was conducted in the 1970s, and as a result, much of the culture seems very alien and peculiar, especially to one familiar with roleplaying only in relatively recent times (in my own experience, since the late 1990s). I find that much of the hidden potential that Fine hints at has come to some fruition, though not completely.

Notes

Fine is a sociologist and this work is an ethnography. Note the goals here: “First, to analyze and describe a contemporary urban leisure subculture. Second, to understand the the development and components of microcultural systems and explore their relationships to the structure of the groups in which they are embedded. Third, to understand the process by which people generate meanings and identities in social worlds.” (p. 1) This last point is the most remarkable about roleplaying, but to get at it, it is necessary to delve into the structure and form of the games and the culture that plays them.

From the preface, Fine describes an interesting conflict in the study: balancing work and play. One one hand, studying a leisure culture might be considered frivolous to those who consider themselves serious sociologists, and conversely, the culture itself may find that the formal study serves to sap the fun or lightness out of the play in question. The success of the study depends on the ability to navigate between these conflicting perspectives. The concern is also particularly relevant to those of us studying video games.

Fine also begins by looking at the history of roleplaying, specifically by investigating war games and the culture that surrounds them. War games connect to simulation games, which, in this context, are frequently used as educational or management tools. The role of simulation games is to encourage the players to see things in terms of positions, not persons. This distinction carries over to the abstract function of player versus character.

On exploring player culture and the role of violence and sublimated aggression within the games: Fine describes a number of situations where players partake and glorify violence in game, but these behaviors are also blanketed with excuses. Some excuse violence by arguing that the game allows the players to simulate and get their hostilities and aggression out within the context of the game. Gary Gygax argues (from an interview) that players, having played these games, know better what violence and war is about, and would thus consider real violence unacceptable. This thread is notable because it compares again to the arguments for and against violent video games.

There is a note on the common interests of the roleplaying community, and Fine describes these as the components of fantasy role-playing gaming. This resounds with Mackay’s findings as well. There is a list of bullet points of interests which are described as relevant: wargaming, fantasy literature, mythology, history, physical science, mysticism, Society for Creative Anachronism experience. A thing to note about these is that many of them are focused around the ideas of model-construction.

On reasons why people play games: there is a large category which is escapism. One of the special items in this category is the idea of escape from self. This idea connects to role-experimentation and identity play that is discussed by Turkle.

Fine also notes, with continuing discomfort, the notable absence of women from fantasy role-playing culture. One note is that women tend towards social settings in play, so, while role-playing would seem to be a natural passtime for female players, there is an emphasis that role-playing is a sublimation of aggressive physical play, which is a sterotypically male developmental pattern.

On the nature of the constructed fantasy in these worlds, Fine notes that there are several “folk ideas” or values that are present or embedded in game worlds:

• Unlimited good. This goodness is in the sense of material or other rewards. There is always infinite possibility for reward in dungeons.
• Oppositional nature of the world. The worlds are framed in the context of good versus evil in clear and stark terms.
• Western morality and culture is identified as good, whereas anything else that is deviant or outside can be cast as evil.
• Prevailing virtue of courage. Courageous behavior is met with increased rewards. Luck is seen as part of it, but success is rationalized with courage.

There is a paradox of reason and logic in fantasy worlds. Fine discusses several layers of logicality: there is realism, where the game is held to certain standards of realistic logic. The example given with this is in the portrayal of medieval worlds. Logic tends to relate to the coherence of the game according to its rules and logical flow. The primary issue at stake is consistency. As long as the realism and logic are consistent, then the game flows appropriately and is not frustrating to the players.

Description of the world setting: The Empire of the Petal Throne, by M. A. R. Barker. The appeal of this setting, as described by Fine seems to be the discovery of the exoticism of the alien world. The appeal of this seems like a social MMOG, where there is a whole culture to learn and be fascinated and surprised by.

Fine references Erving Goffman’s technique of Frame Analysis to examine the styles of interpersonal interaction within the roleplaying games. He also references Alfred Schutz. The essential aspect at stake in this analysis is the idea of engrossment. A frame is a level of interaction in which there is sufficient engrossment. However, the difference between Goffman’s frame analysis and what is conducted in role-playing games is that the engrossment is continually oscillating in the games.

Frames become relevant in managing knowledge. An example given is how game masters aim to keep things secret from the players, to enforce that their characters will remain ignorant, and the players will have the same knowledge as their characters according to a given scenario. Occasionally, GMs try to conceal the rules (specifically numeric probabilities and the statistics of monsters), so that players will not know what to expect. “Some referees extend their concern with the degree of players’ awareness and suggest that, as in ‘real life,’ characters should not know the probabilities in the game world (the rules of the game with their percentages of success). This secretiveness–keeping the player ignorant so that his character will be ignorant–adds to the verisimilitude of the simulation according to some referees.” (p. 191)

On playing characters, there is stress between role-playing and game-playing. This relates to immersion and motivation. When compared to later studies (especially Mackay), the position of pure game-playing seems much more accepted here. Game playing treats the experience as having concrete goals, so the play can be directed around achieving, sometimes even competitively, those goals. This ties back into the way that digital role-playing games, specifically MMORPGs function. In these contexts, the fantasy is a backdrop for the game itself.

Fantasy role-playing games involve a communal construction of culture. Symbolic interaction enables the construction of meaning. The worlds are socially constructed, which means that themes and values are shared by the culture. In personal fantasy, the themes may be idiosyncratic, but in a social construction, the values have been established and are enacted by the group, and the fantasy thus becomes a shared creation. Other social groups construct meaning, but in role-playing the value is fantastic, imaginary, and explicitly formed.

Notes

Conceptual change is introduced as an idea relevant for understanding learning. This is referenced as corresponding to *representational* changes. Representation in this context corresponds to the mental representation of concepts. This means that here, representation is more closely aligned with the establishment of a model (that is, the representation is the model in one’s head), as opposed to a physical representation of the model itself. This distinction is one that is important in discussing the relationship between mental modeling and simulation.

The challenge with articulating conceptual change is understanding the nature of how concepts are held mentally, and the mechanisms by which they are changed. The distinction given is between scientists and children, who are different in the sophistication of their goals and metacognitive strategies. Nercessian describes problem solving as a means by which to articulate how both groups might revise their conceptual models.

Describing concepts: “Concepts provide a means through which humans make sense of the world. In categorizing experiences we sort phenomena, noting relationships, differences, and interconnections among them. A conceptual structure is a way of systematizing, of putting concepts in relation to one another in at least a semi – or locally – coherent manner.”

On mental models: “Loosely construed, a model is a representation of a system with interactive parts with representations of those interactions. Models are representations of objects, processes, or events that capture structural, behavioral, or functional relations significant to understanding these interactions. What is required for something to be an instance of model-based reasoning is that: 1) it involves the construction or retrieval of a model; 2) inferences are derived through manipulation of the model; and 3) inferences can be specific or generic, that is, they can either apply to the particular model or to the model understood as a model-type, representing a class of models.”

Nersessian describes the mind (the cognitive apparatus) as being capable of “modeling, analogy making, abstraction, visualization, and simulative imagining.” She explains that science has leveraged this and incorporated these approaches into the scientific process (and the scientific method, even). This poses the modeling issue as something that is naturally disposed towards science. This is important and valuable, but also conceals the fact that it is used for other types of reasoning as well, particularly social and creative applications.

Nercessian extensively discusses the history of the theory of mental models, and pays attention to the efforts to equate these models computationally or symbolically. Herbert Simon is specifically mentioned, as an example of someone who applied modeling to computational reasoning systems.

One of the first to develop the theory of models in detail is Craik in the 1940s. The central pillar of Craik’s theory seems to be in mental simulations of models, “reasoning about physical systems via mental simulation of analog representations.” Nersessian mentions that simulation is purportedly developed for navigation within an environment (for instance, a rat in a maze simulates the maze in its mind). Due to human linguistics, mental simulation would thus have the capacity to simulate from language.

Reacting to the idea that thinking is rooted in language, Nersessian suggests that instead, language enables certain narratives, of which individuals may form mental models. Texts describe systems and structures, and these are later manipulated as models and used for reasoning. These are called “discourse models” or “situation models”. This idea seems PERFECT for relating to the narrative adaptation work. This descends from Johnson-Laird, 1989, p471. There is a significant bibliography of sources that discuss how readers form models of texts. The last conclusion reinforces the notion of embodiment and the role of embodiment and perspective within reading a text:

“A number of experiments have been conducted to investigate the hypothesis that in understanding a narrative readers spontaneously construct mental models to represent and reason about the situations depicted by the text (Dijk & Kintsch, 1983; Franklin & Tversky, 1990; Johnson-Laird, 1983; Mani & Johnson-Laird, 1982; McNamara & Sternberg, 1983; Morrow, Bower, & Greenspan, 1989; Perrig & Kintsch, 1985; Zwann, 1999; Zwann & Radvansky, 1998). Although no instructions were given to imagine or picture the situations, when queried about how they had made inferences in response to an experimenter’s questioning, most participants reported that it was by means of “seeing” or “being in the situation” depicted. That is, the reader sees herself as an “observer” of a simulated situation. Whether the view of the situation is “spatial”, i.e., a global perspective, or “perspectival”, i.e., from a specific point of view, is still a point of debate, though recent investigations tend to support the perspectival account, that is, the reference frame of the space appears to be that of the body (Bryant & Tversky, 1999; Glenberg, 1997b; Mainwaring, Tversky, & Schiano, 1996).” (p. 24-25)

I think some degree of this relates to abstract problems or narratives (for instance talking about some blocks of various colors and describing where they are in relation to each other), but it is wholly sensible that these ideas be applied to other conceptual areas, especially in fiction.

Looking at models as they relate to artifacts: People use prosthetics to aid in thinking, demonstrative artifacts that help externalize mental information. One suggestion posed here is that people (specifically scientists in her example) do mental manipulation that interacts with the observed visualization. This process serves to construct a mental model that is *constrained* by the visualization. This process involves a certain coupling between internal and external representations of the model. It also suggests a capacity for the model to be bridged to accommodate other things external to it.

Nersessian goes further to discuss the format of working memory, and propose that the format of the information is 1) modal, and 2) embodied. An example of this derives from 3d positioning, which generally maintains an egocentric coordinate system and perspective. A second example is in the representation of concepts, which Lakoff and Johnson describe as something that is literally part of the brain. The researcher described in detail is Barsalou, who argues that mental representations are perceptual, and man cognitive processes are re-enactments or simulation of perceptual states.

Preface

Phenomenology is about essences. This idea seems to be potentially relevant for thinking of essences of meaning to be carried over in translations or adaptations.

Science is a separate order of expressing the world. So, Merleau-Ponty wishes to return to things-in-themselves. So, it is necessary to return to a world that precedes knowledge. Must use describing, instead of analysis or construction. Perception is the background on which all acts are projected.

There is no inner mind, the mind is in the world, and you can only know yourself in the world. Contradicts Descartes argument by insisting that true cognition must be anchored in the world: “The true Cogito does not define the subject’s existence in terms of the thought he has of existing…” (p. xiv)

Consciousness is consciousness of *something*: it rejects that contemplative thought is sufficient. Consciousness is necessarily what connects individuals to the world.

We are condemned to meaning (allusion to Sartre) (p. xxii). There is no such thing as original choice, namely freedom as defined by Sartre (Moran p. 396). We are in a world of meaning, so any experience takes on significance and history. Phenomenology is about having its own foundation, without external things, but like artistic expression, brings truth into being. (?)

Empiricisim denies meaning? Merleau-Ponty wants to avoid the observational approach, but understand the intimate, reflexive relationship between the body and the world. Defining a difference between cause and reason. Empiricism seeks to uncover causes, which reduces, but denies meaning.

Chapter 3: The Spatiality of One’s Own Body and Motility

Merleau-Ponty opens by exploring some aspects of bodily sensation and comprehension. Bodies are comprehended in space, and in relation to each other. Descriptions and terminologies such as “organ” reduce the body into independent disconnected parts. He uses the term “body schema” as a device for collectively relating the wholeness of bodily experience.

What is important is space, not in terms of position, but in terms of situation. (p. 115) Merleau-Ponty wishes to quell the sense of space as composed of points (in the Cartesian sense, imaginably), especially as this orients and imposes values and external objectivity on the situation. Nonetheless, we can imagine reconstructions of coordinates in relative terms (modern physics is especially attentive to this).

Merleau-Ponty spends some time exploring the interactions and self-perceptions of a patient who is “psychically blind” that is, cannot understand the “body schema” without visual reference. This patient requires certain extensive gestures to be able to position himself blindly in space, and cannot distinguish tactile sensation in different parts of the body.

Note: Additional studies have been made of this sort of thing extensively. Eg, “The Disembodied Lady” in Oliver Sacks, “The Man Who Mistook His Wife for a Hat”. The study involved a patient with an intense feeling of disembodiment, and a loss of “proprioception”. However, in her case, she was still able to act very willfully, relearning to perform normal action without the proprioceptive sense. The feeling was distinctly unnatural, but still did not result in the lack of direction and intent described in the patient later.

Normal individuals can do these things automatically, and this is because sensation is reckoned with. The patient must intellectually grapple with motion and action instead of being able to do it automatically. (p. 126) He cannot convert thought of movement into movement, but can both think and move. This means that the patient lacks something in between, a “motor project” or “motor intentionality”. Essentially, a situation of movement in context of the world and the body.

Diagnosis of patient wrt traditional psychology (or empiricism): this is an example of a sort of reduction, illustration of the patient is a kind of denaturalization of the body to illustrate things we take for granted. Empiricism is lacking an understanding of meaning in that sense. Understanding the difference of the self to an object (as the patient might see it)

On concrete and abstract: In the perception of phenomena (namely movement), movement is either “for itself” meaning that stimulus is an object (the individual moved consciously?), or in itself, being objective within the body, unknown externally (like the hand swatting a mosquito). This difference of in-itself vs for-itself is also called “Greifen” and “Zeigen”. (p. 140) Differentiating these cannot be done if the body is categorized as an object. Vision of the body as in-itself reduces the body to a mechanism, vision mind as for-itself reduces the mind to an abstract symbol processor. So, either extreme is fundamentally lacking, and in reality, the line is blurry. (right?)

Matter and form are connected in phenomenology by a relationship of “Fundierung”, a symbolic function that uses vision and ground. This also relates to the concept of analogy, which people can understand without needing to analyze. This is because in normal thought, things are understood in accordance with the analogy of their function. (p. 148)

More on the patient’s condition: he perceives of things in instances, without external connections, or sense of a whole, “He never goes out for a walk, but always on an errand, and he never recognizes Professor Goldstein’s house as he passes it ‘because he did not go out with the intention of going there'”. (p. 155) This discussion connects with the quote Dreyfus used to criticize AI, that life is subtended by an intentional arc.

Note: The patient is basically (and this ties into Dreyfus very neatly) an extreme example of cognitivist learning. He approaches things much the way that traditional AI might try to. That is clearly unnatural, and we can see that in his behavior.

Further, Merleau-Ponty returns to motility: “Motility, then, is, and, as it were, a handmaid of consciousness, transporting the body to that point in space of which we have formed a representation beforehand. In order that we may be able to move our body towards and object, the object must first exist for it, and our body must not belong to the realm of the ‘in-itself’.” (p. 161) Existence for-it, or for-itself, seems to couple nicely with both intentionality and affordance. There is more: “We must therefore avoid saying that our body is in space, or in time. It inhabits space and time.” Movement is not a matter of memory, but of perception?

On habit: Dreyfus looks at Merleau-Ponty in application to learning of skills, the approach to skills as habits is explained in some detail here. Traditional philosophy and mechanistic theory run into problems when understanding habit. The of habit could be understood as a special kind of understanding or significance. In habits, perception is adapted: The feather in a woman’s cap is perceived as a part of herself, the walking stick of a blind man extends as a part of his perception. Explained: “If habit is neither a form of knowledge nor an involuntary action, then what is it? It is knowledge in the hands, which is forthcoming only when bodily effort is made, and cannot be formulated in detachment from that effort. The subject knows where the letters are on the typewriter as we know where one of our limbs is, though a knowledge bred of familiarity which does not give us a position in objective space.” (p. 167) Further, “To understand is to experience the harmony between what we aim at and what is given, between the intention and the performance–and the body is our anchorage in a world.”

Merleau-Ponty

Moran gives an overview and biography of Merleau-Ponty’s life. Merleau-Ponty was influenced by Christian socialism, and later transitioned into Marxism. He studied works on behaviorism and perception, and criticized behaviorism of suffering from “feigned anesthesia”, in that behaviorism requires that the subject feels nothing. Merleau-Ponty is credited with linking gestalt psychology with phenomeonlogical “being in the world”.

Sartre and Merleau-Ponty had a collaborative relationship the Second World War, in their pursuit of existentialism and phenomenology. The two were considered offering rival interpretations of existential phenomenology.

Merleau-Ponty joined Sartre in the French communist party, and they supported the USSR through its various failings. Later, Merleau-Ponty came to see, via the Korean War, Soviet communism as an imperialist force, which was a break with Sartre.

Eventually, they reconciled, and in the later years of Merleau-Ponty’s life, he did work trying to build connections from phenomenology to structuralism. Additionally published on science and how modern science needs to understand its relation with the world.

Merleau-Ponty was primary mission was philosophy, and his overall goal was to use Husserl to uncover the ‘roots of rationality’. Philosophy is a means of understanding awareness, which of course relates strongly to phenomenology.

A challenge is to uncover pre-conceptual experience (which is done in cognitive science, and learning theory, discovering how people build models). Objective thought (reason?) does not generally acknowledge models as being perceived or constructed though. Furthermore, analysis in this level of understanding draws down to the “irreducibility of the real world”: “The real is to be described not constructed or constituted”. Experience requires a self, and this self is inseparable from the world.

Merleau-Ponty was influenced by Levi-Strauss, and by the non-historicality of structuralism that Levi-Strauss posed. Specifically, Merleau-Ponty saw temporal thinking as interfering with language: “The congealing of temporal thinking into language and concepts acts to fix meanings, to give the appearance of absoluteness.” The ideas of structure and system and language were seen as heavily connected to perception.

Merleau-Ponty is rejecting a linear or single-explanation of history. Instead, time is something that is experienced and lived within, and cannot be seen from the outside.

Levi-Strauss did some work on looking at anthropology from a structuralist perspective, composed of binary oppositions. Ie, raw and cooked, or up/down, light/dark, etc. These could potentially be seen as embodied understandings. It still seems odd in association with phenomenology, but connection can be made via embodiment.

Merleau-Ponty is influenced by the method of Husserl, which is the principle of *reduction*.

The human body is an expressive space, which contributes to human action. Speech is not only the expression of ideas, but it may have power of signification.

Phenomenology lives between two extremes- Cartesian, which understands the mind and understands thought as a fully internal thing, which interacts with the outside world through very prescribed means of perception. This has an internalized means of understanding the self as subject and the world and the body as objects. A second extreme is behaviorism, which eliminates the matter of mind entirely, it comes from the scientific perspective, which looks objectively at how behaviors occur.

Phenomenologists reject the traditional subject, object relationship, and reject the position of knowing things abstractly (which operates against Descartes and follows from Kant), and they reject the objectivity of science and demand consciousness (operating against behaviorism).

The Cyborg Manifesto

Haraway’s early work was on primatology and she studied how conventional western metaphors of gender, race, and class had informed primatology and science as a whole. Her goal is not to undermine science as a whole, but expose its concealed lack of objectivity.

Concerning the Cyborg Manifesto, Haraway is still looking at metaphorical terms. Haraway writes of a post-gender world, and of the cyborg as a tangible concept, imbued with many properties, each revolutionary. Our world, while suffused with technology and slowly attaining a networked character, is still heavily embodied and weighted by 19th century notions of race, gender, and class. It is hard to imagine her world as connected to ours, but using the tool of metaphor, we can make that connection.

Haraway’s work is intended to defamiliarize ourselves with naturalness, and shake us out of the conception that we are natural. Her world is still far from ours, but the loss of natural innocence is one trait that we do have in common with hers. Yes, our world is constructed, in terms of gender, class, etc, and in terms of every aspect of our lives: these are composed by the interconnection of many systems. We, however, are not post. The myth of naturalness is still stiffly ingrained in popular imagery, and is romanticized and idealized. Backwards thinking and idealism is heavily present. Is Haraway’s cyborg a utopian vision of the post-hoc?

Haraway’s cyborg is relentlessly self aware of its own construction. It may, at will, deconstruct or reconstruct itself in any manner. The subject of these constructions is the place of the cyborg individual within society, its role with others and as whatever identities are embedded within it. As a cyborg, an individual will recognize that it is part of a machine (can probably connect to Deleuze at this point), a node in a network of many. The cyborg thus is aware of itself and has meta-awareness of its own relationships with the other nodes in its network. The cyborg is thus a totally literate being, in the sense of understanding its relationship and structure with that to which it is connected.

Being a cyborg involves a sort of contact, and being influenced by technology? That sort of argument of is indebted to Foucault. In that sense, even the Amish are cyborg in the sense that they exist as a bubble within a heavily technological society. Everyone within a technological culture is exposed to technology from birth, in that we are affected and informed by technology whether we like it or not. We eat food that has been engineered for millenea, and have been influenced by technology for ages. Through this, we have been constructing ourselves and our relationship to technology, and this has paved the way for us to be cyborgs.

Still, I cannot vouch for language. I really cannot understand what possesses authors to write so incoherently. At least she’s not as bad as Deleuze.

Biopolitics of Postmodern Bodies

Chapter is on the discourses and languages of science- specifically the way that they construct bodies and selves. Haraway describes scientific discourse as ‘lumpy’: they condense and are contested over meanings and practices. Specifically, Haraway is using the metaphor, or subject, of the immune system It is a metaphor in that it represents the idea of difference. This can be extended to self vs other, us vs them, etc. (p. 204)

Haraway first describes Richard K. Gershon (who discovered the T cell), and the 1987 book describing his discovery as an example of the classical western science narrative- of man’s mastery over nature.

From the 19th century to the 1980s, in biology, the concept of bodies (specifically female bodies) has changed from a naturalist idealization for the fulfillment of natural functions towards something different. Bodies changed to be thought of as a much more system-oriented network of conflicting strategies. With immunology, specifically, the model changes from one of a well defined inside and outside, to a much more chaotic interplay.

Haraway looks at Terry Winograd and Fernando Flores in “Understanding Computers and Cognition”, and how they critique rationalism in terms of model construction, in terms of how it applies “commonsense” and embedded ideology to problem solving. Winograd also encouraged a method of modeling, which examines the coupling of the inner and outer worlds of organisms and systems. This idea, which is further explained as depending on context, is deeply influential in modern software design. This places emphasis on structure as opposed to the question of internal/external or self/other.

Examining systems leads one to look at the units that comprise a system. Identifying those units, and finding the levels and strata at which they operate is fairly difficult. Haraway brings up Dawkins who theorizes that units are things which may be replicated (ie, genes, memes, etc). If we are to take this to heart, then, as individuals, we are not units, but rather vehicles for smaller, more atomic entities. This approach serves to denaturalize the concept of the organism.

The effect of expansion undermines the distinction between internal and external. The deeply internal microscopic may be analogized to the very external, the extraterrestrial, both frontiers of science. When these are convoluted to change our perception of ourselves and our own space: not only must we defend against that which is a non-self, but from our own parts. In Expansionist Western medical discourse, that which is colonized came to be seen as an invader in its own territory. Expansionism has come to see the invaded subject as part of the self, with the indigenous as an unforeseen and uncontrollable intrusion. (Can think of some contemporary political examples here.)

A key example for Haraway illustrating this change is the biology of the AIDS virus, which serves to turn a body’s own cells against it, performing a kind of microscopic star wars. These examples continue and persist through the writings of Octavia Butler, who addresses these ideas in science fiction.

Overview

The generalized aim of this book is to look at everyday life. DeCerteau’s aim is to examine ordinary people and uncover how the practice of living can be seen as a subversive activity. At a distance, this work can be seen to counter notions of technological determinism and promote the emergence of unpredicted behavior in controlled systems. Abstractly, popular culture exists in spite of constraints and limitations imposed upon it. This can be seen a guide to understanding the simulation of agents within a system, but also as a guide to seeing how humans might take ownership of products (such as software) and manipulate them to their own ends. This is important to consider in the sense of simulation or adaptation culture.

Notes

This may be a personal thing, but I need to state it somewhere, and the notes may be the best place for it: I find DeCerteau’s opening to sound rather pejorative. Also extremely ironic in his discussion of Wittgenstein and his claims that we must use common language to understand common people, when his language is anything but. On first read, it really seems like he is speaking in terms of objective examination of those prole “nobody”s who are strange anonymous beings who slink and hide within the shadows, unknown to all, even themselves. This may be an unreasonable interpretation, but it’s hard to see it within his language. It may just be the translation, or the fact that he comes from a line of obtuse French humanists, but it is nonetheless immensely strange given his aim to use popular language and uncover popular resistance.

DeCerteau opens by discussing the “everyman” or the “nobody” which is a kind of other. The philosophy of this is anonymity, a sort of being nothing. Simultaneously, the “everyman” is an omnipresent force since it is ubiquitous. There seems to be some mixed feeling towards this everyman, both exalting and pejorative. The ordinary man is noble in the struggle of existence against hostile systems, yet is base and ironic in vulgar simplicity. (p. 1-2)

DeCerteau turns to Freud and his analysis of the “ordinary man” in Civilization and its Discontents. It is tricky to generalize about the “ordinary man”, which seems to idolize ignorance and passivity. (p. 4) Who is an “ordinary man”? Is it you or me, the man in the street? etc?

Turning to “Experts”: These *seem* to be a kind of “ordinary man”, at least in DeCerteau’s supreme generalization of “experts”, in that they are anybody, nobody, just like the ordinary man. I think here he means to indicate them as oppressive forces. Experts use specialized knowledge and language to justify a position of superiority. The expert is a case of mistaken identity, who “confuses social place with technical discourse”. Experts are philosophers and scientists who attempt to explain common experience with specialized knowledge, that is not well suited for the task. Specialized knowledge is used to grant authority to experts. (p. 8)

On Wittgenstein: treats language from a perspective that is not compromised by historicity. We are constrained to language, without the ability to identify in it: “We are subject to, but not identified with, ordinary language. As in the ship of fools, we are embarked, without the possibility of aerial view or any sort of totalization.” In other words, we are bound to our understanding of the world via language. (p. 11) There is some concern about the ability to objectively discuss language while operating within a language. This is interesting, since this has been an element of mathematical logic for some time.

A theory of the ordinary is suggested: “The critical return of the ordinary, as Wittgenstein understands it, must destroy all the varieties of rhetorical brilliance associated with powers that heirarchize and with nonsense that enjoys authority.” thus, to understand ordinary language, we must approach it from within, using ordinary language? This makes us foreigners within our span of ordinary life. Ostensibly the objective of this is to make a critical science of the ordinary. That is meant to abstract and understand the ordinary in some cohesive terms. (p. 13)

DeCerteau turns to looking at history: Describing history in terms of facts and laws. The two mix and mingle, and become confused. One can influence the other in oppressive or subversive ways. (p. 16) An example is voodoo culture in Brazil, which uses superstition to subvert the fatality of the established order. “A (‘popular’) use of religion modifies its functioning.” But, popular culture is opaque, so it is hard to see just how. (p. 17-18)

Starting with proverbs, moving towards games, and then legends. These are all products of a society. In a sense, they are tools of a society, for education, and the promotion and maintenance of values. DeCerteau seems to be weaving a complex thread between social practices and their uses. Namely that the practices are sort of documents for their historical occurrences. (p. 20)

“To be memorized as well as memorable, they [games] are repertoires of schemas of action between partners. With the attraction that the element of surprise introduces, these mementos teach the tactics possible within a given (social) system.” This sounds a lot like role-learning, and relates to Barthes’ mythology. “Tales and legends seem to have the same role. They are deployed, like games, in a space outside of and isolated from daily competition, that of the past, the marvelous, the original. In that space can thus be revealed, dressed as gods or heroes, the models of good or bad ruses that can be used every day. Moves, not truths are recounted.” Tales and games are both used for this learning. In a developmental perspective, it is a form of education and practice. In a social perspective, it is a means of continuing and persisting social knowledge, tradition, and mythology. Often times, these operate not just within one culture, but in context of multiple cultures, with games and traditions of one designed to differentiate and protect against the influence of the other culture. (p. 23)
How does this instrumentalization occur?

DeCerteau transitions to analyzing the “art” of practicing speech, or practicing being in general. DeCerteau is wondering how this art is different, and how to study it. He then goes to talk at great length about the practice of “la perruque”, which is the French term for doing personal work on an employer’s time. This concept is fundamental to his later arguments, which use this idea of la perruque to explain how individuals use existing systems and infrastructure to carve out personal spaces within them. Popular practices, such as la perruque, are devices for turning the social order of a system towards popular ends. (p. 25)

La perruque “introduces artistic tricks and competitions of accomplices into a system that reproduces and partitions through work or leisure.” Essentially, it blurs the line between work and pleasure. La perruque can be applied to not only work, or imposed structures, but consumer infrastructure. (p. 29)

On using products, and looking at consumers: products are visible, but the use and actual interaction with products is much harder to understand. By transgressing onto the product, the consumer can carve out a niche of personal space and territory. (p. 31) DeCerteau is doing something interesting here, shifting the focus from a matter of what people consume, to rather what people make with what they consume. He implies that this reappropriation of things occurs at a significant scale. (p. 31)

On Sassure: The difference between langue and parole is that of system versus act. Langue is the system of language, but parole is the occurrence of speech, which uses language, but also hijacks language in many occasions. (p. 32) It is interesting to note that langue is not really an imposed structure, but it evolved and formed out of parole. It is less flexible, though, so acts as a lattice around which parole grows.

On Strategies and tactics: Strategy is a *calculation* of power relationships that become possible when a “subject with will and power” can be isolated. This implies a place where this subject and its power operate. (p. 36) A tactic is a calculated action that lacks a proper location. It operates in the space imposed from outside. “A tactic is an art of the weak.” (p. 37) Strategies are formations of planning, while tactics are the arts of emergence.

Strategy is essentially rule based and derived from patterns of logic, it equates very well with simulation. A simulation is a strategic instrument, since it imposes a creator’s will and an ordered rule based model onto a simulated world, promoting the model of the simulation cognitively. Simultaneously, tactics could be seen as emergent properties, generally exhibited by players, but potentially could be found within simulated agents themselves. Games like SimCity offer new perspectives, uncovering the strategies and infrastructures (which are normally concealed) that are present in everyday life.

A social simulation system would uncover the social strategies that are persistent in everday life, but also encourage tactical exploration for players. It would also make more transparent the various philosophical models of human interaction and cognition that could be expressed by such a system.

Overview

This excerpt is from NMR. The editors comment that: Suchman made a deep critique of AI, that planning and symbolic manipulation is flawed as models of human intelligence. Suchman argues instead that human reasoning is not based in planning, but rather, action is based on embodiment. Suchman claims that plans are stories used to justify and explain these actions. This critique influenced Philip Agre and David Chapman, who explored AI in consistency with Suchman’s argument.

Notes

The preface, “Navigation” opens with a discussion of Trukese and European navigators. The European navigator operates in accordance with the plan at all times, while the Trukese navigator instead operates only with an objective in mind, using circumstance and conditions to alter his course. The metaphor can inform three possibilities: One, that how to act is purposefully learned, and is different across cultures. Suchman follows to counter this with the counterargument that all activity, even the most analytic is fundamentally embodied. Two, one might argue that planning is used instrumentally, depending on experience or expertise. But this seems to imply that the Trukese navigator would not get anywhere. Three, Suchman’s critique, is that all purposeful actions are situated in their circumstances: we act like the Trukese, but talk like the Europeans. Rather, plans are an ad-hoc resource for the action. This metaphor is an excellent representation of Suchman’s critique, but it also exposes some other qualities that may easily slip by. One is that it was Eurpoean thinking, heavily based in Cartesian dualism, that led to the development of computers and AI. The absence of non-western thinking pushes other forms of reasoning and philosophy into the background in common electronic models of cognition and interaction.

Suchman begins by discussing Turkle’s research on computers as collaborative objects. Computers are reactive, linguistic, and internally opaque: this leads to design challenges, especially with accountability. Computers seem to reason, but the manner of that reasoning is concealed. On Automata: cognitive science has pushed a symbolic metaphor of cognition, with AI and computers being the logical receptors. Cognitive science emphasizes the detachment of rationality from embodiment, and supports the abstract symbolic reasoning pervasive in AI today. Suchman discusses the linguistic metaphor for interaction in HCI. This emphasizes interaction as a dialogue, compared with dialogue between two people. In such case, as Dennett argues, mutual opacity makes intentional explanations much more powerful. Thus, the opacity of computer invites an intentional stance. From the design perspective, artifacts do and should try to explain themselves, but this is muddy water when it comes to intentionality. Opacity, especially in certain untrusted situations can place a user at odds with an treacherous and untrusted world. Obviously, this is not the dominant perception of computers or computation, rather they are seen as extensions or objects. Interaction is not a dialogue, so much as commands and filtering. From an AI perspective, though, the concern is natural.

Suchman finishes off the chapter with a comparison of the computer as an “artifact designed for a purpose” versus “an artifact having purposes”. The former, which is the instrumental approach, evokes embodied and situated reasoning, with the computer as an adaptable tool. The latter approach is that of AI, which (as far as it is instrumental) treats the computer as an intelligent device, which engages with the user reflexively, and is not *usable* in the literal sense. However, again, this form of reasoning gets convoluted when applied to games, which, having no extradiagetic goals, have no purpose. Games are most engaging when reflective and automated, but simultaneously evoke an intense state of situation via their immersive character.

Overview

Dourish writes of “embodied interaction”. This idea is meant to connect the realms of HCI, interfaces, and design with that of continental phenomenology. Dourish’s premise is that HCI has learned a great deal from phenomenology (specifically in the developments of tangible and social computing), but stands to gain a great deal more from applying the principles of embodiment and being to computational artifacts rather than persisting with conventional procedural metaphors. Specifically, Dourish claims that the sense of humans as being subservient to computers (originating in early history of computer science) still remains strongly today. While Dourish’s discussion is meant to give a new perspective on HCI, his book also gives some insight into how simulated agents might experience embodiment and may be “phenomenologically sympathetic”. Embodied interaction may be seen one way as human to computer, but potentially between simulated agents and their artificial worlds.

Notes

A History of Interaction

Dourish opens, describing that “Embodied Interaction is interaction with computer systems that occupy our world, a world of physical and social reality, and that exploit this fact in how they interact with us.” (p. 3) Embodied interaction shifts emphasis from the procedural model to a more process based model, specifically based in things like Milner’s Pi Calculus, and Rodney Brooks’s robotics. (p. 4)

Dourish also makes note that the original “computers” were real people whose careers were doing calculations. (p. 6) This is interesting for comparing computer based simulation to rule based simulation to human imagination. Removing the digital element from computing, there is a lot of room for error and creativity within human processes (necessarily, mechanization tries to stamp this out), but it is ironic to imagine a digital story simulation being carried out by live individuals. The procedural aspect seems necessarily very restrictive in the creative role. However, both stem from this idea of a rule based system, but where humans are challenged and made more creative by the constraints implied by rules, digital systems seem to be limited.

One of the means of interaction that Dourish discusses is the textual form. Here he is primarily referring to command line interfaces, such as those seen on a command shell. These are textual, text, sort of, but not really. Not in the sense that a letter to a friend, IM conversation, or verbal instructions are textual or linguistic. Dourish notes that conversation and dialogue are now integral to our understanding of interactivity. The purpose of such conversation is implicitly assumed to be to give instructions to run computer tasks, not for actual social engagement. “Textual interaction drew upon language much more explicitly than before, and at the same time it was accompanied by a transition to a new model of computing, in which a user would actually sit in front of a computer terminal , entering commands and reading responses. With this combination of language use and direct interaction, it was natural to look on the result as a ‘conversation’ or ‘dialogue’.” (p. 10)

In the discussion of visual metaphor, Dourish describes the visual interaction as a more direct form of engagement, wherein the user interacts with abstract objects in a direct and concrete manner. These objects and interactions are represented symbolically and visually, creating a world of metaphors wherein the system of concepts is complete and consistent. “From these separate element, the designer builds an inhabited world in which users act. Direct manipulation interfaces exploit and extend the benefits of graphical interaction.” (p. 13) While this approach is interesting from an interface perspective, it also renders the computer world as a Baudrillardian simulation. So the representative power of simulation becomes most clear at this level, when metaphors and direct interaction become present in interfaces.

Social and tangible interfaces are grounded in embodied interaction, which is at odds with a positivist Cartesian ‘naive cognitivism’ which gives a very dualist take on interaction, with a heavy emphasis on symbolic representation. Embodiment claims that cognition without a body is fallacy, and embodied interaction exploits the corporeality of its interactors. Embodiment also implies a presence, and hence, participation: “Embodiment, instead, denotes a form of participative status. Embodiment is about the fact that things are embedded in the world, and the ways in which their reality depends on being embedded.” (p. 18) So, we can extend this to thinking about agents and their participative nature.

Being in the World: Embodied Interaction

In the next section, Dourish spends time discussing various phenomenologists (specifically Husserl, Heidegger, Schutz, and Merleau-Ponty), and their potential application towards HCI. He starts with two definitions of embodiment, the second is this: “Embodied phenomena are those that by their very nature occur in real time and real space.” (p. 101) Looking at people interacting with computers, Dourish asserts that people respond physically, directly, and kinetically with the world around them in a tangible manner, and that operating through a computer abstracts this, even in immersive environments, users are operating in opposition to interfaces. We do not need to do planning in engaging with the world as we do with interfaces. However, this seems a dubious claim: many people have great trouble engaging with the world (since it does not have interfaces, but it does have protocols and social conventions). Many people with WoW or SL addictions tend to interact more seamlessly in those worlds than they do in reality. Dourish specifically examines 3d interfaces, wherein a user navigates a world with keyboard and mouse, and notes that our operation with the world is not of this nature- that we do not have a homunculus sitting inside our head observing through our eyes and controlling indirectly our actions (which sounds like Searle and AI). This is the difference between player and avatar, and this relationship, I think, is much more complex and nuanced, and has the capacity to be much tighter than presumed here. (p. 102)

On Husserl: Husserl wanted to carry Cartesian dualsim to address the phenomena of experience. Specifically, he was dissatisfied with the abstraction of mathematics and science, and wanted to “develop the philosophy of experience as a rigorous science”. Moreover, he drew lines between objects of consciousness and objects of intentionality (the Cartesian duals of objects). These sound like they might relate to the socially enacted objects of Mead. Objects of intentionality are “noema”, and our mental consciousness of these objects are “noesis”. Underlying the concept of noema is a platonic conception of essence. (p. 105-106)

On Heidegger: Heidegger rejected Husserl’s dualism, and emphasized that the mental and physical spaces are deeply connected. “Essentially, Heidegger transformed the problem of phenomenology from an epistemological question, a question about knowledge, to an ontological question, a question about forms and categories of existence. Instead of asking, ‘How can we know about the world?’ Heidegger asked, ‘How does the world reveal itself to us through our encounters with it?'”. This change in question is a focal point for HCI and interaction. (p. 107)

On Schutz: Schutz looked on phenomenology as applied to social action. Social enaction is rooted in shared experience, which is phenomenological in nature. Collective action depends on intersubjective understandings of the world. “Schutz argued that the meaningfulness of social action had to emerge within the context of the actor’s own experience with the world.” (p. 111)

On Merleau-Ponty: The focus here is on the phenomenology of perception. Merleau-Ponty wished to reconcile Husserl’s philosophy of essences with Heidegger’s philosophy of being. This involved a change in perspective of the role of the body in experience. “For Merleau-Ponty, the body is neither subject nor object, but an ambiguous third party.” To understand the body, one must understand perception. This is an interesting approach towards embodiment, since his treatment of the body sounds very applicable to the approach necessary for simulated agents. Simulated agents do not *have* bodies, but they must be embodied within their world, and to address this problem, one must turn to the matter of peception. Merleau-Ponty goes on to emphasize a reversibility in perception (that others may perceive ourselves? Can get very Lacanian here), which means that we can apprehend “perceptions of ourselves that we engender in others”. This work was done by Robertson in 1997, and sounds very similar to Goffman’s performance of the self. (p. 114-115)

Ultimately, the phenomeonlogists have explored the relationship between embodied action and meaning. Meaning, to them, is found in the world with which we are in constant contact and engagement. Meaning can be found via the world revealing itself to us and affording for us actions to perform upon it. (p. 116) This sounds very similar to the system of affordances developed by J.J. Gibson and later Don Norman. More on this: “In other words, an affordance is a three-way relationship between the environment, the organism, and an activity. This three-way relationship is at the heart of ecological psychology, and the challenge of ecological psychology lies in just how it is centered on the notion of an organism acting in an environment: being int he world.” (p. 118) And again, this three-way relationship sounds like a model for simulated behavior. Michael Polanyi makes a significant investigation of embodied skills which require a “knowing how” versus “knowing what”. (p. 119)

On Wittgenstein: he used embodiement in relation to language. In the linguistic tradition, “He argues that language and meaning are inseparable from the practices of language users. Meaning resides not in disembodied representations, but in practical occasions of language use.” (p. 124) The return to language is very interesting here, from the departure to visual and tangible interfaces.