Notes

Conceptual change is introduced as an idea relevant for understanding learning. This is referenced as corresponding to *representational* changes. Representation in this context corresponds to the mental representation of concepts. This means that here, representation is more closely aligned with the establishment of a model (that is, the representation is the model in one’s head), as opposed to a physical representation of the model itself. This distinction is one that is important in discussing the relationship between mental modeling and simulation.

The challenge with articulating conceptual change is understanding the nature of how concepts are held mentally, and the mechanisms by which they are changed. The distinction given is between scientists and children, who are different in the sophistication of their goals and metacognitive strategies. Nercessian describes problem solving as a means by which to articulate how both groups might revise their conceptual models.

Describing concepts: “Concepts provide a means through which humans make sense of the world. In categorizing experiences we sort phenomena, noting relationships, differences, and interconnections among them. A conceptual structure is a way of systematizing, of putting concepts in relation to one another in at least a semi – or locally – coherent manner.”

On mental models: “Loosely construed, a model is a representation of a system with interactive parts with representations of those interactions. Models are representations of objects, processes, or events that capture structural, behavioral, or functional relations significant to understanding these interactions. What is required for something to be an instance of model-based reasoning is that: 1) it involves the construction or retrieval of a model; 2) inferences are derived through manipulation of the model; and 3) inferences can be specific or generic, that is, they can either apply to the particular model or to the model understood as a model-type, representing a class of models.”

Nersessian describes the mind (the cognitive apparatus) as being capable of “modeling, analogy making, abstraction, visualization, and simulative imagining.” She explains that science has leveraged this and incorporated these approaches into the scientific process (and the scientific method, even). This poses the modeling issue as something that is naturally disposed towards science. This is important and valuable, but also conceals the fact that it is used for other types of reasoning as well, particularly social and creative applications.

Nercessian extensively discusses the history of the theory of mental models, and pays attention to the efforts to equate these models computationally or symbolically. Herbert Simon is specifically mentioned, as an example of someone who applied modeling to computational reasoning systems.

One of the first to develop the theory of models in detail is Craik in the 1940s. The central pillar of Craik’s theory seems to be in mental simulations of models, “reasoning about physical systems via mental simulation of analog representations.” Nersessian mentions that simulation is purportedly developed for navigation within an environment (for instance, a rat in a maze simulates the maze in its mind). Due to human linguistics, mental simulation would thus have the capacity to simulate from language.

Reacting to the idea that thinking is rooted in language, Nersessian suggests that instead, language enables certain narratives, of which individuals may form mental models. Texts describe systems and structures, and these are later manipulated as models and used for reasoning. These are called “discourse models” or “situation models”. This idea seems PERFECT for relating to the narrative adaptation work. This descends from Johnson-Laird, 1989, p471. There is a significant bibliography of sources that discuss how readers form models of texts. The last conclusion reinforces the notion of embodiment and the role of embodiment and perspective within reading a text:

“A number of experiments have been conducted to investigate the hypothesis that in understanding a narrative readers spontaneously construct mental models to represent and reason about the situations depicted by the text (Dijk & Kintsch, 1983; Franklin & Tversky, 1990; Johnson-Laird, 1983; Mani & Johnson-Laird, 1982; McNamara & Sternberg, 1983; Morrow, Bower, & Greenspan, 1989; Perrig & Kintsch, 1985; Zwann, 1999; Zwann & Radvansky, 1998). Although no instructions were given to imagine or picture the situations, when queried about how they had made inferences in response to an experimenter’s questioning, most participants reported that it was by means of “seeing” or “being in the situation” depicted. That is, the reader sees herself as an “observer” of a simulated situation. Whether the view of the situation is “spatial”, i.e., a global perspective, or “perspectival”, i.e., from a specific point of view, is still a point of debate, though recent investigations tend to support the perspectival account, that is, the reference frame of the space appears to be that of the body (Bryant & Tversky, 1999; Glenberg, 1997b; Mainwaring, Tversky, & Schiano, 1996).” (p. 24-25)

I think some degree of this relates to abstract problems or narratives (for instance talking about some blocks of various colors and describing where they are in relation to each other), but it is wholly sensible that these ideas be applied to other conceptual areas, especially in fiction.

Looking at models as they relate to artifacts: People use prosthetics to aid in thinking, demonstrative artifacts that help externalize mental information. One suggestion posed here is that people (specifically scientists in her example) do mental manipulation that interacts with the observed visualization. This process serves to construct a mental model that is *constrained* by the visualization. This process involves a certain coupling between internal and external representations of the model. It also suggests a capacity for the model to be bridged to accommodate other things external to it.

Nersessian goes further to discuss the format of working memory, and propose that the format of the information is 1) modal, and 2) embodied. An example of this derives from 3d positioning, which generally maintains an egocentric coordinate system and perspective. A second example is in the representation of concepts, which Lakoff and Johnson describe as something that is literally part of the brain. The researcher described in detail is Barsalou, who argues that mental representations are perceptual, and man cognitive processes are re-enactments or simulation of perceptual states.

Preface

Phenomenology is about essences. This idea seems to be potentially relevant for thinking of essences of meaning to be carried over in translations or adaptations.

Science is a separate order of expressing the world. So, Merleau-Ponty wishes to return to things-in-themselves. So, it is necessary to return to a world that precedes knowledge. Must use describing, instead of analysis or construction. Perception is the background on which all acts are projected.

There is no inner mind, the mind is in the world, and you can only know yourself in the world. Contradicts Descartes argument by insisting that true cognition must be anchored in the world: “The true Cogito does not define the subject’s existence in terms of the thought he has of existing…” (p. xiv)

Consciousness is consciousness of *something*: it rejects that contemplative thought is sufficient. Consciousness is necessarily what connects individuals to the world.

We are condemned to meaning (allusion to Sartre) (p. xxii). There is no such thing as original choice, namely freedom as defined by Sartre (Moran p. 396). We are in a world of meaning, so any experience takes on significance and history. Phenomenology is about having its own foundation, without external things, but like artistic expression, brings truth into being. (?)

Empiricisim denies meaning? Merleau-Ponty wants to avoid the observational approach, but understand the intimate, reflexive relationship between the body and the world. Defining a difference between cause and reason. Empiricism seeks to uncover causes, which reduces, but denies meaning.

Chapter 3: The Spatiality of One’s Own Body and Motility

Merleau-Ponty opens by exploring some aspects of bodily sensation and comprehension. Bodies are comprehended in space, and in relation to each other. Descriptions and terminologies such as “organ” reduce the body into independent disconnected parts. He uses the term “body schema” as a device for collectively relating the wholeness of bodily experience.

What is important is space, not in terms of position, but in terms of situation. (p. 115) Merleau-Ponty wishes to quell the sense of space as composed of points (in the Cartesian sense, imaginably), especially as this orients and imposes values and external objectivity on the situation. Nonetheless, we can imagine reconstructions of coordinates in relative terms (modern physics is especially attentive to this).

Merleau-Ponty spends some time exploring the interactions and self-perceptions of a patient who is “psychically blind” that is, cannot understand the “body schema” without visual reference. This patient requires certain extensive gestures to be able to position himself blindly in space, and cannot distinguish tactile sensation in different parts of the body.

Note: Additional studies have been made of this sort of thing extensively. Eg, “The Disembodied Lady” in Oliver Sacks, “The Man Who Mistook His Wife for a Hat”. The study involved a patient with an intense feeling of disembodiment, and a loss of “proprioception”. However, in her case, she was still able to act very willfully, relearning to perform normal action without the proprioceptive sense. The feeling was distinctly unnatural, but still did not result in the lack of direction and intent described in the patient later.

Normal individuals can do these things automatically, and this is because sensation is reckoned with. The patient must intellectually grapple with motion and action instead of being able to do it automatically. (p. 126) He cannot convert thought of movement into movement, but can both think and move. This means that the patient lacks something in between, a “motor project” or “motor intentionality”. Essentially, a situation of movement in context of the world and the body.

Diagnosis of patient wrt traditional psychology (or empiricism): this is an example of a sort of reduction, illustration of the patient is a kind of denaturalization of the body to illustrate things we take for granted. Empiricism is lacking an understanding of meaning in that sense. Understanding the difference of the self to an object (as the patient might see it)

On concrete and abstract: In the perception of phenomena (namely movement), movement is either “for itself” meaning that stimulus is an object (the individual moved consciously?), or in itself, being objective within the body, unknown externally (like the hand swatting a mosquito). This difference of in-itself vs for-itself is also called “Greifen” and “Zeigen”. (p. 140) Differentiating these cannot be done if the body is categorized as an object. Vision of the body as in-itself reduces the body to a mechanism, vision mind as for-itself reduces the mind to an abstract symbol processor. So, either extreme is fundamentally lacking, and in reality, the line is blurry. (right?)

Matter and form are connected in phenomenology by a relationship of “Fundierung”, a symbolic function that uses vision and ground. This also relates to the concept of analogy, which people can understand without needing to analyze. This is because in normal thought, things are understood in accordance with the analogy of their function. (p. 148)

More on the patient’s condition: he perceives of things in instances, without external connections, or sense of a whole, “He never goes out for a walk, but always on an errand, and he never recognizes Professor Goldstein’s house as he passes it ‘because he did not go out with the intention of going there'”. (p. 155) This discussion connects with the quote Dreyfus used to criticize AI, that life is subtended by an intentional arc.

Note: The patient is basically (and this ties into Dreyfus very neatly) an extreme example of cognitivist learning. He approaches things much the way that traditional AI might try to. That is clearly unnatural, and we can see that in his behavior.

Further, Merleau-Ponty returns to motility: “Motility, then, is, and, as it were, a handmaid of consciousness, transporting the body to that point in space of which we have formed a representation beforehand. In order that we may be able to move our body towards and object, the object must first exist for it, and our body must not belong to the realm of the ‘in-itself’.” (p. 161) Existence for-it, or for-itself, seems to couple nicely with both intentionality and affordance. There is more: “We must therefore avoid saying that our body is in space, or in time. It inhabits space and time.” Movement is not a matter of memory, but of perception?

On habit: Dreyfus looks at Merleau-Ponty in application to learning of skills, the approach to skills as habits is explained in some detail here. Traditional philosophy and mechanistic theory run into problems when understanding habit. The of habit could be understood as a special kind of understanding or significance. In habits, perception is adapted: The feather in a woman’s cap is perceived as a part of herself, the walking stick of a blind man extends as a part of his perception. Explained: “If habit is neither a form of knowledge nor an involuntary action, then what is it? It is knowledge in the hands, which is forthcoming only when bodily effort is made, and cannot be formulated in detachment from that effort. The subject knows where the letters are on the typewriter as we know where one of our limbs is, though a knowledge bred of familiarity which does not give us a position in objective space.” (p. 167) Further, “To understand is to experience the harmony between what we aim at and what is given, between the intention and the performance–and the body is our anchorage in a world.”

Merleau-Ponty

Moran gives an overview and biography of Merleau-Ponty’s life. Merleau-Ponty was influenced by Christian socialism, and later transitioned into Marxism. He studied works on behaviorism and perception, and criticized behaviorism of suffering from “feigned anesthesia”, in that behaviorism requires that the subject feels nothing. Merleau-Ponty is credited with linking gestalt psychology with phenomeonlogical “being in the world”.

Sartre and Merleau-Ponty had a collaborative relationship the Second World War, in their pursuit of existentialism and phenomenology. The two were considered offering rival interpretations of existential phenomenology.

Merleau-Ponty joined Sartre in the French communist party, and they supported the USSR through its various failings. Later, Merleau-Ponty came to see, via the Korean War, Soviet communism as an imperialist force, which was a break with Sartre.

Eventually, they reconciled, and in the later years of Merleau-Ponty’s life, he did work trying to build connections from phenomenology to structuralism. Additionally published on science and how modern science needs to understand its relation with the world.

Merleau-Ponty was primary mission was philosophy, and his overall goal was to use Husserl to uncover the ‘roots of rationality’. Philosophy is a means of understanding awareness, which of course relates strongly to phenomenology.

A challenge is to uncover pre-conceptual experience (which is done in cognitive science, and learning theory, discovering how people build models). Objective thought (reason?) does not generally acknowledge models as being perceived or constructed though. Furthermore, analysis in this level of understanding draws down to the “irreducibility of the real world”: “The real is to be described not constructed or constituted”. Experience requires a self, and this self is inseparable from the world.

Merleau-Ponty was influenced by Levi-Strauss, and by the non-historicality of structuralism that Levi-Strauss posed. Specifically, Merleau-Ponty saw temporal thinking as interfering with language: “The congealing of temporal thinking into language and concepts acts to fix meanings, to give the appearance of absoluteness.” The ideas of structure and system and language were seen as heavily connected to perception.

Merleau-Ponty is rejecting a linear or single-explanation of history. Instead, time is something that is experienced and lived within, and cannot be seen from the outside.

Levi-Strauss did some work on looking at anthropology from a structuralist perspective, composed of binary oppositions. Ie, raw and cooked, or up/down, light/dark, etc. These could potentially be seen as embodied understandings. It still seems odd in association with phenomenology, but connection can be made via embodiment.

Merleau-Ponty is influenced by the method of Husserl, which is the principle of *reduction*.

The human body is an expressive space, which contributes to human action. Speech is not only the expression of ideas, but it may have power of signification.

Phenomenology lives between two extremes- Cartesian, which understands the mind and understands thought as a fully internal thing, which interacts with the outside world through very prescribed means of perception. This has an internalized means of understanding the self as subject and the world and the body as objects. A second extreme is behaviorism, which eliminates the matter of mind entirely, it comes from the scientific perspective, which looks objectively at how behaviors occur.

Phenomenologists reject the traditional subject, object relationship, and reject the position of knowing things abstractly (which operates against Descartes and follows from Kant), and they reject the objectivity of science and demand consciousness (operating against behaviorism).

The Cyborg Manifesto

Haraway’s early work was on primatology and she studied how conventional western metaphors of gender, race, and class had informed primatology and science as a whole. Her goal is not to undermine science as a whole, but expose its concealed lack of objectivity.

Concerning the Cyborg Manifesto, Haraway is still looking at metaphorical terms. Haraway writes of a post-gender world, and of the cyborg as a tangible concept, imbued with many properties, each revolutionary. Our world, while suffused with technology and slowly attaining a networked character, is still heavily embodied and weighted by 19th century notions of race, gender, and class. It is hard to imagine her world as connected to ours, but using the tool of metaphor, we can make that connection.

Haraway’s work is intended to defamiliarize ourselves with naturalness, and shake us out of the conception that we are natural. Her world is still far from ours, but the loss of natural innocence is one trait that we do have in common with hers. Yes, our world is constructed, in terms of gender, class, etc, and in terms of every aspect of our lives: these are composed by the interconnection of many systems. We, however, are not post. The myth of naturalness is still stiffly ingrained in popular imagery, and is romanticized and idealized. Backwards thinking and idealism is heavily present. Is Haraway’s cyborg a utopian vision of the post-hoc?

Haraway’s cyborg is relentlessly self aware of its own construction. It may, at will, deconstruct or reconstruct itself in any manner. The subject of these constructions is the place of the cyborg individual within society, its role with others and as whatever identities are embedded within it. As a cyborg, an individual will recognize that it is part of a machine (can probably connect to Deleuze at this point), a node in a network of many. The cyborg thus is aware of itself and has meta-awareness of its own relationships with the other nodes in its network. The cyborg is thus a totally literate being, in the sense of understanding its relationship and structure with that to which it is connected.

Being a cyborg involves a sort of contact, and being influenced by technology? That sort of argument of is indebted to Foucault. In that sense, even the Amish are cyborg in the sense that they exist as a bubble within a heavily technological society. Everyone within a technological culture is exposed to technology from birth, in that we are affected and informed by technology whether we like it or not. We eat food that has been engineered for millenea, and have been influenced by technology for ages. Through this, we have been constructing ourselves and our relationship to technology, and this has paved the way for us to be cyborgs.

Still, I cannot vouch for language. I really cannot understand what possesses authors to write so incoherently. At least she’s not as bad as Deleuze.

Biopolitics of Postmodern Bodies

Chapter is on the discourses and languages of science- specifically the way that they construct bodies and selves. Haraway describes scientific discourse as ‘lumpy’: they condense and are contested over meanings and practices. Specifically, Haraway is using the metaphor, or subject, of the immune system It is a metaphor in that it represents the idea of difference. This can be extended to self vs other, us vs them, etc. (p. 204)

Haraway first describes Richard K. Gershon (who discovered the T cell), and the 1987 book describing his discovery as an example of the classical western science narrative- of man’s mastery over nature.

From the 19th century to the 1980s, in biology, the concept of bodies (specifically female bodies) has changed from a naturalist idealization for the fulfillment of natural functions towards something different. Bodies changed to be thought of as a much more system-oriented network of conflicting strategies. With immunology, specifically, the model changes from one of a well defined inside and outside, to a much more chaotic interplay.

Haraway looks at Terry Winograd and Fernando Flores in “Understanding Computers and Cognition”, and how they critique rationalism in terms of model construction, in terms of how it applies “commonsense” and embedded ideology to problem solving. Winograd also encouraged a method of modeling, which examines the coupling of the inner and outer worlds of organisms and systems. This idea, which is further explained as depending on context, is deeply influential in modern software design. This places emphasis on structure as opposed to the question of internal/external or self/other.

Examining systems leads one to look at the units that comprise a system. Identifying those units, and finding the levels and strata at which they operate is fairly difficult. Haraway brings up Dawkins who theorizes that units are things which may be replicated (ie, genes, memes, etc). If we are to take this to heart, then, as individuals, we are not units, but rather vehicles for smaller, more atomic entities. This approach serves to denaturalize the concept of the organism.

The effect of expansion undermines the distinction between internal and external. The deeply internal microscopic may be analogized to the very external, the extraterrestrial, both frontiers of science. When these are convoluted to change our perception of ourselves and our own space: not only must we defend against that which is a non-self, but from our own parts. In Expansionist Western medical discourse, that which is colonized came to be seen as an invader in its own territory. Expansionism has come to see the invaded subject as part of the self, with the indigenous as an unforeseen and uncontrollable intrusion. (Can think of some contemporary political examples here.)

A key example for Haraway illustrating this change is the biology of the AIDS virus, which serves to turn a body’s own cells against it, performing a kind of microscopic star wars. These examples continue and persist through the writings of Octavia Butler, who addresses these ideas in science fiction.

Overview

The generalized aim of this book is to look at everyday life. DeCerteau’s aim is to examine ordinary people and uncover how the practice of living can be seen as a subversive activity. At a distance, this work can be seen to counter notions of technological determinism and promote the emergence of unpredicted behavior in controlled systems. Abstractly, popular culture exists in spite of constraints and limitations imposed upon it. This can be seen a guide to understanding the simulation of agents within a system, but also as a guide to seeing how humans might take ownership of products (such as software) and manipulate them to their own ends. This is important to consider in the sense of simulation or adaptation culture.

Notes

This may be a personal thing, but I need to state it somewhere, and the notes may be the best place for it: I find DeCerteau’s opening to sound rather pejorative. Also extremely ironic in his discussion of Wittgenstein and his claims that we must use common language to understand common people, when his language is anything but. On first read, it really seems like he is speaking in terms of objective examination of those prole “nobody”s who are strange anonymous beings who slink and hide within the shadows, unknown to all, even themselves. This may be an unreasonable interpretation, but it’s hard to see it within his language. It may just be the translation, or the fact that he comes from a line of obtuse French humanists, but it is nonetheless immensely strange given his aim to use popular language and uncover popular resistance.

DeCerteau opens by discussing the “everyman” or the “nobody” which is a kind of other. The philosophy of this is anonymity, a sort of being nothing. Simultaneously, the “everyman” is an omnipresent force since it is ubiquitous. There seems to be some mixed feeling towards this everyman, both exalting and pejorative. The ordinary man is noble in the struggle of existence against hostile systems, yet is base and ironic in vulgar simplicity. (p. 1-2)

DeCerteau turns to Freud and his analysis of the “ordinary man” in Civilization and its Discontents. It is tricky to generalize about the “ordinary man”, which seems to idolize ignorance and passivity. (p. 4) Who is an “ordinary man”? Is it you or me, the man in the street? etc?

Turning to “Experts”: These *seem* to be a kind of “ordinary man”, at least in DeCerteau’s supreme generalization of “experts”, in that they are anybody, nobody, just like the ordinary man. I think here he means to indicate them as oppressive forces. Experts use specialized knowledge and language to justify a position of superiority. The expert is a case of mistaken identity, who “confuses social place with technical discourse”. Experts are philosophers and scientists who attempt to explain common experience with specialized knowledge, that is not well suited for the task. Specialized knowledge is used to grant authority to experts. (p. 8)

On Wittgenstein: treats language from a perspective that is not compromised by historicity. We are constrained to language, without the ability to identify in it: “We are subject to, but not identified with, ordinary language. As in the ship of fools, we are embarked, without the possibility of aerial view or any sort of totalization.” In other words, we are bound to our understanding of the world via language. (p. 11) There is some concern about the ability to objectively discuss language while operating within a language. This is interesting, since this has been an element of mathematical logic for some time.

A theory of the ordinary is suggested: “The critical return of the ordinary, as Wittgenstein understands it, must destroy all the varieties of rhetorical brilliance associated with powers that heirarchize and with nonsense that enjoys authority.” thus, to understand ordinary language, we must approach it from within, using ordinary language? This makes us foreigners within our span of ordinary life. Ostensibly the objective of this is to make a critical science of the ordinary. That is meant to abstract and understand the ordinary in some cohesive terms. (p. 13)

DeCerteau turns to looking at history: Describing history in terms of facts and laws. The two mix and mingle, and become confused. One can influence the other in oppressive or subversive ways. (p. 16) An example is voodoo culture in Brazil, which uses superstition to subvert the fatality of the established order. “A (‘popular’) use of religion modifies its functioning.” But, popular culture is opaque, so it is hard to see just how. (p. 17-18)

Starting with proverbs, moving towards games, and then legends. These are all products of a society. In a sense, they are tools of a society, for education, and the promotion and maintenance of values. DeCerteau seems to be weaving a complex thread between social practices and their uses. Namely that the practices are sort of documents for their historical occurrences. (p. 20)

“To be memorized as well as memorable, they [games] are repertoires of schemas of action between partners. With the attraction that the element of surprise introduces, these mementos teach the tactics possible within a given (social) system.” This sounds a lot like role-learning, and relates to Barthes’ mythology. “Tales and legends seem to have the same role. They are deployed, like games, in a space outside of and isolated from daily competition, that of the past, the marvelous, the original. In that space can thus be revealed, dressed as gods or heroes, the models of good or bad ruses that can be used every day. Moves, not truths are recounted.” Tales and games are both used for this learning. In a developmental perspective, it is a form of education and practice. In a social perspective, it is a means of continuing and persisting social knowledge, tradition, and mythology. Often times, these operate not just within one culture, but in context of multiple cultures, with games and traditions of one designed to differentiate and protect against the influence of the other culture. (p. 23)
How does this instrumentalization occur?

DeCerteau transitions to analyzing the “art” of practicing speech, or practicing being in general. DeCerteau is wondering how this art is different, and how to study it. He then goes to talk at great length about the practice of “la perruque”, which is the French term for doing personal work on an employer’s time. This concept is fundamental to his later arguments, which use this idea of la perruque to explain how individuals use existing systems and infrastructure to carve out personal spaces within them. Popular practices, such as la perruque, are devices for turning the social order of a system towards popular ends. (p. 25)

La perruque “introduces artistic tricks and competitions of accomplices into a system that reproduces and partitions through work or leisure.” Essentially, it blurs the line between work and pleasure. La perruque can be applied to not only work, or imposed structures, but consumer infrastructure. (p. 29)

On using products, and looking at consumers: products are visible, but the use and actual interaction with products is much harder to understand. By transgressing onto the product, the consumer can carve out a niche of personal space and territory. (p. 31) DeCerteau is doing something interesting here, shifting the focus from a matter of what people consume, to rather what people make with what they consume. He implies that this reappropriation of things occurs at a significant scale. (p. 31)

On Sassure: The difference between langue and parole is that of system versus act. Langue is the system of language, but parole is the occurrence of speech, which uses language, but also hijacks language in many occasions. (p. 32) It is interesting to note that langue is not really an imposed structure, but it evolved and formed out of parole. It is less flexible, though, so acts as a lattice around which parole grows.

On Strategies and tactics: Strategy is a *calculation* of power relationships that become possible when a “subject with will and power” can be isolated. This implies a place where this subject and its power operate. (p. 36) A tactic is a calculated action that lacks a proper location. It operates in the space imposed from outside. “A tactic is an art of the weak.” (p. 37) Strategies are formations of planning, while tactics are the arts of emergence.

Strategy is essentially rule based and derived from patterns of logic, it equates very well with simulation. A simulation is a strategic instrument, since it imposes a creator’s will and an ordered rule based model onto a simulated world, promoting the model of the simulation cognitively. Simultaneously, tactics could be seen as emergent properties, generally exhibited by players, but potentially could be found within simulated agents themselves. Games like SimCity offer new perspectives, uncovering the strategies and infrastructures (which are normally concealed) that are present in everyday life.

A social simulation system would uncover the social strategies that are persistent in everday life, but also encourage tactical exploration for players. It would also make more transparent the various philosophical models of human interaction and cognition that could be expressed by such a system.

Overview

This excerpt is from NMR. The editors comment that: Suchman made a deep critique of AI, that planning and symbolic manipulation is flawed as models of human intelligence. Suchman argues instead that human reasoning is not based in planning, but rather, action is based on embodiment. Suchman claims that plans are stories used to justify and explain these actions. This critique influenced Philip Agre and David Chapman, who explored AI in consistency with Suchman’s argument.

Notes

The preface, “Navigation” opens with a discussion of Trukese and European navigators. The European navigator operates in accordance with the plan at all times, while the Trukese navigator instead operates only with an objective in mind, using circumstance and conditions to alter his course. The metaphor can inform three possibilities: One, that how to act is purposefully learned, and is different across cultures. Suchman follows to counter this with the counterargument that all activity, even the most analytic is fundamentally embodied. Two, one might argue that planning is used instrumentally, depending on experience or expertise. But this seems to imply that the Trukese navigator would not get anywhere. Three, Suchman’s critique, is that all purposeful actions are situated in their circumstances: we act like the Trukese, but talk like the Europeans. Rather, plans are an ad-hoc resource for the action. This metaphor is an excellent representation of Suchman’s critique, but it also exposes some other qualities that may easily slip by. One is that it was Eurpoean thinking, heavily based in Cartesian dualism, that led to the development of computers and AI. The absence of non-western thinking pushes other forms of reasoning and philosophy into the background in common electronic models of cognition and interaction.

Suchman begins by discussing Turkle’s research on computers as collaborative objects. Computers are reactive, linguistic, and internally opaque: this leads to design challenges, especially with accountability. Computers seem to reason, but the manner of that reasoning is concealed. On Automata: cognitive science has pushed a symbolic metaphor of cognition, with AI and computers being the logical receptors. Cognitive science emphasizes the detachment of rationality from embodiment, and supports the abstract symbolic reasoning pervasive in AI today. Suchman discusses the linguistic metaphor for interaction in HCI. This emphasizes interaction as a dialogue, compared with dialogue between two people. In such case, as Dennett argues, mutual opacity makes intentional explanations much more powerful. Thus, the opacity of computer invites an intentional stance. From the design perspective, artifacts do and should try to explain themselves, but this is muddy water when it comes to intentionality. Opacity, especially in certain untrusted situations can place a user at odds with an treacherous and untrusted world. Obviously, this is not the dominant perception of computers or computation, rather they are seen as extensions or objects. Interaction is not a dialogue, so much as commands and filtering. From an AI perspective, though, the concern is natural.

Suchman finishes off the chapter with a comparison of the computer as an “artifact designed for a purpose” versus “an artifact having purposes”. The former, which is the instrumental approach, evokes embodied and situated reasoning, with the computer as an adaptable tool. The latter approach is that of AI, which (as far as it is instrumental) treats the computer as an intelligent device, which engages with the user reflexively, and is not *usable* in the literal sense. However, again, this form of reasoning gets convoluted when applied to games, which, having no extradiagetic goals, have no purpose. Games are most engaging when reflective and automated, but simultaneously evoke an intense state of situation via their immersive character.

Overview

Dourish writes of “embodied interaction”. This idea is meant to connect the realms of HCI, interfaces, and design with that of continental phenomenology. Dourish’s premise is that HCI has learned a great deal from phenomenology (specifically in the developments of tangible and social computing), but stands to gain a great deal more from applying the principles of embodiment and being to computational artifacts rather than persisting with conventional procedural metaphors. Specifically, Dourish claims that the sense of humans as being subservient to computers (originating in early history of computer science) still remains strongly today. While Dourish’s discussion is meant to give a new perspective on HCI, his book also gives some insight into how simulated agents might experience embodiment and may be “phenomenologically sympathetic”. Embodied interaction may be seen one way as human to computer, but potentially between simulated agents and their artificial worlds.

Notes

A History of Interaction

Dourish opens, describing that “Embodied Interaction is interaction with computer systems that occupy our world, a world of physical and social reality, and that exploit this fact in how they interact with us.” (p. 3) Embodied interaction shifts emphasis from the procedural model to a more process based model, specifically based in things like Milner’s Pi Calculus, and Rodney Brooks’s robotics. (p. 4)

Dourish also makes note that the original “computers” were real people whose careers were doing calculations. (p. 6) This is interesting for comparing computer based simulation to rule based simulation to human imagination. Removing the digital element from computing, there is a lot of room for error and creativity within human processes (necessarily, mechanization tries to stamp this out), but it is ironic to imagine a digital story simulation being carried out by live individuals. The procedural aspect seems necessarily very restrictive in the creative role. However, both stem from this idea of a rule based system, but where humans are challenged and made more creative by the constraints implied by rules, digital systems seem to be limited.

One of the means of interaction that Dourish discusses is the textual form. Here he is primarily referring to command line interfaces, such as those seen on a command shell. These are textual, text, sort of, but not really. Not in the sense that a letter to a friend, IM conversation, or verbal instructions are textual or linguistic. Dourish notes that conversation and dialogue are now integral to our understanding of interactivity. The purpose of such conversation is implicitly assumed to be to give instructions to run computer tasks, not for actual social engagement. “Textual interaction drew upon language much more explicitly than before, and at the same time it was accompanied by a transition to a new model of computing, in which a user would actually sit in front of a computer terminal , entering commands and reading responses. With this combination of language use and direct interaction, it was natural to look on the result as a ‘conversation’ or ‘dialogue’.” (p. 10)

In the discussion of visual metaphor, Dourish describes the visual interaction as a more direct form of engagement, wherein the user interacts with abstract objects in a direct and concrete manner. These objects and interactions are represented symbolically and visually, creating a world of metaphors wherein the system of concepts is complete and consistent. “From these separate element, the designer builds an inhabited world in which users act. Direct manipulation interfaces exploit and extend the benefits of graphical interaction.” (p. 13) While this approach is interesting from an interface perspective, it also renders the computer world as a Baudrillardian simulation. So the representative power of simulation becomes most clear at this level, when metaphors and direct interaction become present in interfaces.

Social and tangible interfaces are grounded in embodied interaction, which is at odds with a positivist Cartesian ‘naive cognitivism’ which gives a very dualist take on interaction, with a heavy emphasis on symbolic representation. Embodiment claims that cognition without a body is fallacy, and embodied interaction exploits the corporeality of its interactors. Embodiment also implies a presence, and hence, participation: “Embodiment, instead, denotes a form of participative status. Embodiment is about the fact that things are embedded in the world, and the ways in which their reality depends on being embedded.” (p. 18) So, we can extend this to thinking about agents and their participative nature.

Being in the World: Embodied Interaction

In the next section, Dourish spends time discussing various phenomenologists (specifically Husserl, Heidegger, Schutz, and Merleau-Ponty), and their potential application towards HCI. He starts with two definitions of embodiment, the second is this: “Embodied phenomena are those that by their very nature occur in real time and real space.” (p. 101) Looking at people interacting with computers, Dourish asserts that people respond physically, directly, and kinetically with the world around them in a tangible manner, and that operating through a computer abstracts this, even in immersive environments, users are operating in opposition to interfaces. We do not need to do planning in engaging with the world as we do with interfaces. However, this seems a dubious claim: many people have great trouble engaging with the world (since it does not have interfaces, but it does have protocols and social conventions). Many people with WoW or SL addictions tend to interact more seamlessly in those worlds than they do in reality. Dourish specifically examines 3d interfaces, wherein a user navigates a world with keyboard and mouse, and notes that our operation with the world is not of this nature- that we do not have a homunculus sitting inside our head observing through our eyes and controlling indirectly our actions (which sounds like Searle and AI). This is the difference between player and avatar, and this relationship, I think, is much more complex and nuanced, and has the capacity to be much tighter than presumed here. (p. 102)

On Husserl: Husserl wanted to carry Cartesian dualsim to address the phenomena of experience. Specifically, he was dissatisfied with the abstraction of mathematics and science, and wanted to “develop the philosophy of experience as a rigorous science”. Moreover, he drew lines between objects of consciousness and objects of intentionality (the Cartesian duals of objects). These sound like they might relate to the socially enacted objects of Mead. Objects of intentionality are “noema”, and our mental consciousness of these objects are “noesis”. Underlying the concept of noema is a platonic conception of essence. (p. 105-106)

On Heidegger: Heidegger rejected Husserl’s dualism, and emphasized that the mental and physical spaces are deeply connected. “Essentially, Heidegger transformed the problem of phenomenology from an epistemological question, a question about knowledge, to an ontological question, a question about forms and categories of existence. Instead of asking, ‘How can we know about the world?’ Heidegger asked, ‘How does the world reveal itself to us through our encounters with it?'”. This change in question is a focal point for HCI and interaction. (p. 107)

On Schutz: Schutz looked on phenomenology as applied to social action. Social enaction is rooted in shared experience, which is phenomenological in nature. Collective action depends on intersubjective understandings of the world. “Schutz argued that the meaningfulness of social action had to emerge within the context of the actor’s own experience with the world.” (p. 111)

On Merleau-Ponty: The focus here is on the phenomenology of perception. Merleau-Ponty wished to reconcile Husserl’s philosophy of essences with Heidegger’s philosophy of being. This involved a change in perspective of the role of the body in experience. “For Merleau-Ponty, the body is neither subject nor object, but an ambiguous third party.” To understand the body, one must understand perception. This is an interesting approach towards embodiment, since his treatment of the body sounds very applicable to the approach necessary for simulated agents. Simulated agents do not *have* bodies, but they must be embodied within their world, and to address this problem, one must turn to the matter of peception. Merleau-Ponty goes on to emphasize a reversibility in perception (that others may perceive ourselves? Can get very Lacanian here), which means that we can apprehend “perceptions of ourselves that we engender in others”. This work was done by Robertson in 1997, and sounds very similar to Goffman’s performance of the self. (p. 114-115)

Ultimately, the phenomeonlogists have explored the relationship between embodied action and meaning. Meaning, to them, is found in the world with which we are in constant contact and engagement. Meaning can be found via the world revealing itself to us and affording for us actions to perform upon it. (p. 116) This sounds very similar to the system of affordances developed by J.J. Gibson and later Don Norman. More on this: “In other words, an affordance is a three-way relationship between the environment, the organism, and an activity. This three-way relationship is at the heart of ecological psychology, and the challenge of ecological psychology lies in just how it is centered on the notion of an organism acting in an environment: being int he world.” (p. 118) And again, this three-way relationship sounds like a model for simulated behavior. Michael Polanyi makes a significant investigation of embodied skills which require a “knowing how” versus “knowing what”. (p. 119)

On Wittgenstein: he used embodiement in relation to language. In the linguistic tradition, “He argues that language and meaning are inseparable from the practices of language users. Meaning resides not in disembodied representations, but in practical occasions of language use.” (p. 124) The return to language is very interesting here, from the departure to visual and tangible interfaces.

Overview

Moulthrop comes from a general background of Ludology, and spends much of his time critiquing Janet Murray’s Hamlet on the Holodeck. The general idea of the essay seems to be an attempt to find other means of relevance for game studies, especially as a means of comprehending culture. Moulthrop strongly references Pierre Levy on collective intelligence, and is trying to derive some form of understanding of games and game studies as being critical to the realization of a molecular society.

Notes

Moulthrop opens his essay by noting the contemporary setting and peculiar position of game studies in the (particularly American) political climate. He references Donna Haraway on the current state of the cultural system, as polymorphous and informational rather than industrial. Haraway sees society as transforming to one oriented around play and deadly games instead of work. The idea of using play to express the cultural/political condition in the sort of post-industrial “neo-Taylorist” setting seems, as Moulthrop notes, inherently troublesome.

This idea is the first step towards realizing the molecular society, though. The essence of Moulthrop’s connection is that games are emblematic of a culture of configuration, rather than consumption (or “mere” interpretation). Configuration enables a new response to media, and a new kind of awareness hitherto disabled.

A connection is made between these ideas and the restrictive nature perceived by the role of literary criticism as applied to games. The objection here is the classic cry of attack made by ludologists, but it is posed in the context of this larger cultural dilemma. A division is posed between drama and narrative as compared to play, simulation, and game. The idea is that the so called narrativists expect these media to serve the function of telling stories.

The change in focus that Moulthrop desires is to turn attention to the configurative power that users/players might have over works, as opposed to the interpretive ones. One of the fundamental things that is appealing about games is the process of participatory freedom, at least in ways of engaging with the game in multiple ways. For example, seeing what happens when the puzzle fails rather than succeeds.

However, it is important to note that even in the “most interactive” of games, there are severe limits to this configuration. In GTA, I might have the freedom to commit many kinds of crime, to customize my character, to run free about the city, but there is a significant limitation in that this is all it is ever possible to do. I cannot make the game about anything else. This may be a shallow criticism, but it is exactly the same constraint posed by the practice of interpretation. The only difference is that with configuration, the openness of the work to the player is inscribed and represented within the world of the work itself, but with interpretation it is only in the mind of the reader. This difference too is not severe, since ultimately, nothing is actually *changed* in the game, at least not in the way that others might play it. From the perspective of networked games or communities, this could easily mirror the status of interpretive communities that share and communally shape interpretations.

Further difference is claimed in Alternate Reality games (such as those for the film AI and the Beast for Halo), and emphasizing their procedural appeal. At the same time, narratives to have their procedural mechanics, and furthermore the appeal of these, the mystery, is rarely the ineffability of the “puzzleness” alone. I would argue that the puzzle solving and the fiction of these sorts of games are inseparable, or at least, it is their intricate and deep connection that makes the game so appealing.

Moulthrop finds that one of his main differences with Janet Murray is the role of transparency in games. This perspective of Murray’s seems to originate from Don Norman, who views transparency as ideal. All media eventually find some transparency (ie, over time the television ceases to be a box and turns into a window). However, part of the power of computers (as found by Sherry Turkle) is their immersiveness and holding power, which derive from their inherent opacity. Transparency is furthermore not even the ideal in literature, since open works are deliberately ambiguous and inexact, and even sometimes seemingly contradictory.

A conflict is posed here, between the consumptive values of transparent media versus the values of participatory media. The example given is Citizen’s Broadcast radio, which fell because people supposedly found nothing to say. The trend for passivity seems to change dramatically after the introduction of online participatory culture, but the suddenness of that is debatable. From DeCerteau, we might find that individuals do more with the things they consume than merely absorb them.

Moulthrop’s conclusion nonetheless ties together the values of configuration (what one might even call openness) to the ideas of participation and procedural literacy and criticism.

Overview

The overarching thesis is that the divide between nature and man, as proclaimed by modernism, is artificial and never existed. This divide enabled both science and humanities to make claims for the absolute truth. Concealed by this divide was the emergence of hybrids “quasi-objects” that blurred the lines between human and nature.

The intention of this is to re-examine the role and past of hybrids, and find a new relationship between science and culture. Essentially, to humanize science, and make humanities more scientific.

Notes

Chapter 1: Crisis

Latour opens by looking at the explosion of meaning systems that can be exposed by looking at a magazine. The magazine mixes science and politics and crosses many domains. “All of culture and all of nature get churned up again every day.” This issue stems from an enormous density of connections, where the myriad ways in which science and culture affect each other are exposed, leading a curious observer down the rabbit hole of connections.

Latour seems to be claiming that while this confluence of factors is so densely connected, knowledge is separated from power and politics. There is some sort of perceived divide between the two. Latour mentions a number of writers who have expounded on how technology shapes society, (MacKenzie on guidance systems, Michel Callon on fuel cells, Thomas Hughes on the incandescent lamp, etc.). Latour is trying to find that these relations are more than merely science or politics. The claim seems to be that a culture before a technology and after are very different. And that, despite the connections provided by magazines, the larger connections remain invisible.

Latour is concerned with the feasability of this sort of technological-cultural criticism. He wants to determine whether it is possible to effectively perform this analysis at all or not. His claim is that networks are elusive for deeper anthropological problems.

Latour describes the fall of the Berlin Wall in 1989, and compares it with the first conferences on global ecology. Both socialism and capitalism began with the best intentions (to abolish man’s exploitation of man, and to reorient man’s exploitation of man to an exploitation of nature, respectively), but both paths have gone to such extremes that they turned in on themselves. It then becomes a question over what path to take next. The “antimodern” approach is to no longer dominate nature. The postmodern approach is indecisive and incomplete, while others aim to continue and push towards the modern anyway.

The idea of “modern” is hazy and difficult to define, but it is evocative. Latour’s idea is to define modern by identifying two characteristics. One is the practice of translation, which creates hybrids of nature and culture. The second practice is purification, which aims to isolate meanings and create distinct ontological zones. Translation corresponds to the development of networks, while purification enables criticism.

Notes

Chapter 3: Intention and Artifice

This particular text is on photography, and its role as representing the truth. The credibility of photographs has, after a long period of time of claiming to represent the whole objective truth, or at least representing that something happened. This criticism runs deeper than a rejection of photographs as merely objective. The idea that the camera is a viewpoint, must take a physical position, and is thus inherently subjective has been around and acknowledged. However, the photograph still stood as a proof of existence.

Mitchell is concerned with the constructability of photographs and the fact that they are subject to the same kind of artifice that any other form of evidence is.

Photographs are fossilized light: momentary interpretations that have been made permanant by their exposure on film. They can be seen as records, but like any record, they construct a system and view of reality.

Photographs have a strong connection with matters of convention, for a long while, painting was rooted in matters of convention (Ancient Egyptian figures are a good example), but photography adheres to its own conventions as well. The characteristics of shading and lighting and all of these aspects are firmly grounded in the conventions of the photographic tradition. In turn, these conventions connect with our means of reading and perceiving information from signs encoded in this convention. Mitchell cites Roman Jakobson who explains that over the course of tradition, images become ideograms which are immediately mapped to ideas, and we no longer see pictures. This idea relates again to the traditions of media gradually becoming more transparent over time.

Photographs themselves tend to be deeply connected to their subjects, though: “since photographs are very strongly linked by contiguity to the objects they portray, we have come to regard them not as pictures but as formulae that metonymically evoke fragments of reality.”

This line of reasoning also seems to connect to the ideas of simulation sublimation. Images are seen as semiotic systems, and are internalized, or are, at least, taken to denote the truth when cast as such (which is how we have approached photography culturally). It is only recently that we have begun to exhibit a kind of awareness of the artificiality of the image that we can criticize photographs as non-representative of the truth. I imagine that similar arguments can be made toward simulation, but its artificiality is generally much more apparent.

The part of the camera that lends it its authority is its mechanical nature. Because it is a machine, seemingly untouched by other values or human intervention, it mechanically must capture the truth without the factor of human error. The ramifications of the “superiority” of machine reproduction and capture are documented (for instance with Walter Benjamin), but the near autonomous power of the machine emerges in a great deal of AI criticism.

Cameras impose the existence of a subject, though. It is impossible to take a photograph of “no particular” horse or person, though it might be possible to make a picture of them. The necessity of this particular subject is the essence of the camera’s intentionality. This intentionality is also where the camera’s subjectivity comes into play, and it introduces values based on the question of framing, perspective, and choice. Mitchell explains that this forms a spectrum: “However, Scruton’s distinction between intentional and causal components in image production is helpful, particularly if we do not insist on a clear cut dividing line between paintings and photographs but think rather of a spectrum running from nonalgorithmic to algorithmic conditions–with ideal paintings at one end and ideal photographs at the other.” (p. 29)

The problem that arises in comparing this to computation is that even algorithms have biases and values.

Digital photography and photo editing blur the lines between causal and intentional images. Something which is intentional can be hidden and be made to seem a natural causal element. This perverts the authority of the mechanical process by which photographs tend to derive their legitimacy.

Reading images requires a certain interpretive labor to be undertaken by the observer, and this process is where the influence of convention comes into play. “In forming interpretations of images, then, we use evidence of the parts to suggest possible interpretations of the whole, and we use the context of the whole to suggest possible interpretations of the parts.” (p. 34)

The problem of altering images involves balancing the alteration with the requirements for consistency and coherence in the relation of parts to the whole. Beyond that, we can check for implausibility, by comparing the subjects to that which we already know to be true (or might be able to verify externally). This sort of issue pulls back and seems to reflect ideas being worked through in AI and cognition. When faced with a black box that explains something that we have no means of verifying or checking, but are grounded in a photograph-like mechanic, we have no choice but to accept the result. The examples Mitchell gives are the photographs of the astronauts on the moon, but that idea can be extended to other regions, such as simulation.

Mitchell writes of the general acceptance of images: “In general, if an image follows the conventions of photography and seems internally coherent, if the visual evidence that it presents supports the caption, and if we can confirm that this visual evidence is consistent with other things that we accept as knowledge within the framework of the relevant discourse, then we feel justified in the attitude that seeing is believing.”