Overview

Dourish writes of “embodied interaction”. This idea is meant to connect the realms of HCI, interfaces, and design with that of continental phenomenology. Dourish’s premise is that HCI has learned a great deal from phenomenology (specifically in the developments of tangible and social computing), but stands to gain a great deal more from applying the principles of embodiment and being to computational artifacts rather than persisting with conventional procedural metaphors. Specifically, Dourish claims that the sense of humans as being subservient to computers (originating in early history of computer science) still remains strongly today. While Dourish’s discussion is meant to give a new perspective on HCI, his book also gives some insight into how simulated agents might experience embodiment and may be “phenomenologically sympathetic”. Embodied interaction may be seen one way as human to computer, but potentially between simulated agents and their artificial worlds.

Notes

A History of Interaction

Dourish opens, describing that “Embodied Interaction is interaction with computer systems that occupy our world, a world of physical and social reality, and that exploit this fact in how they interact with us.” (p. 3) Embodied interaction shifts emphasis from the procedural model to a more process based model, specifically based in things like Milner’s Pi Calculus, and Rodney Brooks’s robotics. (p. 4)

Dourish also makes note that the original “computers” were real people whose careers were doing calculations. (p. 6) This is interesting for comparing computer based simulation to rule based simulation to human imagination. Removing the digital element from computing, there is a lot of room for error and creativity within human processes (necessarily, mechanization tries to stamp this out), but it is ironic to imagine a digital story simulation being carried out by live individuals. The procedural aspect seems necessarily very restrictive in the creative role. However, both stem from this idea of a rule based system, but where humans are challenged and made more creative by the constraints implied by rules, digital systems seem to be limited.

One of the means of interaction that Dourish discusses is the textual form. Here he is primarily referring to command line interfaces, such as those seen on a command shell. These are textual, text, sort of, but not really. Not in the sense that a letter to a friend, IM conversation, or verbal instructions are textual or linguistic. Dourish notes that conversation and dialogue are now integral to our understanding of interactivity. The purpose of such conversation is implicitly assumed to be to give instructions to run computer tasks, not for actual social engagement. “Textual interaction drew upon language much more explicitly than before, and at the same time it was accompanied by a transition to a new model of computing, in which a user would actually sit in front of a computer terminal , entering commands and reading responses. With this combination of language use and direct interaction, it was natural to look on the result as a ‘conversation’ or ‘dialogue’.” (p. 10)

In the discussion of visual metaphor, Dourish describes the visual interaction as a more direct form of engagement, wherein the user interacts with abstract objects in a direct and concrete manner. These objects and interactions are represented symbolically and visually, creating a world of metaphors wherein the system of concepts is complete and consistent. “From these separate element, the designer builds an inhabited world in which users act. Direct manipulation interfaces exploit and extend the benefits of graphical interaction.” (p. 13) While this approach is interesting from an interface perspective, it also renders the computer world as a Baudrillardian simulation. So the representative power of simulation becomes most clear at this level, when metaphors and direct interaction become present in interfaces.

Social and tangible interfaces are grounded in embodied interaction, which is at odds with a positivist Cartesian ‘naive cognitivism’ which gives a very dualist take on interaction, with a heavy emphasis on symbolic representation. Embodiment claims that cognition without a body is fallacy, and embodied interaction exploits the corporeality of its interactors. Embodiment also implies a presence, and hence, participation: “Embodiment, instead, denotes a form of participative status. Embodiment is about the fact that things are embedded in the world, and the ways in which their reality depends on being embedded.” (p. 18) So, we can extend this to thinking about agents and their participative nature.

Being in the World: Embodied Interaction

In the next section, Dourish spends time discussing various phenomenologists (specifically Husserl, Heidegger, Schutz, and Merleau-Ponty), and their potential application towards HCI. He starts with two definitions of embodiment, the second is this: “Embodied phenomena are those that by their very nature occur in real time and real space.” (p. 101) Looking at people interacting with computers, Dourish asserts that people respond physically, directly, and kinetically with the world around them in a tangible manner, and that operating through a computer abstracts this, even in immersive environments, users are operating in opposition to interfaces. We do not need to do planning in engaging with the world as we do with interfaces. However, this seems a dubious claim: many people have great trouble engaging with the world (since it does not have interfaces, but it does have protocols and social conventions). Many people with WoW or SL addictions tend to interact more seamlessly in those worlds than they do in reality. Dourish specifically examines 3d interfaces, wherein a user navigates a world with keyboard and mouse, and notes that our operation with the world is not of this nature- that we do not have a homunculus sitting inside our head observing through our eyes and controlling indirectly our actions (which sounds like Searle and AI). This is the difference between player and avatar, and this relationship, I think, is much more complex and nuanced, and has the capacity to be much tighter than presumed here. (p. 102)

On Husserl: Husserl wanted to carry Cartesian dualsim to address the phenomena of experience. Specifically, he was dissatisfied with the abstraction of mathematics and science, and wanted to “develop the philosophy of experience as a rigorous science”. Moreover, he drew lines between objects of consciousness and objects of intentionality (the Cartesian duals of objects). These sound like they might relate to the socially enacted objects of Mead. Objects of intentionality are “noema”, and our mental consciousness of these objects are “noesis”. Underlying the concept of noema is a platonic conception of essence. (p. 105-106)

On Heidegger: Heidegger rejected Husserl’s dualism, and emphasized that the mental and physical spaces are deeply connected. “Essentially, Heidegger transformed the problem of phenomenology from an epistemological question, a question about knowledge, to an ontological question, a question about forms and categories of existence. Instead of asking, ‘How can we know about the world?’ Heidegger asked, ‘How does the world reveal itself to us through our encounters with it?'”. This change in question is a focal point for HCI and interaction. (p. 107)

On Schutz: Schutz looked on phenomenology as applied to social action. Social enaction is rooted in shared experience, which is phenomenological in nature. Collective action depends on intersubjective understandings of the world. “Schutz argued that the meaningfulness of social action had to emerge within the context of the actor’s own experience with the world.” (p. 111)

On Merleau-Ponty: The focus here is on the phenomenology of perception. Merleau-Ponty wished to reconcile Husserl’s philosophy of essences with Heidegger’s philosophy of being. This involved a change in perspective of the role of the body in experience. “For Merleau-Ponty, the body is neither subject nor object, but an ambiguous third party.” To understand the body, one must understand perception. This is an interesting approach towards embodiment, since his treatment of the body sounds very applicable to the approach necessary for simulated agents. Simulated agents do not *have* bodies, but they must be embodied within their world, and to address this problem, one must turn to the matter of peception. Merleau-Ponty goes on to emphasize a reversibility in perception (that others may perceive ourselves? Can get very Lacanian here), which means that we can apprehend “perceptions of ourselves that we engender in others”. This work was done by Robertson in 1997, and sounds very similar to Goffman’s performance of the self. (p. 114-115)

Ultimately, the phenomeonlogists have explored the relationship between embodied action and meaning. Meaning, to them, is found in the world with which we are in constant contact and engagement. Meaning can be found via the world revealing itself to us and affording for us actions to perform upon it. (p. 116) This sounds very similar to the system of affordances developed by J.J. Gibson and later Don Norman. More on this: “In other words, an affordance is a three-way relationship between the environment, the organism, and an activity. This three-way relationship is at the heart of ecological psychology, and the challenge of ecological psychology lies in just how it is centered on the notion of an organism acting in an environment: being int he world.” (p. 118) And again, this three-way relationship sounds like a model for simulated behavior. Michael Polanyi makes a significant investigation of embodied skills which require a “knowing how” versus “knowing what”. (p. 119)

On Wittgenstein: he used embodiement in relation to language. In the linguistic tradition, “He argues that language and meaning are inseparable from the practices of language users. Meaning resides not in disembodied representations, but in practical occasions of language use.” (p. 124) The return to language is very interesting here, from the departure to visual and tangible interfaces.