Speaking Code: Coding as Aesthetic and Political Expression

(by Geoff Cox, with code by Alex McLean)

A review by Kevin Brock, North Carolina State University

An Exigence: The Rise of Code as an Object of Critical Study

Book cover of Speaking Code

Cox, Geoff. Speaking Code: Coding as Aesthetic and Political Expression. Cambridge, MA: MIT Press, 2013. Print.

void speak(Value constraints, Context ecology) {
  audience = new Publics(constraints, ecology);
  audience.reception = argument.delivery(meaning);
  if (audience.reception == "effective") {
    audience.induceAction(meaning);
  } else {
    audience.reconsider(audience, argument.intent);
    argument.revise(meaning);
  }
}

The last decade has seen some extraordinary developments in regards to the critical study of digital technologies—and especially computer software—in relation to objects, and perspectives, more traditionally associated with the humanities. Among the most exciting of these are the arguments put forth, by scholars in the fields of rhetoric and software studies, that software and, just as importantly, the code that comprises it are forms of meaningful communication. This is an important assertion, not in the least because it suggests that code has, and transmits, meaning to human as well as nonhuman audiences.

However, there have been relatively few efforts to engage these two trajectories at any points of convergence (i.e., exploring the ways that code/software work as communication). It is heartening, then, to see a text like Geoff Cox’s Speaking Code, which suggests in its very title that such a discussion is not only timely but necessary for contemporary scholarship. Given the seemingly exponential increase in accessible frameworks for engaging with code languages (e.g. newcomer-friendly integrated desktop environments [IDEs] like Processing or Shoes, microprocessors like Arduino and Raspberry Pi, and even markup-related web packages like Twitter Bootstrap), it is crucial that we consider how software code speaks to us and how we speak to one another with code.

For rhetoricians like myself, these considerations often focus on the possibilities of invention and composition—the means by which we attempt to persuade audiences to engage in certain types of action. Cox offers an exciting and intriguing alternative perspective on this focus in that he attends to the concept of code as a form of speech (and thus meaningful communication) but does so in a way that only rarely incorporates the methods and vocabulary of rhetorical criticism into its fold. I mention this not to complain about Cox’s approach, but instead to draw attention to the distinction between how he discusses code as speech and how other scholars, from other backgrounds, might do so.

This review, then, focuses on Cox’s argument as it pertains to an argument in two parts: first, speech as a form of action (for public- and civic-oriented ends) and second, code as a form of speech. Because Speaking Code exists in print form, the code discussed by Cox is inherently neutered in terms of its potential (since it cannot be executed or modified easily as part of, or to demonstrate, Cox’s point in any particular case). As a result, I have incorporated several code examples to help illustrate how code can, and in some cases cannot, function effectively as speech action.

Addressing the Idea of Code as Speech

Among the most interesting, and commendable, aspects of Cox’s book is the way in which he has incorporated a range of code languages (or at least specific, applied examples from a range of languages) in order to discuss the variety of potential capacities for communicative action through code-based programs. That is, Speaking Code features a direct engagement with multiple code languages used for varied purposes rather than an oblique or incidental discussion about unspecified and uncontextualized code.

With that said, it is also important to note that this is also an extremely problematic quality of the book as well. Cox has not written any of the code included in the book—instead, Alex McLean provided the source for nearly all of the examples. On the one hand, this collaboration provides us with an opportunity to see how critical and productive activities can be expanded through the application of multiple perspectives. On the other hand, one might be led to wonder why Cox did not pursue his own composition of possible code examples. This is not to suggest that Cox needs to have learned all the programming skills necessary for the composition of the included code, but it does raise a question about the accuracy and clarity of Cox’s criticism (how much did Cox know, and how much did McLean need to translate for Cox, about what the various contributed programs “do”?).

This canvas figure requires Javascript to be enabled.

Figure 1. Processing sketch in which “good” code is presented to the user. Click the canvas to clear it and redraw. Source code available here.

Based on Codewell by Annette Vee, originally written in the Scratch language (available via GitHub).

Cox begins by addressing the question of code as a form of speech, teasing out complex potential through an introductory genre of code, the Hello World program meant to help a programmer new to a given language learn its fundamental principles and syntax. By demonstrating and discussing how Hello World functions across languages, Cox grounds his argument for code as speech by focusing on how code allows a programmer to engage in certain communicative activities facilitated by code. (Minor aside: I admit that I hoped for at least a brief discussion of genre studies here, as it would have been entirely appropriate in regards to exploring how Hello World, like other genres, has been constructed through social activity, assuming a set of conventions and rhetorical goals.) In particular, Cox stresses the existence of code as something [m]ore than simply writing, [it] is a special kind of writing (11). This statement is Cox’s central claim: code allows us to talk—to one another and, to a less frequently addressed extent, to technological systems—in ways and for purposes that are simply unavailable in other forms of communication.

If we (i.e., critics of rhetoric, software, and culture) are to consider the potency of code as a type of speech,then it is imperative that we recognize the limitation of one medium in regards to the expressive capabilities of another. Cox writes about code but never contributes himself to any sort of generative or productive code-based action. In other words, he is attempting to speak simultaneously from a position of awareness and power (the voice of the informed critic) and from a position lacking that awareness (the voice of the individual who does not write code). A recognition of this paradoxical stance is key to understanding Cox’s argument, which suggests that code as a means of communication is still not understood well even by those claiming to study it: can a conventional text argument serve as an appropriate, or even a relevant, way to critique code and its potential? This question is never fully answered by Cox, although the fact that the text is a print book may be one implicit means of responding to the question.

Speaking in, and through, Code

Following his introductory chapter, Cox sets out to distinguish prior theory about code (that is, claims about code generally as a type of language) and theory of code as a unique and special means of writing and speaking. He does this by calling on indeterminism and contingency, looking at the distinction between potential and realization; the resulting discussion turns from distinguishing between code and comment lines—by decidedly human authors (i.e., programmers) for arguably differing audiences of human and machine—to purpose-oriented acts in code, as speech, undertaken by human and machine speakers/agents alike.

Graphic of example instructions from John Searle's 'Chinese Room' experiment

Figure 2. Example set of instructions for Searle’s “Chinese Room” experiment.

Figure taken from Anderson & Cox’s Searle and the Chinese Room Argument.

Cox’s chapter turns from indeterminism to intelligence, and Cox questions what it [would] take for a machine to speak convincingly (27). This concern, as Cox notes, has been debated for some time, and Cox hints at the debate between vocability and signification—a distinction (not mentioned at all by Cox) at the center of the work of influential rhetorician Kenneth Burke, who famously set apart nonsymbolic motion from symbolic action. More recently, efforts in Object-Oriented Ontology (OOO) and Object-Oriented Rhetoric (OOR) (for more, see Harman, Tool-Being; Bogost, Alien Phenomenology; or Barnett, Towards an Object-Oriented Rhetoric) have pursued questions of meaningful existence and activity for nonhuman entities, both in relation to and separate from that of humans. There are numerous intriguing connections to be made for scholars interested in exploring how code might work as action (and to facilitate action), and Cox makes the most headway in this regard when he calls on Wittgenstein to help critique John Searle’s reverse Turing test. Specifically, Cox-via-Wittgenstein recognizes the role of a larger environment or ecology on a given speech act: The meanings of words are not derived from an inherent logical structure alone that manipulates symbols into particular sequences (like a program), but also from their social usage (32). This line of argument has its support in the form of the social turn, a theory on discourse popular among scholars of rhetoric and composition (as well as critics in other fields). The social turn refers to a recognition of writing—or any form of communication—as socially constructed, with language (which becomes meaningful through dialogic conventions of specific use, tone, and so on) a fundamental component for thought.

Code act and speech act

Drawing on N. Katherine Hayles’ ideas of code signifiers as interpretations of electrical voltages, Cox takes the curious step of separating out the idea of a code act from that of a speech act, a step that seems to move somewhat backwards from the direction in which he had been traveling through this chapter. It is significant that Cox turns to Hayles to argue for the uniqueness of a code act, which for Cox is a special kind of human-machine writing that makes things happen, although he then states this means that it acts like speech (34-35). As Hayles argued for distinctions between natural language communication and code in multiple texts, most notably My Mother Was a Computer, the influence of her relatively problematic position on Cox’s argument is clear despite his conclusion that code is a kind of speech. (Hayles stressed that code, because of its machinic audience, could not work in the same ways as literature or speech.)

This canvas figure requires Javascript to be enabled.

Figure 3. Example meaningful code act whose color changes based on user activity, based on a combination of red, green, and blue color values. Red and green values are computed from the mouse’s horizontal and vertical position in the window, while clicking the mouse will set a randomized blue value. The bottom half of the canvas is masked in part by a translucent rectangle whose color and opacity values are based on your IP address. Source code available here.

In regards to speech act theory, Cox turns to John Langshaw Austin and his argument on saying as doing, positioning speech as a kind of action that is not merely informative (i.e., transmitting content) but performative, serving as a way to persuade an audience to, in turn, do something with the message provided through a given speech act. Cox notes that speech is highly complex and not simply reducible to instrumental techniques and algorithms, (36) suggesting the same is also true of code, as it means more than the output of its computational operations. This identification of code as doing more is significant as it offers a skeptical reader an entry into understanding why any speech act must be understood as a meaningful event whose components, and impact, extend beyond the bounds of its message content. One may wonder, however, how code—whose performance is based on the sort of truth-value statement that Austin de-emphasizes for performative speech—might be seen as a different kind of saying or doing rather than as an extension of speech and its activity.

Cox’s reliance on Austin is also intriguing in that there are other, more contemporary texts that are arguably more immediately important to the current conversation on code and its relationship to twenty-first century communicative action. (Or, as an alternative, one might wonder why Cox did not title the book How to Do Things with Code or perhaps How to Word Things through Code as a direct homage.) Specifically, Adrian Mackenzie’s work on the performativity of code is conspicuously absent here: Mackenzie is not mentioned until the following chapter, and then only briefly in order to raise questions about agency in relation to performativity—that is, does code as a performative speech demonstrate programmers’ identity or technological systems’ identities working together? (63) While a case might be made that Mackenzie is not really interested in code utterances or acts, Cox’s point would be far better served if he were to draw connections between Austin and Mackenzie’s separate arguments for speaking and programming as performative engagements. Such connections could then explicitly bridge the speech-code gap under examination, whose relationship regarding action and act comprises much of the case made in this chapter.

Code as Work and Action

What may be Cox’s most compelling argument is found in his third chapter, which focuses on the idea of code working for varying purposes and in myriad ways. This argument is split, roughly, into several parts: first, the production of code as a form of labor; second, the performativity of code’s production; and third, the action that code facilitates through its production and its execution. Not surprisingly, the procedural approach Cox takes here to making his argument reflects the procedural nature of computation (with operations building upon operations to reach an anticipated result).

Cox briefly calls upon Hegel and Marx to help set up the idea of a program’s history, the forces that have been exerted upon past generations of agents involved in its creation and use, noting the emergence of possibility for a given code or speech act through contextual contingency or potentiality, resulting in the movement of agents from essence (Hegel’s in-itself) to actuality (Hegel’s for-itself).

Cox draws together Wolfgang von Kempelen’s eighteenth century “mechanical turk”, a chess-playing automated machine secretly run by an individual hiding under the machine’s chess table, and Amazon’s “Mechanical Turk,” an enterprise for the assignment of specific labor tasks to human workers. While admittedly an easy connection to make, Cox’s argument is that workers involved in Amazon’s “Mechanical Turk” are not perceived to be part of a computational labor system but instead are told that what they do is distinct from the menial work of code (as the tasks are often just nuanced enough to make automated algorithmic construction difficult). As a result, workers’ role in this labor system is valorized for being special in this way—when in reality, the human work is another metaphorical cog in the computational structure, which transforms labor into information while stripping away the context of its generation and composition (effectively reducing the worker’s role to an equivalent of the algorithmic activity in automated code).

This sort of labor system is then contrasted with the variety of software licenses that enable different labor philosophies (most notably the Creative Commons set of licenses and the GNU General Public License, or GPL). More specifically, Cox looks at how these licenses attempt to redefine notions of property and labor organization so as to operate against the conventional sociopolitical paradigms underlying action in and around code (or other activity). These licensing movements are compared with the idea of increasingly complicated human-machine assemblages (vis-a-vis Guattari) that do not eradicate human roles but mutate them in accordance with individuals’ emerging abilities—and constraints— as agents in such assemblages.

The accompanying aside (Aside 1) reconfigures this section’s argument through probabilistic Markov chain (n-gram) generation, a way to develop semi-random grammars and texts from a source corpus. In regards to the generated example, one may ask: whose voice is speaking? What is involved in the work of constructing, delivering, and interpreting the reconfigured argument? When you, the reader, are invited to write something new, how are you involving yourself in the writing process? What exactly is being written (or, perhaps more appropriately, enacted)?

Such questions about the participation of a user (or even a developer) in the action that takes place through, and because of, code execution are integral to considerations of code as speech. That Cox is mostly silent on the topic is intriguing (save when he discusses the liberatory power of code through relevant literacies), but there have been several scholars who have explored those considerations. Of particular note is Annette Vee, who examined in her article Text, Speech, Machine the history of code-as-speech in U.S. law (in part contrasting the views that many programmers have of certain populations making and using code with the views that many lawmakers have had over the last several decades). This is not to imply that Cox should have spent more space focusing on the legal ramifications of code as speech—just to note that such an argument has been taken up, and that it has an effect on the sort of conversation that can be had subsequently regarding various publics and their access to code.

The performativity of code as addressed in the third chapter focuses on live coding, an event of simultaneous development and exhibition in which a programmer writes, edits, and iterates his or her code continuously in a looping, mutating demonstration of the effect of code composition on an unfolding activity (which, in the case of live coding, is often code-based musical performance). Live coding offers a means of articulating the influence of programmer on code and code on programmer, with the abilities and constraints of each agent influencing the activities of both. As a developer tweaks his or her code, the changes in the program’s output in turn affect subsequent programming decisions made by the developer to tweak that code (and its output) further. According to Cox, neither agent is entirely in charge, so to speak; the action of a live coding performance is contingent and improvisational, which suggests its definition as a form of speech act. A related act is the performance of Free Radio Linux, in which the entire Linux kernel’s code is read on a webcast via speech synthesizer in a way that simply cannot be imitated or reflected through the code’s distribution through software packages (of various Linux systems) and online repositories of code:

The performance encapsulates the way that coding works in its fuller sense, as already described in relation to live-coding performances, and underpins the tensions around contemporary labor and coding practices. It produces an uncertain relation between the code object and code subject—the program and the programmers—and thus challenges property relations bound to the development and distribution of code. Like live coding, it is an example of machinic expression at the level of embodied communication or speech. (65)

The chapter ends with a discussion on networked production and action (although Cox, like Arendt, distinguishes the context of production-as-making with that of action-as-itself), suggesting that the work practices in which contemporary information workers engage extend far beyond their conscious activities. Specifically, Cox argues that the technological assemblages of which they are a part exert particular influences and tendencies upon them. In many cases—such as those of Apple’s app marketplace, where the manufacturer oversees submissions of applications, or in a compilation of two thousand contributors singing “Daisy Bell” for a task on Amazon’s “Mechanical Turk”—the code/speech act is aggressively locked down, underlining how subjectivitiy in the form of the voice is somehow captured and fragmented in the use of these telecommunication devices (67).

This focus on action as a definitional argument (distinguishing action from making/production) is insightful but not as comprehensive or focused on the quality of action as it could be—although, as noted early in this review, my complaint comes from a rhetoric-oriented perspective. That is, Cox is primarily interested in bringing an economic aspect of labor into the conversation on software and its influence on activity and action; this is perfectly appropriate but does not lead to much discussion of what that labor action is or could be, beyond the scope of working in favor of or against particular systems and paradigms of production.

Code and Its Publics

Cox argues that the public (or, really, the multiple publics that comprise western society) has several major obstacles set before it if there is to be any actually meaningful action and upheaval in and through code. For Cox, the political possibility of code-based action involves several major schemes, each of which is briefly discussed below.

Literacy

Perhaps the greatest obstacle is literacy in relation to code and computational logic—specifically, one’s ability to manipulate the technologies through their symbolic languages. If an individual, or an entire community, cannot gain access to the means of effecting action by which that individual or community could change the terms of a particular situation, then how might that population ever break from its current bounds? As Cox adroitly observes, this literacy is critical if any political action is ever to occur.

Freedom

For Cox, it is this literacy that offers the greatest potential for public freedom, especially in conjunction with models of public domain ownership and peer-based production. This sort of production is best demonstrated, Cox claims, through the Creative Commons and the GPL, both of which move away from a view of ownership tied to individuality common in western society (76). However, as Cox notes, these models do pose some problems for our understanding of traditional productive activities of publics (Ibid).

Control

In any given public, how much freedom is possible, and what forms and systems of control are available (to any entity) in order to constrain that freedom? For code and software, this involves literacy but also a motivation to act, which can be compromised rather easily with online “slacktivism,” a sort of non-action that allows one to feel involved in, say, a political protest without having to leave one’s computer. Or: given the rise of collaborative peer production, how strongly does a desire for autonomy motivate someone to participate—or not?

The examples that Cox turns to in this chapter deal primarily with ways that users can react to, or separate themselves from, the major social networks and websites that exert varying forms of control on them (or that restrict the potential for freedom in expression and action that might otherwise be possible). One of the most interesting discussed is Samy Kamkar’s Samy worm, used to generate hundreds of thousands of MySpace friends in a matter of hours and display a string on their profiles about how Samy was their hero (and this worm would later gain Kamkar a felony conviction). This example is intriguing in that it demonstrates the consequences an individual can face when attempting to circumvent the control systems currently in power. That is, Kamkar’s desire to communicate across the MySpace network with as many other users as possible is viewed as a threat to the authority and security of the network. As Cox notes, publicness is constituted not simply by speaking, writing, arguing, and protesting but also through modification of the domain or platform through which these practices are enacted, making both technology and the law unstable (93). Kamkar’s attempt to engage and create a public through his Samy worm resulted not in a liberated awareness but in his arrest. The autonomy that might be praised as a prominent quality in the code which facilitates most of our online activity is, in fact, already locked down by those in control of our networks and technologies.

Cox raises some significant questions at the end of the chapter about the ways in which an individual—and by extension, a public—can achieve the freedom that code supposedly provides. How it might act for-itself (to recall Hegel and Marx) rather than against forces of oppression and control? (Can it?)

To Code and Code Not

Cox concludes his book by addressing the concept of code for-itself and considers the possibility of code having an activity in the world and a lived body, indeed what is intimately connected to a social body (100). This embodied quality of code is important as it allows us to interrogate the argument of code as speech (and thus also as action). Cox’s critique of the coding public as a necessity that is not yet fully realized is important, as are his suggestions of ways that a code-literate public might emerge through direct and indirect action.

However, at the same time, Cox appears to be himself outside of the coding public—after all, the code he examines was written and spoken by others (from Alex McLean to various individuals and groups on the internet). Further, by printing the code of interest in book form—as ink on paper—it is honestly impossible to see that code fully in action. Those readers who can understand how it works can anticipate the code’s speech activity as it runs, but we are otherwise left examining an artifact frozen in a sort of non-speaking limbo, what Wendy Chun has referred to as a fetish: The knowledge that source code offers is no cure for source code fetishism ... the key question is not ‘what do we know?’ but rather ‘what do we do?’ (53). While Cox has argued for a general potential for action made possible through code, what he has left unstated is a means of engaging code directly as a literacy and a means of acting.

Ultimately, we are left to weigh these prospects: can code actually help individuals, communities, and publics speak in ways that have been heretofore unavailable? Perhaps. But how can they do so, given the obstacles placed between them and their goals? This seems to be the current site of political struggle in relation to speaking in and through digital technologies. Cox provides an intriguing argument that certainly raises awareness of code as a legitimate means of communication but he does not offer a path by which publics might reach the point at which they can use those means for effective, widespread change.

Works Cited

  • Barnett, Scot. Towards an Object-Oriented Rhetoric. Enculturation 7 (2010): Web.
  • Bogost, Ian. Alien Phenomenology, or What It's Like to Be a Thing. Minneapolis: U of Minnesota Press, 2012. Print.
  • Chun, Wendy Hui Kyong. Programmed Visions: Software and Memory. Cambridge, MA: MIT Press, 2011. Print.
  • Cox, Geoff. Speaking Code: Coding as Aesthetic and Political Expression. Cambridge, MA: MIT Press, 2012. Print.
  • Harman, Graham. Tool-Being: Heidegger and the Metaphysics of Objects. Peru, IL: Open Court, 2002. Print.
  • Hayles, N. Katherine. My Mother Was a Computer: Digital Subjects and Literary Texts. Chicago: U of Chicago Press, 2005. Print.
  • Vee, Annette. Text, Speech, Machine: Metaphors for Computer Code in the Law. Computational Culture 2 (2012). Web.