weaver++

Introduction

WEAVER++ (Word Encoding by Activation and VERification) is a computational model designed to explain how humans plan and attentionally control the production of spoken words. The model tries to make explicit the many strands of knowledge and skill that human individuals must weave together in the process of spoken word production and the different mixtures of strands used in different word production tasks. The model falls into the general class of "hybrid" models of human performance in that it combines a declarative associative network and procedural rule system with spreading activation and activation-based rule triggering (cf. ACT-R of John Anderson and colleagues). The distinction between procedural and declarative aspects of word planning is reminiscent of the distinction between mental acts and contents advanced by Külpe based on the work of his Würzburg group in the early 1900s, and it is supported by accumulating empirical evidence (e.g., the work of Michael Ullman and colleagues). The model plans spoken words by activating, selecting, and connecting (weaving together) types of verbal information.

WEAVER++ gives detailed accounts of response time findings on spoken word production—in the tradition of Donders, Cattell, and Stroop. WEAVER++ has also been applied to eye tracking, electrophysiological, and neuroimaging data.

The model has been developed for Germanic languages like Dutch and English, but work on model versions for Mandarin Chinese and Japanese is in progress.

Horizontal and vertical threads

In their classic article "Attention to action: Willed and automatic control of behavior", Norman and Shallice (1986) made a distinction between "horizontal threads" and "vertical threads" in the control of human performance. Horizontal threads are strands of processing that map perceptions onto actions and vertical threads are attentional influences on these mappings. Characteristics of performance arise from interactions between horizontal and vertical threads. WEAVER++ implements specific claims about how the horizontal and vertical threads are woven together in language performance.

A central claim embodied by WEAVER++ is that the control of language performance is achieved through symbolic rules (cf. EPIC of David Meyer and David Kieras) rather than purely associatively. WEAVER++'s lexical network is accessed by spreading activation while the condition-action rules determine what is done with the activated lexical information depending on the task. When a goal symbol is placed in working memory, the attention of the system is focussed on those rules that include the goal among their conditions.

WEAVER++ plans words by incrementally extending verbal goals—i.e., lemmas are selected for lexical concepts, morphemes for lemmas, segments for morphemes, and syllable motor programs for syllabified segments, whereby the syllabification of segments proceeds incrementally from the beginning of a word to its end. The idea of incrementality in language production originates with Wundt. Furthermore, WEAVER++ is an attempt at incremental or cumulative modeling, i.e., an attempt to extend a model in new directions and to new phenomena by building on earlier modeling results.

Kinds of selective attention

In performing tasks requiring selective attention, such as the color-word Stroop task (e.g., name the ink color of the word GREEN; say "red"), WEAVER++ employs at least two kinds of selective attention, referred to as "stimulus set" and "response set" by Donald Broadbent (Decision and Stress book, 1971). Stimulus set ("filtering") refers to selection on the basis of a perceptual attribute, such as spatial location, color, shape, or temporal order. Response set refers to selection on the basis of the vocabulary of allowable responses. Task performance may require one or both of these kinds of selective attention:

Roelofs, A. (2003). Goal-referenced selection of verbal action: Modeling attentional control in the Stroop task. Psychological Review, 110, 88-125. Article (PDF 585K)

From Wernicke to WEAVER++

In the early days of experimental psychology, Wundt (1904, Principles of Physiological Psychology book) criticized the now classic, associative Wernicke-Lichtheim model of word production and perception by arguing that the retrieval of words from memory is an active goal-driven process rather than a passive associative process, as held by the model. According to Wundt, an attentional process centered in the frontal lobes of the human brain controls a word perception and production network located in perisylvian brain areas, described by the Wernicke-Lichtheim model. WEAVER++ builds in many respects on the Wernicke-Lichtheim model, but also addresses Wundt’s critique by implementing assumptions on how the production-perception network is controlled. Characteristics of language performance, such as latencies and errors, arise from interactions between the lexical network and the control system. For example, patterns of speech errors by aphasic and nonaphasic speakers seem determined, at least in part, by self-monitoring, which is an important attentional control function:

Roelofs, A. (2011). Modeling the attentional control of vocal utterances: From Wernicke to WEAVER++. In J. Guendouzi, F. Loncke, & M. J. Williams (Eds.), The Handbook of Psycholinguistic and Cognitive Processes: Perspectives in Communication Disorders (pp. 189-207). Hove, UK: Psychology Press. Article (PDF 1464K)

Roelofs, A. (2004). Error biases in spoken word planning and monitoring by aphasic and nonaphasic speakers: Comment on Rapp and Goldrick (2000). Psychological Review, 111, 561-572. Article (PDF 150K)

WEAVER++ on reading and dyslexia

Since the seminal work of Denckla and colleagues in the early 1970s (based on Norman Geschwind's hypothesis that color naming might predict reading), numerous studies have demonstrated that, in addition to phonological deficits, the majority of children and adults with reading disabilities also exhibit pronounced difficulties on naming-speed tasks, such as tests of "rapid automatized naming" (RAN). These tests require simple objects, colors, letters, or numbers to be named as quickly and accurately as possible. Naming speed is highly correlated with performance on word identification tasks, word reading efficiency measures, and measures of reading comprehension. Naming speed predicts later reading ability and helps identify risk at dyslexia in pre-literate children. Dyslexic readers are also known for their poor performance on Stroop color naming. Reading ability is negatively related to Stroop interference. Evidence suggest that attention mechanisms are critically implicated in reading and that disruption of these mechanisms may play a role in reading difficulties and dyslexia.

WEAVER++ provides functional analyses of object, color, and digit naming as well as word reading, and the model makes explicit how attention determines naming and reading. Moreover, the model provides an account of Stroop task performance and explains the negative linear relationship between reading skill and Stroop interference. It has been suggested that RAN is related to reading because reading recruits object-naming circuits in the left cerebral hemisphere. WEAVER++ makes explicit the connection between reading and object naming, both in functional and anatomical terms.

Roelofs, A. (2006). Functional architecture of naming dice, digits, and number words. Language and Cognitive Processes, 21, 78-111.Article (PDF 176K)

Protopapas, A., Archonti, A., & Skaloumbakas, C. (2007). Reading ability is negatively related to Stroop interference. Cognitive Psychology, 54, 251-282. doi:10.1016/j.cogpsych.2006.07.003

WEAVER++ on specific language impairment

Evidence suggests that (subclinical) attention deficits also contribute to the impaired language performance of individuals with specific language impairment (SLI). This is a disorder of language acquisition and use in children who otherwise appear to be normally developing. The disorder may persist into adulthood. Difficulties concern language production (expressive language disorder) or both production and comprehension (mixed receptive-expressive language disorder). The features of the impaired language performance in SLI are quite variable, but common characteristics are a delay in starting to talk in childhood, deviant production of speech sounds, a restricted vocabulary, slow and inaccurate picture naming, and use of simplified grammatical structures, including frequent omission of articles or plural and past tense endings (for a review, see Leonard’s 1998 book Children with specific language impairment). In general, individuals with SLI seem to have a problem in dealing with relatively complex language structures, in both speech production and comprehension. A prominent account of SLI holds that these difficulties with complexity in language reflect a reduced capacity of systems underlying language processes, resulting from a limitation in general processing capacity (see the work of Laurence Leonard and colleagues). Moreover, it is becoming increasingly clear that attention deficits contribute to SLI.

Individuals with SLI appear to have reduced working memory capacity, as assessed by pseudoword repetition and listening span tasks. Moreover, individuals with SLI have deficits in sustained attention and attentional control. Capacity restrictions concerning language processes, working memory, and attention influence word planning in WEAVER++. For example, a capacity restriction in activating or selecting morphemes for a lemma may result in omission of inflectional morphemes, such as plural and past tense endings. This type of problem will be reinforced by capacity restrictions in working memory and attention. For word planning to be successful in the model, attention needs to be sustained until the phonological form has been planned and syllable motor programs may be accessed. Difficulties in maintaining attention will impede the planning process, especially when a complex mapping between levels is involved (e.g., such as the mapping between lemmas and morphemes).

Janssen, D. P., Roelofs, A., & Levelt, W.J.M. (2002). Inflectional frames in language production. Language and Cognitive Processes, 17, 209-236. Article (PDF 246K)

Levelt, W.J.M., Roelofs, A., & Meyer, A.S. (1999). A theory of lexical access in speech production. Behavioral and Brain Sciences, 22, 1-38. Article (PDF 693K)

Roelofs, A. (1996). Serial order in planning the production of successive morphemes of a word. Journal of Memory and Language, 35, 854-876. Article (PDF 249K)

Roelofs, A. (2006). Context effects of pictures and words in naming objects, reading words, and generating simple phrases. Quarterly Journal of Experimental Psychology, 59, 1764-1784. Article (PDF 172K)

Roelofs, A., & Piai, V. (2011). Attention demands of spoken word planning: A review. Frontiers in Psychology, 2, article 307. Article (PDF 976K), doi: 10.3389/fpsyg.2011.00307

Attention and gaze in dual-task performance

Individuals often perform two tasks concurrently or in close succession, such as talking while driving a car, preparing a meal, or manipulating a computer mouse. The ability to cope with such dual-task situations depends on an individual's ability to coordinate cognitive processes across the tasks at hand. Eye movements need to be coordinated between tasks if the stimuli for the two tasks have different spatial positions. In a simple task used in my laboratory, speakers name stimuli (e.g., the ink color of the word GREEN) and shift their gaze to an arrow (e.g., < or > flanked by two Xs) to manually indicate its direction, see figure below. WEAVER++ describes how an attentional control process coordinates the multiple threads of processing in vocal responding, gaze shifting, and manual responding:

Roelofs, A. (2007). Attention and gaze control in picture naming, word reading, and word categorizing. Journal of Memory and Language, 57, 232-251. Article (PDF 311K)

Roelofs, A. (2008). Attention, gaze shifting, and dual-task interference from phonological encoding in spoken word planning. Journal of Experimental Psychology: Human Perception and Performance, 34, 1580-1598. Article (PDF 377K)

Planning stages

The model distinguishes between conceptual preparation, lemma retrieval, and word-form encoding, with the encoding of forms further divided into morphological, phonological, and phonetic encoding. During conceptual preparation, concepts are flagged as goal concepts. In lemma retrieval, a goal concept is used to retrieve a lemma from memory, which is a representation of the syntactic properties of a word, crucial for its use in sentences. For example, the lemma of the word red says that it can be used as an adjective. Lemma retrieval makes these properties available for syntactic encoding processes. In word-form encoding, the lemma is used to retrieve the morphophonological properties of the word from memory in order to construct an appropriate articulatory program. For example, for red the morpheme <red> and the speech segments /r/, /e/, and /d/ are retrieved and a phonetic plan for [red] is generated. Finally, articulation processes execute the motor program, which yields overt speech.

Assume a speaker wants to refer to the ink color of the word GREEN. This involves the conceptual identification of the color based on the perceptual input and its designation as goal concept (i.e., RED(X)), the retrieval of the lemma of the corresponding word (i.e., red), and the encoding of the form of the word (i.e., [red]). The final result is a motor program for the word "red", which can be articulated.

In performing the color-word Stroop task, aspects of word planning are under attentional (willed, executive) control. The system has to achieve color naming rather than word reading ("goal control") and the irrelevant input—the word in color naming—has to be suppressed ("input control").

Neural correlates

Word planning and attentional control in the Stroop task are mediated through an extensive network of brain areas. The figure below shows a lateral view (top panel) and a medial view (bottom panel) of the left hemisphere of the human brain. The word planning system, associated with a network of perisylvian areas, achieves color naming through color perception (cp), conceptual identification (ci), lemma retrieval (lr), word-form encoding (wfe), and articulatory processing (art); word-form perception (wfp) activates lemmas and word forms in parallel. Word reading minimally involves word-form perception (wfp), word-form encoding (wfe), and articulatory processing (art). The executive system, centered on the anterior cingulate cortex (ACC), achieves goal- and input control. The connectivity shown in the figure reflects theoretically assumed influences of one brain region on another, which does not necessarily reflect the presence of direct anatomical connections. Instead, the influences may be mediated through other brain regions not explicitly included in the figure.

Neuroimaging studies revealed that color-word Stroop performance engages the ACC and the dorsolateral prefrontal cortex for attentional control, the left lingual gyrus for color processing, the left extrastriate cortex for visual word-form processing, and the left perisylvian language areas including the areas of Broca (posterior inferior frontal) and Wernicke (posterior superior temporal) for word planning. Much evidence suggests that the dorsolateral prefrontal cortex serves to maintain goals in working memory. ACC involvement in goal-referenced control agrees with the idea that attention is the principal link between cognition and motivation. For action control, it is not enough to have goals in working memory, but one should be motivated to attain them. Extensive projections from the thalamus and brainstem nuclei to the ACC suggest a role for drive and arousal. Extensive reciprocal connections between the ACC and dorsolateral prefrontal cortex suggest a role for working memory. The motor areas of the cingulate sulcus densely project to the brainstem, spinal cord, and motor cortex, which suggests a role of the ACC in motor control (see the work of Tomáš Paus and colleagues).

Roelofs, A., & Hagoort, P. (2002). Control of language use: Cognitive modeling of the hemodynamics of Stroop task performance. Cognitive Brain Research, 15, 85-97. Article (PDF 438K)

From monkey calls to human words

We vocally communicate through speech but also with our cries and laughs. Whereas speech is learned, cries and laughs are innately specified. Such innate “calls” are observed in diverse vertebrates, including fish, amphibians, reptiles, birds, and mammals. Although the sound-producing organs differ (swim bladder in fish, syrinx in birds, larynx in amphibians, reptiles, and mammals), vertebrates seem to share a common brainstem and spinal cord organization for calls. Vocal learning is common in birds (songbirds, parrots, and hummingbirds), but only humans, bats, cetaceans (dolphins, whales), and pinnipeds (seals) show evidence of vocal learning among mammals. At least in songbirds and humans, some of the forebrain pathways implicated in learned vocalization seem to share homologous components.

Both human and nonhuman primates (monkeys and apes) use their voice for communication. However, whereas an extensive network of perisylvian and medial cortical areas—including the ACC in some circumstances—is involved in the verbal vocal communication of humans (i.e., spoken word production), the only cortical area directly involved in call production by nonhuman primates is the ACC (see the work of Detlev Ploog, Uwe Jürgens, and colleagues). In nonhuman primates, the ACC plays a critical role in the voluntary initiation and suppression of calls (e.g., fear, alarm, aggression, and contact calls), which are all innate. The ACC also controls the innate vocalizations (e.g., crying, laughing, pain shrieking) of humans. The human ACC appears to be the cortical area where the evolutionary older innate-vocalization system and the newer spoken-word production system meet (see the work of Terrence Deacon). For a description of some commonalities of ACC function across call and word production, see

Roelofs, A. (2008). Attention to spoken word planning: Chronometric and neuroimaging evidence. Language and Linguistics Compass, 2, 389-405. Article (PDF 327K)

Declarative aspects

Declarative pieces of information ("facts") about words are stored in a labeled associative network. The network consists of three major strata: a conceptual stratum, a syntactic stratum, and a word-form stratum, corresponding to the major planning steps. The conceptual stratum represents conceptual facts as nodes and labeled links in a semantic network. For example, the concept RED is represented by the node RED(X) connected to COLOR(X) by an "is-a" link. The syntactic stratum contains lemma nodes, such as red, which are connected by a "class" link to nodes for their syntactic class (e.g., adjective).

Finally, the form stratum contains nodes representing morphemes (e.g., <red>), segments (e.g., /d/), and motor programs (e.g., [red]). The figure above shows only a small fragment of the lexical network and omits most of the labels on the links.

Spreading activation

Information is retrieved from the associative network by spreading activation. For example, a perceived color (e.g., red) activates the corresponding concept node (i.e., RED(X)) in the network. Activation then spreads through the network following a linear activation rule with a decay factor d. Each node m sends a proportion r of its activation to the nodes n it is connected to. For example, RED(X) sends activation to other concepts such as GREEN(X) and COLOR(X) and also to its lemma node red.

WEAVER++'s lexical network is accessed by spreading activation while condition-action rules (see below) determine what is done with the activated lexical information depending on the goal. When a goal is placed in working memory, processing in the system is focused on those rules that include the goal among their conditions. The rules mediate attentional influences by selectively enhancing the activation of target nodes in the network in order to achieve mappings of targets onto articulatory programs. For example, in naming the ink color of the word GREEN, the activation of the concept node RED(X) is selectively enhanced.

Attentional activation enhancements

The model assumes that the ACC is implicated in the attentional activation enhancements. The executive control system determines how strongly and for how long the enhancements occur, depending on the allocation policy (cf. Kahneman, Attention and effort book, 1973; EPIC of Meyer and Kieras). Attention is assumed to be sustained to word planning just as long as is needed to achieve acceptable levels of speed and accuracy:

Roelofs, A., Van Turennout, M., & Coles, M. G. H. (2006). Anterior cingulate cortex activity can be independent of response conflict in Stroop-like tasks. Proceedings of the National Academy of Sciences USA, 103, 13884-13889. Article (PDF 350K)

Roelofs, A. (2008). Tracing attention and the activation flow in spoken word planning using eye movements. Journal of Experimental Psychology: Learning, Memory, and Cognition, 34, 353-368. Article (PDF 285K)

Procedural aspects

Procedural knowledge is embodied by condition-action (if-then) rules. The "if-side" of a rule specifies a condition to be satisfied and the "then-side" of a rule specifies an action to be performed when the condition is met.

Verification

Verification means that selections in human performance are accomplished through explicit reference to goals: goal-referenced control. The condition-action rules carry out the selection of nodes. A rule is triggered when its nodes become active. A lemma retrieval rule selects a lemma if the connected concept is flagged as goal concept. For example, red is selected for RED(X) if it is the goal concept and red has reached a critical difference in activation compared to other lemmas. The actual moment in time of firing of the rule is determined by the ratio of activation of the lemma node and the sum of all the others. Thus, how fast a node is selected depends on how active the other nodes are. There is activation-based triggering and firing of condition-action rules.

A morphological rule selects the morpheme nodes that are connected to the selected lemma (<red> is selected for red). Phonological rules select the segments that are connected to the selected morphemes (/r/, /e/, and /d/ for <red>) and syllabify the segments (e.g., /r/ is made syllable onset: onset(/r/)). Finally, phonetic rules select syllable-based articulatory programs that are appropriately connected to the syllabified segments (i.e., [red] is selected for onset(/r/), nucleus(/e/) and coda(/d)/)). The moment of selection of syllable program nodes is also determined by a ratio of activations, such that how fast selection occurs depends on how active other nodes are.

From Wilhelm Wundt via Watt at Würzburg to WEAVER++

Although issues concerning the attentional control of human performance were explored in the early days of experimental psychology by Donders, Cattell, and Wundt (mental chronometry), no real progress was made in understanding the mechanisms of control. Associationist and behaviorist theories, like those of Hume, the Mills, Watson, and Skinner, accounted for action selection by postulating associations between stimuli and responses. However, if all our actions were determined exclusively by stimulus-response associations, goals could not determine which action to make because the strongest association would automatically determine the response. Around 1900, the Würzburg school with Ach, Külpe, and Watt demonstrated the importance of the task ("Aufgabe") in determining a response. However, how exactly task goals directed processing remained unclear. In the 1910s, Müller proposed an account in associative terms, whereas Selz proposed an account in terms of symbolic structures and rules. Later theoretical developments are descendants of these ideas. On the view that dominates the attention and performance literature, goals associatively bias or "sculpt" the activation of one response pathway (e.g., for color naming, in responding to the ink color of GREEN) rather than another (e.g., for oral reading), following Müller. On another view, following Selz, and computationally implemented in WEAVER++, attentional control arises from explicit, symbolic reference to goals, accomplished by condition-action rules. Pictures below (from left to right): Watt, Külpe, Selz, and Newell and Simon.

The idea of goal-referenced control that originated with Selz in the 1910s flourished in the work of De Groot, Newell and Simon, and Anderson, among others, on higher-level cognitive processes like problem solving (e.g., playing chess, proving logic theorems, and solving puzzles such as the Tower of Hanoi), where associative models generally failed. However, due to the traditional partitioning of experimental psychology into cognition, perception, and action, with little communication across the boundaries, the idea of goal-referenced control has had little impact in the perception-action literature. Only recently, goal-referenced control made successful strides into the attention and performance literature.

That goal-referenced control underlies both problem-solving and performing Stroop-like tasks agrees with the strong connection between attentional control and general intelligence (Spearman’s g, see the work of John Duncan and colleagues). Individual differences in general intelligence are most pronounced in behavioral measures when attentional control is required (see the work of Randall Engle and colleagues).

In his dissertation work at Würzburg University, Henry Watt found that when verbal responses of the same intrinsic speed were grouped together, a variation of task had a similar effect across latency groups, although he did not quantify this effect. He stated, "The influence of the task is independent of the rapidity of the tendency to reproduction itself" (Watt, 1906, Journal of Anatomy and Physiology, p. 260, original italics). Watt’s regularity has recently been confirmed and quantified using modern techniques for analysing response time distributions:

Roelofs, A. (2008). Dynamics of the attentional control of word retrieval: Analyses of response time distributions. Journal of Experimental Psychology: General, 137, 303-323. Article (PDF 392K)

Planning latencies

Given the equations for spreading activation and rule firing, the mathematically expected mean planning latencies can be computed.

Frequently asked questions (FAQs)

Question: Why do we need labeled links?

Answer: A mere associative link between two nodes tells nothing about the relation between the entities represented. For example, RED(X) is strongly associated with both GREEN(X) and FIRE(X) but the relationship between RED(X) and GREEN(X) is very different from the relationship between RED(X) and FIRE(X). The importance of representing the relation between entities symbolically was recognized by Otto Selz in the early 1900s, and labeled link have become a central part of semantic networks in Artificial Intelligence since the seminal work of Ross Quillian in the late 1960s. Imagine the Internet without labeled (hyper)links!

Question: Why do we need verification? Isn't it true that condition-action rules seem incompatible with what we know about the human brain? Wouldn't it be better to have a purely associative model?

Answer: Verification is the use of goal-factored rules as a means of selection. Our knowledge of how the human brain works is still so rudimentary that is seems premature to ban condition-action rules and other symbolic entities such as labeled links. Condition-action rules mean nothing more than the operations that they specify. Crucial for the issue of "neural plausibility" is whether we can exclude that the brain performs such operations and I know of no evidence against that. On the contrary, there is increasing evidence that the human brain, in particular prefrontal cortex, supports the use of symbolic rules (e.g., the work of Earl Miller and colleagues).

Question: What do you recommend as background reading?

Question: Where can I find information on WEAVER++?

Answer: Here are a number of references:

Roelofs, A. (2003). Goal-referenced selection of verbal action: Modeling attentional control in the Stroop task. Psychological Review, 110, 88-125. Article (PDF 585K)

Roelofs, A. (2004). The seduced speaker: Modeling of cognitive control. Lecture Notes in Artificial Intelligence, 3123, 1-10. Article (PDF 128K)

Roelofs, A. (2008). Dynamics of the attentional control of word retrieval: Analyses of response time distributions. Journal of Experimental Psychology: General, 137, 303-323. Article (PDF 392K)

Roelofs, A., & Hagoort, P. (2002). Control of language use: Cognitive modeling of the hemodynamics of Stroop task performance. Cognitive Brain Research, 15, 85-97. Article (PDF 438K)

A complete list of articles that report on WEAVER++ simulations can be found here.

Question: What programming language has been used for the model, and is the program available?

Answer: The published WEAVER++ simulations have been programmed in the C/C++ programming language using the Microsoft Visual C++ environment. The programs are available from Ardi Roelofs.

Track record of WEAVER++ (WEAVER++’s Web)