Interference or Integration? Gesture, Sign, and the Missing Hard-of-Hearing Evidence
A methods-forward review of what gesture-sign research shows about transfer, spatial reference, code-blending, and the still-thin evidence on hard-of-hearing people's own gesture production.
The central correction
The literature does not support a simple story in which hard-of-hearing people's ordinary gestures "interfere" with sign language. It supports a more precise claim: manual gesture, depicting sign, pointing, and signed language production share representational resources, and experience in one system can reshape the others. Whether that reshaping is facilitation, transfer, code-blending, discourse adaptation, or interference depends on the population, task, language pair, and the level of analysis.
That distinction is not semantic housekeeping. A movement can look gesture-like because it depicts a referent, yet still be linguistically conventionalized. Conversely, a lexical sign can appear in a spoken interaction without being a random error. Goldin-Meadow and Brentari's landmark review therefore argues that sign should be compared with speech-plus-gesture, not speech alone, while still preserving a distinction between categorical linguistic forms and more imagistic, gradient gestures. 1
This launch review asks three narrower questions:
- What kinds of cross-modal influence have actually been demonstrated?
- Which observations would justify calling an effect "interference" rather than integration or transfer?
- What is known specifically about hard-of-hearing signers, as opposed to adjacent populations such as Deaf native signers and hearing bimodal bilinguals?
Four mechanisms that should stay separate
| Mechanism | What should be observable | Best current interpretation | Main false positive |
|---|---|---|---|
| Gesture-to-sign transfer | A gesture-derived form predicts accuracy or error in a sign parameter, ideally within the same participant and across time | Potentially positive or negative transfer | Whole-sign resemblance hides which parameter was actually transferred |
| Sign-to-gesture influence | Signers produce different handshapes, spatial layouts, viewpoints, or event descriptions while speaking | Cross-modal conceptual or motor influence | Group differences may reflect general fluency, culture, or task familiarity |
| Code-blending | Speech and sign occur simultaneously with coordinated timing and related meaning | A bimodal bilingual production option, not necessarily a breakdown | Calling every manual movement during speech an unintended sign |
| Articulatory or spatial competition | A controlled manipulation produces measurable timing, location, movement, or reference errors | Candidate interference mechanism | Treating a slower or less conventional form as evidence of competition without a baseline |
The field's vocabulary matters here. Code-switching normally refers to alternating languages; code-blending refers to simultaneous speech and sign. Bimodal bilinguals can blend because the two languages use partly separate articulators. In conversational settings, code-blending is often a normal resource, distinct from educational SimCom, where spoken language is typically prioritized and signing may be partial or supportive. 23
What has been demonstrated
Sign experience changes co-speech gesture
The cleanest evidence comes from hearing bimodal bilinguals rather than hard-of-hearing signers. Casey and Emmorey compared 13 native ASL-English bilinguals with 12 non-signing English speakers who retold the same cartoon to a non-signer. Nine of the 13 bilinguals produced at least one identifiable ASL sign. Relative to non-signers, they also produced more iconic gestures, fewer beat gestures, more character-viewpoint gestures, more varied handshapes, and more unmarked handshapes. The study used independent coding of sign versus non-sign gesture, gesture type, viewpoint, and handshape. 4
The result is best described as an interaction between the ASL production system and the co-speech gesture system. It is not yet a causal demonstration that ASL "leaks" into speech. The groups differed in language history, and the cartoon-retelling task invites iconic depiction. Still, the pattern is difficult to explain as a single accidental sign: both the semantic content and the formal inventory of gesture changed.
Weisberg and colleagues strengthened the experience argument by comparing early ASL-English bilinguals, fluent late ASL learners with at least 10 years of signing experience, and monolingual non-signers. Both signer groups produced manual gestures at higher rates, especially iconic gestures, and used a wider range of handshapes. Late learners produced more ASL signs and more handshape varieties than early bilinguals, suggesting that extensive sign experience modifies the gesture system even when sign language was acquired later. 5
The dosage result is also informative. One year of ASL instruction increased adult learners' co-speech gesture production, but did not make them statistically distinct from non-signers. The available comparison suggests that durable changes in the gesture-sign boundary may require extensive use, not mere classroom exposure. 6
Spatial meaning transfers across modalities
Spatial reference is the most plausible place to look for cross-modal influence because both gesture and sign can use location, path, orientation, and viewpoint to represent relations among entities.
In a placement-description study, hearing English speakers learning ASL were compared with Deaf native ASL signers and non-signing English controls. The L2 signers' ASL descriptions resembled native signers' descriptions, consistent with the facilitative role of iconic placement distinctions. At the same time, learners used a wider range of handshapes less appropriately, indicating fuzzy semantic boundaries. Their English co-speech gestures were marginally more ASL-like than those of non-signers. That is a plausible case of bidirectional influence, but the authors explicitly frame the design as correlational. 7
A Mandarin-Chinese Sign Language study points to a different kind of transfer: late bimodal bilinguals used more vertical spatial metaphors for time in co-speech gesture, whereas non-signing Mandarin speakers more often used a horizontal timeline. The important observation is not that one group gestured more, but that a sign-language convention altered the spatial organization of a spoken-language concept. 8
Dutch-NGT research adds a production-planning measure. Hearing bimodal bilinguals produced more ground-first spatial descriptions in speech than hearing non-signers, while their sign descriptions patterned with Deaf NGT signers. Eye-tracking showed more fixation on the ground object during message preparation. This combination links a linguistic preference to a planning signature, although it still does not show that a signer was unable to suppress an alternative form. 9
The comparison must remain cautious. Pointing signs are not simply pointing gestures with a different label. In a large comparison of British Sign Language pointing signs and American English pointing gestures, pointing signs were more consistent across uses, more reduced, and more integrated into sign prosody. Both systems use space, but language-specific conventionalization changes what the spatial act can do. 10
Parameter transfer is not whole-sign copying
A central methodological mistake is to ask whether a gesture and a sign look alike as wholes. Sign languages organize manual forms through parameters such as handshape, location, movement, orientation, and non-manual features. Gesture is often holistic. A gesture can therefore provide a learner with a useful visual-motor analogy without supplying the categorical contrast needed to produce a phonologically well-formed sign.
A 2026 longitudinal study of hearing learners of German Sign Language tested gesture-sign overlap before and after approximately 20 hours of instruction. The researchers varied whether overlap occurred in handshape, location, or movement, and scored production accuracy parameter by parameter. Gesture and iconicity effects depended on the parameter. The authors' interpretation is that gesture supplies phonetic, not phonological, information: it can support a learner's construction of a sign while leaving the sign-language category to be learned. 11
This framework also explains why the ASL studies show both benefit and error. A learner may exploit a transparent spatial relation, yet overgeneralize a handshape or use a classifier-like form outside the native semantic boundary. To call that interference, a study should identify the parameter, define the target category, and compare the learner with a baseline that controls for general L2 proficiency. A global score such as "more gesture-like signing" is not enough.
Depicting signs are especially important because they combine conventional handshape resources with gradient information about path, manner, location, size, or shape. In ASL-English and Libras-Portuguese code-blends, de Quadros and colleagues found that entity and handling depicting signs patterned differently with speech structure. The result supports a single-proposition account of code-blending, not a model in which a gesture simply interrupts an otherwise complete sentence. 12
The hard-of-hearing evidence gap
The direct answer to this channel's founding question is currently limited. The best-established experiments above mostly sample one of three groups: hearing native bimodal bilinguals, hearing late learners of a sign language, or Deaf native/early signers. They are relevant to cross-modal influence, but they are not interchangeable with hard-of-hearing people.
Hard-of-hearing participants can differ in age of hearing loss, access to sound, hearing technology, first language, age of sign-language exposure, family language, educational setting, and whether signing is used with Deaf, hard-of-hearing, or hearing interlocutors. These variables affect both production and audience design. A methods review on sign-language data warns that heterogeneous acquisition histories, simultaneous multi-articulator production, lack of a writing system, and a permeable sign-gesture interface all complicate sampling and interpretation. 13
The gap is clearest for adults with acquired hearing loss. Sparrow, Lind, and van Steenbrugge's review found that existing work focuses largely on how a communication partner's gesture and visual speech cues support comprehension. It identifies the audience design of gesture and gaze as under-researched and does not provide a mature evidence base on how late-deafened or hard-of-hearing adults themselves produce co-speech or co-sign gestures. 14
The same caution applies to co-thought gesture. The literature on gesture during spatial reasoning and problem solving is not a substitute for measuring hard-of-hearing signers' internal planning or silent gesture. In the evidence pass for this issue, I found no direct peer-reviewed study that isolates co-thought gesture production in hard-of-hearing or bimodal bilingual signers. That is a research gap, not evidence that no such behavior exists.
Methods that could make interference testable
Use paired production, not only between-group contrasts
The strongest next design would test the same participant in matched sign, speech-plus-gesture, and silent-gesture conditions. Each condition should use the same spatial or motion stimulus, with interlocutor language and audience knowledge manipulated separately. This would reveal whether a form travels across modalities within an individual, rather than merely correlating with group membership.
Elicit reference under controlled spatial demands
Wordless narratives such as Frog, Where Are You?, cartoon retellings, and placement tasks are useful because they elicit viewpoint, path, location, depicting signs, and reference tracking without forcing a written lexical answer. The task should code new, maintained, and reintroduced referents, pointing function, locus stability, and whether a spatial location is conventionalized or improvised. Cross-linguistic work across Auslan, Irish SL, Finnish SL, Norwegian SL, and Swedish SL shows why a single language's preferred strategy should not be treated as universal. 15
Record motion, but do not let kinematics decide the category alone
Motion capture can add measures that visual coding misses. In Israeli Sign Language, Kinect-based analysis found that lexical signs, constructed action, and classifier constructions differ in velocity, acceleration, displacement, duration, jerk, path length, and curvature. Lexical signs were faster and shorter; classifier constructions occupied an intermediate position. 16
A 2025 ASL-English comparison likewise used high-frequency motion capture and linear mixed-effects models to distinguish communicative from non-communicative manual movement. The groups differed more in temporal signatures than in spatial envelopes. This is a useful warning: a movement can share a spatial path with a gesture while remaining distinct in timing, rhythmicity, or smoothness. 17
Kinematics should therefore be treated as one layer of evidence. It can show that movements differ, but it cannot by itself establish lexical status, communicative intent, or interference.
Annotate form and function on separate tiers
ELAN-style time-aligned annotation is necessary but not sufficient. Gesture research has shown the value of separating form coding, such as handshape, trajectory, and spatial position, from function coding, such as content-carrying, redundant, or supplementary gesture. A sign-language study needs additional tiers for gloss or lexical status, phonological parameters, depicting type, constructed action, gaze, facial expression, speech, interlocutor response, and reference status. 18
For hard-of-hearing samples, the annotation should also record hearing history, device use, sign-language age of acquisition, family signing status, language dominance, and addressee. Without these variables, a pattern may be assigned to gesture when it is actually an acquisition effect, an accommodation to an interlocutor, or a community-specific convention.
Model interference as a contrast, not a metaphor
A credible interference claim needs at least one controlled contrast: a target parameter under competing language histories, a spatial locus under competing reference conventions, or a timing measure under a gesture restriction or audience manipulation. Mixed-effects models with participant and item effects are now standard in this area, but the statistical model cannot repair an underspecified category. Existing studies remain mostly observational or correlational. They can identify systematic co-variation; they rarely establish that gesture caused a sign error.
A research agenda for the channel
The next generation of studies should:
- Recruit hard-of-hearing participants as a stratified population, not as a residual category inside a broad DHH sample.
- Report sign-language age of acquisition, hearing history, language dominance, family language, education, and interlocutor type in every analysis.
- Separate transfer at the levels of concept, spatial reference, phonological parameter, lexical selection, timing, and discourse function.
- Treat code-blending as a possible communicative choice before treating it as failed inhibition.
- Repeat matched tasks across sign languages and spoken-language environments, including languages with different spatial metaphors and reference conventions.
- Combine corpus observations, elicited production, motion capture, gaze, and longitudinal measurement rather than allowing one channel of evidence to stand in for the rest.
Bottom line
The field has good evidence that sign-language experience changes gesture and that gesture can support or reshape sign-language learning. It also has increasingly precise methods for measuring handshape, spatial reference, timing, and depicting structure. What it does not yet have is a sufficiently direct account of hard-of-hearing people's own gesturing habits across signing and speaking contexts, especially for adult acquired hearing loss and co-thought gesture.
For now, "interference" should be a conclusion earned by a controlled contrast. The more defensible starting point is interaction: gesture and sign draw on a partly shared visual-manual repertoire, but language-specific conventionalization, acquisition history, interlocutor design, and timing determine what that interaction becomes.
References
- 1
- 2Bimodal bilingualism
doi.org
- 3Lessons to be Learned from Bimodal Bilingualism
pmc.ncbi.nlm.nih.gov
- 4Co-speech gesture in bimodal bilinguals
pmc.ncbi.nlm.nih.gov
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12Code-blending with depicting signs
pmc.ncbi.nlm.nih.gov
- 13
- 14
- 15
- 16
- 17
- 18
This story was produced automatically by a channel. One sentence is all it takes for Neodrop to keep producing for you.