No Cover Image

Journal article 133 views

Speakers Raise their Hands and Head during Self-Repairs in Dyadic Conversations

Elif Ecem Ozkan Orcid Logo, Patrick G.T. Healey, Tom Gurion, Julian Hough, Lorenzo Jamone Orcid Logo

IEEE Transactions on Cognitive and Developmental Systems, Pages: 1 - 1

Swansea University Author: Julian Hough

Full text not available from this repository: check for access using links below.

Check full text

DOI (Published version): 10.1109/tcds.2023.3254808

Abstract

People often encounter difficulties in building shared understanding during everyday conversation. The most common symptom of these difficulties are self-repairs, when a speaker restarts, edits or amends their utterances mid-turn. Previous work has focused on the verbal signals of self-repair, i.e....

Full description

Published in: IEEE Transactions on Cognitive and Developmental Systems
ISSN: 2379-8920 2379-8939
Published: Institute of Electrical and Electronics Engineers (IEEE) 2023
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa64930
Tags: Add Tag
No Tags, Be the first to tag this record!
first_indexed 2023-11-07T21:42:49Z
last_indexed 2023-11-07T21:42:49Z
id cronfa64930
recordtype SURis
fullrecord <?xml version="1.0" encoding="utf-8"?><rfc1807 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><bib-version>v2</bib-version><id>64930</id><entry>2023-11-07</entry><title>Speakers Raise their Hands and Head during Self-Repairs in Dyadic Conversations</title><swanseaauthors><author><sid>082d773ae261d2bbf49434dd2608ab40</sid><firstname>Julian</firstname><surname>Hough</surname><name>Julian Hough</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2023-11-07</date><deptcode>SCS</deptcode><abstract>People often encounter difficulties in building shared understanding during everyday conversation. The most common symptom of these difficulties are self-repairs, when a speaker restarts, edits or amends their utterances mid-turn. Previous work has focused on the verbal signals of self-repair, i.e. speech disfluences (filled pauses, truncated words and phrases, word substitutions or reformulations), and computational tools now exist that can automatically detect these verbal phenomena. However, face-to-face conversation also exploits rich non-verbal resources and previous research suggests that self-repairs are associated with distinct hand movement patterns. This paper extends those results by exploring head and hand movements of both speakers and listeners using two motion parameters: height (vertical position) and 3D velocity. The results show that speech sequences containing self-repairs are distinguishable from fluent ones: speakers raise their hands and head more (and move more rapidly) during self-repairs. We obtain these results by analysing data from a corpus of 13 unscripted dialogues, and we discuss how these findings could support the creation of improved cognitive artificial systems for natural human-machine and human-robot interaction.</abstract><type>Journal Article</type><journal>IEEE Transactions on Cognitive and Developmental Systems</journal><volume/><journalNumber/><paginationStart>1</paginationStart><paginationEnd>1</paginationEnd><publisher>Institute of Electrical and Electronics Engineers (IEEE)</publisher><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint>2379-8920</issnPrint><issnElectronic>2379-8939</issnElectronic><keywords>Maintenance engineering, Oral communication, Natural language processing, Task analysis, Magnetic heads, Speech recognition, Human-robot interaction</keywords><publishedDay>10</publishedDay><publishedMonth>3</publishedMonth><publishedYear>2023</publishedYear><publishedDate>2023-03-10</publishedDate><doi>10.1109/tcds.2023.3254808</doi><url>http://dx.doi.org/10.1109/tcds.2023.3254808</url><notes/><college>COLLEGE NANME</college><department>Computer Science</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>SCS</DepartmentCode><institution>Swansea University</institution><apcterm/><funders>10.13039/100009149-School of Electronic Engineering and Computer Science (Grant Number: EP/L01632X/1). 10.13039/501100000266-Engineering and Physical Sciences Research Council (Grant Number: EP/R02572X/1 and EP/S00453X/1).</funders><projectreference/><lastEdited>2023-12-04T17:20:09.9214022</lastEdited><Created>2023-11-07T21:40:36.3688528</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Elif Ecem</firstname><surname>Ozkan</surname><orcid>0000-0002-9036-2149</orcid><order>1</order></author><author><firstname>Patrick G.T.</firstname><surname>Healey</surname><order>2</order></author><author><firstname>Tom</firstname><surname>Gurion</surname><order>3</order></author><author><firstname>Julian</firstname><surname>Hough</surname><order>4</order></author><author><firstname>Lorenzo</firstname><surname>Jamone</surname><orcid>0000-0002-1521-6168</orcid><order>5</order></author></authors><documents/><OutputDurs/></rfc1807>
spelling v2 64930 2023-11-07 Speakers Raise their Hands and Head during Self-Repairs in Dyadic Conversations 082d773ae261d2bbf49434dd2608ab40 Julian Hough Julian Hough true false 2023-11-07 SCS People often encounter difficulties in building shared understanding during everyday conversation. The most common symptom of these difficulties are self-repairs, when a speaker restarts, edits or amends their utterances mid-turn. Previous work has focused on the verbal signals of self-repair, i.e. speech disfluences (filled pauses, truncated words and phrases, word substitutions or reformulations), and computational tools now exist that can automatically detect these verbal phenomena. However, face-to-face conversation also exploits rich non-verbal resources and previous research suggests that self-repairs are associated with distinct hand movement patterns. This paper extends those results by exploring head and hand movements of both speakers and listeners using two motion parameters: height (vertical position) and 3D velocity. The results show that speech sequences containing self-repairs are distinguishable from fluent ones: speakers raise their hands and head more (and move more rapidly) during self-repairs. We obtain these results by analysing data from a corpus of 13 unscripted dialogues, and we discuss how these findings could support the creation of improved cognitive artificial systems for natural human-machine and human-robot interaction. Journal Article IEEE Transactions on Cognitive and Developmental Systems 1 1 Institute of Electrical and Electronics Engineers (IEEE) 2379-8920 2379-8939 Maintenance engineering, Oral communication, Natural language processing, Task analysis, Magnetic heads, Speech recognition, Human-robot interaction 10 3 2023 2023-03-10 10.1109/tcds.2023.3254808 http://dx.doi.org/10.1109/tcds.2023.3254808 COLLEGE NANME Computer Science COLLEGE CODE SCS Swansea University 10.13039/100009149-School of Electronic Engineering and Computer Science (Grant Number: EP/L01632X/1). 10.13039/501100000266-Engineering and Physical Sciences Research Council (Grant Number: EP/R02572X/1 and EP/S00453X/1). 2023-12-04T17:20:09.9214022 2023-11-07T21:40:36.3688528 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Elif Ecem Ozkan 0000-0002-9036-2149 1 Patrick G.T. Healey 2 Tom Gurion 3 Julian Hough 4 Lorenzo Jamone 0000-0002-1521-6168 5
title Speakers Raise their Hands and Head during Self-Repairs in Dyadic Conversations
spellingShingle Speakers Raise their Hands and Head during Self-Repairs in Dyadic Conversations
Julian Hough
title_short Speakers Raise their Hands and Head during Self-Repairs in Dyadic Conversations
title_full Speakers Raise their Hands and Head during Self-Repairs in Dyadic Conversations
title_fullStr Speakers Raise their Hands and Head during Self-Repairs in Dyadic Conversations
title_full_unstemmed Speakers Raise their Hands and Head during Self-Repairs in Dyadic Conversations
title_sort Speakers Raise their Hands and Head during Self-Repairs in Dyadic Conversations
author_id_str_mv 082d773ae261d2bbf49434dd2608ab40
author_id_fullname_str_mv 082d773ae261d2bbf49434dd2608ab40_***_Julian Hough
author Julian Hough
author2 Elif Ecem Ozkan
Patrick G.T. Healey
Tom Gurion
Julian Hough
Lorenzo Jamone
format Journal article
container_title IEEE Transactions on Cognitive and Developmental Systems
container_start_page 1
publishDate 2023
institution Swansea University
issn 2379-8920
2379-8939
doi_str_mv 10.1109/tcds.2023.3254808
publisher Institute of Electrical and Electronics Engineers (IEEE)
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science
url http://dx.doi.org/10.1109/tcds.2023.3254808
document_store_str 0
active_str 0
description People often encounter difficulties in building shared understanding during everyday conversation. The most common symptom of these difficulties are self-repairs, when a speaker restarts, edits or amends their utterances mid-turn. Previous work has focused on the verbal signals of self-repair, i.e. speech disfluences (filled pauses, truncated words and phrases, word substitutions or reformulations), and computational tools now exist that can automatically detect these verbal phenomena. However, face-to-face conversation also exploits rich non-verbal resources and previous research suggests that self-repairs are associated with distinct hand movement patterns. This paper extends those results by exploring head and hand movements of both speakers and listeners using two motion parameters: height (vertical position) and 3D velocity. The results show that speech sequences containing self-repairs are distinguishable from fluent ones: speakers raise their hands and head more (and move more rapidly) during self-repairs. We obtain these results by analysing data from a corpus of 13 unscripted dialogues, and we discuss how these findings could support the creation of improved cognitive artificial systems for natural human-machine and human-robot interaction.
published_date 2023-03-10T17:20:10Z
_version_ 1784372695711875072
score 11.016235

Similar Items