No Cover Image

Conference Paper/Proceeding/Abstract 261 views

Speaker Motion Patterns during Self-repairs in Natural Dialogue

Elif Ecem Ozkan Orcid Logo, Tom Gurion Orcid Logo, Julian Hough Orcid Logo, Patrick G.T. Healey Orcid Logo, Lorenzo Jamone Orcid Logo

International Conference on Multimodal Interaction

Swansea University Author: Julian Hough Orcid Logo

Full text not available from this repository: check for access using links below.

DOI (Published version): 10.1145/3536220.3563684

Abstract

An important milestone for any agent in interaction with humans on a regular basis is to achieve natural and efficient methods of communication. Such strategies should be derived on the hallmarks of human-human interaction. So far, the work in embodied conversational agents (ECAs) implementing such...

Full description

Published in: International Conference on Multimodal Interaction
ISBN: 978-1-4503-9389-8
Published: New York, NY, USA ACM 2022
URI: https://cronfa.swan.ac.uk/Record/cronfa64929
first_indexed 2023-11-07T21:33:05Z
last_indexed 2024-11-25T14:15:01Z
id cronfa64929
recordtype SURis
fullrecord <?xml version="1.0"?><rfc1807><datestamp>2024-03-12T14:16:01.6918992</datestamp><bib-version>v2</bib-version><id>64929</id><entry>2023-11-07</entry><title>Speaker Motion Patterns during Self-repairs in Natural Dialogue</title><swanseaauthors><author><sid>082d773ae261d2bbf49434dd2608ab40</sid><ORCID>0000-0002-4345-6759</ORCID><firstname>Julian</firstname><surname>Hough</surname><name>Julian Hough</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2023-11-07</date><deptcode>MACS</deptcode><abstract>An important milestone for any agent in interaction with humans on a regular basis is to achieve natural and efficient methods of communication. Such strategies should be derived on the hallmarks of human-human interaction. So far, the work in embodied conversational agents (ECAs) implementing such signals has been predominantly through imitating human-like positive back-channels, such as nodding, rather than active interaction. The field of Conversation Analysis (CA) focusing on natural human dialogue suggests that people continuously collaborate on achieving mutual understanding by frequently repairing misunderstandings as they happen. Detecting repairs from speech in real-time is challenging, even with state-of-the-art Natural Language Processing (NLP) models. We present specific human motion patterns during key moments of interaction, namely self initiated self-repairs, which would help agents to recognise and collaboratively solve speaker trouble. The features we present in this paper are the pairwise joint distances of head and hands which are more discriminative than the positions themselves.</abstract><type>Conference Paper/Proceeding/Abstract</type><journal>International Conference on Multimodal Interaction</journal><volume/><journalNumber/><paginationStart/><paginationEnd/><publisher>ACM</publisher><placeOfPublication>New York, NY, USA</placeOfPublication><isbnPrint>978-1-4503-9389-8</isbnPrint><isbnElectronic/><issnPrint/><issnElectronic/><keywords>multimodal interaction, non-verbal communication, human motion analysis</keywords><publishedDay>7</publishedDay><publishedMonth>11</publishedMonth><publishedYear>2022</publishedYear><publishedDate>2022-11-07</publishedDate><doi>10.1145/3536220.3563684</doi><url/><notes/><college>COLLEGE NANME</college><department>Mathematics and Computer Science School</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>MACS</DepartmentCode><institution>Swansea University</institution><apcterm>Another institution paid the OA fee</apcterm><funders/><projectreference/><lastEdited>2024-03-12T14:16:01.6918992</lastEdited><Created>2023-11-07T21:30:06.9466828</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Elif Ecem</firstname><surname>Ozkan</surname><orcid>0000-0002-9036-2149</orcid><order>1</order></author><author><firstname>Tom</firstname><surname>Gurion</surname><orcid>0000-0002-7245-4402</orcid><order>2</order></author><author><firstname>Julian</firstname><surname>Hough</surname><orcid>0000-0002-4345-6759</orcid><order>3</order></author><author><firstname>Patrick G.T.</firstname><surname>Healey</surname><orcid>0000-0003-3079-3374</orcid><order>4</order></author><author><firstname>Lorenzo</firstname><surname>Jamone</surname><orcid>0000-0002-1521-6168</orcid><order>5</order></author></authors><documents/><OutputDurs/></rfc1807>
spelling 2024-03-12T14:16:01.6918992 v2 64929 2023-11-07 Speaker Motion Patterns during Self-repairs in Natural Dialogue 082d773ae261d2bbf49434dd2608ab40 0000-0002-4345-6759 Julian Hough Julian Hough true false 2023-11-07 MACS An important milestone for any agent in interaction with humans on a regular basis is to achieve natural and efficient methods of communication. Such strategies should be derived on the hallmarks of human-human interaction. So far, the work in embodied conversational agents (ECAs) implementing such signals has been predominantly through imitating human-like positive back-channels, such as nodding, rather than active interaction. The field of Conversation Analysis (CA) focusing on natural human dialogue suggests that people continuously collaborate on achieving mutual understanding by frequently repairing misunderstandings as they happen. Detecting repairs from speech in real-time is challenging, even with state-of-the-art Natural Language Processing (NLP) models. We present specific human motion patterns during key moments of interaction, namely self initiated self-repairs, which would help agents to recognise and collaboratively solve speaker trouble. The features we present in this paper are the pairwise joint distances of head and hands which are more discriminative than the positions themselves. Conference Paper/Proceeding/Abstract International Conference on Multimodal Interaction ACM New York, NY, USA 978-1-4503-9389-8 multimodal interaction, non-verbal communication, human motion analysis 7 11 2022 2022-11-07 10.1145/3536220.3563684 COLLEGE NANME Mathematics and Computer Science School COLLEGE CODE MACS Swansea University Another institution paid the OA fee 2024-03-12T14:16:01.6918992 2023-11-07T21:30:06.9466828 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Elif Ecem Ozkan 0000-0002-9036-2149 1 Tom Gurion 0000-0002-7245-4402 2 Julian Hough 0000-0002-4345-6759 3 Patrick G.T. Healey 0000-0003-3079-3374 4 Lorenzo Jamone 0000-0002-1521-6168 5
title Speaker Motion Patterns during Self-repairs in Natural Dialogue
spellingShingle Speaker Motion Patterns during Self-repairs in Natural Dialogue
Julian Hough
title_short Speaker Motion Patterns during Self-repairs in Natural Dialogue
title_full Speaker Motion Patterns during Self-repairs in Natural Dialogue
title_fullStr Speaker Motion Patterns during Self-repairs in Natural Dialogue
title_full_unstemmed Speaker Motion Patterns during Self-repairs in Natural Dialogue
title_sort Speaker Motion Patterns during Self-repairs in Natural Dialogue
author_id_str_mv 082d773ae261d2bbf49434dd2608ab40
author_id_fullname_str_mv 082d773ae261d2bbf49434dd2608ab40_***_Julian Hough
author Julian Hough
author2 Elif Ecem Ozkan
Tom Gurion
Julian Hough
Patrick G.T. Healey
Lorenzo Jamone
format Conference Paper/Proceeding/Abstract
container_title International Conference on Multimodal Interaction
publishDate 2022
institution Swansea University
isbn 978-1-4503-9389-8
doi_str_mv 10.1145/3536220.3563684
publisher ACM
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science
document_store_str 0
active_str 0
description An important milestone for any agent in interaction with humans on a regular basis is to achieve natural and efficient methods of communication. Such strategies should be derived on the hallmarks of human-human interaction. So far, the work in embodied conversational agents (ECAs) implementing such signals has been predominantly through imitating human-like positive back-channels, such as nodding, rather than active interaction. The field of Conversation Analysis (CA) focusing on natural human dialogue suggests that people continuously collaborate on achieving mutual understanding by frequently repairing misunderstandings as they happen. Detecting repairs from speech in real-time is challenging, even with state-of-the-art Natural Language Processing (NLP) models. We present specific human motion patterns during key moments of interaction, namely self initiated self-repairs, which would help agents to recognise and collaboratively solve speaker trouble. The features we present in this paper are the pairwise joint distances of head and hands which are more discriminative than the positions themselves.
published_date 2022-11-07T08:31:20Z
_version_ 1822118359280386048
score 11.048388

Similar Items