No Cover Image

Conference Paper/Proceeding/Abstract 95 views

Speaker Motion Patterns during Self-repairs in Natural Dialogue

Elif Ecem Ozkan Orcid Logo, Tom Gurion Orcid Logo, Julian Hough, Patrick G.T. Healey Orcid Logo, Lorenzo Jamone Orcid Logo

International Conference on Multimodal Interaction

Swansea University Author: Julian Hough

Full text not available from this repository: check for access using links below.

DOI (Published version): 10.1145/3536220.3563684

Abstract

An important milestone for any agent in interaction with humans on a regular basis is to achieve natural and efficient methods of communication. Such strategies should be derived on the hallmarks of human-human interaction. So far, the work in embodied conversational agents (ECAs) implementing such...

Full description

Published in: International Conference on Multimodal Interaction
ISBN: 978-1-4503-9389-8
Published: New York, NY, USA ACM 2022
URI: https://cronfa.swan.ac.uk/Record/cronfa64929
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract: An important milestone for any agent in interaction with humans on a regular basis is to achieve natural and efficient methods of communication. Such strategies should be derived on the hallmarks of human-human interaction. So far, the work in embodied conversational agents (ECAs) implementing such signals has been predominantly through imitating human-like positive back-channels, such as nodding, rather than active interaction. The field of Conversation Analysis (CA) focusing on natural human dialogue suggests that people continuously collaborate on achieving mutual understanding by frequently repairing misunderstandings as they happen. Detecting repairs from speech in real-time is challenging, even with state-of-the-art Natural Language Processing (NLP) models. We present specific human motion patterns during key moments of interaction, namely self initiated self-repairs, which would help agents to recognise and collaboratively solve speaker trouble. The features we present in this paper are the pairwise joint distances of head and hands which are more discriminative than the positions themselves.
Keywords: multimodal interaction, non-verbal communication, human motion analysis
College: Faculty of Science and Engineering