“You, Move There!”: Investigating the Impact of Feedback on Voice Control in Virtual Environments

Baxter, Mitchell; Bleakley, Anna; Edwards, Justin; Clark, Leigh; Cowan, Benjamin R.; Williamson, Julie R.

doi:10.1145/3469595.3469609

Conference Paper/Proceeding/Abstract 383 views 82 downloads

“You, Move There!”: Investigating the Impact of Feedback on Voice Control in Virtual Environments

Mitchell Baxter, Anna Bleakley, Justin Edwards, Leigh Clark

, Benjamin R. Cowan, Julie R. Williamson

CUI 2021 - 3rd Conference on Conversational User Interfaces

Swansea University Author: Leigh Clark

DOI (Published version): 10.1145/3469595.3469609

Abstract

Current virtual environment (VEs) input techniques often overlook speech as a useful control modality. Speech could improve interaction in multimodal VEs by enabling users to address objects, locations, and agents, yet research on how to design effective speech for VEs is limited. Our paper investig...

Full description

Published in:	CUI 2021 - 3rd Conference on Conversational User Interfaces
ISBN:	978-1-4503-8998-3
Published:	New York, NY, USA ACM 2021
URI:	https://cronfa.swan.ac.uk/Record/cronfa57576
Tags:	Add Tag No Tags, Be the first to tag this record!

first_indexed	2021-08-10T09:27:24Z
last_indexed	2021-09-14T03:20:39Z
id	cronfa57576
recordtype	SURis
fullrecord	<?xml version="1.0"?><rfc1807><datestamp>2021-09-13T11:40:29.7264696</datestamp><bib-version>v2</bib-version><id>57576</id><entry>2021-08-10</entry><title>“You, Move There!”: Investigating the Impact of Feedback on Voice Control in Virtual Environments</title><swanseaauthors><author><sid>004ef41b90854a57a498549a462f13a0</sid><ORCID>0000-0002-9237-1057</ORCID><firstname>Leigh</firstname><surname>Clark</surname><name>Leigh Clark</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2021-08-10</date><deptcode>SCS</deptcode><abstract>Current virtual environment (VEs) input techniques often overlook speech as a useful control modality. Speech could improve interaction in multimodal VEs by enabling users to address objects, locations, and agents, yet research on how to design effective speech for VEs is limited. Our paper investigates the effect of agent feedback on speech VE experiences. Through a lab study, users commanded agents to navigate a VE, receiving either auditory, visual or behavioural feedback. Based on a post interaction semi-structured interview, we find that the type of feedback given by agents is critical to user experience. Specifically auditory mechanisms are preferred, allowing users to engage with other modalities seamlessly during interaction. Although command-like utterances were frequently used, it was perceived as contextually appropriate, ensuring users were understood. Many also found it difficult to discover speech-based functionality. Drawing on these, we discuss key challenges for designing speech input for VEs.</abstract><type>Conference Paper/Proceeding/Abstract</type><journal>CUI 2021 - 3rd Conference on Conversational User Interfaces</journal><volume/><journalNumber/><paginationStart/><paginationEnd/><publisher>ACM</publisher><placeOfPublication>New York, NY, USA</placeOfPublication><isbnPrint/><isbnElectronic>978-1-4503-8998-3</isbnElectronic><issnPrint/><issnElectronic/><keywords>Virtual Environments, Speech Input, Gesture Input</keywords><publishedDay>27</publishedDay><publishedMonth>7</publishedMonth><publishedYear>2021</publishedYear><publishedDate>2021-07-27</publishedDate><doi>10.1145/3469595.3469609</doi><url/><notes/><college>COLLEGE NANME</college><department>Computer Science</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>SCS</DepartmentCode><institution>Swansea University</institution><apcterm/><lastEdited>2021-09-13T11:40:29.7264696</lastEdited><Created>2021-08-10T10:25:14.4418069</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Mitchell</firstname><surname>Baxter</surname><order>1</order></author><author><firstname>Anna</firstname><surname>Bleakley</surname><order>2</order></author><author><firstname>Justin</firstname><surname>Edwards</surname><order>3</order></author><author><firstname>Leigh</firstname><surname>Clark</surname><orcid>0000-0002-9237-1057</orcid><order>4</order></author><author><firstname>Benjamin R.</firstname><surname>Cowan</surname><order>5</order></author><author><firstname>Julie R.</firstname><surname>Williamson</surname><order>6</order></author></authors><documents><document><filename>57576__20831__e0b65e77feb44765be04cf938284790c.pdf</filename><originalFilename>57576.pdf</originalFilename><uploaded>2021-09-13T11:39:07.0103389</uploaded><type>Output</type><contentLength>2401116</contentLength><contentType>application/pdf</contentType><version>Version of Record</version><cronfaStatus>true</cronfaStatus><documentNotes>© 2021 Copyright held by the owner/author(s). This work is licensed under a Creative Commons Attribution International 4.0 License</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>https://creativecommons.org/licenses/by/4.0/</licence></document></documents><OutputDurs/></rfc1807>
spelling	2021-09-13T11:40:29.7264696 v2 57576 2021-08-10 “You, Move There!”: Investigating the Impact of Feedback on Voice Control in Virtual Environments 004ef41b90854a57a498549a462f13a0 0000-0002-9237-1057 Leigh Clark Leigh Clark true false 2021-08-10 SCS Current virtual environment (VEs) input techniques often overlook speech as a useful control modality. Speech could improve interaction in multimodal VEs by enabling users to address objects, locations, and agents, yet research on how to design effective speech for VEs is limited. Our paper investigates the effect of agent feedback on speech VE experiences. Through a lab study, users commanded agents to navigate a VE, receiving either auditory, visual or behavioural feedback. Based on a post interaction semi-structured interview, we find that the type of feedback given by agents is critical to user experience. Specifically auditory mechanisms are preferred, allowing users to engage with other modalities seamlessly during interaction. Although command-like utterances were frequently used, it was perceived as contextually appropriate, ensuring users were understood. Many also found it difficult to discover speech-based functionality. Drawing on these, we discuss key challenges for designing speech input for VEs. Conference Paper/Proceeding/Abstract CUI 2021 - 3rd Conference on Conversational User Interfaces ACM New York, NY, USA 978-1-4503-8998-3 Virtual Environments, Speech Input, Gesture Input 27 7 2021 2021-07-27 10.1145/3469595.3469609 COLLEGE NANME Computer Science COLLEGE CODE SCS Swansea University 2021-09-13T11:40:29.7264696 2021-08-10T10:25:14.4418069 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Mitchell Baxter 1 Anna Bleakley 2 Justin Edwards 3 Leigh Clark 0000-0002-9237-1057 4 Benjamin R. Cowan 5 Julie R. Williamson 6 57576__20831__e0b65e77feb44765be04cf938284790c.pdf 57576.pdf 2021-09-13T11:39:07.0103389 Output 2401116 application/pdf Version of Record true © 2021 Copyright held by the owner/author(s). This work is licensed under a Creative Commons Attribution International 4.0 License true eng https://creativecommons.org/licenses/by/4.0/
title	“You, Move There!”: Investigating the Impact of Feedback on Voice Control in Virtual Environments
spellingShingle	“You, Move There!”: Investigating the Impact of Feedback on Voice Control in Virtual Environments Leigh Clark
title_short	“You, Move There!”: Investigating the Impact of Feedback on Voice Control in Virtual Environments
title_full	“You, Move There!”: Investigating the Impact of Feedback on Voice Control in Virtual Environments
title_fullStr	“You, Move There!”: Investigating the Impact of Feedback on Voice Control in Virtual Environments
title_full_unstemmed	“You, Move There!”: Investigating the Impact of Feedback on Voice Control in Virtual Environments
title_sort	“You, Move There!”: Investigating the Impact of Feedback on Voice Control in Virtual Environments
author_id_str_mv	004ef41b90854a57a498549a462f13a0
author_id_fullname_str_mv	004ef41b90854a57a498549a462f13a0_***_Leigh Clark
author	Leigh Clark
author2	Mitchell Baxter Anna Bleakley Justin Edwards Leigh Clark Benjamin R. Cowan Julie R. Williamson
format	Conference Paper/Proceeding/Abstract
container_title	CUI 2021 - 3rd Conference on Conversational User Interfaces
publishDate	2021
institution	Swansea University
isbn	978-1-4503-8998-3
doi_str_mv	10.1145/3469595.3469609
publisher	ACM
college_str	Faculty of Science and Engineering
hierarchytype
hierarchy_top_id	facultyofscienceandengineering
hierarchy_top_title	Faculty of Science and Engineering
hierarchy_parent_id	facultyofscienceandengineering
hierarchy_parent_title	Faculty of Science and Engineering
department_str	School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science
document_store_str	1
active_str	0
description	Current virtual environment (VEs) input techniques often overlook speech as a useful control modality. Speech could improve interaction in multimodal VEs by enabling users to address objects, locations, and agents, yet research on how to design effective speech for VEs is limited. Our paper investigates the effect of agent feedback on speech VE experiences. Through a lab study, users commanded agents to navigate a VE, receiving either auditory, visual or behavioural feedback. Based on a post interaction semi-structured interview, we find that the type of feedback given by agents is critical to user experience. Specifically auditory mechanisms are preferred, allowing users to engage with other modalities seamlessly during interaction. Although command-like utterances were frequently used, it was perceived as contextually appropriate, ensuring users were understood. Many also found it difficult to discover speech-based functionality. Drawing on these, we discuss key challenges for designing speech input for VEs.
published_date	2021-07-27T04:13:25Z
_version_	1763753911656644608
score	11.012678

“You, Move There!”: Investigating the Impact of Feedback on Voice Control in Virtual Environments

Similar Items