Conference Paper/Proceeding/Abstract 324 views 80 downloads
Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022
Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
Swansea University Author:
Chat Wacharamanotham
-
PDF | Version of Record
This work is licensed under a Creative Commons Attribution International 4.0 License.
Download (1.38MB)
DOI (Published version): 10.1145/3544548.3580848
Abstract
In recent years, various initiatives from within and outside the HCI field have encouraged researchers to improve research ethics, openness, and transparency in their empirical research. We quantify how the CHI literature might have changed in these three aspects by analyzing samples of 118 CHI 2017...
Published in: | Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems |
---|---|
ISBN: | 978-1-4503-9421-5 |
Published: |
New York, NY, USA
ACM
2023
|
Online Access: |
http://dx.doi.org/10.1145/3544548.3580848 |
URI: | https://cronfa.swan.ac.uk/Record/cronfa63283 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
first_indexed |
2023-04-29T06:11:37Z |
---|---|
last_indexed |
2023-04-29T06:11:37Z |
id |
cronfa63283 |
recordtype |
SURis |
fullrecord |
<?xml version="1.0" encoding="utf-8"?><rfc1807 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><bib-version>v2</bib-version><id>63283</id><entry>2023-04-29</entry><title>Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022</title><swanseaauthors><author><sid>5310be7eb485ebc96c9671f5a45d6f62</sid><ORCID>0000-0003-4831-2516</ORCID><firstname>Chat</firstname><surname>Wacharamanotham</surname><name>Chat Wacharamanotham</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2023-04-29</date><deptcode>SCS</deptcode><abstract>In recent years, various initiatives from within and outside the HCI field have encouraged researchers to improve research ethics, openness, and transparency in their empirical research. We quantify how the CHI literature might have changed in these three aspects by analyzing samples of 118 CHI 2017 and 127 CHI 2022 papers—randomly drawn and stratified across conference sessions. We operationalized research ethics, openness, and transparency into 45 criteria and manually annotated the sampled papers. The results show that the CHI 2022 sample was better in 18 criteria, but in the rest of the criteria, it has no improvement. The most noticeable improvements were related to research transparency (10 out of 17 criteria). We also explored the possibility of assisting the verification process by developing a proof-of-concept screening system. We tested this tool with eight criteria. Six of them achieved high accuracy and F1 score. We discuss the implications for future research practices and education.</abstract><type>Conference Paper/Proceeding/Abstract</type><journal>Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems</journal><volume/><journalNumber/><paginationStart/><paginationEnd/><publisher>ACM</publisher><placeOfPublication>New York, NY, USA</placeOfPublication><isbnPrint>978-1-4503-9421-5</isbnPrint><isbnElectronic/><issnPrint/><issnElectronic/><keywords/><publishedDay>19</publishedDay><publishedMonth>4</publishedMonth><publishedYear>2023</publishedYear><publishedDate>2023-04-19</publishedDate><doi>10.1145/3544548.3580848</doi><url>http://dx.doi.org/10.1145/3544548.3580848</url><notes/><college>COLLEGE NANME</college><department>Computer Science</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>SCS</DepartmentCode><institution>Swansea University</institution><apcterm>Another institution paid the OA fee</apcterm><funders/><projectreference/><lastEdited>2023-11-20T14:15:24.7499697</lastEdited><Created>2023-04-29T07:01:41.9616893</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Kavous Salehzadeh</firstname><surname>Niksirat</surname><orcid>0000-0003-4438-3544</orcid><order>1</order></author><author><firstname>Lahari</firstname><surname>Goswami</surname><orcid>0000-0002-8975-5885</orcid><order>2</order></author><author><firstname>Pooja S. B.</firstname><surname>Rao</surname><orcid>0000-0003-3346-2749</orcid><order>3</order></author><author><firstname>James</firstname><surname>Tyler</surname><orcid>0000-0001-8102-962x</orcid><order>4</order></author><author><firstname>Alessandro</firstname><surname>Silacci</surname><orcid>0000-0001-8121-3013</orcid><order>5</order></author><author><firstname>Sadiq</firstname><surname>Aliyu</surname><orcid>0000-0002-5090-920x</orcid><order>6</order></author><author><firstname>Annika</firstname><surname>Aebli</surname><orcid>0000-0002-8209-9950</orcid><order>7</order></author><author><firstname>Chat</firstname><surname>Wacharamanotham</surname><orcid>0000-0003-4831-2516</orcid><order>8</order></author><author><firstname>Mauro</firstname><surname>Cherubini</surname><orcid>0000-0002-1860-6110</orcid><order>9</order></author></authors><documents><document><filename>63283__27579__62aada8bca9a4a94a9bd6eb2375781dc.pdf</filename><originalFilename>63283.pdf</originalFilename><uploaded>2023-05-23T14:34:11.9744949</uploaded><type>Output</type><contentLength>1445786</contentLength><contentType>application/pdf</contentType><version>Version of Record</version><cronfaStatus>true</cronfaStatus><documentNotes>This work is licensed under a Creative Commons Attribution International
4.0 License.</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>https://creativecommons.org/licenses/by/4.0/</licence></document></documents><OutputDurs/></rfc1807> |
spelling |
v2 63283 2023-04-29 Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022 5310be7eb485ebc96c9671f5a45d6f62 0000-0003-4831-2516 Chat Wacharamanotham Chat Wacharamanotham true false 2023-04-29 SCS In recent years, various initiatives from within and outside the HCI field have encouraged researchers to improve research ethics, openness, and transparency in their empirical research. We quantify how the CHI literature might have changed in these three aspects by analyzing samples of 118 CHI 2017 and 127 CHI 2022 papers—randomly drawn and stratified across conference sessions. We operationalized research ethics, openness, and transparency into 45 criteria and manually annotated the sampled papers. The results show that the CHI 2022 sample was better in 18 criteria, but in the rest of the criteria, it has no improvement. The most noticeable improvements were related to research transparency (10 out of 17 criteria). We also explored the possibility of assisting the verification process by developing a proof-of-concept screening system. We tested this tool with eight criteria. Six of them achieved high accuracy and F1 score. We discuss the implications for future research practices and education. Conference Paper/Proceeding/Abstract Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems ACM New York, NY, USA 978-1-4503-9421-5 19 4 2023 2023-04-19 10.1145/3544548.3580848 http://dx.doi.org/10.1145/3544548.3580848 COLLEGE NANME Computer Science COLLEGE CODE SCS Swansea University Another institution paid the OA fee 2023-11-20T14:15:24.7499697 2023-04-29T07:01:41.9616893 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Kavous Salehzadeh Niksirat 0000-0003-4438-3544 1 Lahari Goswami 0000-0002-8975-5885 2 Pooja S. B. Rao 0000-0003-3346-2749 3 James Tyler 0000-0001-8102-962x 4 Alessandro Silacci 0000-0001-8121-3013 5 Sadiq Aliyu 0000-0002-5090-920x 6 Annika Aebli 0000-0002-8209-9950 7 Chat Wacharamanotham 0000-0003-4831-2516 8 Mauro Cherubini 0000-0002-1860-6110 9 63283__27579__62aada8bca9a4a94a9bd6eb2375781dc.pdf 63283.pdf 2023-05-23T14:34:11.9744949 Output 1445786 application/pdf Version of Record true This work is licensed under a Creative Commons Attribution International 4.0 License. true eng https://creativecommons.org/licenses/by/4.0/ |
title |
Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022 |
spellingShingle |
Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022 Chat Wacharamanotham |
title_short |
Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022 |
title_full |
Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022 |
title_fullStr |
Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022 |
title_full_unstemmed |
Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022 |
title_sort |
Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022 |
author_id_str_mv |
5310be7eb485ebc96c9671f5a45d6f62 |
author_id_fullname_str_mv |
5310be7eb485ebc96c9671f5a45d6f62_***_Chat Wacharamanotham |
author |
Chat Wacharamanotham |
author2 |
Kavous Salehzadeh Niksirat Lahari Goswami Pooja S. B. Rao James Tyler Alessandro Silacci Sadiq Aliyu Annika Aebli Chat Wacharamanotham Mauro Cherubini |
format |
Conference Paper/Proceeding/Abstract |
container_title |
Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems |
publishDate |
2023 |
institution |
Swansea University |
isbn |
978-1-4503-9421-5 |
doi_str_mv |
10.1145/3544548.3580848 |
publisher |
ACM |
college_str |
Faculty of Science and Engineering |
hierarchytype |
|
hierarchy_top_id |
facultyofscienceandengineering |
hierarchy_top_title |
Faculty of Science and Engineering |
hierarchy_parent_id |
facultyofscienceandengineering |
hierarchy_parent_title |
Faculty of Science and Engineering |
department_str |
School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science |
url |
http://dx.doi.org/10.1145/3544548.3580848 |
document_store_str |
1 |
active_str |
0 |
description |
In recent years, various initiatives from within and outside the HCI field have encouraged researchers to improve research ethics, openness, and transparency in their empirical research. We quantify how the CHI literature might have changed in these three aspects by analyzing samples of 118 CHI 2017 and 127 CHI 2022 papers—randomly drawn and stratified across conference sessions. We operationalized research ethics, openness, and transparency into 45 criteria and manually annotated the sampled papers. The results show that the CHI 2022 sample was better in 18 criteria, but in the rest of the criteria, it has no improvement. The most noticeable improvements were related to research transparency (10 out of 17 criteria). We also explored the possibility of assisting the verification process by developing a proof-of-concept screening system. We tested this tool with eight criteria. Six of them achieved high accuracy and F1 score. We discuss the implications for future research practices and education. |
published_date |
2023-04-19T14:15:25Z |
_version_ |
1783092714747723776 |
score |
11.024244 |