Journal article 378 views 108 downloads
Moderating manipulation: Demystifying extremist tactics for gaming the (regulatory) system
Policy & Internet, Volume: 15, Issue: 4, Pages: 478 - 497
Swansea University Author: Ashley Mattheis
-
PDF | Version of Record
© 2023 The Authors. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.
Download (3.75MB)
DOI (Published version): 10.1002/poi3.381
Abstract
Due to its ease of scalability and broad applicability, the use of artificial intelligence (AI) and machine learning in platform management has gained prominence. This has led to widespread debates about the use of deplatforming as the default tool for repeated or severe violations of terms or servi...
| Published in: | Policy & Internet |
|---|---|
| ISSN: | 1944-2866 1944-2866 |
| Published: |
Wiley
2023
|
| Online Access: |
Check full text
|
| URI: | https://cronfa.swan.ac.uk/Record/cronfa65253 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract: |
Due to its ease of scalability and broad applicability, the use of artificial intelligence (AI) and machine learning in platform management has gained prominence. This has led to widespread debates about the use of deplatforming as the default tool for repeated or severe violations of terms or service. But technologically deterministic approaches are not infallible and can be predictable based on their actions. This opens the door for manipulation of media content and technological affordances to become tactical options for actors seeking to subvert regulation. Existing discussions often neglect the topic of manipulation of content, algorithms, or platform affordances as a primary aspect of the strategies used by extremists in relation to the difficulties of moderation from a policy perspective. This paper argues that it is essential to understand how extremists and conspiracy theorists use manipulation tactics to “game” the current policy, regulatory, and legislative systems of content moderation. Developing approaches that attend to manipulation as a strategy and focus on platform and context-specific tactics will generate more effective policies, platform rules, AI developments, and moderation procedures. This paper analyses and demystifies three primary tactics, which the authors categorise as numerology, borderlands, and merchandising, regularly used by extremists online in their strategies to ‘game’ content moderation. We provide case examples from a variety of ideologies including far-right, QAnon, and male supremacism to highlight the tactics rather than ideological nature of such manipulation. We conclude with a discussion of how demystification processes could be incorporated into content moderation settings. This paper contributes new insights about evasion tactics to the content moderation discussion and expands current understanding of how platforms can develop sociotechnical remedial measures. |
|---|---|
| Keywords: |
Artificial intelligence (AI), content moderation, digital culture, extremism, machine learning. manipulation tactics. propaganda. 内容审核 极端主义 人工智能(AI) 机器学习 操纵策略 （政治）宣传 数字文化Moderación de contenidos. extremismo. inteligencia artificial (IA). aprendizaje automático. tácticas de manipulación , propaganda, cultura digital |
| College: |
Faculty of Humanities and Social Sciences |
| Issue: |
4 |
| Start Page: |
478 |
| End Page: |
497 |