No Cover Image

Journal article 202 views 48 downloads

Moderating manipulation: Demystifying extremist tactics for gaming the (regulatory) system

Ashley Mattheis, Ashton Kingdon

Policy & Internet, Volume: 15, Issue: 4, Pages: 478 - 497

Swansea University Author: Ashley Mattheis

  • 65253.VOR.pdf

    PDF | Version of Record

    © 2023 The Authors. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

    Download (3.75MB)

Check full text

DOI (Published version): 10.1002/poi3.381

Abstract

Due to its ease of scalability and broad applicability, the use of artificial intelligence (AI) and machine learning in platform management has gained prominence. This has led to widespread debates about the use of deplatforming as the default tool for repeated or severe violations of terms or servi...

Full description

Published in: Policy & Internet
ISSN: 1944-2866 1944-2866
Published: Wiley 2023
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa65253
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract: Due to its ease of scalability and broad applicability, the use of artificial intelligence (AI) and machine learning in platform management has gained prominence. This has led to widespread debates about the use of deplatforming as the default tool for repeated or severe violations of terms or service. But technologically deterministic approaches are not infallible and can be predictable based on their actions. This opens the door for manipulation of media content and technological affordances to become tactical options for actors seeking to subvert regulation. Existing discussions often neglect the topic of manipulation of content, algorithms, or platform affordances as a primary aspect of the strategies used by extremists in relation to the difficulties of moderation from a policy perspective. This paper argues that it is essential to understand how extremists and conspiracy theorists use manipulation tactics to “game” the current policy, regulatory, and legislative systems of content moderation. Developing approaches that attend to manipulation as a strategy and focus on platform and context-specific tactics will generate more effective policies, platform rules, AI developments, and moderation procedures. This paper analyses and demystifies three primary tactics, which the authors categorise as numerology, borderlands, and merchandising, regularly used by extremists online in their strategies to ‘game’ content moderation. We provide case examples from a variety of ideologies including far-right, QAnon, and male supremacism to highlight the tactics rather than ideological nature of such manipulation. We conclude with a discussion of how demystification processes could be incorporated into content moderation settings. This paper contributes new insights about evasion tactics to the content moderation discussion and expands current understanding of how platforms can develop sociotechnical remedial measures.
Keywords: Artificial intelligence (AI), content moderation, digital culture, extremism, machine learning. manipulation tactics. propaganda. 内容审核 极端主义 人工智能(AI) 机器学习 操纵策略 (政治)宣传 数字文化Moderación de contenidos. extremismo. inteligencia artificial (IA). aprendizaje automático. tácticas de manipulación , propaganda, cultura digital
College: Faculty of Humanities and Social Sciences
Issue: 4
Start Page: 478
End Page: 497