A critical outlook at augmented reality and its adoption in education

.


Introduction
Educational tools have shown to be effective at improving the learning and teaching environment within the classroom and even change the way we think about education.Technologies such as the chalkboard, abacus and textbooks have been closely linked with educational reforms [1] demonstrating the disruptive potential of educational tools.As discussed by Honey, Culp and Carrigg [2], a natural evolution of this is the digitisation of tools to technologies with artifacts such as the the chalkboard and the abacus being replaced by interactive whiteboards and calculators respectively.
The success of these technologies are largely defined by how well teachers perceive them and how well they can fit their goals, teaching strategies and expectations [3,4].Some tools that have shown success in the classroom have been smartphones [5] and the 'Web 2.0' [6].A strategy that was employed in both of these cases was co-design.We define co-design in the context of education as a team-based process where teachers, researchers and developers work together to develop or design new educational tools.Successful applications of co-design within education can be found in science and mathematics [7][8][9].However, education is not the only area where co-design has found significant success.Co-design has also been used to improve tools within healthcare [10] and architecture [11].
One such technology that is experiencing increasing attention in educational research is Augmented Reality (AR).Sırakaya and Alsancak Sırakaya [12] state that between 2011 and 2016 the number of papers investigating the effect of AR in education increased from 8 to 26 annually.Despite this increasing research effort, which largely suggests AR has a positive impact on learners, wide scale adoption of AR has not occurred within educational settings [13].In 2016 it was predicted that the roll-out of AR in education would take two to three years [14], then in 2019 Alexander et al. [13] stated that AR was at "four to five years from adoption".It seems we are moving further away from wide scale adoption of AR in education rather than closer.
There are a number of existing reviews on AR in education, which largely focus on the effect AR has on learners, suggesting that this effect is well understood.For example, Bacca-Acosta, Baldiris, Fabregat, Graf and Kinshuk [15] observed that the most reported advantages for using AR in educational settings were that students had a greater learning performance and increased motivation.Akçayır and Akçayır [16] also found that AR promotes enhanced learning achievement in educational settings.Saltan and Arslan [17] found evidence that the use of AR improves academic performance, student engagement, student motivation and student satisfaction.Fidan and Tuncel [18] found that the most researched variables in AR were student achievement and student attitude which were largely positively effected by AR.Quintero, Baldiris, Rubira, Cerón and Velez [19] reviewed research into the effect of AR on students with disabilities, finding that in general it improved their motivation, interaction and interest.Most recently Garzón, Kinshuk, Baldiris, Gutiérrez and Pavón [20] explored studies which grounded the AR artifact design process in pedagogical theory, and found that AR had a positive impact on learning outcomes.Rather than repeat the work of these existing reviews, we now ask if the literature tells us AR has a positive effect on learning outcomes, why is wide scale adoption moving further away rather than closer?

Our contribution
The existing reviews mentioned above focus largely on the impact AR technology has on student-centred outcomes such as student engagement and student satisfaction.There is limited literature discussing the adoption of the technology outside of the previously mentioned reports by Consortium [14] and Alexander et al. [13], which only note that its time for adoption is increasing.Furthermore, none of the reviews investigate co-design as a strategy being used within the literature.This is important to explore as there is a significant link between adoption of new technologies in education and co-design.Our contribution from this review explores why AR has yet to be adopted despite the research suggesting its benefit in education.We do this by exploring the research goals of a paper, the motivations presented within the respective papers, as well as the exploration of co-design methodologies.This paper identifies that the majority of the literature focuses on student-centred results.This could explain the disconnect between the effectiveness of the technology and its adoption.
The rest of this paper is structured as follows.Section 2 presents some background on the adoption of educational technology in general, a brief introduction to AR, our research aims, questions and approach.Section 3 presents the results of the review, which are discussed in Section 4. Finally, the paper is concluded in Section 5.

Related work
The adoption of digital technologies can drastically change the landscape of education [1], a good example of which is the internet.As Mayfield and Ali [21] comments, before the internet, there was a large focus on books and libraries as a source of knowledge and a way to gather information about a given topic.However, keeping physical media up to date in a school library is costly and time-consuming whereas the internet removes this requirement to update physical media.
The adoption of digital technologies within education is not always feasible -depending both on the context and the technology itself.For example, Slay, Siebörger and Hodgkinson-Williams [22] investigated the feasibility of deploying interactive whiteboards in South African schools.In pilot studies, they found an increase in student engagement through the technology, and teachers also found aspects of the technology useful.However, teachers reported that the main drawback of the interactive whiteboard system was the interactive pen technology itself.In fact, the positive outcomes of the technology simply came from the laptop and projector.Therefore, their conclusion was that the technology was too expensive to warrant investment, especially when considering the resource constrained nature of the schools involved.
Honey et al. [2] state that focusing research efforts into the impact of learning technologies on learning outcomes is both limited and outdated.They observe that with such a large influx of educational technologies, it is now irrelevant to ask how effective a technology is on learning outcomes.Instead, they propose that the priority should be to make technologies more robust so that they can adapt to the systematic changes in educational environments.When viewing the challenge of deploying of technology from a broader perspective within education, Rogers [23] determined that the main barriers to adoption of new technologies are teacher attitudes to the technology, the institutional support provided and the availability or the quality of the hardware and software.Interestingly, one of the least important factors effecting adoption was student learning outcomes.The goal of education is to improve student outcomes by enhancing the learning environment, and research shows that technology can assist greatly in this.However, as discussed above, the integration of technology into education is consistently halted for a number of reasons, but learning outcomes are not at the forefront [24][25][26].
One method of breaking the barriers discussed by Rogers [23] is involving educators in the design of digital artifacts through co-design.Co-design in the context of education can be defined as "a highly facilitated, team-based process in which teachers, researchers and developers work together in defined roles to design an educational innovation, realize the design in one or more prototypes, and evaluate each prototype's significance for addressing a concrete educational need" [27].This approach can aid in creating more robust tools, since teachers are actively involved in the process and are likely to alter the researchers' perceptions of the importance of the educational resource.Teachers have a clearer understanding of the local context than researchers and developers, namely the societal and technical capacity of their schools to support the implementation of technology.This approach has already been done and has shown to be effective.This is important as it shows that teachers are more likely to use such technologies in the long term.We see this in the study by Cviko, McKenney and Voogt [28] which involved kindergarten teachers as co-designers of technology-rich learning activities.During their study they found that the teachers who were involved in the co-design process had a sense of co-ownership over the work, and planned to continue the use of the technology regardless of the continued support of researchers.

Augmented reality
AR is a technology that was first developed over 60 years ago.It allows computer generated imagery to approximately overlay physical objects in real-time [29].In this review we adopt definition of AR proposed by Azuma [30]: "Any system or technology that (a) combines real and virtual imagery, (b) is interactive in real-time, and (c) is registered in three dimensions (i.e., the real world)."Through the rapid development of technology AR has become available on a wide range of platforms.Initially developed on static computers with a display and camera, AR has now made its way onto smartphones, tablets and wearable glasses.These mobile devices allow the design of AR applications with mobility included as an integral part of the design [31].This mobility has resulted in applications of AR in construction [32], medicine [33], retail [34] and, of course, education [35].

Research aim and questions
The aim of this review is to understand why wide-scale adoption of AR in education is moving further away despite increasing academic research.We will not reproduce a further meta review of student learning outcomes, but instead focus on evaluating research efforts into aspects which effect adoption -which are not necessarily student centered.Our review will critically investigate the academic research into educational AR in the context of the work by Honey et al. [2] and Rogers [23] as discussed in Section 2.1.To do this we will answer the following three research questions (RQ1-RQ3): RQ1 and RQ2 will give insight into the general focus of academic research and whether or not this is aligned with factors that effect adoption.By investigating RQ3 we will gain understanding of how much stakeholders within education are involved in the academic research.
Our findings will then be used to make recommendations of where future research efforts should be focused if we wish to accelerate adoption of AR technology within education.

Research methodology for the literature review
Our review methodology is based on the 'PRISMA' strategy [36].Fig. 1 shows a flow diagram of the PRISMA strategy along with the full breakdown of the number of papers, starting from identification through search and the subsequent screening processes.In the following section each stage of the strategy is explained in detail.

Development of review protocol
The review protocol was developed to achieve the following: 1. Maximise the literature covered.
2. Include and identify related work classified as studies (case studies, experiments and others).3. Gather and synthesize significant data from the studies pertaining to the research questions defined in Section 2.2.
This protocol specifies the search strategy, inclusion and exclusion criteria, data extraction and the method of synthesis.

Identification of scope
The inclusion criteria which define our scope were: • Augmented Reality was referenced in either the title or the abstract.
• Papers which are in some way connected to one or more educational institutions.For example, researchers collaborating with a school, or a teacher conducting and reporting on an intervention within their own educational institution.• Papers working with any age group were included.
• Papers were classified as a study and not report, book chapter or abstract.
• The paper was written in English.
• The paper was peer-reviewed.
Exclusion criteria were: • The paper was not available through the university services.
• The paper's primary focus was not education.
The paper by Demir, Ahmad, Calyam, Jiang, Huang and Jahnke [37] is an example which falls outside the scope of our survey.In it they investigate using AR technology to help provide better communication and information between incident commanders and paramedics.Although this technology has been developed to also be a teaching tool, and therefore appeared in our search results, it has been omitted as there is no inclusion of educational institutions and education was a secondary focus.A second example which falls outside our scope is the paper by Noreikis, Savela, Kaakinen, Xiao and Oksanen [38] which investigates gamified AR in public spaces and how their artifact affected the overall visitor experience in a public exhibition.This paper was omitted as it did not involve teachers or students as active participants within the study evaluations.

Search for relevant studies
The search process was carried out in one step and this was to search a series of specific databases.They were: 'Science Direct', 'ACM', 'IEEExplore', 'ERIC', 'Taylor and Francis' and 'Springer'.These databases were used to ensure an effective coverage on both science and education focused research.The following search string was used for all databases: ("Augmented Reality" OR "AR" OR "Mixed Reality") AND ("Education" OR "Learning" OR "Teaching").These terms were selected since they are commonly used in both education and AR research fields and provide a large coverage of the field.At this stage, the search terms resulted in 8778 papers found in the initial search process.For this study, there was no limit on how old a paper could be for its inclusion.
At this stage, the titles and abstracts of the papers were checked.If the papers adhered to the inclusion and exclusion criteria, then the paper was downloaded into a folder as a pdf document.After completing this process, 215 papers remained.The final search was completed on Friday 30th April 2021.

Critical appraisal
The focus of the critical appraisal was on relevance (papers classified as an experiment, case study or studies that used AR in an impactful way in the study), rigor (appropriate research describing scope, methods, execution and research context) and credibility (conclusions based on analysis and reasoning).The critical appraisal was led by the lead researcher.A secondary researcher assisted in the appraisal and to check for consistency, a random sample of papers were reviewed and compared (along with the reasons for rejection).
After the appraisal process, 169 papers were accepted.The reasons for the exclusion fell under one of the following reasons: • Insufficient rigor on the study.
• Insufficient information or description on how AR was used in the study.• The AR technology was not used in an impactful way within the study.

Data extraction
At this stage, data from the 169 papers was extracted by reading through all the papers in detail.To gather this data, it was deposited into an Excel spreadsheet.The data we collected was: The year of publication, the title, the database, the journal, the keywords, the platform choice of AR used (for example, was it a computer and webcam setup, or a smartphone or a tablet), The lead author, the corresponding author, their departments, the research goal of the papers, the motivating factors of the paper and the extent of the involvement of participants in the study.

Synthesis
We employed a content analysis strategy to evaluate the literature, as defined by Elo and Kyngäs [39].Content analysis is a research method that provides a systematic and objective means of describing and quantifying phenomena within a collection of papers.This approach is a systematic and repeatable process [40] therefore making it a suitable approach for this review.An inductive content analysis methodology [41] was followed whereby we did not make any inference of meaning during review, but instead used the literal text in the paper.This methodology was selected predominately to reduce the risk of introducing author bias.The lead author led the analysis, reporting back to the rest of the research team at key milestones and when key themes emerged, to discuss and agree on codes.The lead author and a second researcher were both responsible for the final coding of papers, 100% inter-rater agreement was measured through a random sampling of papers.

Results
This section details the general results from our literature review followed by specific results organised according to each of our three research questions.

General results
In this section, we will present some general findings that whilst not being a part of our research questions, provide some interesting results worthy of discussion.We discuss the distribution of AR papers by publisher, AR publications by lead author affiliation, and different ways AR was deployed into schools and educational systems.

Distributions by publisher and lead author affiliation
Table 1 presents the distribution of papers from publishers.As seen from the table, the publisher ERIC accounts for 55% of papers included in this review, followed by Science Direct at 17%.ERIC is a primarily education focused database, and Science Direct is a database of mixed disciplines.In Table 2, the country origins based on lead author affiliation are presented, USA (16%), Turkey (14%), Taiwan (14%) and Spain (11%) are currently providing the majority of AR research.We note here that in the process of searching for papers in these different databases, we did not personally come across an instance where there were multiple copies of the same paper in different databases.

Modality
In this paper, we refer to modality as the way in which AR is deployed or presented to the participants.Three main categories of AR modality were identified: Fixed Augmented Reality (FAR), Head Mounted Devices (HMD) and Mobile Augmented Reality (MAR).FAR systems often use a static camera connected to a computer or a central processing unit (CPU).These systems are "fixed" in the sense that they are not easy to move, therefore need to be set up prior to the start of a teaching activity.An example of a FAR system can be found in the paper by Cai, Chiang, Sun, Lin and Lee [42], where an AR application was developed using a Microsoft Kinect.The topic of their study was magnets and magnetic fields.The FAR system worked by having a virtual model of a magnet on the screen and when the students waved their hands the magnets would display their magnetic field by showing virtual red lines to represent the various magnetic lines.HMD systems are head mounted units which contain all the required components for AR experience with elements such as a camera, CPU, display, etc being contained in a wearable device.Typically, the virtual information would be displayed on the lenses which were generally over the eyes of the user.An example of a HMD can be found in the paper by Keshav, Vahabzadeh, Abdus-Sabur, Huey, Salisbury, Liu and Sahin [43] who developed smart-glasses for students with autism to help with their attention and social education.The study involved the participant interacting with the teacher.The HMD would help the user pay attention to the teacher by introducing virtual indicator arrows, or by superimposing cartoon-like masks on the face of the teacher.MAR systems are deployed on mobile devices such as smartphones or tablets.These systems are lightweight, easy to use and accessible.For example, in the paper by Fokides and Mastrokoukou [44] Table 1 The distribution of the publishers for papers detailing Augmented Reality research on educational institutions.tablets are used to examine if AR can help students understand the functions of the respiratory and circulatory systems.This was achieved by displaying the content over an individual.Then, students could select specifics parts of those systems and additional information on how they worked and their function would be provided.The distribution of modalities within the papers reviewed is presented in Fig. 2. To determine which modality any given paper used, we reviewed the papers in their methodology or design sections and looked for how they developed the system, and how it was used within the study.The results show a significant majority of the research used MAR systems, the proportion of MAR systems increases rapidly after the release of ARKit1 and ARCore2 MAR development environments.For context, ARKit and ARCore are AR development tools that were pushed by Apple and Google respectively.Both tools can be utilised on multiple development environments, although ARKit only functions on iOS systems whereas ARCore works on Android and iOS systems.

Methodology
Research goals were identified by manually reviewing the contents of the papers.First, we looked to see if a paper had a section titled "research questions."If the paper did not have this, then we then reviewed for phrases such as 'the purpose of this research study was', "the goal of this study was", "the research questions were", "the research goals were" or ' the following hypothesis were'.Once all research questions for all papers were identified, content analysis was used to identify key themes.After the key themes were established, we grouped similar themes into categories with the same core concepts.For the context of this review, we view research goals as the key themes of the research questions in each paper.For example, the paper by Salar, Arici, Caliklar and Yilmaz [45] has a research question which aims to determine if emotional investment positively affects university students' focus of attention.A different paper by Sırakaya and Kilic [46] has a research question which investigates if the use of augmented reality learning material makes a significant difference regarding the course engagement of students.Both examples have the same core concept, namely, they focus on the effectiveness AR technology has on the learning outcomes of students.Thus, both papers fall under the same research goal.Through this review process, we identified 4 categories of research goals: 1. Papers with research questions that investigate the effect of AR on learning outcomes.2. Papers with research questions that investigate the functionality of AR. 3. Papers with research questions that investigate the effect of AR compared to existing methods and technologies.4. Papers with research questions that investigate the views of participants regarding AR.
During analysis we had to consider how to approach papers which had multiple research questions.Our approach to this was to evaluate the research questions individually and determine the categories they would fit into.For the papers that had multiple research questions, if at least one research question satisfied a specific category, then it would be included in that category.Thus, any paper can be within multiple categories.For example, the paper by Lin, Chen and Chang [47], who were investigating the integration of AR technology into teaching activities to design a system for learning solid geometry.They posed the following research questions: 1. Are achievements in math and spatial ability related? 2. Can AR-supported programs improve students' spatial ability?3.For students with various academic achievements, how effective is their learning after the experiment?4. What are the students' perceptions of the systems usability?5. What are the students' perceptions in terms of system task load? 6.Is students' learning effectiveness related to the system's usability and task load?
which fall into multiple research goals; 'papers which investigate the effect of AR on learning outcomes' and 'papers which investigate the functionality of AR technology'.Specifically, research questions 1, 2 and 3 fall into the learning outcomes, and 4,research goals 5 into the functionality of the AR system.Research question 6 was an exception in the sense that it discuses both research goals, however both research goals have already been accounted for here.A counter example is the paper by Chandrasekera and Yoon [48] had two main research questions.They wanted to observe the effect interface type has on technology acceptance, and how learner preference interacts with a certain media type to affect technology acceptance.Here, the research questions aligned and we deemed that they were primarily focused on the functionality of the AR system, so the paper was placed in that single category.

Results
Papers with research questions that investigate the effect of AR on learning outcomes.Papers classed within this category focus on investigating the effect of AR on improving learners academic ability.For example, Nuanmeesri, Kadmateekarun and Poomhiran [49] developed an AR application to assist in teaching human heart anatomy and blood flow.Specifically, they assessed if AR enables the students to gain knowledge and a deeper understanding of a topic.To evaluate the students understanding they use a pre and post-test questionnaire which was evaluated by experts specialised within the fields of information technology and biology.They found that the AR approach was more effective in increasing student achievement compared to the control group which used physical media, pictures and videos.Another example can be found in the paper by Cook [50] where they assess if AR aids in the understanding and conceptualisation of the subject content.The study focused around giving students the task to construct a sound system in a Year 12 music class.They found that the use of AR increased their knowledge about the individual components, able to setup up the sound systems faster than previous cohorts with fewer mistakes.
Papers with research questions that investigate the functionality of AR.In this category we include papers which focused on investigating issues or difficulties regarding the functionality of the AR system designed for the educational system.For example, the paper by Chandrasekera and Yoon [48] investigates how different interface types affect technology acceptance.Specifically, they considered AR and VR interface types, finding that participants found the AR system easier to use and reported that they were more likely to use it again.Another example is the paper by Lin, Chen and Chang [47].One of their research questions looked at the students' perceptions in terms of system task load and another looked at the students' perception of the AR systems usability.They found that the students felt highly satisfied by the AR system and no issues were reported with its usability.Additionally, they found that upon completing the tasks in the experiment, students felt a low to moderate task load through using the technology.
Papers with research questions that investigate the effect of AR compared to existing methods and technologies.In this category we include papers where AR is compared to existing methods and technologies already taking place within a classroom.By methods we are referring to pedagogical techniques and practices.For example, Chen and Wang [51] perform a study in an urban design education class.They focus on using tangible AR technology and its ability to overlay virtual information in the real world to improve the experiential and collaborative learning methods already being employed in the classroom.They found that using an AR system increased the collaborative methods already employed within the classroom.Additionally, they found that it increased the speed at which learners gained experience at designing different urban areas.By technologies we refer to existing tools which are already employed within the classrooms.For example, in the paper by Juan, Alexandrescu, Folguera and García [52], videos were compared to the AR system in the teaching of dental morphology.To evaluate this, they split the participants into two groups, group A and group B. Group A used the video format then took a test, whereas group B used the AR system and then took the test.From the study they found that both groups had an increase in knowledge acquired.However, there was no significant difference in the amount of knowledge gained between the two groups.
Papers with research questions that investigate the views of participants regarding AR.The papers in this category focus on the views participants have regarding AR technology.For example, the paper by Tzima, Styliaras and Bassounas [53] investigated teachers' opinions on: AR technology, the need for continual training, the process of creating 3D models and the feasibility of AR application development by teachers and students in school settings.The study methodology in this case is descriptive and the participants did not engage with an AR application during the study.They found that half of the participants did not know what AR technology was, and half of the teachers who reported that they have not used AR, had in fact used an application that had AR.Another example is the paper by Uygur, Yanpar Yelken and Akay [54] where they investigated the views of teacher candidates' on AR applications in education.They found that the majority of teacher candidates did not know much about AR applications, and those that did studied in the Department of Computer Education and Instructional Technology.Furthermore, the participants who had experience of AR found it entertaining, motivating and able to facilitate learning.The paper distribution can be found in Fig. 3.We found that most papers fell within the category of investigating the effect of AR on learning outcomes, with 123 total entries (63%).The next most researched category was studies that investigated the functionality of AR, with 36 entries (18%).The categories with the fewest papers were those which investigated the views of participants regarding AR technology, and those which compared AR to existing practices within the class room with 21 (11%) and 16 (8%) entries respectively.

RQ2: What are the motivating factors of academic researchers of AR technology in education?
We define a paper's motivating factor as the principle reason or argument for carrying out the research.For example, the paper by Lin, Chen and Chang [47] discusses how some organisations within Taiwan are concerned at the level of ability of students in geometry, and how it has been considered as essential skill for mathematicians.So the motivating factor for their paper is to improve the level of ability of students in geometry.To identify the motivation of a paper we analysed the motivation section, if one existed, or the introduction.Much in the same way as with RQ1, once a motivation for each paper had been determined we grouped these into themes.Three themes of motivation were identified: Papers motivated by positively impacting students and learners.Papers in this category are motivated by student-centred effects, such as improving student motivation, engagement, immersion and/or academic performance.For example, the study by Ibáñez, Ángela Di Serio, Villarán and Delgado Kloos [55] has been included in this category since the primary motivation for the paper was the direct connection between the state of flow of the student and their academic achievement.Additionally, papers such as the one by Cai et al. [42] that were motivated by the difficulty of teaching abstract concepts such as magnetism were included in this category.The reason for this was because they wanted to improve the student experience in learning such topics and reflecting on the existing tools available.
Papers motivated by a lack of empirical data.Here we included papers where the primary motivation for the research came from the lack of existing empirical data in that area.For example, the paper by Yannier, Hudson, Wiese and Koedinger [56] was included as the motivation for this paper was a lack of an empirical basis for explaining why and how 3D physical objects may enhance learning compared to 2D flat-screen representations.Specifically, this paper noted that there was no documentation on the explicit benefits that came from using mixed-reality environments.
Papers motivated by the integration of AR into educational systems.Papers in this category are motivated by how AR technology can be successfully integrated within educational systems.For example, the paper by Tzima, Styliaras and Bassounas [53] satisfies this since they understand that AR research has shown to improvement student learning outcomes.However, its educational use is limited in Greece and they wish to understand the viewpoints of teachers regarding AR for adoption.Another example is the paper by Muñoz-Cristóbal, Jorrín-Abellán, Asensio-Pérez, Martínez-Monés, Prieto and Dimitriadis [57] which is motivated by the increase of technology used within the classroom leading to additional load on teachers.
The paper distribution can be found in Fig. 4. The biggest motivating factor came from positively impacting students and learners at N=126 (75%).This was followed by papers motivated by a lack of empirical data at N=26 (15%).Then, the next category were papers motivated at the possibility of integrating AR into educational systems N=17 (10%).

Methodology
We first split papers into two broad categories, those which adopted traditional design approaches and those which adopted co-design approaches.Traditional design approaches were identified as studies where participants are considered research subjects, and are not actively involved in any of the design or evaluation processes of the study.In contrast, co-design approaches will have some participants who actively engage with the design or evaluation process of the study and/or AR artifact designed.For example, the paper by Kamarainen, Metcalf, Grotzer, Browne, Mazzuca, Tutwiler and Dede [58] is considered a co-design approach as the teachers were actively involved in designing the survey for the students to answer.
We then identified the type of participant/collaborator who took part in each of the papers.In our analysis of the papers we found three main categories of participant: students, teachers and experts.For example, in the paper by González Rogado, Vivar Quintana and Lavandero Mayo [59], they evaluate the use of technology to improve safety in the laboratory.They address this safety concern by incorporating AR technology.In this study the assessed participants are the students since the goal of the study was to evaluate the effectiveness of the new system.In the paper by Uygur, Yanpar Yelken and Akay [54], they wanted to determine the views of pre-service teachers on AR applications in education.Here, the assessed participant are the teachers since the goal of the paper is determine the views of teachers on AR applications.Finally, in the paper by Osuna, Gutiérrez Castillo, Llorente and Valencia-Ortiz [60], they employ a Delphi study to gather the opinions of experts working within education and AR to assess the limitations and obstacles for the integration of AR technology into university education.Here, the assessed participants are neither teachers or students, but expert researchers within the field.They are the participants since they wanted the opinions of researchers who have previously done work with AR.
For the papers which adopted a co-design approach we also identified the various ways participants were involved in the research.The categories of participant involvement are given in the results below.

Results
Out of the papers considered 153 (91%) adopted the traditional design approach and 16 (9%) adopted a co-design approach.We will now explore the breakdown of participant type in each of these broad groups.
Papers adopting a traditional design approach Fig. 5 shows the distribution of participant types in the papers which adopted a traditional design approach.We found that From our results, we found that of the 153 papers using a traditional approach, 42% of papers used higher education students.Then, 41% of participants in this traditional approach were compulsory education students.Then, 8% of participants were teachers in the traditional approach.
Papers adopting a co-design approach All of the papers which adopted a co-design approach involved teachers as participants.There were 3 papers [61][62][63] which also involved students in addition to teachers.Fig. 6 shows the various ways teachers were involved in the research.
Of the papers which adopted a co-design approach, 19% of papers commented on using co-design with teachers.Then, 13% of papers had teachers involved in the evaluation of the data, and another 13% of papers had the participants create AR content in some form.The rest of the papers in this category only had one reported instance.

Discussions
The aim of this review was to understand why AR is yet to be adopted widely within education, despite an increasing attention from academic researchers.In the following sections we will discuss our findings in detail, make several recommendations and finally evaluate our approach by identifying potential biases.

RQ1: What are the research goals of academic research into AR technology in education?
Our results show that the research goals in AR educational technology are predominately focused on improving learning outcomes.We found that 73% of the papers we considered identified understanding the effect of AR on learning outcomes as a research goal.In fact, in all the cases where papers were identified to have multiple research goals, one of these goals was the effect on learning outcomes.The next most common research goal, included in 21% of the papers considered, was investigating the effectiveness of AR as a technology.Of these papers, a significant amount (31%) were focused on the user experience where the student was considered the primary user of the technology.The next most common research goal compared the effect of AR technology to existing learning methodologies and technologies.Even in this category, a significant amount (44%) of papers were focused on improving tools/ teaching methodologies for the students.
These results are unsurprising and confirm the findings of other reviews that AR research is still strongly focused on student outcomes and largely beneficial in this aspect [12,[15][16][17], however this also contributes to the research by identifying that there is little discussion of AR adoption and the challenges present within adoption.

RQ2: What are the motivating factors of academic researchers of AR technology in education?
By far the most frequent motivation of academic research in AR educational technology was improving student learning outcomes.We found that 75% of papers were motivated by providing a positive impact on students and learners.The next most common motivation, with 15% of the papers, was studies motivated by the lack of existing empirical data.Finally, the least common motivation, with 10% of papers, was integrating AR into educational systems.Clearly, it is unsurprising that an increasing volume of research has not led to wide scale adoption, when a minority of research is motivated by integrating AR into educational systems.

RQ3: How is co-design utilised within AR technology in education research?
In the third research question, we identified how co-design is utilised with AR technology in educational research.From our results, we found that only 16 (9%) papers recorded usage of a co-design approach.Further, 5 of those 9% (31%) papers did not actually specify how the participants were involved with the study or with the artefact at all.In fact, none of the papers provided in-depth explanations on how the codesign methodology was used, which led us to believe that it was not a core focus of the paper.
This was a interesting result to find in our analysis, primarily because the problems found in educational technology adoption have been known within the field of AR research for the last few years [53,60,64].However, the research still continues to focus more towards student-centred outcomes even though experts within the AR field understand the barriers to adoption [60].It's clear that to solve these problems, we must include the teachers as part of the process and one approach that has been known to be effective is the use of co-design methodologies [27,28].

Summary of findings
In our analysis of the papers, we found that they were predominately intervention style studies with scope limited to evaluating the student experience in a narrow slice of time.Even with exceptions to this, such as the study by Ruiz-Ariza, Casuso, Suarez-Manzano and Martínez-López [65] which was conducted over 8 weeks, the format of the study was still the same.An artefact would be either developed or found, then it would be utilised in a learning environment and evaluated.None of the studies we analysed observed how the use of that system would develop over time.This isolated approach may result from the majority of research goals being focused on student learning outcomes or the effectiveness of AR technology itself.It is sensible to evaluate such goals in short term studies.Furthermore, the results from RQ2 show that researchers are primarily motivated at providing a positive impact on students, which may also explain the study design observed above.When investigating RQ3, we found that a minority of studies involved co-design.This is likely a byproduct of the study design discussed above and the focus on student-centred outcomes.When focusing on student outcomes, it makes sense to use more traditional design approaches where participants are solely considered research subjects -this approach accounts for 91% of the papers analysed.Additionally, even the papers that adopted the co-design approach did not report the level of involvement of those participants, or how they effected the design process.

Recommendations
In this section we present some recommendations of areas for future research if we wish to see AR adopted widely as an educational technology.
As we discussed in the introduction, there is a vast amount of empirical evidence that shows AR can have a positive effect on student learning outcomes.Whilst student-centred results are important and do have an impact on adoption, these should not be considered in isolation.
As discussed previously, Rogers [23] has identified that student outcomes is the least important factor which influences the adoption of technology within education.If we wish for AR to be adopted we can not continue to focus on student-centred outcomes in isolation.Therefore our first recommendation is for the AR research community to shift away from research solely focused on learning outcomes and towards research which also considers how augmented reality integrates in the teaching environment.
As discussed previously, co-design approaches lead to positive teacher attitudes, co-ownership, continual use of artefacts post-study and positive student outcomes [28].This is likely because teachers have a clearer understanding of the local context within educational institutions than the researchers.Our results show that co-design is under utilised as a technique in AR research.Furthermore, as discussed in the introduction, it is a strategy that has been utilised and been effective in education as well as other fields such as healthcare [10] and architecture [11].Therefore our second recommendation is for the AR research community to explore more co-design strategies.
Our final recommendation concerns the choice of modality in regards to AR deployment strategies.The vast majority of papers analysed did not justify their choice of AR modality.This raises the question of how much we understand the effect of AR modality on learning, teaching and, therefore, adoption.At the time of this review, no research has been completed comparing the different modalities of AR against themselves in the context of education.We should consider researching into modalities since different methods of interaction can result in problems which may affect adoption.This can be seen in other fields, for example, the paper by Zhou, Zhang, Laput and Harrison [66] discusses some of the difficulties with using touch screens on smartwatches as this leads to finger occlusion, effectively reducing the screen size of the smartwatch.Even within the field of AR in education, Ibáñez, Uriarte Portillo, Zatarain Cabada and Barrón [67] found that using the same AR modality provided different results based on if the students went to a public or private schools.This is noteworthy as this could mean that different modalities can have differing levels of impact based on societal background.Therefore our third recommendation is for the AR research community to investigate the effects of AR modality on teaching and learning.

Threats to validity
Our review was carried out according to the method presented in Moher et al. [36] to ensure a systematic approach was taken.This approach ensured a significant coverage of the literature, including all relevant papers are captured and synthesize the data from these papers in a meaningful way.One possible threat to the validity may be in the filtering of papers found from the initial study, and for relevance, rigor and credibility.We evaluated all the papers and the papers that failed to satisfy those requirements were removed.The reader can find all the relevant rejected papers and why they were rejected upon request.
Another possible threat to validity could be the choice of database selection.It is true that some literature may have been missed in the coverage of the literature, however with the databases of ERIC, Elsevier, IEEExplore, ACM, Taylor and Francis and Springer, we have a significant coverage of the literature of AR research from both computer science and education research databases.

Conclusion
To conclude, this paper has presented a literature review to understand why AR technology has not been adopted into educational intuitions despite increasing empirical data suggesting its usefulness.In total, 169 papers were included in the review.To understand this, we presented 3 research questions which were: From RQ1, we found that the majority of AR research was predominantly focused on improving learning outcomes, which were largely successful.Another research goal was found in evaluating effectiveness of AR as a technology.Within this research goal, they also measured the usability of the technology with the focus on student experience.The next research goal was effect of AR technology to existing learning methodologies and technologies.Their focus was to improve tools/ teaching methodologies for the students.RQ2 found that the predominant motivating factor amongst researchers was to positively improve student learning outcomes.Researchers were also motivated by the lack of existing empirical data over a topic, and with integrating AR into educational systems.In RQ3 we found that only a very small sample of papers utilised a co-design approach, and even then, no sufficient detail was provided on how the co-design process was used.The remaining papers used a more traditional approach in involving participants as solely research subjects.Of the papers that adopting this strategy, the participants were primarily students.
The main conclusion from our review is that AR research in education is largely focused on student-centred aspects which are understood to be one of the least important factor for adoption.Therefore it is unsurprising that AR technology has not been widely adopted in education, despite the increase in AR research.Based on our review, our recommendations for the AR in education research community are: (1) a shift away from research focused on learning outcomes and towards research which also considers how augmented reality integrates in the teaching environment, (2) to explore more co-design strategies, and (3) to

Table 3
The distribution of the Research Goals of papers in Augmented Reality research on educational institutions.

Fig. 1 .
Fig. 1.PRISMA Flow Diagram [36] describing the search methodology used in this study.In the diagram the number of papers remaining and/or excluded at each stage of the process is reported.

Fig. 2 .
Fig. 2. Frequency distribution of augmented reality deployment methods/modality and how these have changed over time.The release dates of ARKit and ARCore, two mobile augmented reality (MAR) development environments, is indicated.Since only the first four months of 2021 were included in our review, 2021 has been omitted from this graph.

Fig. 3 .
Fig. 3.The distribution of the Research Goals of papers in Augmented Reality research on educational institutions.

1 .
Papers motivated by positively impacting students and learners.2. Papers motivated by a lack of empirical data.3. Papers motivated by the integration of AR into educational systems.

Fig. 4 .
Fig. 4. The distribution of Motivating Factors of papers in Augmented Reality research on educational institutions.

Fig. 5 .Fig. 6 .
Fig. 5.The distribution of participants as research subjects on papers in Augmented Reality research in educational institutions.

•
What are the research goals of academic research into AR technology in education?• What are the motivating factors of academic researchers of AR technology in education?• How is co-design utilised within AR technology in education research?

Table 2
The country origins of published papers based on lead author affiliation.Countries which represented less than 2% of papers were omitted from this table.