Biologically inspired herding of animal groups by robots

A single sheepdog can bring together and manoeuvre hundreds of sheep from one location to another. Engineers and ecologists are fascinated by this sheepdog herding because of the potential it provides for ‘bio‐herding’: a biologically inspired herding of animal groups by robots. Although many herding algorithms have been proposed, most are studied via simulation. There are a variety of ecological problems where management of wild animal groups is currently impossible, dangerous and/or costly for humans to manage directly, and which may benefit from bio‐herding solutions. Unmanned aerial vehicles (UAVs) now deliver significant benefits to the economy and society. Here, we suggest the use of UAVs for bio‐herding. Given their mobility and speed, UAVs can be used in a wide range of environments and interact with animal groups at sea, over the land and in the air. We present a potential roadmap for achieving bio‐herding using a pair of UAVs. In our framework, one UAV performs ‘surveillance’ of animal groups, informing the movement of a second UAV that herds them. We highlight the promise and flexibility of a paired UAV approach while emphasising its practical and ethical challenges. We start by describing the types of experiments and data required to understand individual and collective responses to UAVs. Next, we describe how to develop appropriate herding algorithms. Finally, we describe the integration of bio‐herding algorithms into software and hardware architecture.


| BACKG ROU N D
People have been attempting to control the movements of wild animals for many centuries, with methods frequently being biologically inspired (Vincent et al., 2006). For example, scarecrows and raptor models have been used to deter birds from farmland for over 3000 years (DeVault et al., 2013;Haining, 1988;Marsh et al., 1992), and people have used dogs to successfully herd livestock for more than 6000 years (Coppinger & Coppinger, 2014). Today, efficient methods for controlling or herding the movement of animal groups have the potential to provide solutions to a variety of animal conservation and management problems at sea (Frixione & Salvadeo, 2021;Horswill et al., 2022), over the land (Fàbregas et al., 2021;Jackson et al., 2012;McKnight, 1995) and in the air (Allan, 2000;Nilsson et al., 2021).
At sea, herding bird flocks has the potential to reduce the negative effects of seabird depredation on fish farms, which can account for >53% of annual yields (Lekuona, 2002) and reduce bird wind-turbine collisions at offshore electricity-generating wind farms (Desholm & Kahlert, 2005). In the air, herding bird flocks away from airports may prevent bird flock collisions with aircraft, estimated to cost the industry $1.2 billion worldwide (Allan, 2000) and resulting in human casualties (Allan, 2000;Dale, 2009;DeVault et al., 2011). Over the land, herding farm livestock can promote efficient use of pastures, protect environmentally sensitive areas, provide ecosystem services and increase profitability (Reichelt, 2018). Herding the movements of free-ranging livestock and wildlife could prevent disease transmission (Vercauteren et al., 2008) and mitigate carnivore-livestock conflicts (Rust & Marker, 2013;Ugarte et al., 2019). However, it is often costly, difficult and even dangerous for humans to directly herd animal groups in many of the contexts mentioned above. This has led to the proposal that herding can instead be performed by robots (Strömbom et al., 2014).

| HOW TO HERD ANIMAL G ROUPS?
A major factor driving the evolution of collective behaviour across different species is predation (Caro, 2005;Krause & Ruxton, 2002;Quinn & Cresswell, 2006). To avoid predators, members of large groups modify their positions relative to each other, often decreasing their nearest neighbour distances (Cavagna et al., 2013;Couzin & Krause, 2003;Hamilton, 1971;King et al., 2012;Quinn & Cresswell, 2006). For example, sheep Ovis aries show a strong attraction towards the centre of their flock when approached by a herding dog Canis lupus familiaris (King et al., 2012). At the same time, individuals may move away from the position or the approach direction of the threat (King et al., 2012;Sankey et al., 2021;Strömbom et al., 2014), as seen in homing pigeons Columba livia that quickly turn away from the heading of an approaching aerial robot (Sankey et al., 2021).
Knowledge of how individuals and groups respond to potential threats can enable humans to control their movement.
Remote-controlled terrestrial or aerial robots have been used to herd a variety of species (Evered et al., 2014;Li et al., 2022;Long et al., 2020;Paranjape et al., 2018;Yaxley et al., 2021; Figure 1). However, the person tasked with controlling the herding robot is limited by what they see directly or via the robot camera system, limiting their scope and applicability. To overcome the need for user control and automate herding through autonomous robots, herding algorithms based on efficient herding strategies are needed.
Inspired by real-world data, a variety of general herding algorithms have been developed (Adachi & Kakikura, 2006;Gade et al., 2015;Bennett & Trafankowski, 2012;Miki & Nakamura, 2006;Strömbom et al., 2014). Unfortunately, due to the challenge of tracking and responding to wild animal groups in real time , most algorithms are tested in computer simulations and have not yet been applied in a real-world setting (though see Pfeifer et al., 1998).
Making a step towards automated herding in the wild, Strömbom and King et al. (2018) conducted laboratory tests using a feedbackcontrolled, image-based, tracking system. To transfer their approach to real-world settings, they suggested using pairs of robots. One robot would collect images and generate information about the position and heading of the animal group. Based on this information, this 'surveillance' robot should direct the behaviour of a second robot, acting as the herder (Strömbom & King, 2018). In this way, the first robot should be aerial and optimised to collect visual data and the second robot should be optimised for herding. We believe that combining this approach with unmanned (or uncrewed) aerial vehicles (UAVs) is a promising framework to achieve automated bioherding. We further provide a roadmap on how to realise it based on our perspective.

| UAVs (DRONE S) FOR B I O -HERD ING
UAVs are being used as tools to support conservation efforts in a wide range of environments (Sandbrook, 2015;Schroeder et al., 2019;Weston et al., 2020): their speed and mobility make them an ideal robot to follow and interact with animal groups at sea, over the land and in the air, that is, to herd them ( Figure 2). Here, we break down this task of bio-herding using UAVs into three steps: (1) system understanding, (2) model development and (3) system integration. First, empirical data should be collected and analysed to provide information on the response and interaction of focal animal groups with UAVs. These data can then be used to develop theoretical models of herding and any specific design features of herding UAVs. The models can then be refined via model inference and selection to create algorithms that will be integrated into software and hardware, setting up the architecture for flying teams of 'surveillance and herding' UAVs. We review each stage in more detail below.  (King et al., 2012;Sankey et al., 2021).

| Understanding of the system
For cases where mounting loggers to animals is not possible, sets of stationary cameras or quadcopter drones can be used to record individual and collective responses. Robust data can then be collected through tests varying the velocities and approach strategies of the UAV, as well as with animal groups of different sizes, geometry and density.
Understanding habituation of animals to the initially novel stimulus of a UAV in the short term and long term is also key to developing systems that work reliably. Existing data give somewhat contradictory information. Experiments with ground-based robots for herding sheep, for example, have shown significant habituation of the animals with the robot shepherd, compared to the presence of a real sheepdog (Evered et al., 2014). Flocks of birds do not always react to drones that show little resemblance to a predator (Egan et al., 2020), and in some cases even show aggressive behaviour (mobbing) towards drones (Frixione & Salvadeo, 2021). However, when a robotic falcon (Figure 1d) was presented to flocks of birds, responses to the F I G U R E 2 Bio-herding. Unmanned aerial vehicle (UAVs) for bio-herding in different contexts, where a surveillance UAV records the position and heading of animal groups and informs the behaviour of the herding UAV to manoeuvre the group. Examples shown: (a) At sea, herding bird flocks to reduce the negative effects of seabirds on fish farms, and reduce bird wind-turbine collisions. (b) Over the land, herding livestock and wildlife for ecosystem services and mitigate carnivore-livestock conflicts. (c) In the air, herding bird flocks away from specific urban areas and away from airports to prevent bird flock collisions. robot predator were stable over time (Sankey et al., 2021;Storms et al., 2022). Herding methods should thus be developed and refined through preliminary tests to be species specific and context specific.
For example, 'barking' drones were designed and used to herd farm animals (Li et al., 2022) after discovering that sheep react to auditory cues when approached by a drone, making their herding more efficient (Yaxley et al., 2021).

| Development of bio-herding models
Once data on animal groups' reactions to UAVs have been collected, researchers can then use these individual-level and group-level observations to produce data-driven models. These models should quantify how individuals interact with each other and respond to each other (Papadopoulou et al., 2022a), and to UAVs (Papadopoulou et al., 2022b;Sankey et al., 2021).
In the case of interactions with each other, while interactions will vary by species and context (Ballerini et al., 2008;Evangelista et al., 2017;Ling et al., 2019), most models of collective motion are based on the balance of attraction, repulsion and alignment of grouping individuals Couzin & Krause, 2003;Hemelrijk & Hildenbrandt, 2012;Papadopoulou et al., 2022a;Strömbom et al., 2014). However, existing herding models only include the standard alignment interaction between neighbours within the animal group which can limit their applicability (Cucker & Smale, 2007;Motsch & Tadmor, 2014). Therefore, a successful bio-herding model must be selected after exploring a variety of modelling approaches from collective behaviour, with agents constantly or intermittently moving (Strömbom & Tulevech, 2022) and being attracted to each other's current (Strömbom et al., 2019) or future position (Bailo et al., 2018;Gerlee et al., 2017;Strömbom & Antia, 2021).
In the case of interactions with UAVs, agents are typically programmed to show simple repulsion from potential threats (Kolokolnikov et al., 2013;Strömbom et al., 2014). Animal behaviour studies can again offer insight into potential modelling rules, for instance, through experiments of fish shoals interacting with biologically inspired and interactive robotic predators (e.g. Polverino et al., 2019Polverino et al., , 2022. However, the interaction dynamics of a herding robot will need to be different to a natural (or simulated) predator, since the goal of bio-herding is to keep the group together and manoeuvre it, while a predator's goal is typically the opposite, to break

F I G U R E 4
Example of a bio-herding model. The figure represents an output of a simulation of interactions between a group of 20 agents (animals) and a herding agent (unmanned aerial vehicle [UAV]) following a simple 'predator-swarm interaction' model proposed by Chen and Kolokolnikov (2014). The animals move based on attraction and short-range repulsion forces among them (to stay together, but also avoid colliding with each other), while also being repulsed by the UAV. The animals' repulsion from the UAV follows a power law, so the repulsion force decays at large distances. The UAV trajectory (dark blue) is determined using a particle swarm optimisation algorithm (MATLAB & Simulink, 2022; Kennedy & Eberhart, 1995) that attempts to steer the animals to the target zone destination in a minimum amount of time. The animal starting positions are shown by red-filled circles, their trajectories in grey and their final positions are shown by light bluefilled circles.
up groups and isolate individuals as prey (Zoratto et al., 2010). We suggest that optimal control theory (Bailo et al., 2018;Totzeck & Pinnau, 2020) may provide a useful framework to generate heuristic rules for the herding strategy of a UAV. For example, we present a herding simulation in which a group of 20 agents interact through attraction and repulsion forces and show repulsion from a UAV ( Figure 4). In this scenario, the trajectory of the UAV is determined by a computational optimisation procedure in real time.
Choosing the 'best' model for a UAV to follow and herd real animal groups will necessarily require some systematic testing and refinement, but Bayesian inference and model selection will speed up this process (Mann et al., 2013). This approach selects models best supported by the data, by specifying a likelihood function (the probability of the data conditioned on a specific model parameterisation) and averaging over the possible parameter values using an appropriate prior distribution to give the marginal likelihood, a measure of model fit that accounts for uncertainty in model parameters, such that different models can be fairly compared (MacKay, 2003 ).
Theoretical reasons to favour one model over another can be incorporated as differing prior probabilities on the models themselves, or more usually these can be equally weighted a priori to give a posterior probability for each model proportional to the marginal likelihood. Bayesian inference can also be used to evaluate the posterior distribution of the parameters within a candidate model, identifying areas of remaining uncertainty, and assisting in the design of new tests to reduce this uncertainty in the most efficient manner (Garnett, 2023).

| Integration of the system
Once herding algorithms are developed, these can be integrated into software and hardware architecture for pairs of UAVs.
Following our perspective of using two UAVs, suitable software integration and automation already exist (Kangunde et al., 2021).
There are open-source autopilots for managing the basics of keeping the two UAVs in the air, including take-off, landing, loitering and flying to waypoints (Day et al., 2015). A ground control station would be needed to communicate between the two UAVs. The first UAV would be optimised to collect visual data, providing information on the formation and dynamics of the group and the relative position of the group to the second UAV. Based on the data fed to the control station, the herding UAV would then be instructed to move towards the next herding-efficient position. Both UAVs can be updated at frequencies greater than 1 Hz, which should allow any algorithm to be rapidly integrated and fed to the herding UAV.
This process continues until the group has been moved to a preassigned location (for example).
The greatest challenge we foresee at this stage relates to the complexity of the computer vision routines necessary to track animals in the field (the job of the surveillance UAV). This is one of the reasons we have proposed a dedicated surveillance UAV-it would take a distant position to optimise the collection of this information (while the herding UAV would likely be closer and with poorer view). Even with a surveillance UAV positioned with a good view of the animals, the perceptual capabilities to detect the animals will be different depending on the context. Walking animals can likely be detected and tracked successfully with proven techniques from computer vision (Redmon et al., 2016). However, flying animals represent a greater challenge in this respect, as they are typically smaller, faster and can be in any direction around UAVs (also challenging its positioning). This may require faster cameras (Gallego et al., 2022) with wide field of view, in combination with active vision strategies (where cameras scan for new information) to avoid a computational effort that is too large. In contexts where partial occlusions are possible (e.g. in the air), additional methods may need to be developed, especially if the algorithms chosen (above) require information like velocity or heading direction to be estimated.

| CHALLENG E S AND E THI C S
In our proposed framework, we have focussed on UAVs because they provide a flexible solution across a variety of contexts. However, there are challenges with the use of UAVs. The nature of these problems highly depends on the system under examination, but in all cases, it is important that researchers consider both the human and wildlife population related to it. Any use of a UAV will require adhering to local Civil Aviation Authority regulations, and any research on wildlife will require local ethical approval and must follow ethical and welfare guidelines for the treatment of animals in research (Stöcker et al., 2017;Vas et al., 2015).
When using UAVs to mitigate negative human-wildlife interactions (e.g. keeping wildlife away from crops or predators away from livestock), there are further ethical concerns regarding the effect of UAVs on the animals they are herding and the local ecosystem. First, the artificial noise of drones may alter the natural behaviour of the target animals, causing vigilance (Schroeder et al., 2019) or inducing stress (Scholten et al., 2020;Yaxley et al., 2021), and may have secondary effects for other species locally (Brunton et al., 2019;Weston et al., 2020). Further research is necessary to minimise these effects. Second, strict legislation should be in place regarding which species can be targeted. If deterring a 'pest' species affects its foraging and ability to meet energy demands, there can be undesirable negative effects for local populations, especially if the species is of conservation value or important from an ecosystem services perspective (Scott et al., 2010;Tauler-Ametller et al., 2017;Zaitzove-Raz et al., 2020).
In settings where wildlife and humans coexist, the operation of UAVs can cause conflicts with civilian privacy and raise complaints (Ly & Ly, 2021;Sandbrook, 2015). To counteract this, researchers must engage early and closely with end users and the public so that all parties understand the use and value of UAVs (Sandbrook, 2015).
In addition, in professional settings, such as airports, flying automated UAVs during operational hours can cause conflict with air traffic and security (Shvetsova & Shvetsov, 2021). In civil airports, aerial traffic rules may need to be redesigned to allow for smooth coexistence of aircrafts and drones (Shvetsova & Shvetsov, 2021;Wang et al., 2020). In military airbases, the sensory systems on drones should be well protected to avoid security breaches (Ly & Ly, 2021).
Finally, the perspective we present here focuses on using UAVs as threats and assumes that animal groups respond primarily via visual cues, allowing us to apply the perspective presented across different species and contexts. Of course, this is an oversimplification.
Regarding the stimuli provided by UAVs, researchers should consider a variety of stimuli that could be delivered by UAVs and their effects on the response of the animals. Apart from using UAVs as a threat, there is also potential in using robots that look like conspecifics to lead/guide groups (as opposed to repelling them) or provide attractive stimuli (e.g. food) to move groups around. This approach may be favourable in situations for which ethical concerns mentioned above apply. We encourage researchers and users to consider these options, but in-depth discussion is beyond the scope of this focussed perspective piece.

| CON CLUS IONS
Although there are challenges to the use of UAVs for bio-herding, their potential is huge. We suggest that flying teams of 'surveillance and herding' UAVs, following state-of-the-art herding algorithms informed by empirical data of animal collective behaviour, could manoeuvre large groups of animals, solving a variety of human-wildlife problems that are currently costly, dangerous and even impossible for humans to deal with directly. Our roadmap provides some principles on how to achieve this. We look forward to future, detailed taxa-and species-specific protocols for the development of robust bio-herding systems.

AUTH O R CO NTR I B UTI O N S
Andrew J. King conceived the article and led the writing of the manuscript with Marina Papadopoulou, after extensive discussion with all authors. All authors contributed critically to the drafts and gave final approval for publication. We thank two anonymous Reviewers and the Editors for their input.

CO N FLI C T O F I NTE R E S T
Authors declare no conflicts of interest.

PEER R E V I E W
The peer review history for this article is available at https://publo ns.com/publo n/10.1111/2041-210X.14049.

DATA AVA I L A B I L I T Y S TAT E M E N T
No datasets are used.