Promise and challenges of risk assessment as an approach for preventing the arrival of harmful alien species

Harmful alien species continue to be a major driver of biodiversity change across the globe, as well as causing enormous economic costs and impacts to human health and livelihoods (Keller et al. 2009; Lodge et al. 2006; Pimentel 2011; Shackleton et al. 2007). The problems related to alien and invasive species have intensified as globalisation has produced a range of vectors that intentionally and unintentionally transport live organisms across borders. The diversity of pathways and organisms involved presents a challenge for national policies that aim to reduce the harm from alien species (Essl et al. 2015; Faulkner et al. 2017; Hulme et al. 2008; Lodge et al. 2006).


Introduction
Harmful alien species continue to be a major driver of biodiversity change across the globe, as well as causing enormous economic costs and impacts to human health and livelihoods (Keller et al. 2009;Lodge et al. 2006;Pimentel 2011;Shackleton et al. 2007).The problems related to alien and invasive species have intensified as globalisation has produced a range of vectors that intentionally and unintentionally transport live organisms across borders.The diversity of pathways and organisms involved presents a challenge for national policies that aim to reduce the harm from alien species (Essl et al. 2015;Faulkner et al. 2017;Hulme et al. 2008;Lodge et al. 2006).
Established alien species are usually difficult and costly to manage, and they are rarely eradicated, which means that their costs should be seen as perpetual (van Wilgen et al. 2016).Preventing the arrival of harmful alien species, as opposed to managing them once they become established, is thus seen as a wise approach.However, preventing the arrival of all new alien species is not desirable because many of the trades that move these species provide large benefits to society.At the other extreme, allowing all alien species for import is not without costs because of the harms that arise from the subset that become harmful invaders (Keller & Springborn 2014).A more desirable goal is to prevent the introduction of species that are likely to become harmful while allowing the import of all others.For most taxonomic groups, the proportion of species that become invasive and have negative impact is relatively small (Kumschick et al. 2015a) so that most species could remain available for trade.
these species need to be accurately identified using only information that is known before they are introduced.This paper focuses on RA approaches that are or could be applied to species intentionally introduced through trade when the identity of the species is known.We do not address RA approaches that have been designed for addressing pathways (e.g.ballast water, pests of nursery plants) of unintentional introduction.
Pre-border RA tools ideally have at least five qualities.Firstly, they should be reasonably accurate so that small proportions of species are misclassified.Secondly, they should be transparent so that the rationale for the result is clear.Thirdly, they should be rapid so that decisions are quickly available and trade interruptions are minimised.Fourthly, they should produce consistent results so that different people performing assessments arrive at the same conclusions.Finally, implementation of these RA tools must require a realistic level of resources in terms of finances and skilled practitioners so that the many thousands of species in trade can be assessed.
RA tools are already implemented in some countries, and more countries are actively developing them (Kumschick & Richardson 2013).Australia and New Zealand have each implemented RA programmes for animal and plant introductions for over a decade and these programmes are some of the best developed globally (Keller & Drake 2009).Analyses have shown that national programmes -such as those in Australia and New Zealand -that identify and keep out harmful invaders can reap large economic benefits for the importing nation (e.g.Keller, Lodge & Finnoff 2007b;Keller & Springborn 2014;Springborn et al. 2015;Springborn, Romagosa & Keller 2011).These economic benefits are in addition to the environmental benefits from prevention of further biodiversity impacts and the benefits to human health and livelihood.Importantly, international agreements and standard setting organisations, such as the International Plant Protection Convention and the World Organisation for Animal Health, explicitly allow restrictions on species in trade if those measures are scientifically justified in the form of RA (Keller & Perrings 2011;Perrings et al. 2005).
The goal of this paper is to evaluate five of the most prominent approaches that have emerged for developing pre-border invasive species RA tools.These tools can be applied to species that are intentionally introduced and for which the identity of the species is known.Following the review of approaches, we discuss some general considerations and South Africa's new National Environmental Management: Biodiversity Act (NEM:BA), Alien and Invasive Species Regulations (Department of Environmental Affairs 2014).We emphasise here that this paper reviews approaches to RA rather than actual tools, and we refer readers to other papers for more comprehensive coverage of the tools available (e.g.Hayes & Barry 2008;Keller & Drake 2009;Kolar & Lodge 2001;Kumschick & Richardson 2013;Leung et al. 2012;Pyse ˇk & Richardson 2007).

History of risk assessment for alien species
Herbert Baker (1965) carried out some of the first work to investigate the link between species traits and invasion potential.Recent work has built upon this foundation and expanded the goals to include making explicit predictions about the likely behaviour of species not yet introduced.To do this, researchers have usually dealt with smaller taxonomic units and geographic areas so that traits associated with invasion are more likely to be consistent (Kolar & Lodge 2001).For example, while Baker (1965) sought generalisations across all plant species and continents, more recent RAs have focused on, for example, predicting the likelihood of fish species spreading into the North American Great Lakes because of climate change (Mandrak 1989) and the likely impacts from woody plant invasions in the South African fynbos (Tucker & Richardson 1995).Baker's (1965) approach was to relate the behaviour of a species within its ecosystem to its traits, and modern RA tool development does the same (but see Detailed and Mechanistic approaches below).Different approaches are applied to search for patterns in traits that can explain the observed invasion history, and if strong correlations between traits and invasion history are found, it is assumed that they will be useful predictors of future invasions.These correlations can then be formalised into specific tools for RA.Although this logic is common across much RA tool development, there are important differences in approaches that in turn represent different beliefs about the relationships between traits and invasiveness.

Trait-based risk assessment
Before a trait-based RA tool can be developed, the taxonomic unit and geographical area need to be set, along with the step(s) in the invasion sequence of interest (Blackburn et al. 2011).Limiting each should lead to traits that are more consistently related to passage through the invasion step, and may thus be more useful for prediction (Kolar & Lodge 2001).For example, the traits associated with fishes moving from introduced to established in South Africa are more likely to be consistent than the traits associated with all vertebrates making the same transition across all of Africa.Conversely, increasing geographic area, taxonomic breadth or the number of invasion sequence steps can be beneficial because there will have been more previous introductions from which data can be gathered for RA development.This trade-off is made more complex because RA tools are usually developed for use within political geographical areas (e.g. a nation or region), and the boundaries of these areas are rarely based on ecological factors.Existing legislation may also present constraints, for example, if an agency controls plants imported for agriculture but not plants imported for ornamental purposes.
These complexities and trade-offs ultimately need to be dealt with on a case-by-case basis.We note here that RA tools have been successfully developed at a large range of geographic areas and taxonomic units, and for all steps in the invasion sequence (Kumschick & Richardson 2013).Once the parameters for the RA are set, the process of developing the RA tool can proceed.Different approaches to this are reviewed in the following section.

Trait scoring
The Trait Scoring approach to RA is based on the belief that many traits can make a species more likely to pass through a step in the invasion sequence and, thus, that species possessing more of these traits are most likely to become established and/or cause harm.This is the most commonly applied approach for developing pre-border RA tools, with the resulting tools usually consisting of a list of questions about the presence or absence of traits (Leung et al. 2012).Presence of a trait is scored as a positive number (usually +1) and absence is scored as a 0 or negative number (usually −1).Once all questions have been answered, the scores for each question are summed to a final score, with higher final scores indicating greater likelihood that the species will become established or harmful.Trait Scoring RAs are thus conceptually simple and can be implemented in a basic spreadsheet.Examples of this approach are the Australian Weed Risk Assessment (WRA; Pheloung, Williams & Halloy 1999), which was developed for all plant introductions to Australia; the Fish Invasiveness Scoring Kit, which was developed in the United Kingdom and has now been applied to many regions (Centre for Environment, Fisheries and Aquaculture Science 2013; XX, 2016 [this volume]); and the New Zealand Aquatic Plants Risk Assessment (NZ AqWRA), which has recently been adapted for the United States (Gantz et al. 2015;Gordon et al. 2012).
The first step in developing these tools is for experts to develop a list of traits that they believe are associated with invasiveness.Scores are assigned to the presence/absence of each trait based on their perceived importance, and this produces a RA tool.Next, the tool is validated by collecting data about the traits of species that have previously been introduced to the region, assessing those species and comparing their scores to the known outcomes from these introductions (i.e.introduced vs. established, benign vs. harmful).If necessary, the RA tool can be tuned by modifying the traits used and the scores assigned to each.If, in the final RA, the group of species that successfully passed through the step in the invasion sequence consistently receive higher scores than the group that failed, a score threshold can be set to discriminate between these groups.This threshold can then be used for prediction when species that have not yet been introduced are assessed.
A drawback to the Trait Scoring approach is that it does not consider interactions among traits.It is likely that some traits influence invasiveness depending on the presence of other traits, and thus that while two traits alone may not be predictive, the combination may be a strong predictor.Although the Trait Scoring approach could in theory consider such interactions, in practice it would be cumbersome and we are not aware of a tool that does this.A second drawback is that correlations among traits may lead to double counting (Leung et al. 2012).For example, in the application of the NZ AqWRA to the Laurentian Great Lakes, plants are assessed on their tolerance for a range of habitats, with a maximum score for a species that can live from dry land to fully aquatic.Plants are separately assessed on their tolerance to periodic flooding and drying, with a maximum score for species with high tolerance (Gantz et al. 2015).Species scoring highly on the first trait are also likely to tolerate flooding, meaning that they effectively receive double points.Despite these logical drawbacks, Trait Scoring approaches have been extensively evaluated and found to be acceptable for policy (Keller & Drake 2009;Kumschick & Richardson 2013).
The Australian WRA (Table 1) is the most prominent example of the Trait Scoring approach and is designed to assess the potential that alien plants will become established and harmful.It consists of 49 questions about invasion history, biology, environmental tolerance, ecology and reproduction (Pheloung et al. 1999).These traits were selected by experts, assembled into an RA and then tested by assessing 370 species that had previously been introduced to Australia.This RA tool has been used to make decisions about plant imports to Australia since 1997 and has been adapted for testing and use in many other regions (Gordon et al. 2008;Kumschick & Richardson 2013).

Statistical approach
Many recent developments in RA have come through the Statistical approach.Development of Statistical RAs begins similarly to Trait Scoring with a list of traits that experts believe are associated with invasiveness.Next, the set of species from the taxonomic unit of interest that have previously been introduced to the geographic area of interest is determined, and a matrix is created that includes trait data about each species and the outcome of each introduction.This matrix is analysed with a statistical or machine learning algorithm to find patterns in traits that are correlated with outcomes.Algorithms used include logistic regression, discriminant analysis, categorical and regression trees and neural networks (Keller, Kocev & Džeroski 2011).Resulting models are most commonly validated with leave-one-out cross-validation to determine performance (Keller et al. 2011).
The logic of this approach differs from Trait Scoring in three important ways.Firstly, the Statistical approach holds that  that it is possible for just one or a few traits to explain invasiveness, and Statistical RA tools generally require data on about one to five traits to perform an assessment (Table 2).An RA using logistic regression for established alien molluscs in the Laurentian Great Lakes, for example, found that annual fecundity was sufficient to explain which species become harmful (Keller, Drake & Lodge 2007a).Similarly, a study of environmentally harmful Cactaceae found that the size of a species' native range is a strong predictor of spread and impacts in South Africa (Novoa et al. 2016).
Secondly, this approach holds that interactions among traits may be important and the algorithms used are designed to find such interactions.As discussed above, such interactions are rarely, if ever, included in Trait Scoring RAs and may even be masked by scoring a species separately on these traits.
Thirdly, the Statistical approach holds that the available data should inform the structure of the RA model.In the Trait Scoring approach experts determine which traits should be included and how they should be scored.In Statistical RA the practitioner determines the traits that will be available to the model, but how these traits are incorporated largely depends on the algorithm, which in turn relies on the historical data.This reduces any potential bias on the part of the RA developer and may lead to surprising and non-intuitive outcomes, providing new insight into the invasion process.
The Statistical approach also has a number of drawbacks.Firstly, the models created are often mathematically complex and based on algorithms that are not widely understood.This can reduce acceptance because managers and policymakers may not be prepared to support methods that they do not fully understand.Secondly, the small number of steps of the tools may be problematic because it conflicts with beliefs that invasion is a highly complex process.Again, this may limit acceptance of the resulting models.Finally, lack of data can be a greater issue for these shorter RA tools because missing data about one trait may make it impossible to reach a conclusion.In comparison, most Trait Scoring tools are robust to some level of missing data.Although many Statistical RA tools have been developed and although they have been shown to have high accuracy and produce rapid results (Keller & Drake 2009;Lodge et al. 2016), we are not aware of any jurisdiction that currently implements them.

Rapid screening
The Rapid Screening approach has seen development in the last few years and shows a lot of promise as a stand-alone approach to RA, for the creation of watch lists, and as a way to prioritise species for more detailed RA.This approach is usually based on just two species attributes.The first of these is climate match -the degree to which the climate in the alien range is similar to that in the species' native range.The second is whether the species has a history of causing harm elsewhere in its alien range.If a species has both strong climate match and a history of impacts, it is designated as likely to cause harm in its new range.If it lacks either, it is considered unlikely to cause harm.The logic of this approach comes from the observation that climate match and invasion history are the two attributes most often correlated to the likelihood that an alien species will become established and cause harm (see Hayes & Barry 2008; Table 3).
The Rapid Screening approach is simple, rapid to implement, appealingly intuitive and can generally be applied across all taxa and geographic areas.However, a major drawback is that it will not be useful for species that may become established for the first time because no records are available about their impacts elsewhere (Kumschick et al. 2015b).The United States Fish and Wildlife Service has developed a Rapid Screening RA tool that addresses this concern by treating assessments as 'Uncertain' if the species in question has not had alien established populations in at least one place for at least 10 years or has not been in trade for at least 10 years (Hoff 2014).

Faulkner et al. (2014) recently published a Rapid Screening
RA tool that they applied to 394 alien species in South Africa.These species come from a range of taxa and habitats, and the RA tool showed reasonable performance.Faulkner et al. (2014) argue that the tool could be used for creating watch lists and that these watch lists could guide import decisions or be used to prioritise species for further RA.
They also note that species can be quickly assessed using readily available data, making this approach particularly applicable to jurisdictions lacking the resources to conduct more involved RAs.

Other approaches to risk assessment
Neither of the following two approaches is explicitly based on species traits, although each requires extensive information about the species being assessed.Firstly, the Mechanistic approach is based on the logic that to become harmful an alien species must cross certain barriers to invasion (as outlined by Blackburn et al. 2011) and be transported, introduced, released, become established, spread and cause negative impacts.This approach treats these steps separately and considers, for example, that if a species is highly unlikely to be introduced, then it poses a low overall risk regardless of   4) and could be readily adapted for use elsewhere.This tool requires users to estimate separately the likelihood that a species will pass through each step in the invasion sequence, with the questions addressing such outcomes rather than traits.Thus, the RA is more transferable between taxonomic units and geographic areas, but it relies strongly on users to make difficult judgements about the likelihoods of specific outcomes.The time taken to perform the RA will depend very much on decisions made by the user about how much detail to include and their expertise on the species.
In the Detailed approach, all available details and information about the species, the region into which it may be introduced and the circumstances of its introduction are included.This usually begins with extensive literature review, may include interviews with experts and proceeds from this information to scenarios of likely outcomes from allowing the species for import.It resembles more closely a risk analysis approach as it often also considers how risks can be managed and the potential benefits of a species (e.g.European Food Safety Authority 2012).Again, the time taken to complete a Detailed RA depends on decisions about how complex it should be and what data are relevant.However, we note that this approach often takes years to complete and is thus far more expensive than other approaches.The main benefit of this approach is that it can provide detailed predictions, for example, that a species will have different impacts in different areas.We believe that its use will only be justified for preborder RA when the species in question has the potential for both benefits and adverse impacts that are considered significant.For example, it was used in Canada to assess the risk posed by five species of Asian carps (Mandrak & Cudmore 2004).These species were all in trade, and were all considered potentially very harmful if they became established.

Important qualities of risk assessment tools
An ideal RA tool is transparent, cheap and rapid to implement, accurate and consistent so that different people assessing the same species arrive at the same conclusion.Trade-offs among these qualities will need to be made because, for example, the fastest RA tool may not achieve required levels of accuracy.
It is ultimately a policy question as to how these qualities should be balanced.
As previously mentioned, the most commonly applied approach to invasive species RA has been Scored Questions.
In Australia and New Zealand, tools based on this approach have been in use for well over a decade with little controversy.Species assessments generally take 1-2 days and the tools have accuracies that are almost always > 80% (Lodge et al. 2016).This level of accuracy has been shown to produce economic benefits in addition to the environmental benefits from keeping out harmful alien species (Keller et al. 2007b).Statistical RA tools achieve similar accuracy and can be completed more quickly because they require fewer data (Lodge et al. 2016).A disadvantage of Scored Questions and Detailed approaches is that development of tools for a given taxa and region can take months to years.The Rapid approach to RA is the fastest and can usually be completed in less than an hour per species and possibly even more quickly if many species are being assessed using the same data source.The accuracy of this approach has not yet been rigorously tested, but preliminary results are encouraging (Lodge et al. 2016).In contrast, the Detailed approach usually requires extensive resources and time, and we are not aware of any attempt to determine its accuracy.Indeed, because this approach is usually different in every application, it is difficult to imagine how its accuracy could be assessed.Finally, the Mechanistic approach is an interesting addition to the RA toolbox, and similar to the Rapid and Detailed approaches, it can be used across taxonomic groups.It is too recent for us to assess its accuracy or time taken to apply it, although we believe it would take quite a lot longer than the first three approaches reviewed.
The large differences summarised above, in RA approaches inevitably make it somewhat confusing for new programmes to decide how to proceed, and existing legislation and administrative structures will need to be considered.We make two further observations that can assist with the development and use of RA tools.Firstly, improvements in the availability of trait and invasion history data can be leveraged.For example, FishBase (Froese & Pauly 2015) is a freely accessible online database with information about the biogeography, invasion history, physical traits and environmental tolerances of most fish species.Pantheria (Jones et al. 2009)  Secondly, cost savings may be possible by adopting, with appropriate modifications, RA tools developed for other regions.The Australian WRA has been shown to be effective in several regions around the world (Gordon et al. 2008;Kumschick & Richardson 2013), although work is usually required to calibrate the threshold between harmful and benign species (e.g.Nishida et al. 2009).The Australian WRA has also been modified to apply to several aquatic taxonomic groups (CEFAS 2013) and the resulting tools have been successfully used in a range of regions (e.g.Lawson et al. 2012) including being applied to fishes in South Africa (Marr et al. 2017).Likewise, an RA tool developed in New Zealand for assessing risks from alien aquatic plants is effective in multiple regions (Gantz et al. 2015;Gordon et al. 2012).

Recommendations for pre-border risk assessment in South Africa
Permits to import alien species to South Africa are given by the Department of Environmental Affairs (DEA) and the Department of Agriculture, Forestry and Fisheries.We focus here on the framework for new importations and permit applications recently produced by DEA (DEA 2014).This framework is part of the NEM:BA, Alien and Invasive Species Regulations published in August 2014 and guides RA for individual species that have been proposed for import and which require a permit (i.e.new imports and species listed as Category 2 under the NEM:BA regulations).Under the framework, a pre-border RA must consider the biology, ecology and invasion history of the species, the proposed use of the species in South Africa, characteristics of the environment that the species is likely to encounter, risks of hitch-hiker species or diseases arriving with the species and several other factors including the cost of control should the species escape.This list covers a broad range of the factors that are known to be important predictors of invasion, and many of the ways that harmful alien species can cause impacts.However, the comprehensiveness of the list means that conducting an RA that meets these standards would most resemble the Detailed approach.The disadvantages of such an approach are detailed above and include the extensive resources that would be required to assess a significant proportion of the species that may enter the country.Additionally, the consistency of such an approach may be low because there is no published guidance as to how different factors should be weighed or the extent and type of information required to adequately assess each factor.However, we note that such guidance could be produced in the future.
While the exact ways that the new NEM:BA regulations will manifest in RA for individual alien species are not yet known, for three reasons we believe that the challenges for alien species introduction to South Africa could be better addressed with other approaches.Firstly, South Africa has a great diversity of ecosystems and species, many of which are already severely impacted by harmful alien species and all of which are at risk from future invasions.Secondly, the resources available for pre-border RA are not sufficient to assess a large proportion of introduced alien species with the Detailed approach.This issue is compounded as the number of species in international trade increases, making it reasonable to expect that the number of species proposed for import to South Africa will likewise increase over coming years.Thirdly, the Detailed approach to RA is difficult to defend in terms of accuracy and consistency.The difficulties for assessing accuracy are described above, and the difficulty for consistency arises because the structure of Detailed RA and the data accessed and used will inevitably differ among users.It would thus be possible for different stakeholders to reach different conclusions while each being able to claim that they are using the process outlined in the NEM:BA regulations.
We suggest that a two-tiered RA system could better meet the needs of South Africa to prevent the arrival of harmful alien species while acknowledging resource limitations.
Our suggested system is similar to that suggested by Faulkner et al. (2014).As a first tier, we suggest that species be assessed with a Rapid RA that could be based on tools already developed (e.g.Faulkner et al. 2014;Hoff 2014).The results of these assessments should be publicised online and in other relevant forums and should be initially used to determine which species are allowed and disallowed for import.The second tier would consist of either a Scored Questions or Statistical RA.This would only be used if there were a request for further assessment, which could come from either a person believing that a species banned from import presents low risks or that a species allowed for import poses unacceptably high risks.In either case, the person could petition the DEA to conduct a second tier RA, which would be final.All RA tools and results from assessments should be reviewed by independent experts prior to implementation, but given our suggestion that relatively simple approaches to RA be used this review could be rapid.
Such an approach, if designed to leverage readily available data, could be used to quickly assess a large number of species in trade.The Rapid RA approach is straightforward to conduct, and it is likely that personnel with a graduate degree in biology could perform the assessments.A main challenge to our suggested approach would be the development of second tier RA tools for all taxonomic groups.However, we note that a tool now exists for fishes (Marr et al. 2017) and that it may be possible to calibrate existing tools, such as the Australian WRA, for use in South Africa.These options would greatly shorten the time to having a full suite of Tier 2 tools available and would reduce costs for development.Alternatively, Mechanistic tools like Harmonia + provide a trade-off between the time needed for assessments and the need to develop tools as they do not require separate tools for different taxa.Such Mechanistic tools could be used until others are available, or over longer periods if they are deemed appropriate and adequate resources are available.

Conclusions
Pre-border RA tools have advanced over recent decades and are now often applied to protect nations from the effects of harmful alien species.As well as the environmental case for implementing these tools, there is strong evidence that they protect the economy of the importing nation.Indeed, we are not aware of an economic analysis of an RA tool that has not shown support for its application.Despite the support for RA, there remain challenges to implementation, including deciding which approach will best meet the needs of the importing nation.
Application of pre-border RA in South Africa presents many challenges but could be extremely beneficial.South Africa contains several unique biomes where alien taxa already cause significant impacts (e.g.Richardson & van Wilgen 2004) and pre-border RA could aid in the protection of this exceptional biodiversity.Additionally, developing countries need to implement cost-effective solutions to potential risks posed to their economies (e.g.van Wilgen et al. 2001) and people's livelihoods (Shackleton et al. 2007).Implementing a robust pre-border RA programme would offer the opportunity to prevent the arrival of additional harmful species and thus reduce economic and social risks.A framework for such a programme has recently been suggested under the legal umbrella of NEM:BA, but it does not explicitly leverage recent advances in RA tools.In particular, it appears to require a Detailed assessment of all species and this likely makes it infeasible to assess and appropriately manage the total number of species that pose risks.We have suggested an alternative framework that builds upon recent advances in RA for alien species and that would make it possible to assess many more species in a much shorter amount of time.While our suggested framework is not without challenges, we believe that it could ultimately be a much more realistic and effective way for South Africa to increase its protection from invasive species.

TABLE 1 :
Five representative trait questions (out of 49 total) from the Australian Weed Risk Assessment, an example of the Trait Scoring approach.

TABLE 2 :
Risk assessment for alien fishes in the North American Great Lakes, an example of the Statistical approach using decision trees.Outcomes are in italics.

TABLE 3 :
(D'hondt et al. 2015;apid Screening approach to risk assessment for alien species.Outcomes are in italics.See text for discussion of development of the Rapid Screening approach its potential impacts.The Harmonia + RA was recently developed for use in Belgium(D'hondt et al. 2015; Table Source: