Journal of Threat Assessment and ManagementThe CTAP, a Threat Assessment Tool for the Initial Evaluation of Concerningor Threatening Communications: Development and Inter-Rater ReliabilityDavid V. James, Philip Allen, Andrew Wolfe Murray, Rachel D. MacKenzie, Junyi Yang, Alice De Silva, and Frank R. FarnhamOnline First Publication, January 31, 2022. http://dx.doi.org/10.1037/tam0000173

CITATIONJames, D. V., Allen, P., Wolfe Murray, A., MacKenzie, R. D., Yang, J., De Silva, A., & Farnham, F. R. (2022, January 31). TheCTAP, a Threat Assessment Tool for the Initial Evaluation of Concerning or Threatening Communications: Developmentand Inter-Rater Reliability. Journal of Threat Assessment and Management. Advance online publication.http://dx.doi.org/10.1037/tam0000173

Don't use plagiarized sources. Get Your Custom Essay on
SOLVED : Inter-RaterReliability.pdf
Just from $10/Page
Order Essay

The CTAP, a Threat Assessment Tool for the Initial Evaluationof Concerning or Threatening Communications:

Development and Inter-Rater Reliability

David V. James, Philip Allen, Andrew Wolfe Murray, Rachel D. MacKenzie,Junyi Yang, Alice De Silva, and Frank R. FarnhamTheseus Risk Management Ltd., Chippenham, United Kingdom

Best practice in threat assessment requires the use of standardised, evidence-based toolsto assist in case formulation. Considerable numbers of such tools have been developed foruse in risk assessment in criminal justice, mental health and community settings:instruments have been published for the threat assessment of cases in specific areas,such as workplace violence, school violence, stalking, and terrorism. However, there hasbeen less attention to a more general and common problem faced regularly by politicians,public companies, those in the talent industry, and others in the public eye. That is howthe recipients of problematic or threatening communications can judge which of theseneed referral to security departments or police for further investigation; and how agenciesreceiving such referrals can reach initial conclusions as to which cases require prioritisa-tion and which do not. Relevant risks in this area concern not simply violence, the focus ofmost threat assessment instruments, but also persistence, escalation, disruption offunction, reputational damage, financial loss, and psychological harm. The Communica-tions Threat Assessment Protocol (CTAP) is a structured professional judgement tool forthe initial threat assessment of unwanted communications, designed to fill this gap. Thisarticle details the background to the CTAP and its development into a manualised threatassessment instrument for use both in the private sphere and in policing contexts. It reportsthe first evaluation of the reliability of the CTAP. The results of this study are promising interms of the use of CTAP in the initial assessment of problematic communications.

Public Significance StatementWhen people in public life receive bizarre or threatening communications, some formof system is needed to help them divide these into ones to bin and ones that needfurther examination by security departments or police. Further, such departments needa means to make an initial threat assessment in order to triage such referrals—and onethat can be done quickly in response to limited information. The CommunicationsThreat Assessment Protocol provides such a system, the background to which andreliability of which is illustrated in this paper.

Keywords: threat assessment, threatening communications, screening tools, structuredprofessional judgement, policing

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

David V. James https://orcid.org/0000-0002-5046-8045Philip Allen https://orcid.org/0000-0003-3912-8071Junyi Yang https://orcid.org/0000-0003-2322-3525The group that designed and conducted this study

included the authors of the CTAP (David V. James, Rachel

D. MacKenzie, Frank R. Farnham). Two of the group (PhilipAllen and Andrew Wolfe Murray) are directors of Theseus,which produces the CTAP manual.Correspondence concerning this article should be

addressed to David V. James, Theseus Risk ManagementLtd., 6 Prince Maurice House, Chippenham SN14 6LH,United Kingdom. Email: david.james5@ntlworld.com

1

Journal of Threat Assessment and Management© 2022 American Psychological AssociationISSN: 2169-4842 https://doi.org/10.1037/tam0000173

In the field of threat assessment, the use ofstandardised tools by multidisciplinary teams de-fines best practice (Meloy et al.,2021,p. 16).Suchtools provide a systematic, structured means ofassessing risk, and comprise an aide-mémoirewhich assists the assessors in making sure thatall relevant factors are considered in their evalua-tion (Skeem & Monahan, 2011), whilst allowingfor the consideration of idiosyncratic factors in agiven case. A considerable number of risk assess-ment instruments is available for examining risksin different spheres. Indeed, Singh et al. (2014)identified at least 400 structured risk assessmentmeasures for violence alone that are used byprofessionals working in health, criminal jus-tice, and the general community. Specialist vo-lumes detail the role of risk assessment tools in avariety of settings, including the workplace, thepublic figure context, schools, stalking, and terror-ism, with a particular emphasis on risk of violence(e.g., Douglas & Otto, 2020; Meloy & Hoffmann,2021; Wormith et al., 2020).In the design of these tools, a differentiation

can be made between risk assessment and threatassessment. Risk assessment generally involvesconsideration of a case in a review setting, wherethere is little time-pressure and a considerableamount of information is available about the case.Threat assessment, by contrast, concerns themaking of decisions quickly, based on limitedinformation, in order to determine the level ofimmediate response. As such, threat assessmenttools incorporate a screening role. Examples ofrisk assessment tools in the area of harassment,intrusive behaviours and the making of threatswould include the Guidelines for Stalking Assess-ment and Management (SAM; Kropp et al., 2008)and the Stalking Risk Profile (SRP; MacKenzie etal., 2009), whereas an example of a threat assess-ment tool would be the Screening Assessment forStalking and Harassment (SASH; James &Sheridan, 2020; McEwan et al., 2017).The risk factors incorporated into these tools are

generally data items which have been shown to bestatistically associated with the outcome in questionin previous research studies. Rather than individualprediction, which is not possible, such instrumentsconcern the allocation, based on group data, of anindividual into a general risk band according to thenumber and/or nature of the risk factors which areshowntobepresentintheircase.Inriskassessment,there tend to be three such groups, high, mediumand low (Meloy et al., 2012). Some risk assessment

instruments incorporate “red flag” indicators, thepresence of one of which is sufficient to determinethat an individual should be considered high risk.With threat assessment instruments, by contrast,what is being assessed is not a comprehensivejudgement as to risk, but a preliminary determina-tion as to the level of concern (and thereforepriority for further assessment or intervention)which should be accorded to the individual underassessment (Scalora, Baumgartner, Zimmerman,et al., 2002). Risk assessment instruments, andsome threat assessment tools, generally enableconclusions to be drawn, not only as to the levelof problem, but as to how it can be managed orreduced.Whatever the type of tool, it is necessary to

ensure that its results are both reliable and valid—in other words, that it produces consistent andreproducible results between investigators andover time, and that it measures what it is supposedto measure. This involves exercises in the mea-surement of inter-rater and test–retest reliability,and of predictive validity. Also relevant is theconcept of construct validity in terms of whetherthe content of the instrument items completelyrepresents the construct, and whether scores cor-relate with those from other instrumentsassessingthe same construct (Cook & Beckman, 2006).Eventually, it is relevant to investigate the utilityof tools, in other words how useful and acceptablethey are found in practice (Hart & Logan, 2011).New tools are usually based upon a systematicreview of the relevant scientific and professionalliteratures, and refined through various forms ofpractical evaluation, before being subject to for-mal reliability and validity testing. The presentstudy concerns the development and reliability ofthe Communications Threat Assessment Protocol(CTAP; James, MacKenzie, et al., 2014), a threatassessment tool for the initial evaluation of con-cerning or threatening communications.

Threatening and ConcerningCommunications

Those in the public eye, or whose jobs arepublic-facing, receive letters and electronic com-munications from members of the general publicbecause of who they are, what they do, or whothey work for. This applies amongst others topoliticians, public officials, the CEOs and em-ployees of major companies, sports and media

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

2 JAMES ET AL.

personalities, and creative artists. The majority ofsuch communications concern legitimate enquiry,reasonable complaint, requests for help, or fanmail. However, a significant minority will bestrange, bizarre, angry or inappropriate in content.In a proportion, the writer may seem mentallyunwell. Some cases will constitute persistent orquerulent complaining, some amount to cam-paigns of harassment, and a few will be inappro-priately amorous. Sometimes, communicationswill be received which contain direct or indirectthreats. These may concern reputational or finan-cial damage, persistent unwanted intrusion, ormore rarely, physical violence. Even where nosuch threats are included, the communicationsmay be psychologically upsetting or distressingtothose receiving them, they may cause disruptionof function, and valuable time and resources maybe consumed in dealing with them.The majority of such inappropriate, aggressive

or threatening communications are harmless,but some are not. The problem is how to separatethe two. The dangers, should this not be doneeffectively, are either that warnings of futureharm are missed, with potential adverse con-sequences, or alternatively that costly investi-gative and security resources are spent on caseswhere they are not required. In the operationalsituation, there are two levels at which practicalguidance is needed as to which cases warrantfurther attention and which can safely beignored. The first concerns those who open orreceive correspondence, whether that be indi-vidual addressees, their correspondence staff, orpublic-facing departments such as customerrelations or complaints handlers. Here, a briefand simple screening process or filter is neces-sary for deciding which cases to refer on tosecurity departments or risk managers. Thesecond level concerns guidance as to the initialthreat assessment and management of cases bythe risk managers, security staff or policingteams to which such cases are referred. Itspurpose is to avoid missing risk, to preventunnecessary distress from over-estimating risk,to avoid unnecessary expense, and to point towardsa suitable investigative response. Rather than anuanced risk assessment involving detailedinformation and an absence of time pressure,it concerns the making of quick decisions inresponse to limited information in a dynamic,real-time setting. The CTAP was evolved to fulfilthese functions.

The Knowledge Base

Literature

There is a considerable background body ofpublished research to assist in the construction ofsuch tools. This has been expertly summarisedelsewhere (Meloy, 2014; Meloy et al., 2004, 2011,2021), and the account here will be limited to thebroad brush. Formal study of the attacks on publicfigures began in the nineteenth century with thework of Régis in France and Laschi and Lombrosoin Italy (Farnham & Busch, 2016; James, 2015;Laschi & Lombroso, 1886; Régis, 1890). Modernresearch into this area was greatly advanced byDietz and Martell’s (1989) report “MentallyDisordered Offenders in Pursuit of Celebritiesand Politicians,” an unpublished work of 619pages plus appendices in which was buried atreasure-trove of information about the character-istics of concerning correspondence and asso-ciations withsubsequent approach. Unfortunately,this was robbed of context in the two associatedpublications (Dietz, Matthews, Martell, et al.,1991; Dietz, Matthews, Van Duyne, et al., 1991),as mental illness was excluded from the accountfor“securityreasons”(Dietz&Martell,2010).TheU.S. Secret Service’s Exceptional Case StudyProject on lethal and near-lethal attacks, whichhad considerable influence, examined an historicalcohort of 83 cases and found that 61% of indivi-duals had a history of psychiatric problems, 43%of delusional ideas, and 10% of violent com-mand hallucinations (Fein & Vossekuil, 1998):this compares with a point prevalence of psychoticillness in the general community of around 0.4%(Kirkbride et al., 2012). A further historical exer-cise looking at judicial figures was contributed byCalhoun (1998). Scalora and colleagues then pro-duced a series of studies of problematic contactstowards members of the U.S. Congress and factorsassociated with subsequent approach (Marquez &Scalora, 2011; Scalora, Baumgartner, Hatch-Maillette, et al., 2002; Scalora, Baumgartner,Zimmerman, et al., 2002; Scalora et al., 2003;Schoeneman et al., 2011; Schoeneman-Morriset al., 2007), which form the foundation of recentwork in this area and aided subsequent research inthe United Kingdom. As regards sectors otherthan politicians, Meloy and colleagues advancedknowledge of problematic communications tocelebrities (Meloy, Mohandie, et al., 2008), andthere are the beginnings of work on inappropriate

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 3

approaches to corporate figures (Hoffman &Sheridan, 2008; Meloy, Amman, et al., 2021).In parallel, conceptual work has taken place onthe subjects of warning behaviours (Meloy et al.,2012) and leakage (Meloy & O’Toole, 2011).Inthe U.K., the Fixated Research Group (FRG)

produced a 300-page report in 2006 and publishednine research papers in peer-reviewed journals(James, 2010; James et al., 2007, 2008, 2009;James, McEwan, et al., 2010; James, Meloy,et al., 2010; James, Mullen, et al., 2010; Meloyet al., 2004; Mullen et al., 2009). These papersconcerned concerning communications to theBritish royal family and to politicians, the role ofmentalillness,andfactorsthatpredictedpersistence,approach and attack. The relation of the FRG’sfindings to the U.S. literature is set out by Mullenet al. (2008). It is from the work of the FRG, both intermsofitsinvestigationsandresearchpublications,that the CTAP was initially developed.

Threat Assessment Services

A number of threat assessment services in thepublic and private sectors over the last 25 yearshas published broad detail of their threat assess-ment approach (for instance, Cawood & Corcaran,2003; Calhoun & Weston, 2003, 2009; de Becker,1996;Fein&Vossekuil,2000;McElhaney,2004).Accounts are available of the work of the LosAngeles Police Threat Assessment Unit (Bixleret al., 2021; Dunn, 2014); the U.S. Secret Service(Fein & Vossekuil, 1998); the U.S. Capitol PoliceThreat Assessment Section (Scalora et al., 2008);and a range of other threat assessment services andapproaches (Meloy & Hoffmann, 2014, 2021;Meloy, Sheridan, et al., 2008a, 2008b). Perhapsthe most has been written in recent years aboutthe Fixated Threat Assessment Centres(FTACs), which developed out of the work ofthe Fixated Research Group. Based on its re-commendations, the U.K.’s Office of Security andCounter-Terrorism (OSCT) and the Departmentof Health financed the establishment of a unit toassess and manage the risk to politicians and theroyal family from individuals who write con-cerning letters or make concerning approaches.This, the U.K. FTAC, is a police unit, within theMetropolitan Police Service’s protection com-mand, but one which is jointly staffed by policeofficers and forensic psychiatrists, psychologistsandnursesfromtheU.K.’sNationalHealthService(NHS).The rationale and operating practices of the

FTACs have been extensively described (Barry-Walshet al., 2020; James et al., 2014; Wilson et al.,2018, 2019, 2021). It was within this unit that thefoundations of the CTAP were developed.TheFTACmodelwassubsequentlyreplicatedin

the Netherlands (Nelen et al., 2012, 2013; Sizoo &vanNobelen,2021),Queensland(Pathéetal.,2015,2017; Pathé, Haworth, et al., 2016; Pathé, Lowry,et al., 2016), in the Australian state of Victoria,in Western Australia, in New South Wales, inCanberra (Riddle et al., 2019), and in Wellington,New Zealand. The extent of the problem of threatsand disturbing correspondence to members ofparliament from lone individuals has meanwhilebeen established in a number of jurisdictions(Akhtar & Morrison, 2019; Bjelland & Bjørgo,2014; Brottsförebygganderådet(CrimePreventionCouncil), 2012; De Groot et al., 2010; Every-Palmer et al., 2015; Hamers et al., 2010; James,Farnham, et al., 2016; James, Sukhwal, et al., 2016;Muller et al., 2010; Narud & Dahl, 2015; Nijdamet al., 2008; Wallin & Wallin, 2014). Outcomestudies from the U.K. FTAC have been published(James & Farnham, 2016; James, Kerrigan, et al.,2010),andfurtherresearch onriskfactorshas beenproduced from the FTAC database (Clemmowet al., 2021; Gill et al., 2021). Other researchinvolving the FTACs has demonstrated the closesimilarities between non-ex-intimate stalkers inthe general population and those who harass orpursue public figures (James et al., 2014; James,McEwan, et al., 2010; McEwan et al., 2012; Pathé,2017). This is important in that it enables researchon risk factors in stalking to be translated acrossinto the field of risk assessment in public figures.

Assessment Tools

Individual threat assessment units and depart-ments will have their own internal data collectionand assessment procedures, though few will besubject to research scrutiny. However, a numberof structured assessment approaches have beendeveloped and published in recent years to aid inthe assessment and management of risk in situa-tions where inappropriate communications andapproaches occur. For instance, the aforemen-tioned SAM, SRP and SASH concern stalking:the Workplace Assessment of Violence Risk-21(WAVR-21; White & Meloy, 2016) and theCawood Assessment Grid (CAG; Cawoodet al., 2020) concern workplace violence; theTerrorist Radicalization Assessment Protocol-18

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

4 JAMES ET AL.

(TRAP-18) concerns lone terrorist activity(Meloy, 2016). Exercises have been undertakento establish their reliability and validity (Cawoodet al., 2020; Meloy, 2018; Meloy et al., 2013).With the notable exception of the stalking instru-ments, most emphasis in this area is focussed ontheriskofviolenceandseriousphysicalharm.Thisis understandable in terms of their development inenvironments where firearms can be acquired bymembers of the public. However, it leaves some-thing of a gap. As described above, those in thepublic eye are in need of some general means ofdifferentiating between concerning communica-tions which they can safely ignore, and thosewhich they need to take seriously and pass on totheirsecuritydepartments.Securitydepartmentsinmajor companies and other organisations need ameans of threat (rather than risk) assessment toaid in their initial prioritisation of responses toworrying contacts, a form of “screening in” and“screeningout”(Shafferetal.,2020).Andwhereasviolence is evidently important, it is rare and otherforms of risk (e.g., persistence, escalation, dis-ruption of function, reputational damage, financialloss, and psychological harm) are often of moreimmediate day-to-day concern. Attending to theseneedsistherolethattheCTAPwasdesignedtofill.

Nature of the CTAP

The CTAP comprises two parts: a preliminaryQuick Correspondence Screen (QCS), and themain CTAP itself. The QCS is designed to becompleted by those to whom communications areaddressed or their staff. It comprises 18 itemsconcerning the content of the communication. Ifany item on the list is present, the letter shouldbe referred to the security or threat assessmentdepartment for more detailed examination, usingthe CTAP. If none are present, the communicationneedsnofurtheraction.TheCTAPitselfisdesignedto be used by security managers, threat assessors orpolice who have undergone training in its use. Itcontains 25 items, each of which is associatedwith increased risk. The CTAP is contained in a55-paged manual which includes: an introduction,instructions for use, descriptions of the 25 CTAPitems, instructions for allocation of concern level,frameworkforinitialmanagementplans,andguide-linesforcaseswhichdonotproveamenabletorapidresolution.The nature of each CTAP item is described in

turn on a separate page, with illustrations where

necessary, and the specification for each possiblecoding set out. For most items, there are twopossible codings—yes or no. For others, a third“?” coding is defined. Guidance is given as to howto allocate cases into low, medium or high concerncategories. The CTAP is not totalled into a simple“score. ” Whilst a higher number of positives islikely to be associated with a higher level ofconcern, the items are not equal in value, nor inlikelihood of occurrence. Some of the factors arerarely encountered: however, their significancewhen present is such that it is important that theyare always considered. Five of the factors are“red flag” items associated with a high risk ofviolence, such that the presence of one item aloneis sufficient to render the case of high concern untilshown otherwise. Use of the CTAP requires priorattendance at an approved, standardised trainingsession, in person or on-line, lasting 4 hr andincorporating discussion about the practical appli-cation of the CTAP that may be peculiar to thedelegates’ organisation, most usually concerningissues of business structure, reporting processes,and company policy regarding communicationwith the general public. The CTAP manual hasbeen translated into Dutch and Norwegian.The CTAP has been in use over the last 7 years

by major international corporations, financialinstitutions and non-governmental organisations,family offices and the management teams of arange of sporting, acting, presenting and musicaltalent, security departments of elected assemblies,by policing departments, including the NetherlandsNational Police, and by various public agencies andcharities.

Aims of the Study

ThisarticlesetsoutthebackgroundtotheCTAPand its development, before presenting the find-ings of inter-rater and test–retest reliability studies,and considering ways of evaluating its ability effec-tively to accord casesan appropriatelevelofpriority.It is the first part of the evaluation project, the secondpart of which will look at the validity of the CTAP.

Development of the CTAP and QCS

This section describes the identification of theproblem with which the CTAP was designed todeal, the sources used in its construction, its evo-lution, the factors determining its format, and itsadaptation to the non-governmental sector.

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 5

The Nature of the Problem

The ur-version of what was subsequently devel-oped into the CTAP was a product of the U.K.FTAC. The first problems faced by the develop-ment team in establishing FTAC were twofold.How should referrals to the new unit (mainlyconcerning correspondence, but also people tryingto gain access to Royal and government buildings)be selected (or limited)? And how could a stan-dardised system be set up for deciding which caseswarranted further assessment and determining thepriority that they were afforded within limitedresources? The FRG had already identified thestark problems within the existing systems. Theabsence of a suitable system for filtering casesfor referral had seriously hampered attempts onthe part of Royalty Protection police to assessinappropriate and threatening communications tomembersoftheroyalfamily.Theirhouseholdshadbeen forwarding all letters that seemed odd orconcerning, these amounting to some 11,000 ayear. The few officers who were attempting todeal with them were overwhelmed, and the back-log had stretched to 6-months, by which time therewas littlethatcouldbedone withmostofthe lettersother than logging them. By contrast, the cor-respondence office at 10 Downing Street waswell-staffed and efficient, and correspondencehandlers were adept at identifying at least a pro-portion of thosecommunicationswhich weremostconcerning. However, the system fell down in thewaythesewerehandled.Themostworryingletterswere given to the Prime Minister’s personal pro-tection officers who, it transpired upon investiga-tion, placed them in a cardboard box under a desk.Such “out of sight, out of mind” approachesproved to be a relatively common phenomenon,not just in government departments, but in privatecompanies, where worrying letters were oftensimply placed in the “mad drawer” without anyfurther action. Given the low base-rate of vio-lence, inaction in the great majority of suchcases will be without consequence. However,the consequences of inaction can be so severethat systems need to be in place to assess risk,even where the chances of an adverse outcomeoccurring are low. The task, then, was to design ascreening or filter instrument for use by correspon-dence offices, and to create a risk assessmentprotocol for the initial assessment of concerningcommunications in the new FTAC.

Examining Best Practice Elsewhere

The development team visited relevant organi-sations in Europe and the U.S. engaged in similarforms of threat assessment in order to study theirprocesses and schedules. Some proved more rele-vant to FTAC’s purposes than others. The U.S.Secret Service placed such value on their mainprotectee that their methods appeared over-inclu-sive. The methods of the Behavioral Analysis Unitin Virginia were focussed on their various func-tions in assisting U.S. law enforcement agencies ininvestigations of specific crimes and were over-concentrated for FTAC purposes on the mainte-nance of specialised databases and the use ofrelatively complicated methods, such as textualanalysis and the employment of algorithms. Theexperience of the Swedish Security Police inStockholm was directly relevant in that thecountry possessed both a parliament and a con-stitutional monarchy, and there had been devel-opments in threat assessment procedures in theaftermath of the assassination of Olaf Palme,including a collaboration in case assessmentbetween the Security Police and forensic psy-chologists and psychiatrists from the KarolinskaInstitute. A similar, but less developed, servicewas operating in the Norwegian Police SecurityService. However, the most useful source ofprocedural material was the Capitol Police ThreatAssessment Section in Washington D.C. (Scaloraetal.,2008),whereMarioScaloraoftheUniversityof Nebraska had devised and supervised a sched-ule with a prominent psychological componentto evaluate threatening and harassing contactstowards members of Congress (Marquez &Scalora, 2011; Scalora, Baumgartner, Hatch-Maillette, et al., 2002; Scalora, Baumgartner,Zimmerman, et al., 2002; Scalora et al., 2003,2008; Schoeneman et al., 2011). The trove ofmaterial collected on these trips comprised a seriesof protocols, lists of risk factors, risk grids, andof less practical relevance, lists of key words toidentify in correspondence (including multiplesynonyms for kill, bomb, shoot, and engagementin procreational activity).

Examining the Literature

Extensive searching of the literature on riskassessment, threat assessment, violence, violenceand mental illness, and stalking and harassment

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

6 JAMES ET AL.

was conducted. Opinions and advice were alsosought directly from experts in the field. Theresearch publications by the FRG referred toabove were directly relevant to the identificationof statistical associations with adverse outcomes,these associations ineffect constituting riskfactors.An example is that of “end-of-tether” languagewhich, since its inclusion in the FRG publications,has been recognised as an important form ofwarning behaviour (Meloy et al., 2012) and beenadopted asan indicator for violence risk, notjust intheCTAP,butinotherriskassessmentinstrumentssuchas the Stalking RiskProfile(MacKenzie et al.,2009) and the TRAP-18 (Meloy, 2016; Meloy &Gill, 2016). Other than statistical analyses, his-torical case studies were also informative in theconsideration of relevant warning behaviours, par-ticularly where the phenomena in question wererare. An illustration is the factor which eventuallybecame item 23 of the CTAP, “Delusions involv-ing horrific or frightening beliefs, which threatenthe individual’s or others’ existence or physicalintegrity.” Two of a series of examples were Adel-heid Streidel’s stabbing of politician, OscarLafontaine, to bring attention to the abduction ofpeople to underground killing factories where newbeings were constructed from their body parts, theremaining flesh being sold as sausages in butchers’shops across Germany: and Dieter Kaufmann’sshootingoftheGermaninteriorminister,WolfgangSchäuble, to try and stop the beaming into hisbody by state transmitters of alternating pain andintense homosexual lust. Both had taken mattersinto their own hands when their multiple com-plaints to the authorities had come to nothing(James et al., 2007).Table 1 sets out some links to supporting

literature for each of the 25 CTAP items, togetherwith references to other risk assessment instru-ments in which similar items have been incor-porated. This list is intended as illustrative, notexhaustive. It should be noted that the QCS itemsare simplified versions of CTAP items. As such,the table is also relevant to the locating of the QCSitems within the literature.

Parallel Development With Other RiskAssessment Instruments

Two other assessment instruments were devel-oped in parallel with the CTAP, the Stalking RiskProfile and the SASH. The three CTAP authors

include the lead author and a co-author of the SRP,and two of the four authors of the SASH. Thework on the Stalking Risk Profile contributed threeimportant ideas which assisted in the constructionof the CTAP: that it is important to considerdomains of risk other than violence, specificallythe risks of persistence, escalation, disruptionof function, reputational damage, financial loss,and psychological harm to those upon whomthe unwanted behaviours are focussed; that riskfactors will differ according to the domain of risk;and that risk in each domain is likely to varyaccording to underlying motivation. The SRPalso provided a model in terms of the descriptionand scoring of risk items and their manualisation,and it contains an addendum on the structuredassessment of risk to public figures. The SRP andthe SASH, like the CTAP, incorporate the idea of“red flag” items, the presence of any one of whichautomatically means that the case is rated as beingof high concern. There is an overlap in risk itemsincluded in the three instruments, and the CTAPincorporates four of the five “red-flag” items forviolence contained in the SRP. The SASH usesthe low, moderate and high concern levels adoptedby the CTAP, rather than risk levels. The SASHmanual follows the same format as that of theCTAP.Reliabilityandpredictivevaliditystudiesofboth the SRP and the SASH have been published(Hehemann et al., 2017; McEwan et al., 2018).

Evolution of the Fixated Threat AssessmentCentres Correspondence Screen and InitialThreat Screen

Correspondence Screen

Initial experience at FTAC of dealing withcorrespondence offices demonstrated the futilityof trying to impart an effective understanding ofthreat assessment to their staff, or of relying onindividuals’ experience and personal judgement.The rapid turn-over of employees in these rolesmeant that it was unrealistic to expect them toengage in any evaluation process, and instead itwas necessary to have a simple screen for them tofollow which covered no more than one side ofpaper. The items had to be written in a way thatwas easy to comprehend. In order to flag up a casefor referral, the presence of any one characteristicdescribed—in other words, the scoring of a singleitem as present—had to be the determining factor,so that no further judgement was needed. FTAC

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 7

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

Tab

le1

CTAPItem

sandLinks

totheLiterature

CTAPitem

Somelinksto

theliterature

1.Threats

Threatsin

stalkingareassociated

withviolence,particularlyin

ex-intimates

(e.g,.Churcher&

Nesca,2013).Threatsto

publicfigureshaveyet

tobedisaggregated

accordingto

motivation;in

undifferentiated

samples,threatsare

negativelyassociated

withviolence.Seriousviolence

iscommonly

preceded

bythreats,even

ifmostthreatsarenot

followed

byviolence.Threatsshould

alwaysbetaken

seriouslyin

publicfigure

risk

assessment(M

eloyet

al.,2011).

Threatsarearisk

factorin

theSAM

(Kropp

etal.,2008)

andtheSASH

(McE

wan,Strand,et

al.,2017),as

aredirect

threatsin

theTRAP-18(M

eloy,

2016).

2.Declarationofintention

Declarationofintentto

approachisasignificantassociationwithapproachin

publicfigurecases(Schoenem

anetal.,2011),

andastrongassociationofapproachin

theFTACdata(G

illet

al.,2021).Itisassociated

withapproachcasesin

Dietz

andMartell’s

(1989)work

onpoliticians,as

isstatingaparticulartimeorplace

when

somethingwould

occur.

Itisarisk

factorfordisruptionin

thepublicfigure

sectionoftheSRP(M

acKenzieet

al.,2009).

3.Evidence

ofdisplacement

Mention

oftravel

isassociated

withapproachto

politicians(D

ietz

&Martell,1989),as

areknownprevioustrips

tothevicinityofthepolitician.Seekingproximityisarisk

factorin

theSASH

(McE

wan,Strand,et

al.,2017)

4.Extrem

esofanger

Anger

isadriver

ofviolentoffendingin

general

(Novaco,2011,2013),arisk

factorforviolence

inpsychoticpatients

(Monahan

etal.,2001,

p.58),andamediatorbetweenvariousform

sofdelusionandviolence

(Coid

etal.,2013).

Elevated

anger

isarisk

factorin

theSRP(M

acKenzieet

al.,2009)andin

theSAM

(Kropp

etal.,2008).Moral

outrageisarisk

factorin

theTRAP-18(M

eloy,

2016).

5.Escalationin

anger

orincreasingpreoccupation

Escalationhas

beenfoundto

bean

associationofapproachin

celebrities

(Meloy,

Mohandie,et

al.,2008,p.41).

Anescalation

ofbehaviourisarisk

factorintheSAM

(Kroppetal.,2008)andintheSASH(M

cEwan,S

trand,etal.,2017).

6.Highly

personal

questforjustice

Themajor

role

of“fixation”

inthisfieldhas

beendescribed

(Mullen

etal.,2009)together

withitsoverlap

withpersistentcomplainingandwithquerulance,whichisassociated

withescalationandsometim

esviolent

outcomes

(Mullen

&Lester,2006).Reference

toinjusticeorviolatedrightsareastrongindicationofapproach

inpublicofficial

cases(Schoenem

anet

al.,2011).Those

pursuingan

agendain

theroyal

familystudieswere

more

likelysuccessfullyto

breachsecurity

barriers,to

achieveproxim

ityto

theirtarget,to

carryaweapon

andto

engagein

intimidatingbehaviour(Fixated

ResearchGroup,2006).

7.Dem

andsto

change

behaviour

Theim

portance

ofdem

andsto

change

behaviour

isnotonlythat

they

areinappropriateandintrusivein

crossinga

boundarybetweenpublicandpersonal

roles,but

that

they

show

evidence

ofasense

ofentitlem

entwhichim

plies

alack

ofconstraintbysocial

norm

s.Such

dem

andsareoften

infusedwithanger.Theassociated

sense

ofentitlem

ent

isarisk

factorforpersistence

intheSRP(M

acKenzieet

al.,2009).

8.Dem

andformoney/apology

Themakingofdem

andsandclaimsofentitlem

entareassociated

withapproachinpolitician

cases(Schoenem

anetal.,2011),

asarehelp-seekingrequests.Multiple

approaches

aresignificantly

more

likelythan

single

approaches

inthose

usingdem

andlanguagein

royal

familycases(Fixated

ResearchGroup,2006).Dem

andsforspecificoutcomes

whichareunlikely

tobemet

aremore

likelyto

engender

anger

andbeassociated

withpersistence

andescalation.

Failuresto

satisfycomplaintsarearisk

factorin

theSRP(M

acKenzieet

al.,2009).

9.Prolificcorrespondence

Itisaconsistentfindingin

thisfieldthat

intensity

ofcontact

isassociated

withapproach(M

eloy

etal.,2011).

Thiscanbemanifestin

volumeofcommunications,number

ofpeoplecontacted,andnumber

ofdifferentmethods

ofcontact

used.

Multiple

communicationsareassociated

withpersistentapproachto

mem

bersoftheRoyal

Fam

ily

(Jam

eset

al.,2009).Multiple

communicationsarearisk

factorin

thepublicfiguresectionoftheSRP

(MacKenzieet

al.,2009).

8 JAMES ET AL.

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

Tab

le1(continued)

CTAPitem

Somelinksto

theliterature

10.Awarenessofpersonal

details

Knowledgeofprivatedetailsnotin

thepublicdomainrepresentsadegreeofdeterminationandactiveintrusion.

Awarenessofvictim

location

isaprerequisiteto

approach.Itisarisk

factorin

theSRP(M

acKenzieet

al.,2009).

11.History

ofintrusive

behaviours

Previous

stalkinghas

beenlinked

toincreasedlikelihoodoffuture

stalking.A

history

ofintrusivebehaviours

isarisk

factorin

theSRP(M

acKenzieet

al.,2009)

andin

theSASH

(McE

wan,Strand,et

al.,2017).

12.End-of-tether

language

Theim

portance

oflastresortthinkingwas

recognised

incase

materialfrom

theFixated

ResearchGroup(2006),

andwas

adoptedas

ared-flag

risk

factorin

theSRP(M

acKenzieet

al.,2009)andin

theSASH

(McE

wan,

Strand,et

al.,2017).Itisawarningbehaviourin

theclassificationofMeloy

etal.(2012),andincorporatedinto

the

TRAP-18(M

eloy,2016).

13.Suicidal

ideation

Stalkersareatincreasedrisk

ofsuicide,anduptoathirdreportsuicidalideation

(McE

wan

etal.,2010;Mohandieetal.,2006).

Cases

ofattempted

orcompletedhomicide-suicidehavebeenreportedin

celebrity

cases(Schlesinger,2006)and

widelyin

thepress,as

wellas

beingarecognised

phenomenon

inex-intimatestalkingcases.Suicidal

ideation

may

actas

adisinhibitingfactor,in

that

those

notexpectingto

liveareunder

fewer

constraints.Itcanalso

be

associated

withideasofdem

onstratingvictim

status

andinducingregretin

thosewhoareperceived

ashaving

harmed

one.

Suicidal

ideationisared-flag

indicatorin

theSRP(M

acKenzieet

al.,2009).

14.Sexually

aggressivelanguageorfantasies

Sexuallyinappropriatelanguageisnotassociated

withmoreintrusive

behavioursorapproach,butitcauses

particulardistress

whichrequires

intervention.More

aggressivefantasies

arearisk

factorforassaultin

predatory

stalkercasesin

theSRP(M

acKenzieet

al.,2009).

15.Interestin

attackersorviolentextrem

ism

Interestin

attackersandviolentextrem

ism

isrecognised

asarisk

factorin

loneactorgrievance.Itisawarningbehaviourin

theclassificationofMeloy

etal.(2012),andarisk

item

intheTRAP-18(M

eloy,

2016).

16.Referencesto

weapons

Therelevance

ofweaponswas

noted

byMeloy,

Sheridan,et

al.(2008b,

p.25).Possession

offirearmsisassociated

withapproach(Schoenem

anet

al.,2011).A

history

ofweaponpossessionwas

significantly

associated

with

attemptingto

beach

security

barriersam

ongstFTACapproachersandtherewas

asignificant

associationbetween

approachandhistory

ofconcealingweapons(G

illet

al.,2021).Accessto

oraffinitywithweaponsisarisk

factorin

theSRP(M

acKenzieet

al.,2009).

17.Knownhistory

ofviolence

Previous

violence

isapredictoroffuture

violence

inboth

mentallydisordered

andnon-disordered

offenders

(Bontaetal.,1998).Stalkerswithahistory

ofviolence

aremorelikelytobeviolent(seemeta-analysis:Churcher&Nesca,

2013).Problematic

approachersto

publicfiguresaresignificantly

more

likelyto

haveahistory

ofviolence

than

communicators(G

illetal.,2021;S

calora,B

aumgartner,H

atch-M

aillette,etal.,2002).P

revious

violence

isarisk

factorin

theTRAP-18(M

eloy,

2016)

andin

theSASH

(McE

wan,Strand,et

al.,2017).

18.Homicidal

ideation

Enquiryabouthom

icidal

ideation

isastandardelem

entofviolence

risk

assessmentandincorporatedinto

risk

assessmenttoolssuch

astheClassificationofViolence

Risk(COVR;Monahan

etal.,2006).Homicidalideationisgreatly

increasedin

seriouspsychoticillnesses(Carboneet

al.,2020).Itwas

foundto

bean

omnibusrisk

factorin

aprison

sample

(DeL

isiet

al.,2017).Itisaredflag

item

forviolence

risk

intheSRP(M

acKenzieet

al.,2009).Itisgenerally

conceptualised

along

similar

lines

tosuicidal

ideation.

19.Delusionsoflovingrelationship

Erotomania

isassociated

withintrusivenessandwithahighdegreeofpersistence

intheabsence

ofintervention

(McE

wan

etal.,2009;McE

wan,Strand,etal.,2017),as

isintimacy-seekingmotivation(Jam

es,McE

wan,etal.,2010).

Erotomania

isarisk

factorin

theSRP(M

acKenzieet

al.,2009).

(table

continues)

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 9

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

Tab

le1(continued)

CTAPitem

Somelinksto

theliterature

20.Delusionsofjealousy

Thisitem

includes

delusional

jealousy

andjealousy

inrelationshipswhicharedelusional.Pathological

jealousy

isassociated

withassault(M

ullen

&Maack,1985;S

ilvaetal.,1998),and

jealousy

withindelusionalrelationships(although

uncommonin

other

settings)

ispresumed

tobesimilar.

21.Beliefin

shared

pastordestiny

Non-erotomanic

delusionsofshared

pastordestinyareassociated

withincreasedintrusivenessandpersistence.

Forinstance,thosewithdelusional

familyties

weresignificantlymore

likelyto

approachin

Royal

Fam

ilyCases

and

significantly

more

likelyto

makemultiple

rather

than

single

approaches.(Fixated

ResearchGroup,2006).

Delusionsofaspecialrelationship

areincluded

under

“delusionsrelatedto

stalking”as

arisk

factorin

theSRP

(MacKenzieet

al.,2009).

22.Beliefpeopleareim

posters

orpossessed

Theassociationsofdelusional

misidentificationsyndromes

withviolence

havebeennoted

incollectionsofcase

studies

(Silvaet

al.,1996)andfound

incommunitysamplesondelusionsandviolence

(Coid

etal.,2013).

23.Threat

topersonal

integrity

Findingsabouttheassociationbetweenthreat-control-overridesymptomsandviolence

(Linket

al.,1998)

appear

tobebornout

infurther

work

(e.g.,Hodginset

al.,2003).Such

high-riskpsychotic

phenomenaareared-flag

indicatorforviolence

risk

intheSRP(M

acKenzieet

al.,2009).Threat-control-overridesymptomsareassociated

withproblematic

approachbehaviourtowardspublicofficials(Schoenem

anet

al.,2011).

24.Beliefin

owndivinityoronadivinemission

Seriousviolence

has

beensignificantlyassociated

withgrandiosity

inthose

withschizophrenia

(Swansonet

al.,2006).

Grandiosity

significantlydifferentiated

betweenthose

whoapproached

U.S.celebrities

from

thosewhodid

not

(Dietz

&Martell,1989).Itisassociated

withapproachin

publicofficial

cases(Schoenem

anet

al.,2011)andin

problematicapproachtowardamem

beroftheBritish

RoyalFam

ily(Jam

es,M

cEwan,etal.,2010).Those

withdelusions

ofdivinepurpose

wereidentified

intheFixated

ResearchGroup

(2006)reportas

aparticularlydifficultgroup

untram

melledbyinhibitionsorboundaries.Extrem

esofgrandiosity

arearisk

factorin

thepublicfigure

section

oftheSRP(M

acKenzieet

al.,2009).

25.Gutreaction

Gutfeelingsconstituteassociativeprocesses

whichareintuitiveandnotim

mediately

accessible

toconscious

awareness,butarevaluable,in

particularin

those

withexperience

ofagiven

field(Slovic

etal.,2004).Whereas

they

arenosubstitute

forevidence-based

assessment,they

arean

adjunct

whichshould

not

beignored

(McE

wan,Strand,et

al.,2017).

Note.

CTAP=CommunicationsThreatAssessm

entProtocol;SAM

=StalkingAssessm

entandManagem

ent;SASH=ScreeningAssessm

entforStalkingandHarassm

ent;FTAC=

Fixated

Threat

Assessm

entCentre;

TRAP-18=

TerroristRadicalizationAssessm

entProtocol-18;SRP=

StalkingRiskProfile.

10 JAMES ET AL.

adopted a liaison system, with a police officer anda community psychiatric nurse assigned to each ofthe main referring correspondence sections. Thisprovided a feed-back mechanism as to whichfeatures or wording of the screen caused problemsto any individuals. In addition, the referrals toFTAC each day, made using the screen, consti-tuted a daily exercise in examining its efficacy incorrectly identifying cases suitable for referral—inother words, in the identification of false positives.Such cases were then discussed with the originat-ing correspondence offices to ascertain the reasonsfor the case being referred. The FTAC liaison staffalso undertook regular examination during thedevelopment period of samples of letters fromtheir linked correspondence offices which hadbeen thought inappropriate in some way but hadnotbeen referred toFTACon the basis of using thescreen. In other words, this was a regular exercisein ascertaining the presence of false negatives.The wording in the screen was adapted through

experience of its use on several thousand cases. Inaddition, the original list of some thirty items inthe screen was reduced over time for two reasons:firstly, someitemswerenever scoredaspositive—largely because they necessitated knowledge whichcould rarely be obtained from the presentingmaterial; secondly, some items almost alwaysco-occurred becausethey wereineffectmeasuringthe same thing. It proved possible to add inadditional items which concerned a pattern ofbehaviour, because the royal households kept acomputerised database of correspondents. Thefinal version of the screen covered similar contentto that of the CTAP, but in simplified form.As a specific exercise in searching for false

negatives, a sample of the Home Secretary’s mailwas examined, the correspondence office havingremoved all that simply concerned core business.FTAC police staff used the correspondence screenonasampleof283inappropriatecommunications:those that were not flagged up by the screen werethen examined by FTAC consultant forensicpsychiatrists to ascertain whether cases had been“missed” by the screen which should have occa-sioned sufficient concern for referral. None wasidentified in the sample.

Initial Threat Screen

The initial FTAC threat assessment tool wasapplied to all new referrals, enabling a judgementas to whether the case was of low, moderate or

high concern, this determining the priority ofintervention. Low concern cases were logged,but no other action taken. Moderate concern caseswere taken on for further investigation and man-agement. High concern cases were dealt with as anurgent priority.The threat assessment tool started as a grid, but

then changed in form into a series of items con-cerning the subject’s characteristics or behaviour,grouped into thematic sections and incorporating“red-flag” items. The list of items was developedfrom the sources described above. It concernednot simply violence, the base rate of which is low,but also the areas of persistence, escalation, dis-ruption, impairment of function, psychologicalharm, and reputational damage. The original listcomprised 65 items and concerned the assess-ment both of concerning communications andconcerning approaches. Its length was unwieldyand impractical, and as it was developed over6 years and used on some 6,000 cases, its wasreduced to 38 items through processes similar tothose described for the correspondence screenabove—co-occurrence and unavailability of data.It was necessary for the items to include bothcharacteristics which were relatively commonand those which were rare but of considerableimportance when present.This, together with theuse of “red-flag” items, meant that the determiningof concern level using the tool could not involve asummation of scores, but required a defined pro-cess,withinstructionsastowhich cases should fallinto which level. This determined that those usingthe tool required formal instruction in its use.

Development of the CTAP and QCS for Usein the Private Sector

The catalyst for the development of the pub-lished CTAP from the initial work at FTAC wasthe recognition of the need for a structured profes-sional judgement tool concerning initial assess-ment of unwanted communications for use in theprivate sphere, given that no sufficiently specia-lisedtoolwasavailableinthatarea,whilstitshouldremain relevant to police settings. It needed to be acomprehensive, fully-fledged and self-containedpackage, rather than a step in the operating proto-colofapolicingunit.And,whilsttheproblemsthatthe CTAP seeks to address are fundamentally thesame in the private sphere, there are differenceswhich need to be taken into account. Private firms

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 11

tend to be larger, more complex and more multi-national in nature than their public sector counter-parts: they rely heavily on on-line platforms andelectronicformsofcommunication,andhavedoneso for longer than organisations in the publicsphere, and the management of complaint andgrievance tends to be part of core business to agreater extent than in the public sphere. Of note,private organisations tend to be more sensitive to abroader range of risks, in particular the reputa-tional, the financial and the risk of disruption, andthey do not have access to the same range ofinformation sources and powers of interventionthatpolicingorganisationsroutinelyemploy.Thereis also a wider range of organisational roles inprivate organisations to which the use of a struc-tured threat assessment process is relevant—forinstance, security, communications, operations,human resources and legal managers, so care-fully thought-out training packages needed to beincorporated into the development of a relevantsystem. A clearer recognition and understandingof the needs of organisations in the private spherewas achieved by the authors through its interac-tions with corporate entities and through corporatecase-work, insights which were incorporatedinto the development work on the CTAP. Takinginto account the above concerns, the eventualCTAP was produced incorporating 25 core items.The development of the QCS paralleled that of theCTAP. Both cover the same ground, in that theQCS reflects in simpler fashion the content ofthe CTAP.

Reliability Testing: Method

Study Sample

The study sample concerned 102 consecutivecases referred for risk assessment and/or manage-ment to Theseus, a risk assessment company. Thiswaschosenbecausethecompany was setupontheFTAC model and adopted FTAC’s data collectionand risk assessment methodology. It was also thesetting in which the final form of the CTAP wasdeveloped. Fifty of the cases came from the cor-porateworld, 38concerned the talent sector and14were private clients. Each of the 102 cases wasexamined and scored independently on both theCTAP and QCS by two separate raters, neither ofwhom was a trained psychologist or psychiatrist,and neither of whom was an author of the CTAP.

Both worked at Theseus and had been trained inthe use of the CTAP. This was not an exercise thatinvolved vignettes or that was necessarily limitedto single pieces of correspondence: rather, it con-cerned “real world” cases, in which the informa-tion available to the raters was that presented atreferral. The amount of material varied betweencases from very little to considerable, generallyaccording to how long the behaviour of concernhad been occurring. Some cases included summa-riesfromthereferrers,whilstotherswerelimitedtothe correspondence itself. The number of items ofcorrespondence per relevant case ranged fromone to several hundred. Of the 102 cases rated,97 (95.1%) involved correspondence, of which 41also involved some form of intrusive behaviour,ranging from turning up at the person’s place ofwork to attempts at approaching the addressee.Five cases involved some form of approach, with-out correspondence. Whilst the CTAP is intendedfor written communications, these were includedinthesample,usingrecordsofverbaloutput,asthesample was intended to represent the “real world”of threat assessment work.

Inter-Rater Reliability Procedure

Both the CTAP and the QCS were rated foreach case, and the results (together with a rangeof other data items) were entered into a databaseusing SPSS, version 26 (IBM Corp, 2019). Inter-rater reliability was then calculated for each itemin the CTAP and the QCS. Particular note wastaken of the level of agreement on “red flag”items, as the presence of one of these determinesan overall “high concern” finding, independentof any other scoring. Inter-rater reliability wasthen examined for the outcome measures of theQCS and of the CTAP. For the QCS, the out-come decision was whether the case neededreferral to someone else, this being determinedby the scoring of at least one QCS item as positive.For the CTAP, inter-rater agreement on concernlevel judgement in each case was compared.This was seen as a particularly important metric,both because it represented the main outcomemeasure of the CTAP rating, and because it incor-porated a further element of judgement by eachrater, the translation of the findings on the individ-ualCTAPitemsintoanoverall concernjudgementin the manner set out in the CTAP manual.The total scores for the QCS and the CTAP

were then calculated in each case for each rater,

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

12 JAMES ET AL.

and the inter-rater reliability of the total scoreswas calculated. It should be noted that the total-ling of scores is not part of the procedure for usingeither theQCSortheCTAP. TheQCSis designedto give a “positive” result, if only one item isjudged present. And the CTAP contains a mixtureof rare and relatively common risk factors, whichcannot simply be summated, as well as incorpo-rating “red flag” items, the presence of one alonebeing sufficient for the case to be judged as beingof high concern. Nevertheless, it was decided toinclude this measure as a further form of check oninter-rater consistency.

Test–Retest

Twenty cases were then randomly selectedfrom the first part of the database and rated againon the CTAP by one of the original raters at aminimum of 2 years after the original rating,without reference to the original rating results.The two ratings were then compared to give mea-sures of test–retest reliability for the individualCTAP items and for the concern judgement. Theperiod of 2 years was selected as long enough forthe rater no longer to retain memory of the originalratings.

Concern Levels

Lastly, the concern level from the initial CTAPrating for each case was compared with the con-cernlevelatthepointTheseusclosedthecasefromits books or last received feedback on progress.Thereasoningforthiswasasfollows.TheCTAPisnot a predictive instrument, given that it does notaim to reach a judgement on future risk, but ratherto aid those engaged in the initial assessment of acase to decide upon the need for, and priority tobe accorded to, any initial intervention. As such,the CTAP is not open to any simple exercise inassessing predictive validity. However, it ispossible to use a change in concern levels togain some indication as to its performance. TheCTAP is balanced conservatively to accord pri-ority to a case presenting with certain definedfeatures until further information can be gath-ered which will either confirm the need forfurther intervention, or enable the concern levelto be reduced. In other words, it is designed toavoid false negatives. Were it to be failing to dothis, then it would be expected that there wouldbe a proportion of cases in which the concern

level would rise subsequent to the initial assess-ment. Consequently, the absence of any increasein concern level between initial and final contactwould constitute evidence that the CTAP wasnot underestimating initial concern. Alterna-tively, if the CTAP were consistently overesti-mating the priority that should be accorded tocases, then it would be expected that there wouldbe a high proportion of cases initially rated asmoderate or high concern which were subse-quently reduced to low. Finally, if the initiallevel of priority suggested by the initial CTAPapplication were appropriate, then a proportionof cases would have been reduced in concernlevel through Theseus intervention, given thatpart of its purpose is to aid in the drawing up andexecution of management plans to reduce orcontain threat. It would not be anticipated thatall cases would be reduced to a low level ofconcern, in part because risk containment doesnot necessarily equate to risk reduction, and inpart because those receiving suggested manage-ment plans may choose not to put them intoeffect.

Statistical Analysis

To ascertain a coefficient of agreement betweenthe two raters on categorical variables, specificallythe ratings for each item on CTAP and QCS,Cohen’s kappa (κ) was used (Cohen, 1960),kappa being a means of ascertaining the propor-tion of cases in which there was agreementwhich takes account of the level of agreementthat would be expected based simply on chance(Agresti, 2013). Kappa values greater that zerorepresent better-than-chance agreement betweenthe two raters, to a maximum value of +1, whichindicates perfect agreement. Different classifi-cations have been suggested for the strength ofagreement based on the value of Cohen’s κ. Here,the guidelines used were taken from Altman(1999), originally adapted from Landis andKoch (1977). A value of less than 0.20 is classi-fied as poor agreement; 0.21–0.40 fair; 0.41–0.60 moderate; 0.61–0.80 good; and 0.81–1.00very good. Confidence intervals were calculatedas ±(1.96 × SE).To compare total scores for the CTAP between

raters and total scores for the QCS between raters,the intraclass correlation coefficient (ICC) wasused, ICC being a measure which reflects bothdegree of correlation and agreement between

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 13

measurements. Selecting from the ten forms ofmodel set out by McGraw and Wong (1996), atwo-way random model was used, absoluteagreement type, single rater, this being selectedto allow generalisation from the current raters to alarger set of possible raters. The ICC values wereinterpreted according to the thresholds set out byShrout (1998): virtually none (0.00–0.10), slight(0.11–0.40),fair(0.41–0.60),moderate(0.61–0.80),and substantial (0.80–1.0).To examine the correlation between total

CTAP scores and total QCS scores for each rater,and to compare judgements as to concern level,Kendall’s tau-b (τb) was used (Agresti, 2010;Kendall, 1938). Test–retest reliability was exam-ined using Cohen’s κ.

Results

Inter-Rater Reliability for CTAP Items

The percentage agreement and kappa values,with 95% confidence intervals, are given for theCTAP in Table 2, together with the proportion ofpositive responses by each rater for each item.Some items were infrequently scored as present,with there being no cases in this sample whichwere positive for Item 24. Of the 25 items, kappawas not calculated for three items for the followingreasons: Item 20, sample too small to calculate;Item 24, no data; Item 25, score for Rater 1 is aconstant. For 16 of the 25 items, the kappa valuefell withinthe band 0.81–1.00. Items1, 6, 9, 14, 15fell within the band 0.61–0.80. One item, Item 21,producedtheresultκ=0.499(0.301–0.697)whichis in the “moderate” agreement band. Percentageagreement fell below 95% for only five items (1, 6,9, 11, and 21).

“Red Flag” CTAP Items

Of the five items, the percentage agreementrangedbetween97%and100%.Thespecifickappavalues for each item with 95% confidence limitswere as follows: end-of-tether’ language 0.830[0.642–1.000]; suicidal ideation 1.000; homi-cidal ideation 0.913 [0.756–1.000]; and threat topersonal integrity 0.936 [0.811–1.000]. One redflag item (delusions of jealousy) was present intoo few cases (two) to allow the calculation of akappa value.

Inter-Rater Reliability for QCS Items

The percentage agreement and kappa values forthe 18 QCS items, with 95% confidence intervals,are given in Table 3, together with the proportionof cases in which each rater found each itempositive/present. For Items 1 and 15, kappa wasnotcalculated,asatleastoneraterproportionwasaconstant. For the remaining 16 items, kappa fellinto the agreement band 0.81–1.00. All casesscored at least one positive item for each rater.Percentage agreement for the eighteen items laybetween 90% and 100%

Total CTAP and QCS Scores

The intraclass correlation coefficient (ICC) fortotal CTAP score between raters was 0.962, 95%CI [0.944–0.974]. For total QCS score, the ICCwas 0.954, 95% CI [0.932–0.969].

Level of Concern

There was a strong positive correlationbetween the judgements as to level of concernτb = 0.899, p < .001. There was absolute agree-ment in 92% of cases.

Test–Retest Reliability

Therewas100%agreementbetweentheoriginaltestandthere-teston19ofthe 25items.Percentageagreement and Kappa values (all p < .005), with95% confidence intervals, were as follows for theremaining items: Item 1; 80%, 0.718 [0.432–1.000]: Item 3; 95%, 0.898 [0.704–1.000]: Item5; 95%, 0.900 [0.710–1.000]: Item 11; 95%, 0.886[0.670–1.000]: Item 19; 90%, 0.750 [0.427–1.000]: Item 21; 95%, 0.623 [0.166–1.000].

Change in Concern Levels

At initial assessment, 14 cases (13.7%) wereassessed as being of low concern, 57 cases(55.9%) as of moderate concern, and 31 cases(30.4%) as of high concern. Theseus interventionlasted less than 2 months in 86 cases (84.3%),between 2 months and 1 year in 12 cases (11.7%),including one case which was still open, andbetween 1 and 3 years in 4 cases (3.9%). Atthe end of Theseus’ involvement, all the lowconcern cases remained as low concern. Of the57 cases of moderate concern, 22 (39%) remained

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

14 JAMES ET AL.

of moderate concern and 35 (61%) had beenreduced to low concern. None had increased tohighconcern.Ofthe31casesofhighconcern,onecase (3%) remained of high concern, this beingthe case that was still open: of the remainder, 23(74%) had been reduced to moderate concern and7 (23%) to low concern.

Discussion

The CTAP is unusual in being designed spe-cifically for the purpose of allocating priority inthe initial assessment of concerning communica-tions and for the making of quick decisions inresponse to limited information in a dynamic,

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

Table 2Inter-Rater Reliability of CTAP Items

Item (brief characterisation)Percentageagreement

Proportion of yesresponses

(Rater 1, Rater 2) (%)

Kappa[95% CI]

in all cases, p < .005

1. Threats 88.24 48, 51 0.806[0.708–0.904]

2. Declaration of intent 97.06 42, 45 0.940[0.873–1.000]

3. Evidence of displacement 96.08 43, 45 0.920[0.844–0.996]

4. Extremes of anger 97.06 17, 17 0.929[0.833–1.000]

5. Escalation in anger 97.06 36, 39 0.937[0.866-1.000]

6. Quest for justice 91.00 17, 18 0.744[0.587–0.901]

7. Demands to change behaviour 95.00 36, 37 0.892[0.800–0.984]

8. Demand for money/apology 97.03 26, 23 0.919[0.829–1.000]

9. Prolific correspondence 91.18 27, 26 0.771[0.628–0.914]

10. Awareness of personal details 96.04 36, 32 0.911[0.827–0.995]

11. History of intrusive behaviours 92.08 28, 34 0.818[0.698-0.938]

12. End-of-tether 97.06 6, 8 0.830[0.642–1.000]

13. Suicidal ideation 100 8, 8 1.00014. Sexually aggressive 95.01 6, 7 0.666

[0.411–0.921]15. Interest in attackers 99.00 2, 3 0.795

[0.403–1.000]16. Reference to weapons 97.06 10, 9 0.942

[0.828–1.000]17. History of violence 98.97 3, 4 0.905

[0.721–1.000]18. Homicidal ideation 99.02 4, 4 0.913

[0.756–1.000]19. Delusions of love 96.08 19, 18 0.900

[0.804–0.996]20. Delusions of jealousy 98.04 1, 1 Sample too small to calculate21. Belief in shared destiny 85.30 8, 7 0.499

[0.301–0.697]22. Imposters or possessed 100 3, 3 1.00023. Threat to personal integrity 99.02 8, 9 0.936

[0.811–1.000]24. Belief in own divinity 100 0, 0 No data25. Gut reaction 98.04 100, 99 Not calculable: score for Rater 1 a constant

Note. CTAP = Communications Threat Assessment Protocol.

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 15

real-time setting. It constitutes a promising instru-ment designed to have considerable utility in arange of settings, its strengths being that: it is ageneral tool looking at risk as a whole, rather thansimply violence; it is designed to have wide appli-cability; it incorporates standardised approachesboth for recipients of communications (or theircorrespondence staff),and forthe securityor threatassessment personnel who are required to conductan initial evaluation of correspondence referred tothem; it has been evolved over a number of yearsthrough practical application, which renders itstraightforward and user-friendly.The foundations of the CTAP were laid through

dailyuseofearlieriterationsinapolice/NHSthreatassessment setting over an 8-year period on real

cases as they unfolded, the cases being manythousand in number. It incorporates knowledgeaccumulated by leading threat assessment agen-ciesontwocontinents,aswellasleadingexpertsinthe fields of threat, harassment, stalking and vio-lence and mental illness. The items of the CTAPare firmly anchored in research evidence, as illus-trated in Table 1. It was developed alongside twoother instruments, the SRP and the SASH, withwhich it shares some content and structural fea-tures, and both of which have been shown to haveuseful reliability and predictive validity. Its even-tual development into the comprehensive, self-containedCTAPpackagefollowedtherecognitionof the need for a structured professional judgementtool for use in the private sphere, and involved a

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

Table 3Inter-Rater Reliability for QCS Items

ItemPercentageagreement

Proportion of yes responses(Rater 1, Rater 2) (%)

Kappa[95% CI]

in all cases, p < .005

1. 100 100, 100 Not computed as case proportions areconstants.

2. 99% 53, 54 0.980[0.941–1.000]

3. 98% 25, 25 0.947[0.875–1.000]

4. 96% 45, 45 0.920[0.845–0.996]

5. 99% 12, 13 0.954[0.866–1.000]

6. 98% 34, 32 0.955[0.894–1.000]

7. 95% 28, 31 0.883[0.783–0.983]

8. 90% 26, 26 0.742[0.591–0.893]

9. 99% 11, 10 0.947[0.843–1.000]

10. 100% 7, 7 1.00011. 98% 6, 6 0.823

[0.584–1.000]12. 98% 13, 11 0.906

[0.777–1.000]13. 96% 33, 33 0.912

[0.828–0.996]14. 97% 8, 10 0.878

[0.712–1.000]15. 99% 0, 1 Not computed: figure for Rater 1 is a constant.16. 94% 25, 27 0.845

[0.725–0.965]17. 94% 21, 24 0.857

[0.736–0.976]18. 96% 30, 34 0.911

[0.823–0.997]

Note. QCS = Quick Correspondence Screen.

16 JAMES ET AL.

further step—the incorporation of the perspectiveand requirements of non-public organisations.Additional experience of its use has been accu-mulated over 7 years in the assessment of casesinvolving prominent individuals, celebrities andcorporate entities. Overall, its provenance andresearch roots go some way to establishing itscontent validity.This is the first study that has examined the

reliability of the CTAP. The results for inter-raterreliability are encouraging; the CTAP demon-strated very good agreement for most of its items.One CTAP item (belief in shared destiny) pro-duced a disappointing kappa value of 0.499, 95%CI [0.301–0.697], which is in the moderatelygood agreement band, with confidence intervalsranging from fair to good. This item requires therater to make a judgement as to the degree ofconviction that is present in the mind of the indi-vidual under consideration. It may be that this is amore difficult judgement to make than with factualitems: or it may be that the wording of the CTAP orthe training module offer insufficient guidance.It is of note that the test–retest reliability was alsopoorest on this item.It is particularly important that there should be

good inter-rater reliability on the “red flag” items,given their particular role in determining that acase should be judged of high concern. Agree-ment was in the very good range for the four itemsfor which it was possible to calculate results.However, for one red flag item (delusions ofjealousy), there were insufficient cases in thesample of 102 to allow analysis. This is under-standable in terms of the way the CTAP is con-structedtoincludebothrelativelycommonfeaturesand uncommon characteristics which, when pres-ent, are of such importance that their incorporationinto a threat assessment instrument is warranted. Alarger sample and/or repetition of this exercise witha sample of different origin would be necessary totest reliability more robustly for this item.The test–retest reliability for the CTAP was

examined on a smaller sample of twenty caseswith at least 2 years between evaluations. Agree-ment fell in the “very good” range for 21 of the25 items and in the “good” range for the remain-der. This is promising, though repeating theexercise on a larger sample would be desirable.But of particular note in this study was the highlevel of agreement between raters as to level ofconcern. This is a separate part of the CTAPprocess, for which instructions are given in the

manual.Evidently, it is not only of importance thatthere is inter-rater agreement on the individualCTAP items: there needs also to be consistencybetween raters in the manner that these are con-verted into a judgement as to level of concern, thisafter all being the final “product” of the exercise.The final part of this study concerned an indi-

cator as to whether the level of concern allocated islikely tohave been appropriate. The purpose of theCTAP is to triage cases at initial assessment intothree concern levels as an indication of the prioritythat they should be accorded. As explained indetail above, the pattern of change in priority levelbetween initial assessment and case closure canoffer information as to whether the CTAP is notpronetogivingfalsenegatives,ora preponderanceof false positives. The results indicate that neitheris likely to have been the case. There is little in theliterature with which to compare these particularmeasures (James, Kerrigan, et al., 2010), but thisstudy suggests that they may be of use. Given thatthe CTAP’s purpose is not a predictive one, ex-ercises in predictive validity present problems indesign. A companion paper to this one examinesthe ability of CTAP items to distinguish betweenthose concerning correspondents who go on toapproach and those that do not, and examines therelationship between CTAP items and persistence.

Limitations

This is the first such study of the CTAP andconfirmation of its results by other groups wouldbe desirable in order to offer a firmer foundationforitsconclusions andtoensure thatitsresultscanbe generalised. The study involved two raters,and its repetition with a greater number would beinstructive. As regards the QCS, the results hereshould be treated with caution, in that a corre-spondence screen intended for use by correspon-dence handlers was completed by individualsdirectly involved in threat assessment. This mighthave resulted in an artificially good performanceof the QCS in terms of inter-rater reliability.A limitation of the CTAP itself is that it em-

ploys the use of a manual and requires standar-dised training. However, it is not unique in thisrespect, thetraininglastshalfaday,andtheCTAPitself can be completed quickly by those familiarwith its use. A limitation of the present study isthat there were insufficient cases concerning twoof the CTAP items to produce kappa values. Thisis the consequence of using real world cases,

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 17

rather than constructed vignettes. However, theuse of real cases in a threat assessment setting isalso one of the strengths of the study, given thatinitial case material does not always contain cleardata and often involves a degree of ambiguitywhich can make cases more difficult to rate.

Directions for Future Research

As stated above, this study has been the first ofa two-part project, the second part of whichconcerns the predictive validity of the CTAP.The first need is for the repetition of these studies,using similar methodology and training, to see ifthe results are replicated. This should ideally beundertaken on case series from different origins,bothinterms ofsector(e.g.,policing) andintermsof country, population configuration, and lan-guage. It would be preferable if these were inde-pendent of the authors of the CTAP, given thatthere is some evidence of a researcher allegianceeffect in studies conducted by the authors of riskassessment instruments (Singh et al., 2013). AsfortheQCS,thereisevidentlyscopeforrepetitionof inter-rater reliability exercises using lightlytrained correspondence personnel (the skillsetfor which it is intended), rather than threat asses-sors,asthis wouldgive amore accurate indicationof real-world conditions. Finally, it is likely thatthere are other approaches to the standardisedassessment of concerning correspondence whichhave yet to be published. It would be a usefulexercise if the performance of these could bemeasured against the CTAP, given the currentabsence of published instruments with which theCTAP could be compared.

Conclusion

The CTAP has been widely used in differentsettings over the last 7 years, and developmentversions for a longer period still. The CTAPappears to be a useful tool in the identificationand initial prioritisation of concerning commu-nications, both in the public and private spheres.However, this is the first published study relatingits content to the background literature on riskfactors and evaluating the tool’s reliability. Itsresults are encouraging. It suggests that CTAPhas a firm, up-to-date research base and that boththe CTAP and the QCS can be applied reliably.The CTAP appears to be a useful addition to the

stock of instruments which encourage systematicand defensible threat assessment practice.

References

Agresti, A. (2010). Analysis of ordinal categoricaldata (2nd ed.). Wiley. https://doi.org/10.1002/9780470594001

Agresti, A. (2013). Categorical data analysis (3rded.). Wiley.

Akhtar, S., & Morrison, C. M. (2019). The prevalenceand impact of online trolling of UK Members ofParliament. Computers in Human Behavior, 99,322–327. https://doi.org/10.1016/j.chb.2019.05.015

Altman, D. G. (1999). Practical statistics for medicalresearch. Chapman & Hall/CRC Press.

Barry-Walsh, J., James, D. V., & Mullen, P. E. (2020).Fixated Threat Assessment Centers: Preventing harmand facilitating care in public figure threat cases andthose thought to be at risk of lone-actor grievance-fueled violence. CNS Spectrums, 25, 630–637.https://doi.org/10.1017/S1092852920000152

Bixler, B. S., Dunn, J., & Grundland, T. (2021).Operations of the Los Angeles Police Threat Man-agement Unit and Crisis Support Response Section.In J. R. Meloy & J. Hoffmann (Eds.), Internationalhandbook of threat assessment (2nd ed., pp. 545–570). Oxford University Press.

Bjelland, H. F., & Bjørgo, T. (2014). Trusler og trus-selhendelser mot politikere: En spørreundersøkelseblant norske stortingsrepresentanter og regjerings-medlemmer [Threats and threat events against politi-cians: A survey of Norwegian members of parliamentand government officials]. Politihøgskolen.

Bonta, J., Law, M., & Hanson, K. (1998). The predictionof criminal and violent recidivism among mentallydisordered offenders: A meta-analysis. PsychologicalBulletin, 123(2), 123–142. https://doi.org/10.1037/0033-2909.123.2.123

Brottsförebyggande rådet (Crime Prevention Council).(2012). Politikernas Trygghets undersökning 2012:Förtroendevaldas utsatthet och oro för hot, våld ochtrakasserier Rapport 2012:14 [Politicians: The secu-rity survey 2012: Elected officials vulnerability andconcern for the threats, violence and harassment.Report 2012:14]. Fritzes.

Calhoun, F. S. (1998). Hunters and Howlers: Threatsand violence against federal judicial officials in theUnited States, 1789–1993. Department of Justice,U.S. Marshals Service.

Calhoun, F. S., & Weston, S. W. (2003). Contemporarythreat management. Specialized Training Services.

Calhoun, F. S., & Weston, S. W. (2009). Threatassessment and management strategies. CRC Press.

Carbone, J. T., Holzer, K. J., Vaughn, M. G., &DeLisi, M. (2020). Homicidal ideation and forensicpsychopathology: Evidence from the 2016 Nation-wide Emergency Department Sample (NEDS).

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

18 JAMES ET AL.

Journal of Forensic Sciences, 65(1), 154–159.https://doi.org/10.1111/1556-4029.14156

Cawood, J. S., & Corcaran, M. H. (2003). Violenceassessment and Intervention: The practitioner’shandbook. CRC Press. https://doi.org/10.1201/9780203500057

Cawood, J. S., Scalora, M. J., & Viñas-Racionero, R.(2020). The Cawood assessment grid for organisa-tional (workplace) violence: Initial testing fir inter-rater reliability and predictive validity. Journal ofThreat Assessment and Management, 7, 200–213.https://doi.org/10.1037/tam0000153

Churcher, F. P., & Nesca, M. (2013). Risk factors forviolence in stalking perpetration: A meta-analysis.FWU Journal of Social Sciences, 7, 100–112.

Clemmow, C., Gill, P., Corner, E., Farnham, F.,Taylor, R., Wilson, S., Taylor, A., & James, D.(2021) A data-driven classification of outcome be-haviours in those who cause concern to Britishpublic figures. Psychology, Public Policy, andLaw, 27(4), 509–521. https://doi.org/10.1037/law0000269

Cohen, J. (1960). A coefficient of agreement fornominal scales. Educational and PsychologicalMeasurement, 20, 37–46. https://doi.org/10.1177/001316446002000104

Coid, J. W., Ullrich, S., Kallis, C., Keers, R., Barker,D., Cowden, F., & Stamps, R. (2013). The relation-ship between delusions and violence: Findings fromthe East London first episode psychosis study.JAMA Psychiatry, 70(5), 465–471. https://doi.org/10.1001/jamapsychiatry.2013.12

Cook, D. A., & Beckman, T. J. (2006). Currentconcepts in validity and reliability for psychometricinstruments: Theory and application. The AmericanJournal of Medicine, 119(2), 166.e7–166.e16. https://doi.org/10.1016/j.amjmed.2005.10.036

de Becker, G. (1996). The gift of fear. Random House.De Groot, I. N. J., Drost, L. F., Boutellier, J. C. J., &Voerman, B. E. (2010). Bedreigers van politici:Risicos en interventiemogelijkheden [People whothreaten politicians: Risks and intervention possi-bilities]. Verwey-Jonker Instituut.

DeLisi, M., Tahja, K., Drury, A. J., Caropreso, D.,Elbert, M. J., & Heinrichs, T. (2017). The criminol-ogy of homicidal ideation: Associations with criminalcareers and psychopathology among federal correc-tional clients. American Journal of Criminal Justice,42, 554–573. https://doi.org/10.1007/s12103-016-9371-5

Dietz, P., & Martell, D. A. (2010). Commentary:Approaching and stalking public figures—A pre-requisite to attack. The Journal of the AmericanAcademy of Psychiatry and the Law, 38(3), 341–348. http://www.jaapl.org/content/38/3/341

Dietz, P., Matthews, D., Van Duyne, C., Martell, D.,Parry, C., Stewart, T., van Duyne, C., & Crowder, J.

(1991) Threatening and otherwise inappropriate let-ters to Hollywood celebrities. Journal of ForensicSciences, 36, 185–209. https://doi.org/10.1520/JFS13019J. ISSN 0022-1198.

Dietz, P. E., & Martell, D. A. (1989). Mentally disor-dered offenders in pursuit of celebrities and poli-ticians. National Institute of Justice.

Dietz, P. E., Matthews, D. B., Martell, D. A., Stewart,T. M., Hrouda, D. R., & Warren, J. (1991). Threat-ening and otherwise inappropriate letters to membersof the United States Congress. Journal of ForensicSciences, 36(5), 1445–1468. https://doi.org/10.1520/JFS13165J

Douglas, K. S., & Otto, R. K. (Eds.). (2020). Handbookof violence risk assessment (2nd ed.). Routledge.https://doi.org/10.4324/9781315518374

Dunn, J. (2014). The Los Angeles Police DepartmentThreat Management Unit. In J. R. Meloy &J. Hoffmann (Eds.), International handbook ofthreat assessment (pp. 285–298). Oxford UniversityPress.

Every-Palmer, S., Barry-Walsh, J., & Pathé, M.(2015). Harassment, stalking, threats and attackstargeting New Zealand politicians: A mental healthissue. The Australian and New Zealand Journal ofPsychiatry, 49, 634–641. https://doi.org/10.1177/0004867415583700

Farnham, F. R., & Busch, K. G. (2016). Internationalperspectives on the protection of public officials.Behavioral Sciences & the Law, 34, 597–601.https://doi.org/10.1002/bsl.2266

Fein, R. A., & Vossekuil, B. (1998). Preventing attackson public officials and public figures: A secretservice perspective. In J. R. Meloy (Ed.), The psy-chology of stalking (pp. 175–191). Academic Press.

Fein, R. A., & Vossekuil, B. (1999). Assassination inthe United States: An operational study of recentassassins, attackers, and near-lethal approachers.Journal of Forensic Sciences, 44(2), 321–333.https://doi.org/10.1520/JFS14457J

Fein, R. A., & Vossekuil, B. (2000). Protective intel-ligence and threat assessment investigations: Aguide for state and local law enforcement officials.U.S. Dept. of Justice, Office of Justice Programs,National Institute of Justice.

Fixated Research Group. (2006). Report of the fixatedresearch group. Home Office.

Gill, P., Corner, E., Farnham, F., Wilson, S., March-ment, Z., Taylor, A., Taylor, R., & James, D.(2021). Predictors of varying levels of risks posedby fixated individuals to British public figures.Journal of Forensic Sciences, 66(4), 1364–1376.https://doi.org/10.1111/1556-4029.14708

Hamers, H., Borgesius, E., & Verschure, S. (2010).Mensen met een psychische stoornis die publiekepersonen bedreigen: Een aanpak gericht op zorg[People with mental disorders who threaten public

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 19

figures: an approach focused on facilitating care].Zorg Consult.

Hart, S., & Logan, C. (2011). Formulation of violencerisk using evidence-based assessments: the structuredprofessional judgement approach. In P. Sturmey &M. McMurran (Eds.), Forensic case formulation(1st ed., pp. 83–107). Wiley. https://doi.org/10.1002/9781119977018.ch4

Hehemann, K., van Nobelen, D., Brandt, C., & McEwan,T. E. (2017). The reliability and predictive valid-ity of the Screening Assessment for Stalking andHarassment (SASH). Journal of Threat Assessmentand Management, 4(3), 164–177. https://doi.org/10.1037/tam0000085

Hodgins, S., Hiscoke, U. L., & Freese, R. (2003). Theantecedents of aggressive behavior among men withschizophrenia: A prospective investigation of pa-tients in community treatment. Behavioral Sciencesand The Law, 21(4), 523–546. https://doi.org/10.1002/bsl.540

Hoffman, J., & Sheridan, L. (2008). Stalking, threat-ening and attacking corporate figures. In J. R.Meloy, L. Sheridan, & J. Hoffman (Eds.), Stalking,threatening and attacking public figures: A psycho-logical and behavioral analysis (pp. 123–142).Oxford University Press. https://doi.org/10.1093/med:psych/9780195326383.003.0006

IBM Corp. (2019). SPSS statistics, version 26.0.James, D. V. (2010). Protecting the prominent? Aresearch journey with Paul Mullen. CriminalBehaviour and Mental Health, 20(3), 242–250.https://doi.org/10.1002/cbm.769

James, D. V. (2015). Another helping of “intellectualhash”: Commentary on “Assassins of Rulers” (Mac-Donald, 1912). Journal of Threat Assessment andManagement, 1, 228–240. https://doi.org/10.1037/tam0000027

James, D. V., & Farnham, F. R. (2016). Outcome andefficacy of interventions by a public figure threatassessment and management unit: A mirrored studyof concerning behaviors and police contacts beforeand after intervention. Behavioral Sciences & theLaw, 34, 660–680. https://doi.org/10.1002/bsl.2255

James, D. V., Farnham, F. R., Sukhwal, S., Jones, K.,Carlisle, J., & Henley, S. (2016). Aggressive/intrusivebehaviours, harassment and stalking of membersof the United Kingdom parliament: A prevalencestudy and cross-national comparison. Journal ofForensic Psychiatry & Psychology, 27, 177–197.https://doi.org/10.1080/14789949.2015.1124908

James, D. V., Farnham, F. R., & Wilson, S. P. (2014).The Fixated Threat Assessment Centre: Implement-ing a joint policing and psychiatric approach to riskassessment and management in public figure threatcases. In J. R. Meloy & J. Hoffmann (Eds.), Interna-tional handbook of threat assessment (pp. 299–320).Oxford University Press.

James, D. V., Kerrigan, T., Forfar, R., Farnham, F., &Preston, L. (2010). The Fixated Threat AssessmentCentre: Preventing harm and facilitating care. Journalof Forensic Psychiatry & Psychology, 21, 521–536.https://doi.org/10.1080/14789941003596981

James,D. V.,MacKenzie, R. D.,& Farnham,F.R. (2014).Communications Threat Assessment Protocol-25:Guidelines for the initial assessment of problem-atic communications. Theseus.

James, D. V., McEwan, T. E., MacKenzie, R. D.,Meloy, J. R., Mullen, P. E., Pathé, M. T., Farnham,F. R., Preston, L. F., & Darnley, B. (2010).Persistence in stalking: A comparison of associa-tions in general forensic and public figure samples.Journal of Forensic Psychiatry & Psychology,21, 283–305. https://doi.org/10.1080/14789940903388994

James, D. V., Meloy, J. R., Mullen, P. E., Pathé, M. T.,Farnham, F. R., Preston, L. F., & Darnley, B. J.(2010). Abnormal attentions toward the BritishRoyal family: Factors associated with approachand escalation. The Journal of the American Acad-emy of Psychiatry and the Law, 38, 329–340. http://www.fixatedthreat.com/perch/resources/factors-associated-with-approach-and-escalation.pdf

James, D. V., Mullen, P. E., Meloy, J. R., Pathé, M. T.,Farnham, F. R., Preston, L., & Darnley, B. (2007).The role of mental disorder in attacks on Europeanpoliticians 1990–2004. Acta Psychiatrica Scandi-navica, 116, 334–344. https://doi.org/10.1111/j.1600-0447.2007.01077.x

James, D. V., Mullen, P. E., Meloy, J. R., Pathé, M. T.,Preston, L., Darnley, B., Farnham, F. R., & Scalora,M. J. (2010). Stalkers and harassers of Britishroyalty: An exploration of proxy behaviours forviolence. Behavioral Sciences & the Law, 29(1),64–80. https://doi.org/10.1002/bsl.922

James, D. V., Mullen, P. E., Pathé, M. T., Meloy, J. R.,Farnham, F. R., Preston, L., & Darnley, B. (2008).Attacks on the British Royal family: The role ofpsychotic illness. The Journal of the AmericanAcademy of Psychiatry and the Law, 36, 59–67.https://fixatedthreat.com/perch/resources/attacks-on-the-british-royal-family.pdf

James, D. V., Mullen, P. E., Pathé, M. T., Meloy, J. R.,Preston, L. F., Darnley, B., & Farnham, F. R.(2009). Stalkers and harassers of royalty: The roleof mental illness and motivation. PsychologicalMedicine, 39(9), 1479–1490. https://doi.org/10.1017/S0033291709005443

James, D. V., & Sheridan, L. P. (2020). What worksin risk assessment in stalking cases? In J. S.Wormith, L. A. Craig, & T. E. Hogue (Eds.),The Wiley handbook of what works in violence riskmanagement: Theory, research and practice (pp.527–542). Wiley. https://doi.org/10.1002/9781119315933.ch27

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

20 JAMES ET AL.

James, D. V., Sukhwal, S., Farnham, F. R., Evans, J.,Barrie, C., Taylor, A., & Wilson, S. P. (2016).Harassment and stalking of members of the UnitedKingdom Parliament: Associations and consequences.Journal of Forensic Psychiatry & Psychology,27, 309–330. https://doi.org/10.1080/14789949.2015.1124909

Kendall, M. (1938). A new measure of rank correlation.Biometrika, 30(1–2), 81–89. https://doi.org/10.2307/2332226

Kirkbride, J. B., Errazuriz, A., Croudace, T. J., Morgan,C., Jackson, D., Boydell, J., Murray, R. M., & Jones,P. B. (2012). Incidence of schizophrenia andother psychoses in England, 1950–2009: A sys-tematic review and meta-analyses [Bottom of Form].PLOS ONE, 7(3), Article e31660. https://doi.org/10.1371/journal.pone.0031660

Kropp, P. R., Hart, S. D., & Lyon, D. R. (2008).Guidelines for Stalking Assessment and Management(SAM): User manual. ProActive ReSolutions Inc.

Landis, J. R., & Koch, G. G. (1977). The measurementof observer agreement for categorical data. Biometrics,33, 159–174. https://doi.org/10.2307/2529310

Laschi, M., & Lombroso, C. (1886). Le délit politique[Political crime]. In Actes du Premier CongrèsInternational d’Anthropologie Criminelle, Rome,1885 (pp. 379–389). Bocca Frères.

Link, B. G., Stueve, A., & Phelan, J. (1998). Psychoticsymptoms and violent behaviors: Probing the com-ponents of “threat/control-override” symptoms.Social Psychiatry and Psychiatric Epidemiology,33(Suppl. 1), S55–S60. https://doi.org/10.1007/s001270050210

MacKenzie, R. D., McEwan, T. E., Pathé, M. T.,James, D. V., Ogloff, J. R. P., & Mullen, P. E.(2009). The stalking risk profile: Guidelines for theassessment and management of stalkers. Stalk Inc.

Marquez, A., & Scalora, M. J. (2011). Problematicapproach of legislators: Differentiating stalking fromisolated incidents. Criminal Justice and Behavior,38(11), 1115–1126. https://doi.org/10.1177/0093854811420847

McElhaney, M. (2004) Aggression in the workplace:Preventing and managing high-risk behaviour.Author House.

McEwan, T. E., MacKenzie, R. D., Mullen, P. E., &James, D. V. (2012). Approach and escalation instalking. Journal of Forensic Psychiatry & Psychology,23(3), 392–409. https://doi.org/10.1080/14789949.2012.679008

McEwan, T. E., Mullen, P. E., & MacKenzie, R.(2009). A study of the predictors of persistence instalking situations. Law and Human Behavior,33(2), 149–158. https://doi.org/10.1007/s10979-008-9141-0

McEwan, T. E., Mullen, P. E., & MacKenzie, R.(2010). Suicide amongst stalkers. Journal of

Forensic Psychiatry & Psychology, 2(4), 514–520.https://doi.org/10.1080/14789940903564370

McEwan, T. E., Shea, D. E., Daffern, M., MacKenzie,R. D., Ogloff, J. R. P., & Mullen, P. E. (2018). Thereliability and predictive validity of the stalking riskprofile. Assessment, 25(2), 259–276. https://doi.org/10.1177/1073191116653470

McEwan, T. E., Strand, S. M., MacKenzie, R. D., &James, D. V. (2017). Screening Assessment forStalking and Harassment (SASH): Guidelines forapplication and interpretation. Stalk Inc.

McGraw, K. O., & Wong, S. P. (1996). Forminginferences about some intraclass correlation coeffi-cients. Psychological Methods, 1, 30–46. https://doi.org/10.1037/1082-989X.1.1.30

Meloy, J. R. (2014). Approaching and attacking publicfigures: A contemporary analysis of communica-tions and behavior. Journal of Threat Assessmentand Management, 1(4), 243–261. https://doi.org/10.1037/tam0000024

Meloy, J. R. (2016) TRAP-18 (Terrorist Radicali-zation Assessment Protocol-18). User manual(version 1.0). Global Institute of ForensicResearch, Inc.

Meloy, J. R. (2018). The operational development andempirical testing of the Terrorist RadicalizationAssessment Protocol (TRAP-18). Journal of Per-sonality Assessment, 100(5), 483–492. https://doi.org/10.1080/00223891.2018.1481077

Meloy, J. R., Amman, M., & Hoffman, J. (2021).Public figure stalking and attacks. In J. R. Meloy &J. Hoffmann (Eds.), International handbook ofthreat assessment (2nd ed., pp. 166–188). OxfordUniversity Press. https://doi.org/10.1093/med-psych/9780190940164.003.0009

Meloy, J. R., & Gill, P. (2016). The lone-actor terroristand the TRAP-18. Journal of Threat Assessmentand Management, 3(1), 37–52. https://doi.org/10.1037/tam0000061

Meloy, J. R., & Hoffmann, J. (Eds.). (2014). Inter-national handbook of threat assessment. OxfordUniversity Press.

Meloy, J. R., & Hoffmann, J. (Eds.). (2021). Inter-national handbook of threat assessment (2nd ed.).Oxford University Press. https://doi.org/10.1093/med-psych/9780190940164.001.0001

Meloy, J. R., Hoffmann, J., Deisinger, E. R. D., &Hart, S. (2021). Threat assessment and threat man-agement. In J. R. Meloy & J. Hoffmann (Eds.),International handbook of threat assessment (2nd ed.,pp. 3–21). Oxford University Press. https://doi.org/10.1093/med-psych/9780190940164.003.0001

Meloy, J. R., Hoffmann, J., Guldimann, A., & James,D. (2012). The role of warning behaviors in threatassessment: An exploration and suggested typology.Behavioral Sciences & the Law, 30, 256–279. https://doi.org/10.1002/bsl.999

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 21

Meloy, J. R., James, D. V., Farnham, F. R., Mullen,P. E., Pathé, M., Darnley, B., & Preston, L. (2004). Aresearch review of public figure threats, approaches,attacks, and assassinations in the United States.Journal of Forensic Sciences, 49(5), 1086–1093.https://doi.org/10.1520/JFS2004102

Meloy, J. R., James, D. V., Mullen, P. E., Pathé, M. T.,Farnham, F. R., Preston, L. F., & Darnley, B. J.(2011). Factors associated with escalation and prob-lematic approaches toward public figures. Journalof Forensic Sciences, 56(Suppl. 1), S128–S135.https://doi.org/10.1111/j.1556-4029.2010.01574.x

Meloy, J. R., Mohandie, K., & Green, M. (2008). Aforensic investigation of those who stalk celebrities.In J. R. Meloy, L. Sheridan, & J. Hoffman (Eds.),Stalking, threatening and attacking public figures(pp. 37–54). Oxford University Press. https://doi.org/10.1093/med:psych/9780195326383.003.0002

Meloy, J. R., & O’Toole, M. E. (2011). The concept ofleakage in threat assessment. Behavioral Sciences &the Law, 29(4), 513–527. https://doi.org/10.1002/bsl.986

Meloy, J. R., Sheridan, L., & Hoffman, J. (Eds.).(2008a). Stalking, threatening and attacking publicfigures: A psychological and behavioral analysis.Oxford University Press. https://doi.org/10.1093/med:psych/9780195326383.001.0001

Meloy, J. R., Sheridan, L., & Hoffman, J. (2008b).Public figure stalking, threats and attacks: the stateof the science. In J. R. Meloy, L. Sheridan, & J.Hoffman (Eds.), Stalking, threatening and attackingpublic figures (pp. 3–34). Oxford University Press.https://doi.org/10.1093/med:psych/9780195326383.003.0001

Meloy, J. R., White, S. G., & Hart, S. (2013).Workplace assessment of targeted violence risk:The development and reliability of the WAVR-21.Journal of Forensic Sciences, 58(5), 1353–1358.https://doi.org/10.1111/1556-4029.12196

Mohandie, K., Meloy, J. R., McGowan, M. G., &Williams, J. (2006). The RECON typology of stalking:Reliability and validity based upon a large sampleof North American stalkers. Journal of ForensicSciences, 51, 147–155. https://doi.org/10.1111/j.1556-4029.2005.00030.x

Monahan, J., Steadman, H., Silver, E., Appelbaum, P.,Robbins, P., Mulvey, E., Roth, L., Grisso, T., &Banks, S. (2001). Rethinking risk assessment: TheMacArthur study of mental disorder and violence.Oxford University Press.

Monahan, J., Steadman, H. J., Appelbaum, P. S.,Grisso, T., Mulvey, E. P., Roth, L. H., Robbins,P. C., Banks, S., & Silver, E. (2006). The classifica-tion of violence risk. Behavioral Sciences & the Law,24(6), 721–730. https://doi.org/10.1002/bsl.725

Mullen, P., James, D., Meloy, J. R., Pathé, M. T.,Farnham, F. R., Preston, L., Darnley, B., & Berman,

J. (2009). The fixated and the pursuit of public figures.Journal of Forensic Psychiatry & Psychology, 20,33–47. https://doi.org/10.1080/14789940802197074

Mullen, P. E., James, D. V., Meloy, J. R., Pathé, M.,Farnham, F., Preston, L., & Darnley, B. (2008). Therole of psychotic illnesses in attacks on publicfigures. In J. R. Meloy, L. Sheridan, & J. Hoffman(Eds.), Stalking, threatening and attacking publicfigures: A psychological and behavioral analysis(pp. 55–82). Oxford University Press. https://doi.org/10.1093/med:psych/9780195326383.003.0003

Mullen, P. E., & Lester, G. (2006). Vexatious litigantsand unusually persistent complainants and peti-tioners: From querulous paranoia to querulousbehaviour. Behavioral Sciences & the Law, 24(3),333–349. https://doi.org/10.1002/bsl.671

Mullen, P. E., & Maack, L. H. (1985). Jealousy,pathological jealousy and aggression. In D. P.Farrington & J. Gunn (Eds.), Aggression and dan-gerousness (pp. 103–126). Wiley.

Muller, E. R., Bron, R. P., de Hoog, D., & van derVarst, L. P. (2010). De aanpak van verwardebedreigers in Nederland [The approach to threatsby people with mental health disorders in TheNetherlands]. COT Instituut voor Veiligheids-enCrisismanagement [COT Institute for Safety andCrisis Management].

Narud, K., & Dahl, A. V. (2015). Stalking experiencesreported by Norwegian members of parliamentcompared to a general population sample. Journalof Forensic Psychiatry & Psychology, 26(1), 116–131. https://doi.org/10.1080/14789949.2014.981564

Nelen, H., Leeuw, B., Bakker, F., & Herrenberg, T.(2012). Pilot Dreigingsmanagement: Een ex anteevaluatie [Pilot threat management: an ex anteevaluation]. WODC, Ministerie van Veiligheid enJustitie [Research and Documentation Centre, Ministryof Justice].

Nelen, H., Mol, M., Plaisier, J., & Peters, M. (2013).Pilot dreigingsmanagement. De implementatie enwijze van uitvoering onder de loep [Threat man-agement pilot. Implementation and performanceunder the microscope]. WODC, Ministerie vanVeiligheid en Justitie [Research and DocumentationCentre, Ministry of Justice].

Nijdam, M. J, Olff, M., de Vries, M., Martens, W. J., &Gersons, B. P. R. (2008). Psychosociale effectenvan dreigingen beveiliging. [Psycho-social effects ofthreats and security]. NCTb [National Coordinatorfor Counter-Terrorism].

Novaco, R. W. (2011). Anger dysregulation: Driver ofviolent offending. Journal of Forensic Psychiatry &Psychology, 22(5), 650–668. https://doi.org/10.1080/14789949.2011.617536

Novaco, R. W. (2013). Reducing anger-related offending:What works. In L. A. Craig, L. Dixon, & T. A. Gannon(Eds.), What works in offender rehabilitation: An

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

22 JAMES ET AL.

evidence-based approach to assessment and treatment(pp. 211–236). Wiley Blackwell. https://doi.org/10.1002/9781118320655.ch12

Pathé, M. (2017). Stalking public figures: The fixatedloner. In W. S. G. Petherick (Ed.), The psychologyof criminal and antisocial behavior (pp. 295–320).Academic Press. https://doi.org/10.1016/B978-0-12-809287-3.00009-2

Pathé, M., Haworth, D., & Lowry, T. (2016). Managingthe risk posed by fixated persons at major events: Ajoint police-mental health intelligence approach.Journal of Policing, Intelligence and Counter Terror-ism, 11(1), 63–72. https://doi.org/10.1080/18335330.2016.1161226

Pathé, M. T., Haworth, D. J., Goodwin, T., Holman,A. G., Amos, S. J., & Day, L. (2017). Establishing ajoint agency response to the threat of lone-actorgrievance-fuelled violence. Journal of ForensicPsychiatry & Psychology, 29(1), 37–52. https://doi.org/10.1080/14789949.2017.1335762

Pathé, M. T., Lowry, T. J., Haworth, D. J., Webster, D.,Winterbourne, P., & Briggs, C. J. (2015). Assessingand managing the threat posed by fixated personsin Australia. Journal of Forensic Psychiatry &Psychology, 26(4), 425–438. https://doi.org/10.1080/14789949.2015.1037332

Pathé, M. T., Lowry, T. J., Haworth, D. J., Winter-bourne, P., & Day, L. (2016). Public figure fixation:Cautionary findings for mental health practitioners.Behavioral Sciences & the Law, 34(5), 681–692.https://doi.org/10.1002/bsl.2252

Régis, E. (1890). Les régicides dans l’histoire et dansle présent [‘Regicides’ in history and the present day].Maloine.

Riddle, F. J., Ferriman, M., Farmer, D. G., & Baillie,K. M. (2019). Towards a national strategy formanaging fixated persons in Australia. The Journalof Psychiatry & Law, 26(3), 457–467. https://doi.org/10.1080/13218719.2018.1506722

Scalora, M. J., Baumgartner, J. V., Hatch-Maillette,M. A., Covell, C. N., Palarea, R. E., Krebs, J. A.,Washington, D. O., Zimmerman, W., & Callaway, D.(2002). Risk factors for approach behavior towardthe U.S. congress. Journal of Threat Assessment,2, 35–55. https://doi.org/10.1300/J177v02n02_03

Scalora, M. J., Baumgartner, J. V., & Plank, G. L.(2003). The relationship of mental illness to targetedcontact behavior toward state government agenciesand officials. Behavioral Sciences & the Law, 21,239–249. https://doi.org/10.1002/bsl.525

Scalora, M. J., Baumgartner, J. V., Zimmerman, W.,Callaway, D., Hatch Maillette, M. A., Covell, C. N.,Palarea, R. E., Krebs, J. A., & Washington, D. O.(2002). An epidemiological assessment of prob-lematic contacts to members of congress. Journalof Forensic Sciences, 47, 1360–1364. https://doi.org/10.1520/JFS15575J

Scalora, M. J., Zimmerman, W. J., & Wells, D. G.(2008). Use of threat assessment for the protectionof the United States Congress. In J. R. Meloy, L.Sheridan, & J. Hoffman (Eds.), Stalking, threat-ening and attacking public figures: A psychologicaland behavioral analysis (pp. 425–434). OxfordUniversity Press. https://doi.org/10.1093/med:psych/9780195326383.003.0019

Schlesinger, L. B. (2006). Celebrity stalking, homi-cide, and suicide: A psychological autopsy. Inter-national Journal of Offender Therapy andComparative Criminology, 50(1), 39–46. https://doi.org/10.1177/0306624X05276461

Schoeneman, K. A., Scalora, M. J., Darrow, C. D.,McLawsen, J. E., Chang, G. H., & Zimmerman,W. J. (2011). Written content indicators of prob-lematic approach behavior toward political officials.Behavioral Sciences & the Law, 29, 284–301.https://doi.org/10.1002/bsl.977

Schoeneman-Morris, K. A., Scalora, M. J., Chang,G. H., Zimmerman, W. J., & Garner, Y. (2007). Acomparison of email versus letter threat contactstoward members of the United States Congress.Journal of Forensic Sciences, 52, 1142–1147.https://doi.org/10.1111/j.1556-4029.2007.00538.x

Shaffer, C. S., Fuller, E. K., & Guy, L. S. (2020).Brief and emerging violence risk assessmentmeasures. In K. S. Douglas & R. K. Otto (Eds.),Handbook of violence risk assessment (2nd ed.,pp. 52–69). Routledge. https://doi.org/10.4324/9781315518374-4

Shrout, P. E. (1998). Measurement reliability andagreement in psychiatry. Statistical Methods inMedical Research, 7, 301–317. https://doi.org/10.1177/096228029800700306

Silva, A. J., Ferrari, M. M., Leong, G. B., & Penny, G.(1998). The dangerousness of persons withdelusional jealousy. The Journal of the AmericanAcademy of Psychiatry and the Law, 26(4),607–623.

Silva, J. A., Harry, B. E., Leong, G. B., & Weinstock, R.(1996). Dangerous delusional misidentificationand homicide. Journal of Forensic Sciences, 41(4),641–644.

Singh, J. P., Desmarais, S. L., Hurducas, C., Arbach-Lucioni, K., Condemarin, C., Dean, K., Doyle, M.,Folino, J. O., Godoy-Cervera, V., Grann, M., Ho,R. M. Y., Large, M. M., Nielsen, L. H., Pham, T. H.,Rebocho, M. F., Reeves, K. A., Rettenberger, M.,de Ruiter, C., Seewald, K., & Otto, R. K. (2014).International perspectives on the practical applicationof violence risk assessment: A global survey of44 countries. International Journal of ForensicMental Health, 13, 193–206. https://doi.org/10.1080/14999013.2014.922141

Singh, J. P., Grann, M., & Fazel, S. (2013). Authorshipbias in violence risk assessment? A systematic review

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 23

and meta-analysis. PLOS ONE, 8, Article e72484.https://doi.org/10.1371/journal.pone.0072484

Sizoo, B. B., & van Nobelen, D. (2021). Effectivenessof interventions by the Threat Management Team ofthe Netherlands Police Agency: A retrospectivestudy. Journal of Threat Assessment and Management,8(1–2), 36–47. https://doi.org/10.1037/tam0000159

Skeem, J., & Monahan, J. (2011). Current directions inviolence risk assessment. Current Directions inPsychological Science, 20(1), 38–42. https://doi.org/10.1177/0963721410397271

Slovic, P., Finucane, M. L., Peters, E., & MacGregor,D. G. (2004). Risk as analysis and risk as feelings:Some thoughts about affect, reason, risk, and ratio-nality. Risk Analysis: An Official Publication of theSociety for Risk Analysis, 24(2), 311–322. https://doi.org/10.1111/j.0272-4332.2004.00433.x

Swanson, J. W., Swartz, M. S., Van Dorn, R. A., Elbogen,E. B., Wagner, H. R., Rosenheck, R. A., Stroup, T. S.,McEvoy, J. P., & Lieberman, J. A. (2006). A nationalstudyof violent behavior inpersons withschizophrenia.Archives of General Psychiatry, 63(5), 490–499.https://doi.org/10.1001/archpsyc.63.5.490

Wallin, L., & Wallin, S. (2014). Politikernas trygghet-sundersökning 2013: Utsatthet och oro för trakas-serier, hot och våld [Politicians security survey 2013:Vulnerability and concern—Harassment, threats andviolence]. National Council for Crime Prevention.

White, S. G., & Meloy, J. R. (2016). The WAVR-21: Astructured professional guide for the workplaceassessment of violence risk (3rd ed.). SpecialisedTraining Services.

Wilson, S., Dempsey, C., Farnham, F., Manze, T., &Taylor, T. (2018). Stalking risks to celebrities andpublic figures. British Journal of Psychiatric Advances,24, 152–160. https://doi.org/10.1192/bja.2017.22

Wilson, S., Farnham, F., Taylor, A., & Taylor, R.(2019). Reflections on working in public-figurethreat management. Medicine, Science and theLaw, 59(4), 275–281. https://doi.org/10.1177/0025802419861168

Wilson, S. P., Pathé, M. T., Farnham, F. R., & James,D. V. (2021). The fixated threat assessment centers.In J. R. Meloy & J. Hoffmann (Eds.), Internationalhandbook of threat assessment (2nd ed., pp. 475–491).Oxford University Press.

Wormith, J. S., Craig, L. A., & Hogue, T. E. (Eds.).(2020). The Wiley handbook of what works inviolence risk management: Theory, researchand practice. Wiley. https://doi.org/10.1002/9781119315933

Received June 22, 2021Revision received November 30, 2021

Accepted December 12, 2021 ▪

ThisdocumentiscopyrightedbytheAmerican

Psychological

Associationoroneofitsallied

publishers.

Thisarticleisintended

solely

forthepersonaluse

oftheindividual

userandisnotto

bedisseminated

broadly.

24 JAMES ET AL.

Ray writers

Order your essay today and save 30% with the discount code ESSAYSHELP