Skip to main content Accessibility help
×
Home

Regulating terrorist content on social media: automation and the rule of law

  • Stuart Macdonald (a1), Sara Giro Correia (a1) and Amy-Louise Watkin (a1)

Abstract

Social-media companies make extensive use of artificial intelligence in their efforts to remove and block terrorist content from their platforms. This paper begins by arguing that, since such efforts amount to an attempt to channel human conduct, they should be regarded as a form of regulation that is subject to rule-of-law principles. The paper then discusses three sets of rule-of-law issues. The first set concerns enforceability. Here, the paper highlights the displacement effects that have resulted from the automated removal and blocking of terrorist content and argues that regard must be had to the whole social-media ecology, as well as to jihadist groups other than the so-called Islamic State and other forms of violent extremism. Since rule by law is only a necessary, and not a sufficient, condition for compliance with rule-of-law values, the paper then goes on to examine two further sets of issues: the clarity with which social-media companies define terrorist content and the adequacy of the processes by which a user may appeal against an account suspension or the blocking or removal of content. The paper concludes by identifying a range of research questions that emerge from the discussion and that together form a promising and timely research agenda to which legal scholarship has much to contribute.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Regulating terrorist content on social media: automation and the rule of law
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Regulating terrorist content on social media: automation and the rule of law
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Regulating terrorist content on social media: automation and the rule of law
      Available formats
      ×

Copyright

Corresponding author

*Corresponding author. E-mail: s.macdonald@swansea.ac.uk

References

Hide All
Alaimo, K (2015) How the Facebook Arabic page ‘We are all Khaled Said’ helped promote the Egyptian revolution. Social Media + Society 1, 110.
Anderson, D (2014) The Terrorism Acts in 2013: Report of the Independent Reviewer on the Operation of the Terrorism Act 2000 and Part 1 of the Terrorism Act 2006. London: The Stationery Office.
Balkin, JM (2017) The three laws of robotics in the age of big data. Ohio State Law Journal 78, 12171241.
Barak, A (2002) Foreword: A judge on judging: the role of the Supreme Court in a democracy. Harvard Law Review 116, 19162.
Berger, JM (2013) Zero degrees of al Qaeda: how Twitter is supercharging jihadist recruitment, Foreign Policy, 14 August.
Berger, JM and Morgan, J (2015) The ISIS Twitter Census: Defining and Describing the Population of ISIS Supporters on Twitter. Washington, DC: The Brookings Institution.
Berger, JM and Perez, H (2016) The Islamic State's Diminishing Returns on Twitter: How Suspensions Are Limiting the Social Networks of English-speaking ISIS Supporters. Washington, DC: George Washington University Program on Extremism.
Bickert, M (2018) Publishing our internal enforcement guidelines and expanding our appeals process, Facebook News, 24 April. Available at https://newsroom.fb.com/news/2018/04/comprehensive-community-standards/ (accessed 22 February 2019).
Bickert, M and Fishman, B (2017) Hard questions: how we counter terrorism, Facebook News, 15 June. Available at https://newsroom.fb.com/news/2017/06/how-we-counter-terrorism/ (accessed 22 February 2019).
Bickert, M and Fishman, B (2018) Hard questions: how effective is technology in keeping terrorists off Facebook?, Facebook News, 23 April. Available at https://newsroom.fb.com/news/2018/04/keeping-terrorists-off-facebook/ (accessed 22 February 2019).
Bloom, M, Tiflati, H and Horgan, J (2017) Navigating ISIS's preferred platform: Telegram, Terrorism and Political Violence. Available at https://doi.org/10.1080/09546553.2017.1339695 (accessed 22 February 2019).
Browne, M (2017) YouTube removes videos showing atrocities in Syria, New York Times, 22 August.
Brownsword, R (2011) Lost in translation: legality, regulatory margins, and technological management. Berkeley Technology Law Journal 26, 13211365.
Brownsword, R (2015) In the year 2061: from law to technological management. Law, Innovation and Technology 7, 151.
Brownsword, R (2016) Technological management and the rule of law. Law, Innovation and Technology 8, 100140.
Carlile, A (2007) The Definition of Terrorism, Cm 7052. London: The Stationery Office.
Chandler, A (2017) The racist algorithm. Michigan Law Review 115, 10231045.
Citron, DK (2008) Technological due process. Washington University Law Review 85, 12491314.
Citron, DK (2018) Extremist speech, compelled conformity, and censorship creep. Notre Dame Law Review 93, 10351071.
Citron, DK and Pasquale, F (2014) The scored society: due process for automated decisions. Washington Law Review 89, 133.
Conway, M et al. (2017) Disrupting Daesh: Measuring Takedown of Online Terrorist Material and Its Impacts. Dublin: VOX-Pol Network of Excellence.
Dan-Cohen, M (1984) Decision rules and conduct rules: on acoustic separation in criminal law. Harvard Law Review 97, 625677.
Deep, Dot Web (2017) After being shut down by Google and GoDaddy, major neo-Nazi site moves to Darknet, Deep Dot Web, 31 August. Available at https://www.deepdotweb.com/2017/08/31/shut-google-godaddy-major-neo-nazi-site-moves-darknet/ (accessed 22 February 2019).
Desai, DR and Kroll, JA (2017) Trust but verify: a guide to algorithms and the law. Harvard Journal of Law & Technology 31, 164.
Edwards, J (2010) Justice denied: the criminal law and the ouster of the courts. Oxford Journal of Legal Studies 30, 725748.
Facebook (2016) Partnering to help curb spread of online terrorist content, Facebook News, 5 December. Available at https://newsroom.fb.com/news/2016/12/partnering-to-help-curb-spread-of-online-terrorist-content/ (accessed 22 February 2019).
Fletcher, G (2006) The indefinable concept of terrorism. Journal of International Criminal Justice 4, 894911.
Gerbaudo, P (2012) Tweets and the Streets: Social Media and Contemporary Activism. London: Pluto Press.
GIFCT (2018) Global Internet Forum to Counter Terrorism: an update on our efforts to use technology, support smaller companies and fund research to fight terrorism online, Global Internet Forum to Counter Terrorism Press, 8 June. Available at https://gifct.org/press (accessed 22 February 2019).
Greenfield, P (2018) Home Office unveils AI program to tackle ISIS online propaganda, The Guardian, 13 February.
Grinnell, D et al. (2018) Who Disseminates Rumiyah? Examining the Relative Influence of Sympathiser and Non-sympathiser Twitter Users, paper presented at the 2nd European Counter Terrorism Centre (ECTC) conference on online terrorist propaganda, 17–18 April 2018, at Europol Headquarters, The Hague. Available at https://www.europol.europa.eu/publications-documents/who-disseminates-rumiyah-examining-relative-influence-of-sympathiser-and-non-sympathiser-twitter-users (accessed 22 February 2019).
Grinnell, D, Macdonald, S and Mair, D (2017) The Response of, and on, Twitter to the Release of Dabiq Issue 15, paper presented at the 1st European Counter Terrorism Centre (ECTC) conference on online terrorist propaganda, 10–11 April 2017, at Europol Headquarters, The Hague. Available at https://www.europol.europa.eu/publications-documents/response-of-and-twitter-to-release-of-dabiq-issue-15 (accessed 22 February 2019).
Hardy, K and Williams, G (2011) What is ‘terrorism’? Assessing domestic legal definitions. UCLA Journal of International Law and Foreign Affairs 16, 77162.
Haynes, J (2017) Backdoor access to WhatsApp? Rudd's call suggests a hazy grasp of encryption, The Guardian, 27 March.
Hildebrandt, M (2008) Legal and technological normativity: more (and less) than twin sisters. Techne 12, 169183.
Hildebrandt, M (2018) Law as computation in the era of artificial legal intelligence: speaking law to the power of statistics. University of Toronto Law Journal 68(Suppl 1), 1235.
HM Government (2015) The Government Response to the Annual Report on the Operation of the Terrorism Acts in 2013 by the Independent Reviewer of Terrorism Legislation, Cm 9032. London: The Stationery Office.
HM Government (2018) CONTEST: The United Kingdom's Strategy for Countering Terrorism, Cm 9608. London: The Stationery Office.
Hodgson, JS and Tadros, V (2013) The impossibility of defining terrorism. New Criminal Law Review 16, 494526.
Home Office (1998) Legislation Against Terrorism: A Consultation Paper, Cm 4178. London: The Stationery Office.
Horder, J (2016) Ashworth's Principles of Criminal Law, 8th edn. Oxford: Oxford University Press.
ICT4Peace and Counter-Terrorism Committee Executive Directorate (2016) Private Sector Engagement in Responding to the Use of the Internet and ICT for Terrorist Purposes: Strengthening Dialogue and Building Trust. Geneva: ICT4Peace.
Jigsaw (2017) The Redirect Method: A Blueprint for Bypassing Extremism. Available at www.redirectmethod.org (accessed 22 February 2019).
Joh, E (2016) The new surveillance discretion: automated suspicion, big data, and policing. Harvard Law & Policy Review 10, 1542.
Kavanagh, C et al. (2017) Terrorist use of the Internet and cyberspace: issues and responses. In Conway, M et al. (eds), Terrorists’ Use of the Internet: Assessment and Response. Amsterdam: IOS Press.
Kayyali, D and Althaibani, R (2017) Vital human rights evidence in Syria is disappearing from YouTube, VOX-Pol Blog, 22 November. Available at http://www.voxpol.eu/vital-human-rights-evidence-syria-disappearing-youtube/ (accessed 22 February 2019).
Kim, PT (2017) Auditing algorithms for discrimination. University of Pennsylvania Law Review Online 166, 189203.
Kroll, JA et al. (2017) Accountable algorithms. University of Pennsylvania Law Review 165, 633705.
Lehr, D and Ohm, P (2017) Playing with the data: what legal scholars should learn about machine learning. University of California, Davis, Law Review 51, 653717.
Levitt, G (1986) Is ‘terrorism’ worth defining? Ohio Northern University Law Review 13, 97116.
Lorenzo-Dus, N and Macdonald, S (2018) Othering the West in the online jihadist propaganda magazines. Inspire and Dabiq: Journal of Language, Aggression and Conflict 6, 79106.
Macdonald, S (2009a) The unbalanced imagery of anti-terrorism policy. Cornell Journal of Law and Public Policy 18, 519540.
Macdonald, S (2009b) Why we should abandon the balance metaphor: a new approach to counterterrorism policy. ILSA Journal of International and Comparative Law 15, 95146
Macdonald, S (2014) Prosecuting suspected terrorists: precursor crimes, intercept evidence and the priority of security. In Jarvis, L and Lister, M (eds), Critical Perspectives on Counter-terrorism. Abingdon: Routledge, pp. 130149.
Macdonald, S and Lorenzo-Dus, N (forthcoming) Purposive and performative persuasion: the linguistic basis for criminalising the (direct and indirect) encouragement of terrorism. In Fuller, C and Finkelstein, C (eds), Using Law to Fight Terror: Legal Approaches to Combating Violent Non-state and State-sponsored Actors. Oxford: Oxford University Press.
Malik, N (2018) Terror in the Dark: How Terrorists Use Encryption, the Darknet, and Cryptocurrencies. London: The Henry Jackson Society.
May, T (2018) Theresa May's Davos address in full, World Economic Forum, 25 January. Available at https://www.weforum.org/agenda/2018/01/theresa-may-davos-address/ (accessed 22 February 2019).
Morozov, E (2013) To Save Everything, Click Here: Technology, Solutionism, and the Urge to Fix Problems that Don't Exist. London: Allen Lane.
Aoláin F, (2018) Mandate of the Special Rapporteur on the promotion and protection of human rights and fundamental freedoms while countering terrorism, OL OTH 46/2018, 24 July. Available at https://www.ohchr.org/Documents/Issues/Terrorism/OL_OTH_46_2018.pdf (accessed 22 February 2019).
Nouri, L (2018) Britain first and Facebook: banned but not solved, Europe Now, 2 October. Available at https://www.europenowjournal.org/2018/10/01/britain-first-banned-on-facebook-but-not-solved/ (accessed 20 March 2019).
O'Callaghan, D et al. (2013) Uncovering the wider structure of extreme right communities spanning popular online networks, WebSci ’13, Proceedings of the 5th Annual ACM Web Science Conference, New York, 276285.
Pasquale, F (2015) The Black Box Society: The Secret Algorithms that Control Money and Information. Cambridge, MA: Harvard University Press.
Pearson, E (2017) Online as the new frontline: affect, gender, and ISIS-take-down on social media. Studies in Conflict & Terrorism. Available at https://doi.org/10.1080/1057610X.2017.1352280 (accessed 22 February 2019).
Posner, EA and Vermeule, A (2007) Terror in the Balance: Security, Liberty, and the Courts. Oxford: Oxford University Press.
Posner, RA (2006) Not a Suicide Pact: The Constitution in a Time of National Emergency. Oxford: Oxford University Press.
Prucha, N (2016) IS and the jihadist information highway: projecting influence and religious identity via telegram. Perspectives on Terrorism 10, 4858.
Ragan, S (2015) After Paris, ISIS moves propaganda machine to Darknet, CSO News, 15 November. Available at https://www.csoonline.com/article/3004648/security-awareness/after-paris-isis-moves-propaganda-machine-to-darknet.html (accessed 22 February 2019).
Rich, M (2013) Should we make crime impossible? Harvard Journal of Law and Public Policy 36, 795848.
Rich, M (2016) Machine learning, automated suspicion algorithms, and the Fourth Amendment. University of Pennsylvania Law Review 164, 871929.
Rosenthal, D (2011) Assessing digital preemption (and the future of law enforcement). New Criminal Law Review 14, 576610.
Roth, A (2016) Trial by machine. Georgetown Law Journal 104, 12451305.
Saul, B (2006) Defining Terrorism in International Law. Oxford: Oxford University Press.
Schwartz, B (2016) Google's Paul Haahr: we don't fully understand RankBrain, Search Engine Roundtable, 8 March. Available at https://www.seroundtable.com/google-dont-understand-rankbrain-21744.html (accessed 22 February 2019).
Shane, S (2017) In ‘watershed moment,’ YouTube blocks extremist cleric's message, The New York Times, 12 November.
Spigelman, JJ (2003) The rule of law and enforcement. University of New South Wales Law Journal 26, 200209.
Stalinsky, S and Sosnow, R (2016) The jihadi cycle on content-sharing web services 2009–2016 and the case of Justpaste.it: favored by ISIS, Al-Qaeda, and other jihadis for posting content and sharing it on Twitter – jihadis move to their own platforms (Manbar, Nashir, Alors.Ninja) but then return to Justpaste.it, MEMRI Inquiry & Analysis Series No 1255, 6 June. Available at https://www.memri.org/reports/jihadi-cycle-content-sharing-web-services-2009-2016-and-case-justpasteit-favored-isis-al (accessed 20 March 2019).
Tadros, V (2007) Justice and terrorism. New Criminal Law Review 10, 658689.
Tech, Against Terrorism (2017) UK launch of tech against terrorism at Chatham House, Tech Against Terrorism, 12 July. Available at https://www.techagainstterrorism.org/2017/07/12/tat-at-chatham-house/ (accessed 22 February 2019).
Twitter (2016) Combating violent extremism, Twitter blog, 5 February. Available at https://blog.twitter.com/official/en_us/a/2016/combating-violent-extremism.html (accessed 22 February 2019).
Twitter (2017) New data, new insights: Twitter's latest #Transparency report, Twitter Public Policy, 19 September. Available at https://blog.twitter.com/official/en_us/topics/company/2017/New-Data-Insights-Twitters-Latest-Transparency-Report.html (accessed 22 February 2019).
Twitter (2018a) Expanding and building #TwitterTransparency, Twitter Public Policy, 5 April. Available at https://blog.twitter.com/official/en_us/topics/company/2018/twitter-transparency-report-12.html (accessed 22 February 2019).
Twitter (2018b) Our range of enforcement options, Twitter Help Center. Available at https://help.twitter.com/en/rules-and-policies/enforcement-options (accessed 22 February 2019).
United Nations Security Council (2017) Fourth Report of the Secretary-General on the Threat Posed by ISIL (Da'esh) to International Peace and Security and the Range of United Nations Efforts in Support of Member States in Countering the Threat, S/2017/97. New York, NY: United Nations.
Waldron, J (2003) Security and liberty: the image of balance. Journal of Political Philosophy 11, 191210.
Wojcicki, S (2017) Expanding our work against abuse of our platform, YouTube Official Blog, 4 December. Available at https://youtube.googleblog.com/2017/12/expanding-our-work-against-abuse-of-our.html (accessed 22 February 2019).
Youmans, WL and York, JC (2012) Social media and the activist toolkit: user agreements, corporate interests, and the information infrastructure of modern social movements. Journal of Communication 62, 315329.
YouTube (2018) Community Guidelines strike basics, YouTube Help. Available at https://support.google.com/youtube/answer/2802032?hl=en-GB (accessed 22 February 2019).
Zedner, L (2005) Securing liberty in the face of terror: reflections from criminal justice. Journal of Law and Society 32, 507533.

Keywords

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed