Analysis: What can we learn from the online response to the Halle terrorist attack?

Below we summarise tech sector responses to the use of their services by the Halle terrorist and sympathisers in the wake of the attack on 9 Oct 2019.

The 9 Oct terrorist attack in Halle, Germany, demonstrates that the tech sector has improved its response in preventing the spread of viral terrorist content - however many of these efforts are undermined by a minority of fringe small platforms and some mainstream media.

Summary

• Since the Christchurch attack great efforts have been made to coordinate tech sector response to content incidents however more work is required to coordinate behaviour across mainstream media, academia, government, and the broader tech industry

• Both smaller platforms and larger platforms were prompt in dealing with the proliferation of the video - it was only circulated widely with little or no moderation on smaller fringe platforms

• Cross-sector initiatives, like the GIFCT hash-sharing consortium and their Content Incident Protocol, are shown to be effective in dealing with crisis events with a risk of content virality by providing smaller platforms with the necessary resources to quickly detect terrorist content. The Terrorist Content Analytics Platform (TCAP) and its alert function will be instrumental in this

• Mainstream media outlets like the Daily Mail immediately published CCTV footage of the event, as well as photos of a victim’s dead body. Neither have been removed

• The response by smaller tech platforms and larger tech platforms to prevent virality are being undermined by both the mainstream media and fringe platforms

How did the attack unfold online?

Before and during the attack

• A post announcing the attack including a manifesto that is allegedly connected to the attacker was initially posted on the image board site Meguca.org.
• The attack itself was live-streamed on Twitch, for about 35 minutes. According to Twitch’s initial report on the attack (Fig. 1), the livestream was watched by only 5 people.

*

*

Smaller platform response

• The livestream footage was automatically uploaded as a video on the Twitch site, which remained up for nearly 6 hours. Within this period, Twitch stated that 2200 people watched the video. Although the video did not surface in any Twitch recommendations or directories, copies had already been downloaded and shared elsewhere on the internet.

• The video and manifesto then spread to other smaller platforms, including Telegram. Telegram is a platform often acting as a beacon in the dissemination cycle of online terrorist content. Researcher Megan Squire found that the video was watched on Telegram by c.15,625 accounts – excluding private channels (Fig. 2). The video is still accessible on Telegram.

*

Figure 2: Megan Squire's analysis of the attacker's video on Telegram

*

• The video and manifesto are also still available on fringe sites Kiwifarms, 4chan, and Bitchute. In the analysis that Tech Against Terrorism carried out on the tech sector’s response to the Christchurch attack, the same platforms were also identified as having failed to respond adequately.
• However, other than this handful of smaller fringe platforms, most smaller tech platforms reacted promptly. We have ensured that various smaller platforms that we have worked with in the past are aware and have taken action, many of whom have now reported expeditious takedowns. Several of these platforms were helped to detect the video by cross-sector initiatives with larger platforms.

*

Cross-sector and large platform response 

• Edited versions of the video were originally shared on Twitter throughout the day on Wednesday.
• Twitch said that after they removed the video, they shared the hash with the GIFCT’s hash sharing consortium to help prevent the proliferation of this content. Twitch is a member of the GIFCT via its parent company, Amazon.
• The GIFCT immediately activated their Content Incident Protocol (CIP),  stating that they were “actively removing perpetrator-created content related to the attack, hashing it and placing the hashes into a shared database for Hash Sharing Consortium members, to prevent its viral spread across our services.”

What is currently being done to prevent attacks like this from going viral?

The GIFCT’s CIP is one of a number of cross-sector iniatives and protocols that are being developed to ensure that content does not reach similar levels of virality post Christchurch, including those led by Tech Against Terrorism, GIFCT, Europol, and the Christchurch Call to Action.

To support smaller platforms in identifying terrorist content in near real-time, Tech Against Terrorism is developing the Terrorist Content Analytics Platform (TCAP) with the support of Public Safety Canada.  The TCAP will be a secure online platform that hosts terrorist material including verified terrorist content (imagery, video, PDFs, URLs, audio) collected from open-sources and existing datasets. The purpose of the TCAP is to facilitate secure information sharing between platforms, academia, and data scientists. As well as archiving historical content to support academic analysis and the development of improved content classifiers, the TCAP will provide a real-time alert service to inform smaller internet platforms of public content discovered on their services.

Last month, Tech Against Terrorism, along with govenrments and tech platforms, was invited to contribute to a table-top exercise organised by the EU Internet Referral Unit under the auspices of the EU Internet Forum at Europol’s headquarters in the Hague. The exercise aimed to help inform the development of a coordinated continent-wide response to the dissemination of viral online terrorist content connected to terrorist attacks.

Europol and EU Home Affairs is developing a “Response Protocol to Online Crises” by engaging with Member States, law enforcement, the tech industry and Tech Against Terrorism, which will allow Member States and online platforms to respond rapidly and in a coordinated manner to the dissemination of terrorist content online in the event of a terrorist attack.

A response protocol is also being drafted under the Christchurch Call to Action, which was initiated by the Governments of New Zealand and France in the aftermath of the Christchurch attack. The Christchurch Call is an action plan that commits government and tech companies to a range of measures, including developing tools to prevent the  upload of terrorist and violent extremist content; countering the roots of violent extremism; increasing transparency around the removal and detection of content, and reviewing how companies’ algorithms  direct users to violent extremist content.

Mainstream media response

Despite efforts from many parts of the global tech industry, some elements of mainstream media, such as the Daily Mail, have shared CCTV footage and pictures of dead victims. When tackling terrorist use of the internet it is important to consider mainstream media’s role - with the Christchurch attack, analysis carried out by Tech Against Terrorism found that some British and Australian tabloids were instrumental in amplifying the virality of the attack video and manifesto.

Adam Hadley, Director at Tech Against Terrorism, said:

"It is disappointing to see that UK media is again sharing terrorist content, primarily because this undermines the work that is being done by the tech sector. 

In this instance, Twitch acted quickly to remove the stream and the recording. Through collaboration with the Global Internet Forum to Counter Terrorism (GIFCT), the video was hashed and put into the GIFCT's hash-sharing consortium which seems to have prevented the virality of the video.

Since Christchurch, Tech Against Terrorism, GIFCT, Europol, and the Christchurch Call to Action have been developing protocols to ensure that content does not reach similar levels of virality in the case of another attack. Some positive effects of these efforts were seen in the aftermath of the Halle attack, however they were undermined by the decisions made by UK media to actively publish content that glamorises terrorist violence.

This event together with the Christchurch attack, where UK media was instrumental in ensuring virality of the attack video and manifesto, suggests that there is a need to reform editorial guidelines in the UK."

Tech Against Terrorism in the news

Adam Hadley also appeared in an interview with France24 the following day (Fig. 3), speaking about the attack in Halle and how the tech sector's "Content Incident Protocol" minimised re-uploads of the video. He also noted that there is much more to be done to support smaller platforms and discourage mainstream media sensationalisation. You can watch the whole interview here.

Adam Hadley was also quoted in the following articles covering the tech sector's response to the attack:

• Vice News, "The German Synagogue Shooter's Twitch Video Didn't Go Viral. Here's Why", 11 October
• MIT Technology Review, "Germany's Synagogue Shooting Was Live-Streamed on Twitch - But Almost No One Saw It", 10 October

For any media enquiries, please get in touch on contact@techagainstterrorism.org

**N.B. This post was edited on Jan 3 2020 to amend two initial findings in an earlier version, including: that the manifesto was first posted on German image board site Kohlchan; and, that the video remained on Twitch for close to 30 minutes. These findings were initially alleged by the ICSR and Twitch respectively, however, both corrected these statements at a later date with the updated and accurate facts.