2 min read

Blog Post: Understanding Hash List Sizes

Combating terrorist and violent extremist content (TVEC) online requires both sophisticated technical solutions and robust collaborative approaches.

At Tech Against Terrorism (TAT), we contribute to this effort through a range of initiatives and partnerships, one of which is our Content Hash List. We maintain a carefully curated hash list to help platforms identify and remove TVEC. This list is powered by the TCAP Archive, the world’s first verified archive of TVEC, which currently holds 53,990 files and has enabled us to hash 213,512 pieces of content. We use a combination of manual and ML-based automated classification to ensure that our hashes are accurate and actionable. This verification process is crucial given the complex nature of TVEC, where context often determines whether content violates platform policies or laws. 

While our database and associated hash-sharing capabilities have enabled us to make a meaningful impact in countering terrorist use of the internet, the broader use of hash-sharing in the counterterrorism space remains relatively underdeveloped. Existing databases are comparatively small, especially when contrasted with similar efforts to combat other forms of online harm, such as Child Sexual Abuse Material (CSAM), where databases often contain millions of hashes. This disparity reflects important and distinct challenges inherent to addressing terrorist content online. 

The Nature of Terrorist Content 

Terrorist content is highly variable and context-dependent. Terrorist groups release content irregularly and through controlled channels, making it more challenging to identify and hash systematically. This variability requires a more nuanced and targeted approach to verification and classification. 

Historical Context and Infrastructure 

Efforts to counter other forms of online harm have benefited from decades of established legal frameworks, enforcement mechanisms, and sustained institutional investment. While we commend this pioneering work and continue to learn from their experiences and best practices, an equivalent coordinated response to TVEC remains in its early stages. As such, we are actively working to build comparable capabilities through sustained innovation and collaboration. 

Looking Forward: Unified Collaborative Efforts 

As we continue to expand our database, we remain committed to maintaining quality and accuracy. We are actively working to enhance both our historic and future hash collection capabilities and are pursuing partnerships with other hash list providers to strengthen our collective fight against Terrorist and Violent Extremist Content (TVEC). Our focus remains on providing platforms with reliable, verified hashes that serve as a critical resource in content moderation efforts against online terrorism. 

We firmly believe that combating TVEC requires a unified, collaborative approach. Researchers, counter-terrorism organisations, law enforcement agencies, and tech platforms should work together openly, sharing information and resources to tackle this global threat effectively. Withholding crucial information for commercial gain undermines the collective security interests we all share. Genuine collaboration and transparent information sharing are vital for advancing the prevention of terrorist content online. With increased resources and continued collaboration with established organisations, we are committed to expanding our impact and maintaining the high standards that ensure our hash list remains a valuable tool in this ongoing effort. 

Microsoft and Tech Against Terrorism Partner to Leverage Latest AI Technologies to Improve Terrorist and Violent Extremist Content Detection

Microsoft and Tech Against Terrorism Partner to Leverage Latest AI Technologies to Improve Terrorist and Violent Extremist Content Detection

Tech Against Terrorism and Microsoft are partnering on the development of an AI-powered tool for detecting terrorist or violent extremist content...

Read More
Tech Against Terrorism alert tech platforms to online terrorist and violent extremist content from Israel-Palestine

Tech Against Terrorism alert tech platforms to online terrorist and violent extremist content from Israel-Palestine

In response to the continuing violence in Israel and Gaza since Saturday, 7 October 2023, Tech Against Terrorism has monitored online threats to life...

Read More

Public consultation on the Terrorist Content Analytics Platform (TCAP)

We are seeking feedback from tech companies, civil society, and academics. Deadline for submission is 23:59 (GMT) on 2 December 2019.

Read More