By Chelsea Quilling

Abstract

In the digital age, new technologies and advancements in computing power have transformed the nature of potentially relevant evidence of atrocities evaluated in international criminal law. The International Criminal Court is presently underprepared to meet the challenges of authenticating digital evidence. This paper outlines the challenges and dangers of the ICC’s current approach to digital evidence authentication and verification, explores the debate among scholars over the analysis of scientific evidence as an analogous problem, and identifies policy recommendations for improving the Court’s capacity and capability to authenticate digital evidence.

---

Introduction 

In the digital age, new technologies and advancements in computing power have transformed how potentially relevant evidence of atrocities will be evaluated in international criminal law. New technologies make possible the aggregation of data related to armed conflict from a rich array of sources including, but not limited to, satellite and geospatial imagery, Global Positioning Systems (GPS), mobile phone data, videos, photographs, social media, and other Internet sources (Koenig et al. 2021, 14-21). The International Criminal Court (ICC or “the Court”) is presently underprepared for the increasingly central challenges of authenticating digital evidence and has struggled to adapt to these changes.

Established in 1998, the ICC is the world’s first permanent international criminal court, charged with investigating and prosecuting the gravest crimes of concern to the international community: genocide, crimes against humanity, and war crimes, and the crime of aggression (ICC 2022). At the time of its formation, it may have been impossible for the ICC to foresee the coming revolution in digital technologies and its effects on the Court. However, over two decades later, the Court’s evidentiary procedures and judicial review processes for authenticating evidence are ill-suited for the modern era. While new technology holds great promise in transforming the judicial process related to international crimes in many ways, the Court is presently underprepared to undertake the complex task of authenticating and verifying digital evidence (Koenig and Murray 2020, 8).

International courts like the ICC have long struggled to evaluate complex domains outside its area of expertise including, inter alia, forensics, ballistics, DNA, and more (HRC UC Berkeley 2012). The proper degree of judicial review that should be exercised in these kinds of cases remains a salient issue. Debates regarding the evaluation of scientific evidence serve as a useful analogous starting point to develop the appropriate judicial review framework for the authentication of digital evidence. The aim of this paper therefore is to: (1) outline the challenges and dangers of the ICC’s current approach to digital evidence authentication and verification; (2) explore the debate among scholars over the analysis of scientific evidence as an analogous problem; and (3) identify the most pragmatic judicial review approach for digital evidence authentication moving forward.

Scope

While there are many significant challenges in bringing digital evidence before the Court, most notably in the collection and preservation of digital evidence, the paper’s core focus will be on the authentication of digital evidence. The Office of the Prosecutor (OTP) has its own manifold problems with its processes of collecting, analyzing, and preserving digital evidence. However, the OTP’s investigative activities are not the focus of this paper. Instead, this paper will examine the pragmatic concerns over how the Court’s Chambers should begin to approach the authenticity and verification of the digital evidence. This paper focuses on pragmatic concerns for two main reasons. First, as will be explained in detail in subsequent sections, there are immediate and serious remedies the Court can and should take to improve its procedures for authenticating digital files. Second, while the ICC has not yet had to decide a case where digital evidence plays a central role, it will undoubtedly in the near future.

While new technology holds great promise in transforming the judicial process related to international crimes in many ways, the Court is presently underprepared to undertake the complex task of authenticating and verifying digital evidence.

Definitions 

Before proceeding, it is important to clarify the terms that will be presented in subsequent sections. 

Authentication is both a cybersecurity and legal term used to describe the process of proving that a digital file is genuine or not forged. In essence, authentication ensures the item in question is what it purports to be and has not been manipulated or altered (Dubberley, Koenig, and Murray 2020, 10).

Verification is the process of ensuring that a claim, or assertion statement in some medium of communication, is reliable and/or true (Dubberley, Koenig, and Murray 2020, 10). 

Verification and authentication are highly interrelated, but distinct. For example, it may be possible that an authentic digital file is either truthful or false; however, an inauthentic piece of evidence should be assumed presumptively unverifiable because that digital file has been found to be forged, manipulated, or altered.

The ICC's Approach to Evidence 

At the ICC, judges are given wide discretion to admit evidence as they see fit. The Rome Statute (the Statute) is the ICC’s founding treaty and serves as the Court’s guiding legal instrument. The adjoining Rules of Procedure and Evidence Rules of Procedure and Evidence (RPE or “The Rules”) offer additional specificity regarding the admission and handling of evidence. Rule 63(2) of the Rules states: “A Chamber shall have the authority, in accordance with the discretion described in Article 64, paragraph 9, to assess freely all evidence submitted in order to determine its relevance or admissibility in accordance with Article 69.” [1] In other words, judges hold the power to decide on any issues that arise in either the authenticity or verifiability of digital evidence. The Court relies heavily on the discretion and expertise of judges to appropriately weigh admitted evidence. This flexible approach to the admission of evidence is also a product of the limitations of the ICC’s authority. Unlike domestic criminal investigations where law enforcement has authority to compel parties through subpoenas and search warrants, ICC investigative teams hold no such authority (Freeman 2020, 57). 

The ICC’s general approach to admissibility of evidence involves a sequential, three-part test (Mehandru and Koenig n.d.) in which each of the following criteria must be met:

Relevance: Under Articles 64(9)(a) and 69(4) of the Rome Statute, and further articulated in the adjoining Rules of Procedure and Evidence, evidence will be considered relevant if the “evidence tendered makes the existence of a fact at issue more or less probable.” [2] In other words, evidence may be considered relevant if it is prima facie relevant to the case. [3] 

Probative value: Probative value is generally understood as whether an exhibit is sufficiently useful in proving an important part of the trial (Wex, n.d.). In essence, probative value measures the extent to which the proffered evidence is likely to influence a determination of a fact or issue (Mehandru and Koenig, n.d.). Under Article 69(4) of the Statute, the probative value of an item must be assessed before it can be admitted into evidence. [4] There is no exhaustive list of criteria under which probative value is assessed. Instead, “probative value is determined on the basis of a number of considerations pertaining to the inherent characteristics of the evidence.” [5] The Court must therefore balance the probative value of the item against its prejudicial effect to accused parties. [6]

Weighing probative value against prejudicial effect: Pursuant to Rules 69(4) and 63(2), evidence presented must be “sufficiently relevant and probative to outweigh any prejudicial impact or effect its admission might cause”. [7] In other words, the weight afforded to the evidence must fully respect the rights of all parties and not be obviously unfair to either the prosecution or defense or be prejudicial to the overall fairness of the trial (Krzan 2021, 171). 

The focus of this paper is on the mechanisms and challenges of authenticating digital evidence and the appropriate processes for judicial review. However, concepts of verification and authenticity have the potential to become entangled as judges begin deciding on the admissibility, relevance, probative value, and weight of digital evidence. 

One key reason for the convergence of authenticity and verification is the ICC’s flexible approach to the admissibility of evidence. While the Rome Statute attempts to balance civil (codified legal code) and common law (uncodified and reliant on legal precedent) traditions in its conceptualization of fair trial and due process, the Court leans heavily on the civil law system’s approach to evidentiary procedure (Jackson and Summers 2012, 33-39). Civil legal tradition, and by extension the ICC, favors a system of the “utmost flexibility” with very few restrictions on the kind of evidence that may be admitted (Krzan 2021, 171). In concordance with civil legal procedure, there is no jury system at the ICC. In contrast, a common law tradition relies on a jury system wherein jury members are not expected to be experts in legal code, meaning there is good reason to restrict the type of evidence admitted to the proceedings for fear that some may be overly prejudicial to the accused party or parties (Jackson and Summers 2012, 39). This is not the approach taken by the ICC. Instead, parties submit proposed evidence to the Chamber and then the judge considers the relevance and admissibility of the evidence. Once evidence is admitted, a judge is responsible for appropriately weighing the evidence, establishing the facts of the case, and applying the appropriate legal code (Apple and Deyling 1995, 27).

In practice, this flexible admissibility standard means that most evidence even remotely relevant to the case will likely be admitted (Krzan 2021, 172). For example, in Prosecutor v. Jean-Pierre Bemba Gombo, the OTP tendered ten audio recordings of radio broadcasts to establish the background and context of the conflict. When the defense raised objections to the admission of these recordings, the Chamber ruled that “recordings that have not been authenticated in court can still be admitted, as in-court authentication is but one factor for the Chamber to consider when determining an item’s authenticity and probative value.” [8] However, a determination of the admissibility of any evidence has no bearing on the evidentiary weight it is afforded by the Chamber. [9] Evidentiary weight refers to the relative importance attached to a piece of admitted evidence in making a determination about whether a certain issue has been proven or not. Therefore, unlike probative value, the weight of evidence is assessed by judges at the end of a trial, after having heard the totality of the evidence admitted in the case. [10] Given the Court’s flexible approach to evidence, determining authenticity of evidence is ultimately left to the discretion of judges. This becomes problematic in the evaluation of digital evidence, as discussed in detail in subsequent sections. 

In Prosecutor v. Jean-Pierre Bemba Gombo, the ICC affirmed that judges do not have a strict requirement or responsibility to rule separately on the authenticity of evidence presented. [11] The core reasoning given by the Chamber in that case was the facilitation of a fair and speedy trial, as required by Article 64(2) of the Statute. [12] If the tendered items, on their face, are sufficiently authentic or reliable, that is enough for the item to be admitted (Ashouri, Bowers, and Warden 2014, 4). [13] However, where disagreements between parties arise, the authenticity of the evidence must be examined (Ashouri, Bowers, and Warden 2014, 5). 

Article 52 of the Statute created the Regulations of the Court to articulate the components necessary for the “routine functioning of the Court.” [14] Under Regulation 26, the Court directed the establishment of “a reliable, secure, efficient electronic system which supports its daily judicial and operational management and its proceedings.” [15] Thereafter, the Unified Technical Protocol or (“E-Court Protocol” or “Protocol”) was created to serve as the technical protocols and means for determining the authenticity of digital evidence (Freeman and Vazquez Llorente, 2021).

E-Court Protocol 

Most of the E-Court Protocol is a relatively standard description of naming conventions and procedures for preparing and submitting documents in the Court’s electronic system. However, the Protocol’s mandated authentication methods warrant careful examination. 

The e-Court Protocol requires all digital files uploaded to the electronic system to be assigned a digital signature that “may be used to verify authenticity of evidence, if authenticity is challenged.” [16] A digital signature is a “mathematical algorithm routinely used to validate the authenticity and integrity of a message” (CISA, n.d.). Digital signatures are unique to a particular individual or entity and are used to protect and securely authenticate the origin, integrity, and signatory non-repudiation of a digital file. The e-Court Protocol requires tendering parties to authenticate files with a digital signatures hashing algorithm called MD5. [17] While it is generally good security practice for the Court to require digital signatures for authentication purposes, the use of MD5 should be seen as a problem of immediate and serious concern for the ICC. 

MD5 is a hashing function program originally created in 1992. Every digital signature generates a “hash function,” or a string of numbers and letters generated by the algorithm unique to the file or document. A hash is a one-way function—meaning that the process that created the hash cannot be reversed to find other files that generate the same hash value (CISA, n.d.). MD5 uses a 128-bit “digital fingerprint” to create this one-way hash, which, by modern standards, is a relatively low number of bits. MD5 fails one of the basic requirements of any cryptographic hash function—that it should be computationally impossible to find two distinct files with the same hash value (Wang and Yu 2005, 19-20). This phenomenon, where multiple files have matching hash functions, is known as collision (Wang and Yu 2005, 19-21). Because of its known collision vulnerability, by 2008, MD5 was internationally recognized as cryptographically broken (Dougherty, 2009).

In 2011, the Internet Engineering Task Force (IETF), the premier international internet protocol organization, warned all computer users that “MD5 is no longer acceptable where collision resistance is required such as digital signatures” (IETF 2011). In 2012, collision vulnerabilities in MD5 were exploited on a massive scale with a sophisticated malware called Flame, which at the time was considered “one of the most complex threats ever discovered” (Lee 2012). Flame, widely believed to be an Iranian state-sponsored attack on its own people, infected networks in Iran, Israel, Sudan, Syria, Lebanon, Saudi Arabia, and Egypt, allowing hackers to record audio, take screenshots, control keystrokes, and open sensitive files (Lee 2012). Flame exploited MD5’s weak cryptography and deceived infected computers into believing that the malware was presenting a valid digital signature. Flame’s collision attack renewed pleas from researchers to stop using MD5 for authenticating digital signatures for any purpose. In summary, MD5 is dangerously outdated and cannot be trusted to fulfill the basic function of its intended use—to authenticate digital files (Wang and Yu, 2005).

The implications of MD5’s insecurity to the ICC cannot be understated. MD5 makes the ICC’s store of digital evidence needlessly insecure. The ICC’s failure to update its broken cryptography is akin to putting a broken lock on gates to evidence of the world’s worst crimes. The data in the ICC’s electronic systems are highly sensitive and may implicate state actors who are adversarial to the Court (Freeman 2020, 57). The use of MD5 not only makes authentication of digital evidence technically impossible, but also undermines the legitimacy of the Court and the professionalism of its judicial process. Further, any party wishing to challenge the authenticity of any digital evidence submitted to the Court could point to MD5’s well-publicized and well-known insecurity and immediately dismiss the evidence as inauthentic, effectively negating the tremendous amount of time and energy expended by the OTP and the defense to collect, protect, and submit evidence to the Court. 

Replacing MD5 with a more reliable cryptographic program would mitigate the immediate security vulnerability in the system. However, removing and replacing MD5 with more robust cryptographic standards requires both cryptographic agility and interoperability (National Academies of Sciences, 2017). Cryptographic agility describes the ability for machines to add new cryptographic algorithms or features to existing hardware or software and to efficiently retire vulnerable or obsolete cryptographic systems (The National Academies Press, 2017). Interoperability describes the ability to communicate and exchange information between different systems (The National Academies Press, 2017). This is not a straightforward task for all parties. Some legacy machines and systems may not support security updates, or users may be unwilling to pay for security upgrades. The Court must also be mindful of the time and expense of updating security protocols for less technologically advanced parties. For instance, the strength of more favored cryptography may slow down older machines, making the e-Court filing system less accessible to less technologically advanced users (The National Academies Press, 2017). Therefore, any updates to the e-Court protocol must carefully balance the logistical and security considerations of all tendering parties. 

The Court’s failure to update its outdated and insecure authentication procedures demonstrates that the Court is still woefully underprepared for the authentication challenges it faces in the digital age. This undermines the ICC’s credibility and capacity to handle future cases in which digital evidence is guaranteed to play a central role.

The development of a system that balances cryptographic agility and interoperability is a necessary step toward a more secure electronic filing system. However, it does not solve the problem of how the Court should handle all documents and files authenticated with MD5 for pending cases. There may only be two possible mitigation strategies available for such cases. The Court could require all parties to re-submit every piece of digital evidence under new, more secure hashing algorithm(s). This would be a painstakingly time-consuming process for parties and may violate Article 64(2) of the Statute, which provides: “the Trial Chamber shall ensure that the trial is … expeditious.” [18] To put this monumental task into perspective, in 2011 in response to the Prosecution’s request for an amendment to the e-Court Protocol in Prosecutor v. Callixte Mbarushimana, the Court noted that any changes would affect “hundreds of thousands of electronic items collected” that were relevant to the case. [19] The volume of evidence necessary to prove the facts of the case continues to grow, increasing the enormity of the task of resubmission (Freeman and Vazquez Llorente 2021, 170). The second option, a situation in which every digital file for every case authenticated with MD5 is presumed inauthentic, is far worse. While the Court may have been forgiven for requiring the use of MD5 had it been one of the secure industry standards at the time of its adoption, this was not the case. At the time digital signatures requirements were added to the e-Court Protocol around 2012, the security vulnerabilities of MD5 were widely known (IETF 2011). Since 2012, the Court has revised the e-Court protocol several times—most recently in 2019—and still has not retired MD5 or updated its digital signatures protocols. The Court’s failure to update its outdated and insecure authentication procedures demonstrates that the Court is still woefully underprepared for the authentication challenges it faces in the digital age. This undermines the ICC’s credibility and capacity to handle future cases in which digital evidence is guaranteed to play a central role.

Why Authentication and Verification Matter 

Unlike traditional forms of physical, documentary, and testimonial evidence, digital evidence is far more susceptible to compromise, forgery, alteration, manipulation, and deletion (Freeman and Vazquez Llorente 2021, 171). While forgery may be an issue in physical, documentary, and testimonial evidence, most digital evidence today is born-digital, meaning there may be no analog document or hardware to refer to in determining authenticity. In the trial against Congolese warlord Thomas Lubanga, a video recording between 2002 and 2003 was used to show that children visibly under age 15 were recruited as soldiers and military bodyguards (Ng 2020, 143). Investigators were able to support the timeframe of the video based on the specific formation of VHS tape on which the videos were recorded (Ng 2020, 143). This kind of authentication and verification is not possible for videos uploaded online and posted to social media. Further, the information age has enabled a polluted information ecosystem — one in which disinformation (purposely spreading misleading or false information) and misinformation (unintentional amplification of disinformation) are commonplace (Freeman and Vazquez Llorente 2021, 167). These complexities hold true in the data generated in an armed conflict, where rapidly changing circumstances make authenticating and verifying information extremely difficult.

Polluted Information Environment 

In an armed conflict, belligerent parties have leveraged false claims, disinformation, and misinformation to their advantage by misattributing attacks, manipulating digital images or videos, and purposefully distorting the context of images, video, or speech in ambiguous or misleading ways. For instance, in the recent Tigrayan conflict in Ethiopia, parties manipulated images by adding flags of other countries to still images, adding weapons systems to images, and posting old images to social media from other conflicts and attributing them to adversarial actors (BBC News, 2020). In 2015, investigative journalism group Bellingcat was able to prove Russia’s Ministry of Defense had manipulated geospatial imaging related to the downing of Malaysian Airlines flight MH17 including altering the terrain, removing the presence of Russian military vehicles, and obscuring important features of the airplane crash site with fake clouds (Higgins 2015). Further, in 2014, an online video called “Syrian Hero Boy” went viral, purportedly portraying a young Syrian boy rescuing a girl who was trapped during active shooting in the Syrian civil war. The video was shared by several major media outlets, including the BBC, and became further amplified on social media. However, the video was later revealed to have been made on the set of the film Gladiator by a Norwegian film director (McDermott, Koenig, and Murray 2021, 103). 

The susceptibility to the amplification of false information by reputable news outlets is a significant challenge for the ICC. In early stages of a trial, judges are open to hearing evidence from media reports from outlets with perceived legitimacy like the BBC and New York Times to set the context or help establish the integrity of other evidence. While judges may weigh the probative value of media reports at the “bottom of the value-chain,” these outlets help shape the dominant narrative related to international crimes, running the risk of entrenching potentially misleading or false information in eyewitnesses, investigators, and judges (McDermott, Murray, and Koenig 2019, and Freeman 2020, 56). Wrong facts can have serious consequences in the legal sphere. In criminal proceedings, bad information could lead to wrongful convictions, failure to convict perpetrators of grave crimes, and other serious miscarriages of justice (Freeman 2020, 64). Given the vast amount of potentially relevant data generated over the course of an armed conflict in the digital age, it has never been more important or difficult to distinguish what information is real and what is fake.

Temporal Challenges 

Another significant challenge to authenticating and verifying digital evidence is the temporal scope of many international criminal cases. Most international criminal trials are not tried until years or decades after the initial digital evidence is captured. This means the risks of data loss and/or manipulation increase significantly over time (Koenig and Murray 2020, 144). Brewster Kahle, founder of Internet Archive, estimates the average lifespan of a webpage is ninety-two days (Carlson 2017). Hyperlinks can “rot,” web pages are taken down, hardware malfunctions or becomes obsolete, and information may be lost forever (Carlson 2017). Content on social media platforms is always at risk of being taken down by machine learning algorithms for violations of terms of service, especially in documenting serious human rights violations, which may contain graphic or violent content (Murray and Koenig 2020, 143–44). In a video to the New York Times, Syrian activist Hadi Al Khatib pleaded with platforms like YouTube and Facebook to improve their content moderation systems after ten percent of the archive documenting the Syrian civil war’s violence, amounting to tens of thousands of videos, was lost to YouTube’s automated takedowns in 2017 (Khatib and Kayyali 2019). The risks of data loss diminish the ability of the ICC to fulfill its core judicial functions. Without access to critical evidence, the Court runs the risk of enabling premature acquittals, false accusations, or other missed opportunities for accountability and justice. 

Using cryptography to authenticate digital evidence is also not a static task, as industry standards for authentication today could quickly become obsolete in the future (Moriarty 2021). Encryption algorithms are bound to the technological sophistication and known attack vectors at the time of their writing, meaning that, over time, these algorithms can become compromised in new ways, unsupported by the vendor, or otherwise insecure (Murrow 2020). As discussed in detail in previous sections, the ICC is currently using an encryption algorithm called MD5 that was at one time widely used but has since become dangerously outdated and vulnerable (Wang and Yu 2005).

Verification of digital evidence also requires an understanding of how cultural and social context changes over time. A poignant example of how central contextual understanding can be is the International Criminal Tribunal for Rwanda (ICTR)’s prosecution of Prosecutor v. Jean-Paul Akayesu regarding the Rwandan genocide. [20] In the case’s proceedings, the ICTR’s Chamber noted that Rwandan witnesses were often reluctant or unwilling to plainly affirm that the word Inyenzi in the official local Kinyarwanda language meant “cockroach.” [21] Linguistic experts helped the Court understand that Inyenzi, whose ordinary meaning is “cockroach,” had morphed over time to take on an abusive connotation, attaching the word “cockroach” to all Tutsi people. [22] Without the context to know that this word had been weaponized to incite genocide against the Tutsis in the 1990s, the Chamber would have missed a critical component of the case. In the digital age, the rapid speed at which online norms and coded language evolve means that the social and temporal context of digital content that is potentially relevant to an ICC case could be easily missed or misunderstood. 

Memes also play an important role in communicating and creating cultural narratives around sensitive geopolitical issues and, importantly, about war. For instance, in early 2020, after the United States assassinated Qassem Soleimani, memes expressing anxiety about a potential war between Iran and the United States and prospects of a military draft spread quickly across social media (Romano 2020). Social media researcher and professor at Michigan State University Saleem Alhabash told Vox in a 2020 interview, “[I]n any kind of political tension, whether it is local, regional, national or global, social media is part of the warfare. And this is something to look for in any future crisis” (Romano 2020). On social media platforms, the growing phenomenon of “algospeak” is changing the meaning of ordinary words and how users communicate about sensitive or violent issues. Coined by journalist Taylor Lorenz, “algospeak” refers to code words or phrases users have adopted to circumvent content moderation algorithms (Lorenz 2022). Users, particularly on TikTok, commonly use words like “unalive,” instead of dead, or “SA,” instead of sexual assault, to avoid content being removed or downranked (Lorenz 2022). Evolutions in language surrounding sensitive or violent online content are highly significant to the Court. These changes could have a major impact on judges’ ability to make full and accurate assessments of the veracity of digital evidence (Cole 2015). If judges are unable to verify the claims of digital content because they do not understand the meaning or context of the evidence, they could miss significant opportunities for justice and accountability. 

Lastly, technological leaps in computational power and data storage will undoubtedly continue. The World Economic Forum estimates that by 2025, the world will generate roughly one billion gigabytes each day (Desjardins 2019). To put that into perspective, the totality of evidence from the International Criminal Tribunal for the Former Yugoslavia (ICTY) generated an unparalleled 8 million pages of documents related to the conflict (ICTY - UNICRI 2009). In 2017, German reporters published a series of stories related to the “Paradise Papers” based on more than 13.4 million documents (Freeman and Vazquez Llorente 2021, 170). In just eight weeks of the 2022 war in Ukraine, archivists at the nonprofit Mnemonic have collected and verified more than 500,000 videos alone (Lovett and Ojewska 2022). The vast volume of data created in the digital age creates significant challenges for ICC investigators, who need to collect, authenticate, verify, and preserve potentially relevant data that is highly impermanent and vulnerable. Given this monumental task facing the OTP, it is vital that the Court do all it can to support the investigative efforts of the OTP by ensuring the integrity and security of digital evidence.

The Rise of Open Source Investigations 

Another significant change to digital evidence authentication is the burgeoning practice of open source investigations. Open source information is “publicly available information that any member of the public can observe, purchase or request without requiring special legal status or unauthorized access” (HRC UC Berkeley and UN OHCHR 2020, 6). Open source investigations leverage open source information to perform critical information and gather potential evidence (HRC UC Berkeley and UN OHCHR 2020, 8). 

Traditionally, the OPT has relied on open source information in a particular conflict from reports compiled by international and nongovernment organizations (NGOs). However, the Court has become increasingly critical of the OTP’s overreliance on NGO reports, which judges contend lack sufficient probative value (Freeman 2020, 54). Frustrations boiled over in Prosecutor v. Gbagbo, when investigators submitted NGO, UN, and news reports based on a significant amount of anonymous hearsay without taking any additional investigative steps to corroborate their authenticity or veracity (Freeman 2020, 54). The Chamber noted that, while these reports and news articles help establish context, “such pieces of evidence cannot in any way be presented as the fruits of a full and proper investigation.” [23]

Open source investigations therefore offer significant promise toward more fruitful information and evidence collection. Open source investigations allow a variety of actors from civil society to play an active role in the collection, investigation, and analysis of digital evidence. This opens new doors for justice to victims of mass atrocities with innovative methods and means of collecting evidence and corroborating witness testimony (Cole 2015).In 2015, researchers from Amnesty International used Google Earth satellite imagery to pinpoint the exact location of a mass grave in Burundi captured by a user-generated video and further corroborated by eyewitness testimony. This evidence was able to counter official government narratives downplaying or denying extrajudicial state killings—a feat that would have been impossible without open source information (Koettl, Murray, and Dubberley 2020, 24). 

The benefits of new methods and sources of open source investigation are clear. In recent years, there have been encouraging steps taken toward professionalizing open source investigations for documenting human rights violations. In 2017, the Atlantic Council developed innovative open source methods to document the siege of Aleppo, Syria (Czuperski et al. 2017). These methods were recently leveraged to help analyze Russian siege tactics used in Russian attacks on the Ukrainian city of Mariupol (Hendrix 2022). Perhaps most notably, in 2020 the Berkeley Protocol on Digital Open Source Investigations created standards and practical guidance for practitioners on the proper and ethical collection and use of open information (United Nations OHCHR and HRC UC Berkeley 2020). However, with so many new entrants into open source investigations, concerns over the reliability, integrity, and authenticity also increase (Hendrix 2022). Many of the innovations in open source investigations were pioneered by actors outside of the legal field (Freeman 2020, 62). Intelligence analysts, journalists, activists, and members of civil society are untrained in evidentiary standards and could inadvertently undermine investigative efforts to capture potential evidence (Hendrix 2022). The ICC must therefore evaluate open source information with caution and skepticism by accounting for which actors are involved, the technologies they rely on, and the possibilities for forgeries and digital evidence manipulation (United Nations OHCHR and HRC UC Berkeley 2020).

Relevant ICC Cases 

The ICC has already begun grappling with questions of how to evaluate digital evidence in two notable cases, Al-Mahdi and Al-Werfalli (Freeman and Vazquez Llorente 2021, 179).

In the Al Mahdi case, videos from the internet, along with satellite imagery, were used to bring charges against an alleged member of an Al-Qaeda affiliate, Ahmad Al Faqi Al Mahdi, for the destruction of cultural heritage property in Timbuktu, Mali (Zarmsky 2021, 214). However, because the defendant pleaded guilty, the authenticity of the evidence was never challenged, nor were any technical expert witnesses called to testify on the digital evidence presented (Zarmsky 2021, 216). Similarly, in the 2017 case, Prosecutor v. Mahmoud Mustafa Busyf Al-Werfalli, the Court relied heavily on several videos posted to Facebook to issue an arrest warrant for a high-ranking Libyan military officer, Major Mahmoud Mustafa Busyf Al-Werfalli, for the murder of 33 people (Freeman and Vazquez Llorente 2021, 179). Al-Werfalli is not presently in ICC custody, so the effect of the introduction of social media evidence remains to be seen. Al-Mahdi and Al-Werfalli demonstrate the Courts’ willingness to evaluate digital evidence in substantially new ways. In the case of Al-Werfalli, there would likely not be a case at all without the Facebook videos of these alleged crimes (Freeman 2020, 57). While digital evidence has not yet played a central role in the final judgment of a case before the ICC, these two cases demonstrate that this change is inevitable (Freeman and Vazquez Llorente 2021). 

An Analogous Debate: Weighing Scientific Evidence 

No clear road map exists to help judges resolve disputes over digital evidence authenticity. However, the Court can draw upon existing literature associated with the related domain of scientific evidence to help establish a framework for ICC judges to resolve authenticity disputes. Debates between international legal scholars and experts regarding scientific evidence serve as a useful framework to help ICC judges evaluate digital evidence in the face of technical uncertainty. While not all elements of the debates surrounding the evaluation of scientific evidence are perfectly analogous to the ICC and digital evidence authentication, the problem of “different truths” between legal and scientific domains is nonetheless highly instructive. 

When dealing with scientific uncertainty and “different truths” in scientific understanding, international legal expert, Yuka Fukunaga, argues judges should never decide on the science (Boisson de Chazournes 2012, 479). Fukunaga’s research is a response to debates over scientific uncertainty at the World Trade Organization (WTO) related to contentious topics like the use of growth hormones in cattle or genetically-modified organisms in the production of food and feed (Fukunaga 2012, 559–576). As she rightly points out, scientific inquiry does not naturally lend itself to determinations of singular or universal truth (Fukunaga 2012, 567). While some areas of science may have general scientific consensus, scientific research is, by nature, open-ended (Wagner 2016, 156-159). In contrast, international law is a truth-seeking enterprise, requiring strict adherence to standards and procedures and ultimately a final ruling (Wagner 2016, 158-159). Given the conflicting values between scientific inquiry and legal standards, Fukunaga proposes a judicial test to probe the coherence, objectivity, and proportionality of scientific assessment methods to meet the standard of proof beyond a reasonable doubt (Boisson de Chazournes 2012, 479). By plainly stating that judges are to review the arguments presented, and not decide on scientific truth, the Chamber is spared the time, expense, and frustration of learning complex domains outside their scope of expertise. However, this approach still does not relieve the Court from deciding how exactly to exercise judicial review in the face of significant scientific or technological disagreement and uncertainty. 

A proposed counterargument for evaluating scientific evidence was developed by international lawyer Caroline Foster in her book Science and the Precautionary Principle in International Courts and Tribunals. Expert Evidence, Burden of Proof and Finality (Foster 2011). In it, she advocates for adjudication that is “plainly exercised”— not transformed into a process of review (Boisson de Chazournes 2012, 480). She argues that judges should engage with scientific concepts, embrace scientific uncertainty, and adjust the way they apply the rules of burden of proof beyond a reasonable doubt to accommodate inherently uncertain scientific issues (Perez 2013, 971). This approach has a clear advantage of increasing the literacy of judges in basic aspects of science or technology. If judicial proof must be “capable of dissection, element by element, so that it becomes completely clear what are the premises, what are the conclusions, why the latter are supposed to follow from the former” (De Smet, 2015, 861), then judges should be prepared to engage with all of the elements (including scientific and technical) that constitute such judicial decisions. Applying Foster’s logic to digital evidence authenticity, Lindsey Freeman points out that “the ability of the Judges to exercise their adjudication power will increasingly depend on their capacity to interrogate technology systems, enhance their familiarity with digital evidence, and increase their understanding of new sources of information” (Freeman 2021). Increasing judges’ capacity to understand and interrogate digital evidence would require some form of continuing judicial education, something that has not gained widespread support among ICC judges (IER 2020, 139).

If the ICC were to follow the recommended judicial test advocated by experts like Yuka Fukunaga, uncertainty may be reduced by choosing the more coherent and well-reasoned argument between the prosecution and defense. However, this approach runs the risk of creating space for judges to unwittingly accept well-reasoned, coherent, and objective legal arguments that are technically or scientifically unsound. By applying this line of reasoning to digital evidence authentication, judges could unwittingly legitimize a judicial review process completely divorced from the technology with no effective mechanisms for determining whether a digital file has been forged or manipulated. 

Conversely, Caroline Foster’s approach would require judges to modify their standard for burden of proof and accept a level of technological uncertainty. Such an accommodation of doubt also has its dangers. ICC judges would benefit from an increased understanding of the basic concepts of the technologies or scientific processes they are charged with evaluating. However, altering the standard of proof raises questions of how much uncertainty is acceptable, and whether this uncertainty still complies with Article 66(3) of the Rome Statute that requires guilt beyond a reasonable doubt. The beyond reasonable doubt standard has always been a complicated concept, particularly at the ICC with its hybrid legal system and combined variety of legal traditions (De Smet 2015, 1). Taking a reasonably subjective, flexible approach to standard of proof would allow judges the freedom to determine the standard based on the type of evidence or the facts involved (De Smet 2015, 18). This approach also aligns well with the Court’s general affinity toward flexibility. Nevertheless, it is important to be mindful that the argument for “plainly exercised” judicial review risks degrading any common understanding of what “beyond a reasonable doubt,” as articulated in Article 66(3) of the Statute, reasonably requires.

Policy Recommendations 

While the ICC clearly has many challenges ahead related to the authentication of digital evidence, the Court has the potential to meet some of these challenges by repurposing or redirecting existing resources and increasing transparency on its procedures related to digital evidence authentication. These recommendations do not advocate for a radical realignment of the Court’s core duties, nor do they seek to overburden an already tightly resourced organization. However, these policy recommendations will go a long way toward addressing the inadequacies of the Court’s current digital authentication procedures. 

Of immediate and pressing concern is the Court’s continued use of the highly insecure and outdated digital signatures algorithm, MD5. The risks of weak cryptography are not well-understood by the Court at present. The consequences of a data breach, destruction, or manipulation of the Court’s digital evidence would be severe. Loss of confidential information could put the lives of victims, witnesses, and investigators in danger. Further, data exfiltration, damage, or destruction would threaten a core function of the Chamber: ensuring a fair and expeditious trial. [24] The fact that the ICC requires digital signatures at all suggests, at some level, that the Court recognizes the importance of data integrity and security. The purpose of cryptography is to protect the confidentiality, authenticity, and integrity of digital information—functions that are critical to fulfilling the Court’s mission (Johnson and Millett 2017, 25). Therefore, it is imperative that the ICC prioritize robust cryptography standards to secure its electronic filing system. 

Since many stronger, favored cryptographic algorithms for digital signatures are available, the ICC should retire MD5 and identify the ICC’s favored industry standards from a reputable source as soon as possible (Moriarty 2021). International standards bodies like the International Organization for Standards (ISO), the Institute of Electrical and Electronics Engineers (IEEE), and the Internet Engineering Task Force (IETF) could offer guidance because they regularly publish up-to-date lists of secure algorithms used by organizations around the world. As described previously, the Court must work toward implementing protocols to achieve algorithm agility, migrating from one algorithm suite to another as cryptographic algorithms become weak or obsolete. Achieving this goal would align with one of the ICC’s state goals in the Court’s Strategic Plan 2019-2021: “improving organizational performance”, and more specifically, to “further strengthen professionalism, dedication, and integrity in all of the Court’s operations” (ICC 2019, 3). 

The IETF suggests that protocol designers pursue a “modular” implementation scheme, meaning that the protocol can easily accommodate the insertion of new algorithms or suites of algorithms and seamlessly retire older algorithms (Housley 2015). The ICC’s e-Court User Group could be an appropriate internal body for this task. The e-Court User Group was established to allow users to convene and discuss practical issues regarding the operations of the e-courtroom (Dillon and Beresford 2014, 6). The User Group’s role in resolving these issues could be twofold. First, the User Group could help the Court decide on favored cryptography algorithm(s) based on international standards and the needs of the ICC. Second, they could collectively agree on how best to achieve algorithmic agility in a way that is the least costly and disruptive to all parties, particularly for parties with limited resources or technical capability. More broadly, the e-Court Protocol should be systematically reviewed by cybersecurity experts for other vulnerabilities and revised to be vendor agnostic, meaning no specific software or hardware solution is favored in case of future cybersecurity vulnerabilities in specific programs (Moriarty 2021). Algorithm agility is important not only to properly protect highly sensitive data, but also to bolster judges’ ability to meaningfully resolve disagreements over authenticity and veracity of digital evidence. 

Debates regarding the evaluation of scientific evidence are instructive toward the ICC’s resolution of disputes over the authenticity of digital evidence and strengthening its judicial review process. Based on an assessment of these two arguments, Caroline Foster’s approach is more persuasive. There is an obvious and significant need for improved technological literacy at the ICC. For judges, this should include basic familiarity with data security, metadata, data storage, digital forensics, and how technologies influence the degree to which digital evidence can be authenticated. Article 7(3) of the Court’s Code of Judicial Ethics says, “Judges shall take reasonable steps to maintain and enhance the knowledge, skills, and personal qualities necessary for judicial office.” [25] Judges across many jurisdictions generally accept the need for “continuing judicial education” (IER 2020, 139). Professional development is now the norm in many other countries and high courts (IER 2020, 139).In the Independent Expert Review of the ICC and Rome Statute Final Report released in 2020, experts recommend judges use the ICC’s annual retreat as a starting point for introducing a professional development program (IER 2020, 140). The IER also recommends a series of events at The Hague where judges can engage with experts in international law and other professions “to address matters of interest relevant to the development of their professional, scientific and cultural knowledge, skill and experience” (IER 2020, 141).

These recommendations may be met with resistance by some judges. Some judges may view continuing education as unnecessarily burdensome on their time or even an affront to their credentials, knowledge or skills as premier experts in international law (IER 2020, 139). But the purpose of continued judicial education is in no way to question the integrity or authority of judges. Rather, continuing judicial education should be considered an essential competency enhancement tool to achieve the proper administration of justice. Given the significant technological challenges the Court faces in the digital age, it has never been more important for judges to engage with technological concepts. Judges have the final say in determining the authenticity of digital evidence. They are therefore the best hope and weakest link for ensuring the proper resolution of authentication disputes. For these reasons, judges must improve their basic literacy of digital evidence authentication. 

Improving judges’ technological literacy requires a review of the most promising resources available to assist the Court in these improvements. In 2014, the ICC and the OTP established the ICC’s Scientific Advisory Board composed of an international group of forensics experts from specialized scientific organizations to assist the Office with investigative and prosecutorial work (ICC 2014). According to ICC press releases, the Board meets once a year with the OTP and creates an annual report discussing the scientific priorities for the OTP (ICC 2020). However, it does not appear that these annual reports are available to the public, nor does the ICC publish general information about the Board or its activities. In a 2012 workshop on the ICC’s use of scientific evidence at the Human Rights Center at the University of California, Berkeley, School of Law, participants recommended that the Office of the Prosecutor establish said Scientific Advisory Board to advise prosecutors, analysts, and investigators on cutting-edge scientific and technological developments relevant to their investigations (HRC UC Berkeley 2012, 11). Participants included representatives from the Human Rights Center, the ICC, other international justice tribunals and experts in remote sensing, information technology, DNA analysis, forensics, and digital evidence, among others (HRC UC Berkeley 2012, 4). Notably, workshop participants “agreed that the Court . . . will need to work with outside experts to develop guidelines on the reliability and admissibility of social media and video evidence” (HRC UC Berkeley 2012, 7). 

While the Court clearly established the Scientific Advisory Board, it is not clear whether the Court has ever convened a group of experts to develop the HRC’s proposed social media and video evidence guidelines. In a 2020 press release on the Scientific Advisory Board’s annual meeting, the ICC disclosed a list of eleven participating organizations at the annual meeting. Of those, eight were international or regional forensic science organizations, two were legal medicine organizations, and one, the Europol European Cybercrime Centre EC3, was focused on cybercrime (ICC 2020). While there is no doubt a great need for forensic expertise at the ICC, the overrepresentation of forensic sciences demonstrates that the Board is not focused on advancing or resolving technological issues. Given the urgent need for digital evidence guidance, the Board should consider expanding its technological advisory role—offering expertise and digital evidence best-practices to the Court, as envisioned by the HRC’s 2012 workshop. Unfortunately, the Board only meets once a year. So, while Board resources may be highly useful in providing general guidance to the Court, it is not well suited to advise on technological issues on an urgent or regular basis. 

The ICC’s e-Court User Group may be a more appropriate resource to advise on issues related to digital evidence authentication. The e-Court User Group is responsible for the following duties: 

1. Propose and decide on technical improvements/enhancements to the systems comprising the Court's e-court
2.  Propose and decide on changes of the data structure of the systems;
3. Propose changes to the e-Court protocol(s) to the relevant Chamber(s) where required; and
4. Follow up on the implementation of the above proposed changes when required (Dillon and Beresford 2014,7).

Very little public information is available regarding the activities of the User Group. It remains unclear who qualifies as a “user” to contribute to the activities of the User Group (Dillon and Beresford 2014, 7). Based on the limited information available, it appears the Registry manages individuals invited to Group meetings, which may include representatives from the OTP, the defense, and victims as special parties (Dillon and Beresford 2014, 7). The e-Court User Group has already played an important part in improving the e-Court Protocol. In response to a 2011 request from the Prosecution in Prosecutor v. Callixte Mbarushimana, the e-Court User Group successfully worked with the Registry to amend the Protocol, including provisions related to the disclosure of metadata, electronic image formatting, and more. [26] The User Group could fulfill a more central, continuous role in ensuring evidence is properly authenticated and associated arguments presented to judges by the prosecution and defense in complex areas outside judges’ expertise do not fall prey to logical traps or misrepresentations. 

The ICC should also consider taking steps toward improving transparency of the functions and activities of the Scientific Advisory Board and the e-Court User Group. There is little value in making the Board’s annual reports inaccessible to the public. Greater transparency would allow researchers and technologists to also play a more active support role in improving the operations of the Court.

In sum, the ICC should strongly consider the policy recommendations outlined above (1) appoint the e-Court User Group to lead efforts on algorithm agility, and continuous development of authentication issues; (2) expand the technological advisory role of the Scientific Advisory Board; (3) create routine workshops and seminars to improve the technical competency of judges; and (4) increase transparency surrounding the Scientific Advisory Board and e-Court User Group’s activities. These steps would greatly improve the legitimacy and authority of the Court’s digital evidence authentication and verification methods. 

Conclusion 

The ICC’s engagement with digital evidence began when it signaled its willingness to consider new forms of digital evidence in Al-Mahdi and Al-Werfalli (Freeman 2020, 57). However, the Court has not sufficiently adapted to the unique challenges that digital evidence poses to authentication and verification. Not only is the e-Court Protocol’s main mechanism for authenticating digital files cryptographically broken and unfit for use, but judges are also underprepared to fulfill their role in resolving disagreement over the authenticity of digital evidence. Rapid advancements in technology, a polluted information environment, and vast amounts of vulnerable, transient, and time-sensitive data mean that disputes over the authenticity of digital evidence are inevitable. New forms of digital evidence increase the level of doubt in authentication and verification, raising serious questions about how the ICC should best approach complex areas of technology and what judicial review under such circumstances should entail. 

The ICC must not be paralyzed by fear and uncertainty of new technologies and methods for digital authentication. For the Court, the only way forward is through the technical morass, not around it.

The ICC must not be paralyzed by fear and uncertainty of new technologies and methods for digital authentication. For the Court, the only way forward is through the technical morass, not around it. At a minimum, the cryptographic standard for digital file authentication must be replaced with a strong digital signature program and its e-Court Protocol should be systematically reviewed for other vulnerabilities. These preliminary measures, along with the suggested policy recommendations outlined in this paper, will go a long way toward ensuring the ICC is better prepared for the future challenges that digital evidence will bring to international criminal justice. The ICC’s improved processes will have reverberating effects and help inform other international, regional, and national bodies adjudicating human rights abuses and other overlapping issues (Cole 2015). For these reasons, the ICC must seize this moment to establish robust procedural safeguards and judicial review processes for digital evidence authentication in today’s fast-evolving international criminal justice landscape.

*This article was edited by Michelle Zhang (Princeton University), Lynne Guey (Princeton University), and Chester Eng (Tufts University). 

---

About the Author

Chelsea Quilling is a soon-to-be graduate of the Master of Law and Diplomacy program at the Fletcher School of Tufts University where she studied technology policy and international security. Her research interests center on international governance and policy issues related to artificial intelligence, privacy, and cybersecurity. At Fletcher, she was co-president of Fletcher Women in Business, a co-leader of the Murrow Conference on Digital Media Policy for Future Crises, and a part-time research and data science intern for the U.S. Army War College’s Strategic Studies Institute. Chelsea has over nine years’ experience working in the legal field, primarily in business immigration. She is an honors graduate of Gonzaga University and was born and raised in Boulder, Colorado.

(Return to Top)

---

Notes

[1] Rules of Procedure and Evidence. ICC, Rule 63(2). (Return to Note)

[2] Prosecutor v. Germain Katanga. Trial Chamber II. “Judgment pursuant to article 74 of the Statute.” ICC-01/04-01/07. (2014) ¶ 16 (Return to Note)

[3] Prosecutor v. Thomas Lubanga Dyilo. Trial Chamber I. “Decision on the admissibility of four documents.” ICC-01/04-01/06-1399. (2008) ¶¶ 27-32 (Return to Note)

[4] Rome Statute, Art 69(4). (Return to Note)

[5] Prosecutor v. Germain Katanga and Mathieu Ngudjolo Chui. Trial Chamber II. “Decision on the Prosecutor's Bar Table Motions.” ICC-01/04-01/07. (2010) ¶ 13 (Return to Note)

[6] The Prosecutor v. Germain Katanga. Trial Chamber II. “Judgment pursuant to article 74 of the Statute.” ICC-01/04-01/07. (2014) ¶ 11 (Return to Note)

[7] Prosecutor v. Jean-Pierre Bemba Gombo. Trial Chamber III. “Decision on the admission into evidence of items deferred in the Chamber's ‘Decision on the Prosecution's Application for Admission of Materials into
Evidence Pursuant to Article 64(9) of the Rome Statute.’" ICC-01/05-01/08. (2013) ¶ 13 (Return to Note)

[8] Prosecutor v. Jean-Pierre Bemba Gombo. Trial Chamber III. “Decision on the admission into evidence of materials contained in the prosecution’s list of evidence.” ICC-01/05-01/08. (2010) ¶ 17 (Return to Note)

[9] Ibid., ¶ 9 (Return to Note)

[10] Ibid., ¶ 9 (Return to Note)

[11] Prosecutor v. Jean-Pierre Bemba Gombo. Trial Chamber III. “Decision on the admission into evidence of materials contained in the prosecution’s list of evidence.” ICC-01/05-01/08. (2010). ¶ 9 (Return to Note)

[12] Prosecutor v. Jean-Pierre Bemba Gombo. Trail Chamber III. ICC-01/05-01/08, “Decision on the admission into evidence of items deferred in the Chamber's ‘Decision on the Prosecution's Application for Admission of Materials into Evidence Pursuant to Article 64(9) of the Rome Statute.’" (2013). ¶ 2 (Return to Note)

[13] Ibid.,¶ 7 (Return to Note)

[14] Rome Statute, Art 52. (Return to Note)

[15] Regulations of the Court. Regulation 26. International Criminal Court. ICC-BD/01-05-16. (Return to Note)

[16] Unified Technical Protocol (‘E‐court Protocol’) for the Provision of Evidence, Witness and Victims Information in Electronic Form. ICC-01/14-01/18-64-Anx 23-01-2019. (Return to Note)

[17] Unified Technical Protocol (‘E‐court Protocol’) for the Provision of Evidence, Witness and Victims Information in Electronic Form. ICC-01/14-01/18-64-Anx 23-01-2019. ¶ 32 (Return to Note)

[18] Rome Statute, Article 64(2). (Return to Note)

[19] Prosecutor v. Callixte Mbarushimana. Pre-Trial Chamber I. “Decision amending the e-Court Protocol.” ICC-01/04-01/10-124. (2011). Pg. 4 (Return to Note)

[20] Prosecutor v. Jean-Paul Akayesu. International Criminal Tribunal for Rwanda (ICTR). Chamber I. “Judgment.” ICTR-96-4-T. (1998). (Return to Note)

[21] Ibid., ¶ 156. (Return to Note)

[22] Ibid., ¶ 148. (Return to Note)

[23] Prosecutor v. Laurent Gbagbo. ICC. Pre-Trial Chamber I. “Decision adjourning the hearing on the confirmation of charges pursuant to article 61(7)(c)(i) of the Rome Statute.” ICC-02/11-01/11 (2013). ¶ 35. (Return to Note)

[24] Rome Statute, Article 64(2). (Return to Note)

[25] Code of Judicial Ethics. International Criminal Court. Article 7(2). ICC-BD/02-01-05. (Return to Note)

[26] The Prosecutor v. Callixte Mbarushimana. “Decision amending the e-Court Protocol.” ICC-01/04-01/10. (2011). (Return to Note)

---

References

Apple, James G. and Deyling, Robert P. 1995. A Primer on the Civil-Law System. Washington, D.C.: Federal Judicial Center.

Ashouri, Aida, Caleb Bowers, and Cherrie Warden. 2014. “An Overview of the Use of Digital Evidence in International Criminal Courts.” Digital Evidence and Electronic Signature Law Review 11 (0): 115–27. https://doi.org/10.14296/deeslr.v11i0.2130.

BBC News. 2020. “Ethiopia’s Tigray Conflict Sparks Spread of Misinformation,” November 11, 2020, sec. Africa. https://www.bbc.com/news/world-africa-54888234.

Boisson de Chazournes, Laurence. 2012. “Introduction: Courts and Tribunals and the Treatment of Scientific Issues.” Journal of International Dispute Settlement 3 (3): 479–81. https://doi.org/10.1093/jnlids/ids020.

Carlson, Frank. 2017. “Internet History Is Fragile. This Archive Is Making Sure It Doesn’t Disappear.” PBS NewsHour. January 2, 2017. https://www.pbs.org/newshour/show/internet-history-fragile-archive-maki….

Cole, Alison. 2015. “Technology for Truth: The Next Generation of Evidence.” International Justice Monitor. March 18, 2015. https://www.ijmonitor.org/2015/03/technology-for-truth-the-next-generat….

Cybersecurity and Infrastructure Security Agency (CISA). 2020. “Security Tip (ST04-018): Understanding Digital Signatures.” August 24, 2020. https://us-cert.cisa.gov/ncas/tips/ST04-018.

Czuperski, Maksymilian, Emma Beals, Faysal Itani, Ben Nimmo, and Eliot Higgins. 2017. “Breaking Aleppo.” Washington: Atlantic Council.

Desjardins, Jeff. 2019. “How Much Data Is Generated Each Day?” World Economic Forum. April 17, 2019. https://www.weforum.org/agenda/2019/04/how-much-data-is-generated-each-….

Dillon, Mark, and David Beresford. 2014. “Electronic Courts and the Challenges in Managing Evidence. A View From Inside The International Criminal Court.” International Journal for Court Administration 6 (1): 29–36. https://doi.org/10.18352/ijca.132.

Dougherty, Chad R. 2001. “MD5 Vulnerable to Collision Attacks, Vulnerability Note VU#836068.” Carnegie Mellon University, Software Engineering Institute, CERT Coordination Center. January 21, 2001. https://www.kb.cert.org.

Dubberley, Sam, Koenig, Alexa, and Murray, Daragh. 2020. “Introduction: The Emergence of Digital Witnesses.” In Digital Witness: Using Open Source Information for Human Rights Investigation, Documentation, and Accountability, 1st ed., 3–11. Oxford University Press.

Foster, Caroline E. 2011. Science and the Precautionary Principle in International Courts and Tribunals: Expert Evidence, Burden of Proof and Finality. Cambridge Studies in International and Comparative Law. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511973680.

Freeman, Lindsay, and Raquel Vazquez Llorente. 2021. “Finding the Signal in the Noise: International Criminal Evidence and Procedure in the Digital Age.” Journal of International Criminal Justice 19 (1): 163–88. https://doi.org/10.1093/jicj/mqab023.

Freeman, Lindsay. 2021. “How to Prepare the International Criminal Court for Our Digital Future.” Opinio Juris (blog). October 12, 2021. https://opiniojuris.org/2021/10/12/how-to-prepare-the-international-cri….

Freeman, Lindsay. 2020. “Prosecuting Atrocity Crimes with Open Source Evidence : Lessons from the International Criminal Court.” In Digital Witness : Using Open Source Information for Human Rights Investigation, Documentation, and Accountability, 1:48–67. Oxford, United Kingdom: Oxford University Press.

Fukunaga, Yuka. 2012. “Standard of Review and ‘Scientific Truths’ in the WTO Dispute Settlement System and Investment Arbitration.” Journal of International Dispute Settlement 3 (3): 559–76. https://doi.org/10.1093/jnlids/ids016.

Hendrix, Justin. 2022. “Ukraine May Mark a Turning Point in Documenting War Crimes.” Just Security. March 28, 2022. https://www.justsecurity.org/80871/ukraine-may-mark-a-turning-point-in-….

Higgins, Eliot. 2015. “New July 17th Satellite Imagery Confirms Russia Produced Fake MH17 Evidence.” Bellingcat. June 12, 2015. https://www.bellingcat.com/news/uk-and-europe/2015/06/12/july-17-imager….

Housley, Russ. 2015. “Guidelines for Cryptographic Algorithm Agility and Selecting Mandatory-to-Implement Algorithms.” BCP 201, RFC 7696, DOI 10.17487/RFC7696 Internet Engineering Task Force. https://www.rfc-editor.org/info/rfc7696. 

ICTY - UNICRI. 2009. “ICTY Manual on Developed Practices.” Turin, Italy: International Criminal Tribunal for the Former Yugoslavia (ICTY) and United Nations Interregional Crime and Justice Research Institute (UNICRI).

Independent Expert Review. 2020. “Independent Expert Review of the International Criminal Court and the Rome Statute System Final Report.” https://asp.icc-cpi.int/sites/asp/files/asp_docs/ASP19/IER-Final-Report….

International Criminal Court “About the ICC.” 2022. April 27, 2022. https://www.aba-icc.org/about-the-icc/about-the-icc/.

International Criminal Court. 2019. “Strategic Plan 2019-2021.” https://www.icc-cpi.int/sites/default/files/itemsDocuments/20190717-icc….

International Criminal Court. 2014. “The Office of the Prosecutor of the International Criminal Court Establishes a Scientific Advisory Board.” Accessed April 12, 2022. http://www.icc-cpi.int/news/office-prosecutor-international-criminal-co….

International Criminal Court. 2020. “The Scientific Advisory Board of the Office of the Prosecutor holds 7th annual meeting.” Accessed April 12, 2022. http://www.icc-cpi.int/news/office-prosecutor-international-criminal-co….

Internet Engineering Task Force (IETF). 2011. “Updated Security Considerations for the MD5 Message-Digest and the HMAC-MD5 Algorithms.” Request for Comments: 6151 ISSN: 2070-1721. https://datatracker.ietf.org/doc/html/rfc6151.

Jackson, John D., and Sarah J. Summers, eds. 2012. “The Common Law Tradition.” In The Internationalisation of Criminal Evidence: Beyond the Common Law and Civil Law Traditions, 30–56. Law in Context. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9781139093606.005.

Johnson, Anne Frances and Millett, Lynette I., eds. 2017. “Cryptographic Agility and Interoperability: Proceedings of a Workshop.” In Forum on Cyber Resilience: Workshop Series. The National Academies of Sciences, Engineering, and Medicine. Washington, D.C.: The National Academies Press. https://doi.org/10.17226/24636.

Khatib, Hadi Al, and Dia Kayyali. 2019. “Video: Opinion | YouTube Is Erasing History.” The New York Times, October 23, 2019, sec. Opinion. https://www.nytimes.com/video/opinion/100000006702129/syria-youtube-con….

Koenig, Alexa, Emma Irving, Yvonne McDermott, and Daragh Murray. 2021. “New Technologies and the Investigation of International Crimes: An Introduction.” Journal of International Criminal Justice 19 (1): 1–7. https://doi.org/10.1093/jicj/mqab040.

Koettl, Christoph, Murray, Daragh, and Dubberley, Sam. 2020. “Open Source Investigation for Human Rights Reporting : A Brief History.” In Digital Witness : Using Open Source Information for Human Rights Investigation, Documentation, and Accountability, 1st ed., 12–31. Oxford, United Kingdom: Oxford University Press.

Krzan, Bartlomiej. 2021. “Admissibility of Evidence and International Criminal Justice.” Revista Brasileira de Direito Processual Penal 7 (1): 161. https://doi.org/10.22197/rbdpp.v7i1.492.

Lee, David. 2012. “Flame: Massive Cyber-Attack Discovered, Researchers Say.” BBC News, May 28, 2012, sec. Technology. https://www.bbc.com/news/technology-18238326.

Lima, Lucas Carlos. 2015. “The Evidential Weight of Experts before the ICJ : Reflections on the Whaling in the Antarctic Case.” Journal of International Dispute Settlement 6 (3): 621–35. https://doi.org/10.1093/jnlids/idv022.

Lorenz, Taylor. 2022. “Internet ‘Algospeak’ Is Changing Our Language in Real Time, from ‘Nip Nops’ to ‘Le Dollar Bean.’” Washington Post, April 8, 2022. https://www.washingtonpost.com/technology/2022/04/08/algospeak-tiktok-l….

Lovett, Ian and Ojewska, Natalia. 2022. “Citizens’ Images of Potential War Crimes in Ukraine Flood the Internet, but Might Not Hold Up in Court.” WSJ. Accessed April 30, 2022. https://www.wsj.com/articles/citizens-images-of-potential-war-crimes-in….

McDermott, Yvonne, Koenig, Alexa and Murray, Daragh. 2021. “Open Source Information’s Blind Spot: Human and Machine Bias in International Criminal Investigations.” Journal of International Criminal Justice 19 (1): 85–105. https://doi.org/10.1093/jicj/mqab006.

McDermott, Yvonne, Murray, Daragh, and Koenig, Alexa. 2019. “Digital Accountability Symposium: Whose Stories Get Told, and by Whom? Representativeness in Open Source Human Rights Investigations.” Opinio Juris. December 19, 2019. http://opiniojuris.org/2019/12/19/digital-accountability-symposium-whos….

Mehandru, Nikita, and Koenig, Alexa. n.d. “Open Source Evidence and the International Criminal Court.” Harvard Human Rights Journal. Accessed November 25, 2021. https://harvardhrj.com/2019/04/open-source-evidence-and-the-internation….

Moriarty, Kathleen. 2021. “Why Are Authentication and Authorization So Difficult?” Center for Internet Security. October 18, 2021. https://www.cisecurity.org/blog/why-are-authentication-and-authorizatio….

Murrow, Susan. 2020.“Security Risks of Outdated Encryption: Is Your Data Really Secure?” Infosec Resources. Accessed January 31, 2022. https://resources.infosecinstitute.com/topic/security-risks-of-outdated….

Ng, Yvonne. 2020. “How to Preserve Open Source Information Effectively.” In Digital Witness : Using Open Source Information for Human Rights Investigation, Documentation, and Accountability, 1st ed., 143–64. Oxford, United Kingdom: Oxford University Press.

Perez, Oren. 2013. “Caroline Foster. Science and the Precautionary Principle in International Courts and Tribunals. Expert Evidence, Burden of Proof and Finality.” European Journal of International Law 24 (3): 971–74. https://doi.org/10.1093/ejil/cht056.

Romano, Aja. 2020. “World War 3 Memes as Therapy: Coping with War and Crisis through Memes - Vox.” Vox. January 17, 2020. https://www.vox.com/2020/1/17/21065113/world-war-3-memes-iran-2020-sale….

De Smet, Simon. 2015. “Part V Fairness and Expeditiousness of ICC Proceedings, 34 The International Criminal Standard of Proof at the ICC—Beyond Reasonable Doubt or Beyond Reason?” In The Law and Practice of the International Criminal Court. Oxford Public International Law. 861-889. http://opil.ouplaw.com/view/10.1093/law/9780198705161.001.0001/law-9780….

The Human Rights Center at the University of California, Berkeley, School of Law. 2012. “Beyond Reasonable Doubt: Using Scientific Evidence to Advance Prosecutions at the International Criminal Court.” Workshop Report. https://www.law.berkeley.edu/files/HRC/HRC_Beyond_Reasonable_Doubt_FINA…

The Human Rights Center, University of California, Berkeley, School of Law and United Nations High Commissioner for Human Rights. 2020. “Berkeley Protocol on Digital Open Source Investigation: A Practical Guide on the Effective Use of Digital Open Source Information in Investigating Violations of International Criminal, Human Rights and Humanitarian Law.” 2020. Advanced Version HR/PUB/20/2. New York and Geneva. https://www.ohchr.org/sites/default/files/Documents/Publications/OHCHR_….

Wagner, Markus. 2011. “Law Talk v. Science Talk: The Languages of Law and Science in WTO Proceedings,” Fordham International Law Journal 35, no 1. 151-200. 

Wang, Xiaoyun, and Hongbo Yu. 2005. “How to Break MD5 and Other Hash Functions.” In Advances in Cryptology – EUROCRYPT 2005, edited by Ronald Cramer, 19–35. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer. https://doi.org/10.1007/11426639_2.

Wex. n.d. “Pobative Value.” Wex Legal Information Institute (LII), Cornell Law School. Accessed April 27, 2022. https://www.law.cornell.edu/wex/probative_value.

Zarmsky, Sarah. 2021. “Why Seeing Should Not Always Be Believing: Considerations Regarding the Use of Digital Reconstruction Technology in International Law.” Journal of International Criminal Justice 19 (1): 213–25. https://doi.org/10.1093/jicj/mqab048.