From the Blog

Digital Investigation & Risk of Confirmation Bias


We at Blue Lights Digital are passionate about raising awareness about key areas of investigation that can be of use to our clients and that can spark meaningful debate and interest. When we first began our blog back in 2015, we wrote about a subject that we find extremely interesting as it can be transferred across all areas of investigation, including digital investigation – confirmation bias.

Many of you will be familiar with this concept (perhaps you even began your understanding when reading our 2015 blog post!), but nonetheless, it is always a useful topic to come back to occasionally, if only to remind ourselves of its huge importance. If you are aware of the definitions, then please do feel free to skip past our opening two sections, What is Digital Investigation? and What is Confirmation Bias? and begin where the two meet in the third and fourth section.

 

What is Digital Investigation?

In today’s digital world, it is now accepted that every investigation, whether criminal or civil, ranging from the low level to the high profile and complex, has a ‘digital footprint’. A digital footprint is the data or trail that is left by any user (passive or active) interacting with a digital device, product or service. This includes, but certainly is not limited to, areas such as, communications data, device forensics, email and internet usage, social media and so on. This digital footprint creates an opportunity for investigators (both law enforcement and civil) to acquire and analyse data – a process known as digital investigation.

When undertaking a digital investigation, an effective Digital Investigator will not only be well-versed with the relevant legislation, policy and principles, they will have a good knowledge of emerging and evolving technologies with the ability to identify potential data opportunities. These skills and knowledge will be used to their advantage alongside traditional investigation techniques. They will combine these areas to build a collaborative and cross-referenced evidential picture, which can define, then underpin or undermine a case.

With the explosion of smart mobile devices, the convergence of different technologies, and the wide range of operating systems, the divide between personal and enterprise devices and data has been blurred. When combing this with the increasing trend towards home or remote working, the challenge of conducting a comprehensive digital investigation that is both proportionate and lawful is further intensified. This means that confirmation bias needs to be taken incredibly seriously.

What is Confirmation Bias?

So if confirmation bias is so critical to investigations, why haven’t I heard of it? Quite probably you have heard of it, without it being named, and have potentially experienced a level of it in your everyday life. Put simply, confirmation bias is the tendency people have to seek, interpret and remember information that confirms, rather than brings into doubt, their preconceptions. This can be both subconscious and conscious.

“A man sees what he wants to see and disregards the rest” – Paul Simon

Confirmation bias happens when people give more weight to evidence that supports their hypothesis and therefore undervalue (or ignore) evidence that could disprove it. As noted before, it is applicable to many day-to-day elements of our lives, and often occurs without any significant or life changing impact on others. Some interesting examples are:

  • Conspiracy theorists. It is well-known that despite overwhelming evidence to the contrary, conspiracies are based around facts and events being interpreted to fit into a particular version of events.
  • Police officers and the full moon. In the police world, it is often said that an exceptionally busy shift at the time of a full moon means the full moon is responsible for the hike in crime that night, whereas little attention is paid to the moon on other busy nights.
  • A dog and a postman. Every day the postman arrives to bring the post. Every day the dog who lives there goes to the door and barks untill the postman leaves. The dog believes that because the postman goes away after she barks, the postman must have gone away because she barks – confusing correlation with cause. The dog has developed a theory and does not look elsewhere to disprove it, therefore never finding the truth.

 

Confirmation Bias in Investigation

We now know that confirmation bias runs at a subconscious level, but this actually occurs at all times, often harmlessly, but when it is applied to the world of investigation, this can bring up challenges. Consider for a moment the potential impact on a case if an investigator had a pre-determined outcome, whether formed at the onset or during the investigation… Obviously, this could be detrimental to a case so it is essential to begin to aim to notice if this is happening.

There are different ways in which confirmation bias can undermine a fact-finding process in an investigation, therefore leading an investigator towards a dangerous remit of directing efforts to confirm guilt, rather than investigating a case with an open mind. Some ways that this can happen are by:

  1. Skewing the search, identification and acquisition of additional evidence. If this happens in a case, and it confirms a favoured hypothesis, other avenues that may undermine the case and hypothesis may be overlooked and therefore not pursued.
  2. Causing interpretation of ambiguous or contradictory evidence in a way that is consistent with the guilt of an already identified suspect or in line with a favoured hypothesis.

It has happened to all of us… consider the flutter you get when you find a compelling piece of evidence confirming your suspect’s guilt, versus the moment your heart sinks when you realise there is compelling material to the contrary. It is easy to get caught up in confirming what you believe rather than disproving it.

The best investigators are able to think independently and view the facts as they are, rather than what they want them to be – but this is not always possible, even for experienced investigators. For example, here we see two high-profile cases where confirmation bias has been applied:

The DNA of David Camm

Camm had been initially accused, then charged, with the murder of his wife and children in September 2000. However, 13 years later, his defence successfully argued that this conviction had been as a result of significant confirmation bias applied during the investigation and he was acquitted. Camm had been arrested three days after the murders took place because investigators identified his DNA on a sweatshirt found at the crime scene, so started to build their case against him. Tragically, the investigations overlooked that on the same sweatshirt was also the DNA, nickname and ID of a recently released convicted criminal with a history of similar MO. They allowed their confirmation bias let a guilty person run free and a traumatised innocent man be charged.

The fingerprints of Brandon Mayfield

In 2004, a US Attorney named Brandon Mayfield was linked to the Madrid Train Bombings after two latent fingerprints were found by Spanish authorities and shared via Interpol. The FBI conducted analysis and proposed 20 possible matches, one of which was Mayfield – despite his print not being identical to that recovered.
Details of Mayfield’s life convinced the FBI that he must have been involved in some way. In a post-9/11 culture, Mayfield had converted to Islam having met his [Egyptian] wife, and in his professional capacity he had represented in a child custody case one of the Portland Seven, a group of men who tried to travel to Afghanistan to fight for Al Qaeda. This circumstantial evidence condemned Mayfield as a terrorist sympathiser, and he was arrested and detained in the US for weeks under the Patriot Act without charge.
The Spanish authorities disagreed throughout with the FBI’s findings and proactively informed the FBI they had other suspects in the case, none of whom were linked in any way to anyone in the US. The FBI continuously disregarded all of the information supplied by the Spanish authorities and proceeded to focus on Mayfield anyway.
 A later FBI review acknowledged serious errors and confirmation bias in the investigation, and resulted in Mayfield receiving a formal apology and a $2million settlement

 

Confirmation Bias in Digital Investigation

Both of the above cases lay bare the potential extreme impact of applying confirmation bias, but neither clearly outlines the use of digital investigation. The fact these offences occurred around 15 years ago will most likely be the main reason for the omission of clear digital investigation. Consider how different, but also how complex these investigations would be if happening today, in 2017, given the increased opportunities to criminals and investigators through digital interactions and investigation. Even rewriting this post from its original two years ago, there are significant changes in digital technology in that short space of time. Also consider the multiple disciplines of digital investigation, and how significant a risk confirmation bias could be when it is recognised that ‘statistics and big data can be used to prove any hypothesis’…

“Do not put your faith in what statistics say until you have carefully considered what they do not say” – William W. Watt

Let’s take the power of social media intelligence and evidence on its own. 50% of internet traffic in the UK is occupied by Facebook, and the average user tweets 4.4 times a day (that’s over 1600 per year). Think how many times you as an individual have shared, commented on, liked or retweeted an interaction and what that could say about you when viewed independently or taken out of context. Now ask yourself, if a detective had a pre-determined mindset to prove their hypothesis… how simple would it be to do so?

  • Could retweeting a story about immigration portray you as having right wing views?
  • Could liking a meme that was shared by an extremist page, even if unrelated, allude to you agreeing with their views?
  • Could an innocent but negative comment made about an associate indicate motive if that associate was the victim of an anonymous assault a few days later?

All these interactions could tell a story about someone’s character, but they could also be an innocent second on the internet, never to be thought of again. Both these angles must be visited to get the clearest picture.


Now let’s look at modern communication methods. In one day, the average business employee spends over 200 minutes interacting with their smartphone, sends over 50 text messages, and sends and receives over 100 emails. If we now apply some civil investigation context, what conclusions can be drawn?

Example: a solid case is gathered against an employee accused of IP theft, but the investigator believes that a second employee, a close friend of the accused, is also involved. Because of this assumption, the investigator is determined to prove a conspiracy between the two, and works gathering evidence to prove their hypothesis. If the review of the communications data between the two reveals vast numbers of phone calls, text messages and emails between the two parties prior to and around the offence, suddenly ceasing when the employee is accused, this hypothesis seems to have evidence to support it.

If the investigator then has access to [and the permission to] review email communications, then it may even demonstrate dialogue arranging to meet and discuss things discreetly outside of an email chain. A compelling case could now be compiled supporting the hypothesis. But before this is done, and when not embroiled and caught up in the emotion of this investigation, further questions could and should be considered:

  • What evidence is there other than circumstantial?
  • How many other people does the suspect communicate with in the same way?
  • How many emails or communications have been exchanged between the parties without any reference of this or any other action or offence?

And, most importantly in modern culture:

  • How would your hypothesis stand up when applying the ‘reasonable third person’ test or further still in an employment tribunal, employee grievance or criminal court?

Identifying a hypothesis and believed sequence of events is a fundamental part of any successful investigation. Effective digital investigation provides an incredible opportunity to support your investigative hypothesis with factual, indisputable and contextual evidence but, as outlined above, it also presents significant risk of wrongful application if not carefully considered, challenged and cross-referenced. Totally avoiding confirmation bias is impossible, but for an effective and balanced digital investigation, being aware of its presence, considering its risks and putting appropriate safeguards in place to mitigate its impact are all key.

 

Recommendations to mitigate against Confirmation Bias

The fictional character, Sherlock Holmes, said: Eliminate all other factors, and the one which remains must be the truth. We recommend that this Holmes principle is adopted and that investigators work to overcome confirmation bias through attempting to disprove their theories, instead of trying to confirm them. This can be done through the following eight steps:


1.

Apply the ABC of Investigation:

  • Assume Nothing
  • Believe Nothing
  • Challenge Everything

2.

Sift through all of the available evidence before you settle on an opinion. The danger lies in forming a hypothesis of innocence or guilt too early in an investigation.


3.

Review all available data and evidence, not just that which fits with your hypothesis. Create timelines and repeatedly test each piece against fact and testimony.


4.

Appoint an investigator who has no emotional stake in the outcome, and therefore can act impartially throughout the investigation. If as an investigator you are unable to look objectively at the circumstances, then remove yourself from the case or obtain continual reviews from an independent third party who will challenge your assumptions.


5.

Keep an open mind throughout, especially to what you do not want to hear. Encourage an atmosphere of open inquiry where the goal is to remain impartial and neutral. Ensure your investigators have a thick skin and are prepared to accept objections, doubt, and criticism in order to find the truth.


6.

Where appropriate, seek expert advice for complex subject matters rather than assuming your findings are factually correct. Do not fail to consider the full value or alternative evidential opportunities within the data.


7.

Create training that familiarises your investigators with the concept of confirmation bias, along with its risks and how to mitigate against it.


8.

Where possible, utilise automated digital investigation products. Expert systems present only facts and do not apply bias to any data set without being led to do so by closed filters being applied by the user.


 

Overall, we as a community must accept that confirmation bias does exist whether we like it or not, and therefore the way we can combat the risks it poses is to develop an awareness and an acknowledgement of it. Here at Blue Lights Digital, our job is to help you and your organisation get the best results from your digital investigation. If you feel our expertise can help you, contact us directly and we can discuss your needs.

 

Please note that these thoughts and recommendations are by no means exhaustive and are designed to promote discussion within the investigative community.

Thank you for reading and engaging. Please feel free to provide feedback, add comments or discuss your investigative experiences from where you have encountered confirmation bias in investigations, whether criminal or civil – here on LinkedIn, or by contacting us.

 

Leave a Reply

Your email address will not be published. Required fields are marked *