Blog Layout

On Wolves and Huskies

There’s a well-known story that people tell when they want to argue that Artificial Intelligence is untrustworthy. It usually goes something like this:

A group of AI researchers wanted to test their latest piece of software, so they trained it up on images of huskies and wolves to the point where it could distinguish one from the other. It got pretty good, to the point where it was getting it right all the time.  
However, it then got one wrong, and identified a husky as a wolf. When the computer scientists investigated why, it turned out that they had trained the system on pictures of wolves in snow, and on huskies in other terrain. Which meant that when they showed it a picture of a husky in a snowy field, it saw the snow and answered ‘wolf’ because all the other pictures with snow had been wolves.
In fact, they’d built an AI tool that detected snow!

This is often cited as an example of how the data fed to a machine learning system can result in systematic bias. In this case, you might say that because of an unnoticed correlation in the sample data, the resulting algorithm was biased against identifying huskies in the snow as huskies. Such mistakes, are a real and dangerous issue. They may, for example, serve to replicate real-world prejudices.  

For example, using training data that features only female infant school teachers will train the system to conclude that all such teachers are female. More worryingly, if current processes for assessing bank loan applications are unfairly prejudiced against black and other minority applicants, then using historic loans data without modification could cause a new AI solution to replicate those biases, unfairly rejecting good candidates.  

The Information Commissioner has focused on this issue in recent Guidance on Artificial Intelligence and Data Protection. This is a thorough and helpful piece of work, and I recommend it fully.

Unfortunately, although this is a real and very important issue, it isn’t what the husky and wolf story is really about at all. It’s become misinterpreted in the telling, and in some ways the original is much more interesting.

In fact, the researchers knowingly fed the machine learning tool biased data – and it wasn’t a new tool. They wanted to have a system that threw up errors. Because their research was about how users of such systems reacted when those errors occurred, and what strategies could be used to create confidence in AI systems.

In this research, Marco Tulio Ribeiro, Sameer Singh and Carlos Guestrin, argue that:

Despite widespread adoption, machine learning models remain mostly black boxes. Understanding the reasons behind predictions is, however, quite important in assessing trust, which is fundamental if one plans to take action based on a prediction, or when choosing whether to deploy a new model. Such understanding also provides insights into the model, which can be used to transform an untrustworthy model or prediction into a trustworthy one. 

In other words, the research was about human reactions to AI, and how trust in AI might be earned. For without trust, machine learning won’t be successful.  

In the 2016 paper, ‘“Why Should I Trust You?” Explaining the Predictions of Any Classifier’ , they go on to propose a new trust model of their own, and suggest techniques for building trust in predictive models through explanatory techniques. They give examples from text (eg random forests) and image classification (e.g. neural networks) – and it is the latter which throws up the example of the huskies and wolves.

They trained the system on hand selected images of huskies and wolves, to intentionally create the false correlation of wolves with snow, and then allowed a test group of subjects to use the system. The research looked at how explanatory material on what the system was doing affected the test subjects’ confidence in the system when false results were found.

This is important. There is a real risk in deploying and using AI systems without also providing some insights into how they are reaching the conclusions they do. Mistakes will occur, but by building an understanding into deployments, trust can be built, users can be more confident of the outcomes, and they can also, hopefully, spot erroneous results and guard against following the machine too-slavishly.


Share by: