Thoughts on Data: Perception, Bias, and Interpretation.
February 28, 2019
Annie Duke describes ‘resulting’ as judging the quality of a decision based on the outcome rather than the processes and reasons that went into the decision. The problem with judging our decisions solely on outcomes is that it fails to take into account probability and chance. If unchecked, the practice of resulting can lead to confirmation bias and a failure to objectively evaluate the quality of our decision-making. The distinction between the quality of a decision and its result is how we analyze and discover insights from data.
Sherlock Holmes, arguably the most well-known fictional detective, stressed the importance of data throughout the stories and many of his points are not only relevant to but are part of today’s best practices in data science. Holmes would reiterate that you couldn’t draw conclusions without data, and without sufficient data risk skewing facts to support theories of speculation.
Understanding your data is essential to knowing what kind of insights you are able to discover, data can be manipulated and many variables can be presented in a way that is misleading beyond the difference between correlation and causality. It is important to think logically about how you are interpreting patterns and be sceptical of your prospective conclusions until you are confident in the results. Consider all possible explanations and do not take things for face value, because all data has limitations.
We also need to remember that while analytics and statistics are really good for finding patterns, trends, or probabilities, they often cannot offer an explanation to those insights. Each dataset is effective in its own way but cannot be a solution to every problem. When we want to answer why-type questions, especially with human behaviour or rationale, we need qualitative data because those explanations cannot be counted, quantified, or measured.
“A lie can travel halfway around the world while the truth is still putting on its shoes” though often attributed to Mark Twain or Winston Churchill, the quote is actually by satirist Jonathon Swift. Whether or not it is intentionally deceptive, misinformation is the spreading of false or inaccurate information, and with the growth of social media, is often spreading faster than people can bring fact-checking into the conversation.
I remember my statistics professor quoting Spiderman during our final lecture, that with great power comes great responsibility, then he proceeded to demonstrate to us how easily data could be manipulated and presented contradictory findings from the same dataset. That is the importance of peer review, and also why I enjoy working with social perception and interpretation. We all have unconscious biases and perceptions, it doesn’t necessarily make us good or bad people to acknowledge that fact, our beliefs and received wisdom inform our assumptions about the world. It is our responsibility to identify, understand, and challenge those belief systems and biases in order to make fair and rational decisions. Until you can see your biases, you will only see through them. It is this kind of reflection that helps us to differentiate between beliefs and facts, and in data science, is key to letting evidence inform our findings and developing theories that explain them.