How do you really know if someone is trying to fool you? Sometimes it’s easy. You know the kid took the cookie. You know the employee wasn’t sick. You know corporate is just cutting costs. But big data makes it harder to know what to believe. The raw data takes hours to read and is in a specialization that is not your area. Everyone who works with the data is beholden to an interest. And what if that cool thing the data scientists have figured out his how to scam you as a target? Thankfully, there is help.
A recent article from the New Yorker advises on How to Call B.S. on Big Data. It’s a great summary of a course at the University of Washington which became available in January 2017. In the spirit of public education, you can access a large amount of the materials in the course’s web site with videos, tools, and case studies.
There are some simple protective measures that are known to those in the number-crunching fields. Watch out for unfair comparisons; remember that correlation doesn’t imply causation; and beware of the hubris of those making bold claims. On average you need to keep information in context and ask for a plausible theory about why a fact would be true. The plausible theory is your hypothesis, and the scientific method is to test the hypothesis. No plausible theory; no science.
Amongst the precautions is that data taken from the general public will often re-create prejudice. I see this all the time when looking at inequalities in women’s promotions and salaries. Superficially it does appear that many women are self-selecting into less onerous careers. Deeper into the analysis, you tend to find that women do more than their share of the caring (in all of its forms) and that it’s a pervasive imposition, a subtle stereotype, and just about everyone is causing this to happen.
I’m glad to know that there is a growing desire among non-quants to consume information in a more sophisticated manner. For my own work, this doesn’t worry me. I’m honest and my motives are transparent [about stuart] If anything, I’m intimidated by the volume of work ahead of me. I used to have a clear sense of the amount of work required to seek the truth in the data and share what I found with a sincere audience. Since the rise of fake news and the increasing complexities of social media, a Pandora’s Box has been opened people like me are obliged to investigate ten times as many topics. We may be asked to fact-check nonsensical statements, defend controversial findings that were created in a neutral setting, or spend excess hours establishing credibility.
The most unsettling concept raised in the article is the BS Asymmetry Principle coined by Alberto Brandolini: the amount of energy needed to refute BS is an order of magnitude bigger than that needed to produce it. And so begins the new hybrid skill set of doing good math, and then talking about it properly.
Stuart Murray Speaks 