We live in the age where big data and data science are used to predict everything from what I might want to buy on Amazon to the outcome of an election.
The results of the Brexit referendum caught many by surprise because pollsters suggested that a “stay” vote would prevail. And we all know how that turned out.
History repeated itself on Nov. 8 when U.S. president-elect Donald Trump won his bid for the White House. Most polls and pundits predicted there would be a Democratic victory, and few questioned their validity.
The Wall Street Journal article, Election Day Forecasts Deal Blow to Data Science, made three very important points about big data and data science:
- Dark data, data that is unknown, can result in misleading predictions.
- Asking simplistic questions yields a limited data set that produces ineffective conclusions.
- “Without comprehensive data, you tend to get non-comprehensive predictions.”