Happy Friday, everyone.
There have been a couple of events lately that, at least tangentially, made me think about information and what we do with it. There have been a series of DDOS attacks on popular sites, at least one of which was driven by a blind army of smart devices. The other is the volatile and ultimately inaccurate polling leading into the US Presidential election. Both of these hint at the Wild West nature of technology — its flexibility and newness offers a lot of promise and a lot of unknown risks. So the theme for this week is — what is the quality of data and analytics and how do we do it “right.”
I’d love a really long, detailed analysis of the polling data for this election, but this morning-after article touches on something relevant — you can only get good analysis from good data. The 2012 election was a triumph of big data, but two major elections this year — Brexit in June and the US Presidential election — threw pollsters for a loop. As this article points out, the algorithms were accurate in some ways but missed others. Relying too heavily on those algorithms without understanding the limits of the data coming in was one of the ways that prognosticators missed the ultimate outcome.
This paper is a couple of years old, but it really hits at the importance of data both in quality and accessibility for operational and business intelligence. This covers architectural patterns, business intelligence requirements, and data integration, without getting too heavily into technical details. It keeps the overall picture in mind.
This article really doesn’t need to be limited to marketing — it truly applies to almost any strategic decision that can be influenced by data. The article itself is brief and uses a very concise and clear, real-world example, so read the whole thing. The great takeaway, though is that the best ideas do not come from data; they are refined by good data.
This is the second in a two-part series, and both worth reading, but I’m calling out part two because of what seemed an off-the-wall question about shadow IT. The CIO (Mike Giresi) had been talking about how he wanted to change the perspective of IT, to have them look at their technologies and strategies from the perspective of customer needs and to work more collaboratively with business analysts, and the interviewer said that sounded a lot like shadow IT. Somehow, that became an example on how companies should use data effectively. Quote: “[W]ith data, IT must ensure that the right data is collected and that it’s reported in a way that teams like marketing can use to make strategic decisions.” It’s a simple statement, but it distills something that’s easy to miss in a more-is-more culture. It’s not about amassing big quantities of data — it’s about collecting the right data, from the right places, and making that data usable.
This compares data analytics to a flawed applied Newtonian systems model, and that alone makes it totally worth reading. Although he doesn’t use the word, there is an implied need for approaching analytics: humility. We cannot understand all data and our models cannot truly reflect reality. This is the argument for adaptive processes. Rather than assuming that you can create a model or system that covers everything, you define a way to process information and adapt to changing conditions. It’s a shift from trying to impose order to being responsive and resilient in changing conditions.