Thursday, October 30, 2014

Big Data v. Godzilla

Who Wins the Battle For The Right to Use or Kill Information

Com·pe·ti·tion: the act or process of trying to get or win something (such as a prize or a higher level of success) that someone else is also trying to get or win: the act or process of competing (Merriam Webster)

Steven Wright mused that “you can't have everything, where would you put it?” But there are many in the IT world that think otherwise. Larger companies this year will grow their Information Footprint by 25-50% on average, which is about how much their data store grew last year and the year before that. Smart business people believe this path of keeping all their information is a good thing. Some even go farther, believing that all their information is essential to effectively use analytics technology (referred to as Big Data) to connect the dots to solve business problems. That is because not only does Big Data crave, well, big data, but also because answers to important business questions may dwell within the deep recesses of unstructured data piles that may seem unimportant to the casual or even the sophisticated observer. In other words, within all sorts of Information Parking Lots dwell all sorts of valuable information nuggets that only technology can harness. Getting rid of any information is tantamount to ridding the company of a competitive advantage that comes from harvesting the business answers.

The Information Competition Becomes a Conflict

But there is a whole different group of smart business folks that look at Big Data as a big risk and liability. Sure there may be value in finding the needle in the massive information haystack, but at what costs? These people seemingly take the exact opposite position, that more is not merrier and that at some point information which is “valueless” must be disposed. Defensible Disposition or “Rightsizing Your Information Footprint” is needed for risk reduction and reducing costs. The more information the company retains, the greater the likelihood that personal customer data may be compromised or someone will successfully hack our corporate information Crowned Jewels. Or the more information the larger the e-discovery headache. Or keeping everything forever undermines the records management program. Indeed, Privacy Officers generally think the right answer is for companies to keep less information for shorter periods of time. While Big Daters think about keeping more information longer periods of time.  Core to Records Management is that records go away at the end of its period of retention no matter what, unless it’s needed as evidence in a lawsuit of investigation.  More ill managed IP means more risk of losing company trade secrets.
And the “information use” battle waging is not limited to companies trying to predict the colors customers will want next season based on past buying habits. In an October 16, 2014 Wall Street Journal Story, entitled “FBI Chief Warns Against Phone Encryption,” it makes clear that the conflict over who gets to decide how information is managed is a real life and death situation pitting privacy advocates against the government. While government uses Big Data tools to crawl and unearth terrorists, privacy advocates and some phone companies want phone data encrypted. Similarly in an article entitled, “Privacy in the Internet of Things era: Will the NSA know what’s in your fridge,” Wojtek Borowicz, points out that “we’ve already entered the Internet of Things: a world where everything is connected, with billions of devices storing and exchanging data about each other and about their users – i.e. us. As it matures, it’s going to be hugely convenient, not only to the average Joe, whose smart home will always remember to lock the door and switch the lights off, but also to huge organizations. However, one of the main concerns associated with it is the security of IoT platforms and devices. But it’s not only preventing hackers from accessing these systems we should be discussing: What about privacy, government surveillance and the creepy vision of Big Brother hiding in my smart fridge?”

So Who Wins and Who Loses in This Conflict?

We find ourselves in an information Olympics where the best of the best of every information use and misuse is congregating to duke it out, though they may not even know it. The Big Data team is trying to tie together disparate chunks of information to answer business questions, while the storage guy screams, “No Mas”.  I think the business people win. I think Big Data wins where it adds value. But that said, I believe that maybe making the seemingly divergent interests of information use can be accommodated. Either way, we will see soon enough. But for now, conflict or competition, information is being used for different purposes by different sides of the company and this new reality needs management attention ASAP.


Adam Brudo said...

Big data is also changing hiring. Take Catalyst IT Services, a Baltimore-based technology outsourcing company that assembles teams for programming jobs. This year, the company will screen more than 10,000 candidates. Not only is traditional recruiting too slow and cumbersome, the company says, but also the subjective choices of hiring managers too often result in new employees who aren't the best fit.
Also, if you are interested in secure file sharing you may be needed vdr, and, as for me, iDeals data room provider is the best.