eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
I went to the Big Data Summit hosted by the Mass Technology Leadership Council recently in Cambridge, Mass., and had a chance to get an update on big data from an East Coast perspective. This was the fourth year of the summit, and during that time big data has emerged from concept to buzzword applied to nearly every business segment. “The full potential of big data is still unknown,” said Oracle’s Big Data Strategist Paul Sonderegger in the conference’s keynote presentation.
And the theme of the potential of big data was consistent throughout the event, as data scientists, entrepreneurs and users, including the city of Boston interim CIO Justin Holmes, described how they see the potential unfolding. In an era when buzzword answers to business problems (“just Hadoop it”) have become all too common, the Summit was refreshing in explaining how the nebulous concept of big data is currently most effective in being applied to solve small, but nagging issues, and the future is in companies figuring out how to meld existing internal data structures with data pulled in from nontraditional outside sources.
Also, amid all the discussion regarding the potential benefits of big data—including advanced analytics and forecasting—the old standbys of privacy, security and compliance need to be remembered. Those considerations were part of the Cambridge panel discussions and reflected the experience of the panelists who have dealt with data and privacy since before the big data buzz started.
“Our challenge is how to leverage big data without being big brother,” said Holmes. City governments collect a lot of data, and the current task is figuring out how to make that data accessible without violating privacy rights. Projects range from collecting pothole problems using the sensors contained in smartphones, allowing residents to adopt a fire hydrant to uncover following snowstorms, providing restaurant inspection data for out-of-town visitors to collecting information that carries more gravity, such as 911 calls to highlight potential high crime areas.
In the case of the 911 scenario, the public data has to be both made anonymous and connected to broader areas than specific addresses. “Who owns the data? In what format and context was the data collected? And what is the value versus risk equation?” said Holmes as he outlined the issues surrounding the transition of data from collection to a useful application.
Where should businesses begin their excursion into big data? Start with the business problem you wish to solve, said Ingo Mierswa, CEO of data analytics firm RapidMiner.
Mierswa’s view was supported by Sonderegger, who was the chief strategist at Endeca before that company was bought by Oracle. He advocated a hybrid approach to using data based on traditional data gathering and analysis techniques to “run the business,” while new methods can be used to “change the business.” Each type of data gathering and analysis has its place, just as alternating current and direct current electricity transmission methods share a compatible rather than competitive position.
“Predictive analytics do not predict great things; most of the time analytics makes a very small prediction millions of times,” said Sonderegger. Those predictions can include price changes or call center interactions, but while the predictions are small, the incremental activity improves the business. The oncoming rush of “datafication” from the “social data of thoughts and opinions, the data of things and the data of activities” will spur the opportunities for companies to rethink their business strategies based on information not previously available, he said.
While technologies such as Hadoop and data analysis engines were discussed, the Cambridge big data summit was much more about how businesses can use big data today. And that is what made the event successful.
Eric Lundquist is a technology analyst at Ziff Brothers Investments, a private investment firm. Lundquist, who was editor-in-chief at eWEEK (previously PC Week) from 1996 to 2008, authors this blog for eWEEK to share his thoughts on technology, products and services. No investment advice is offered in this blog. All duties are disclaimed. Lundquist works separately for a private investment firm, which may at any time invest in companies whose products are discussed in this blog, and no disclosure of securities transactions will be made.