At Strata+Hadoop World in New York, which drew its usual crowd of end users, vendors, and other industry participants, a message that has been growing the past year was dominant: big data requires enterprise credibility in the form of solid, ROI-focused use cases. It was not, in this analyst's opinion, the only message – the role of sourcing and preparing data for analysis was evident in abundance.
Big data needs to behave like an enterprise adult
In many ways, big data solutions have, for the past couple of years, existed outside the bounds of enterprise reality. By this I mean they were not constrained by the usual rules associated with enterprise IT-friendly criteria, namely, ROI and enterprise-grade features. Big data technology stands out further as it brings new challenges to the party, not least a growing library of new and unfamiliar data types and sources. The next two to three years are critical – big data technology either pays its way or will be consigned to extreme use cases that make it a novelty.
This year's Strata gathering, while retaining many of the "start-up charms" of a new industry, has begun to mature. With that in mind, Ovum has observed and continues to observe a range of tactical use cases that do one of two things: deliver on the promise of a previously suggested but undeliverable use case (e.g. "customer 360"); or bring a novel solution to a tactical problem (e.g. enterprise data warehouse augmentation). To achieve either of these outcomes, however, the technology has to mature in several key areas. It needs to simplify deployment, ease of operation, security, and – critically – its data management.
Data management – in the holistic sense – has been sadly lacking in big data technologies. Ironic perhaps, given that its genesis was the management of data too big for traditional approaches and technology solutions to manage. Sourcing, formatting, managing, and – critically – making big data sets available to users for analysis has been the exclusive preserve of hard-to-find data scientists and engineers. It is Ovum's opinion that if big data technology is to become a mainstream and accepted enterprise tool, three things are needed:
data management that shields users from the complexity of big data without inhibiting their ability to benefit from its analysis
use cases that justify the investment in big data solutions
non-expert, likely visual, analytical tools, which grant the broadest audience the benefits of data-driven analysis while blurring the divide between big and small / traditional data sources.
The first two are visible and tangible parts of the big data market. The third is a work in progress – in many cases (and with exceptions), tools designed for working with "small" data sets have been adopted to work with big data. Ovum suggests that its proposed new category, exploratory analytics, is the means by which a toolbox that unifies the traditional world of BI, the emerging "data democracy," and big data will enable better, data-driven decisions for all.
Tom Pringle, Practice Leader, Information Management