There's no denying that technology, especially these days, is moving fast. New developments emerge on a regular basis and developments that were formerly new find their metaphorical chrome peeling away to reveal either flashes in the pan or the solid underpinnings of stable technologies. When the Gartner (News - Alert) Hype Cycle--a process by which many technologies can be tracked from new and shiny to mature and stable and even to obsolete if one goes out far enough--was applied to big data, the unexpected came back: big data is further along than many predicted.
Recently, Svetlana Sicular, Gartner's research director, set out to track big data on the Hype Cycle and made the somewhat unpleasant discovery that her studies were already behind the curve. While she was expecting to find that big data was at a point known as the Peak of Inflated Expectations, (the Hype Cycle has five key points: Technology Trigger, Peak of Inflated Expectations, Trough of Disillusionment, Slope of Enlightenment and Plateau of Productivity) what she actually found was that big data had cleared that particular point and was now firmly in level three, or the Trough of Disillusionment.
On a certain level, this is a good thing; it means that the technology is maturing at a much greater level than was expected, and that there's not much time left before big data becomes a big part of many companies' infrastructures. Sure, right now, it's not so good--companies are discovering that big data couldn't do all the things they were hoping it would do back in the wild-eyed early days--but as realistic expectations assert themselves, the overall system will exhibit better reliability and more overall utility as companies use it the right way.
What tipped Sicular off was a discussion at a meeting with several of her colleagues at various firms, who started having some less-than-positive things to say about Hadoop; the limitations of MapReduce, for example, or that Hadoop was "primitive and old-fashioned.” These weren't the only points cited, of course, but the fact that they were there, and in that concentration, showed that disillusionment had begun to spread, at least, as far as Sicular was concerned. A report from Ovum (News - Alert) that came out days later suggested that the sentiment around Hadoop was still positive, regardless of what, as Ovum put it, "skeptics" had to say about it. Yet at the same time, Ovum's analysis might have dovetailed with Gartner's to a surprising degree, as both were discovering a large amount of good feeling toward the end of last year. Gartner had really only begun to notice the drop recently, while the story of big data in general was getting legs as a business story as well as a technology story, so Gartner could have spotted the drop on the tech side, while Ovum, which was getting most of its information on the matter from aggregated tweets, was seeing more diverse expressions of positivity.
The critical point here is that, as more businesses and more technology companies start getting a better handle on what big data can--and can't--do, they'll start adopting better measures to use it. More standardization will take place, best practices will arrive and the system will take on much greater refinement. The exact point on a "hype cycle" may be subjective and somewhat in dispute, but big data itself is rapidly showing itself to be a powerful new addition to the business toolbox.
Edited by Rachel Ramsey