While all those are good things and very worthwhile pursuits, they are not the reasons our industry is so enamored with the construct of big data. The reason we love big data is far less complex.
There are really two reasons: first, because Oracle decided to get into the game and second, because once we figured out why they wanted into the game, we realized there was a ton of money to be made–or lost.
A quick history lesson:
Big data means nothing. It’s a well meaning term for (literally) big piles of data, sitting in various massive balls of infrastructure, randomly scattered around our enterprise. More common terms include data warehouses or decision support systems, etc.
The IT industry has been built on the back of transactional systems: the big iron, big money, big visibility systems that run our companies. Those are the most expensive, most important systems in our worlds and as such have the best people on them, the most expensive components–both software and hardware–and have the most risk (perceived or real) to our livelihoods, which is why they are the most reliable.
Transactions occur once. We make sure our systems scale to meet transactional demands–once. We pay a lot to over-provision in every aspect because transactional scale is not a nice to have, it’s mandatory.