Wednesday, June 05, 2013

Big Data Primer and why you should care about it

As your business grows, your systems will grow as well.

Unfortunate thing is complexity is not a linear, but an exponential function.  What does that mean?  It means that if you have 2 stores, the amount of work that needs to be done to manage 2 stores is not 2x of what it takes to manage one store.. You will need to hire a manager per store, institute formalized processes and procedures, implement reporting systems, put in inter-office communication, mind the politics, etc.. And if you get multiple regions, you'll need regional managers, who will probably want to institute some of their own rules, procedure, and systems.  And if you add a reseller channel, they will come with their own expectations and influence the way things are done.

Same in the computer world - having one system on one server stuck in a data closet is fine, but you quickly move beyond that with need to support multiple systems from different vendors, different version of those systems, set up for growth and spikes, maybe integrate the computer systems from the companies you've acquired, deal with legacy data and backups - this quickly becomes quite a hurdle requiring a large stuff of highly qualified and expensive IT personel to take care of it all on an around-the-clock basis.

That's why big data is important - it:

  1. greatly reduces the load that's placed on your IT department by making scaling automatic 
  2. eliminates or greatly reduces the system integration challenges, and 
  3. gives you a much quicker access to the data so that you can make the business decisions quicker