Friday, July 26, 2013

The Big Data Explosion Meets the Cloud - Takeaways from NY Tech Panel: Part 1 of 5

The intersection of Big Data Analytics and Cloud Computing has great potential, yet unlocking the potential of these technologies requires capabilities, frameworks, and methodologies most companies haven't yet mastered.  At a recent New York Tech Council event moderated by Jaime Fitzgerald, panelists discussed the opportunities, challenges, and themes engendered by the rise of big data cloud solutions.

 I followed the discussion from the front row, live blogging highlights and recording intriguing themes. This is the 1st of 5 blog posts in which I'll share what I learned.

By Carmen Augustine -- July 26, 2013

Companies are scrambling to capitalize on the potential of Big Data, but what does it actually mean for your business? But what specifically should companies do I do with all this data, processing power, and technology? And how do the advantages and disadvantages of the cloud change the game?  Should companies be using the cloud to approach Big Data opportunities?  If so, how and under what circumstances does this make the most sense?

Jaime Fitzgerald (Founder & Managing Partner at my firm, Fitzgerald Analytics) led the panel discussion for the New York Technology Council  at the 10gen offices in New York's times square. Panelists included cloud analytics experts EdouardServan-Schreiber, Director for Solution Architecture at 10gen, Scott Rose, Client Partner at Think Big and Stuart Sim, Senior VP of Technology and Engineering at PlaceIQ, who provided a fascinating range of opinions and case studies. From budgeting for new data infrastructure to discussing how your Fitbit might be used to price health insurance, the discussion run the gamut of Big Data applications. 

 Executive summary:
  • Growing yet embryonic domains: Although both Big Data and Cloud Technology have been rapidly expanding in the past few years, both trends remain in the early stages of their development and adoption.  As a result, they are simultaneously "major," "immature," and fast-evolving.
  • Pros, Cons, and Constraints of cloud solutions: Cloud storage has both advantages, disadvantages, and limitations. "One size doesn't fit all needs," but the flexibility, scalability, and diversity of options make cloud-based big data solutions an important option for many firms and analytic use-cases.
  • Creative applications: We heard about numerous creative and new uses of big data and cloud-based big data solutions, from natural disaster detection to tailor-made advertising campaigns.
  • Innovation and iteration: Although the potential of big data is significant, unlocking this potential will require significant innovation.  This innovation, in turn, will require lots of experimentation, trials, and errors... which means organizations need to build a culture in which failure is accepted, experimentation is encouraged, and agile use of big data analytics is made possible.
About the discussion:

Panelists came from a variety of backgrounds and covered a range of topics within the field...and each of them shared unique 1st hand experiences in the use of Big Data to achieve better results. From a high school math team captain to a former Scott Henson puppet builder, the participants each had different career paths and provided a unique perspective on industry trends and growth.