Big Data Meets the Cloud: Takeaways Pt 2
(see the first article in the series, Big Data Explosion in the Cloud)
Imagine you are a marketing executive at a large retailer. How do you know who to target, when to target them, and what products they want to buy? Now imagine you know everything each customer has purchased, when they purchased it and even have a solid understanding of why. You have millions and millions of data points profiling your customers, giving you the opportunity to individually target each one – you could send the recent graduate a coupon for hampers, the new mom an e-mail about baby food.
(see the first article in the series, Big Data Explosion in the Cloud)
Imagine you are a marketing executive at a large retailer. How do you know who to target, when to target them, and what products they want to buy? Now imagine you know everything each customer has purchased, when they purchased it and even have a solid understanding of why. You have millions and millions of data points profiling your customers, giving you the opportunity to individually target each one – you could send the recent graduate a coupon for hampers, the new mom an e-mail about baby food.
Sounds good, and it’s
possible with today’s technology, data and analytic innovation, but getting to
this enviable potential requires navigation of numerous options (to use cloud
or not? What platform? Data? Methods?). With so much information available, what
do you do first? How do you turn this data into an actionable solution, and
what are the tools you need to do it?
By Carmen
Augustine -- August 12, 2013
Big Data itself is a relatively new phenomenon – the term has become
synonymous with the recent explosion in the size, diversity and spread of both
creation and processing of available data – which often requires new approaches
to data storage and processing. Now that technology has been developed to take
advantage of big data in cloud
architecture platforms, the question
is: how do we make use of what we’ve built? Who can benefit, and to what degree
does the cloud democratize big data?
In last week’s article, The Big Data Explosion Meets the Cloud, I introduced the growing role of Big Data in the business world and how cloud computing platforms have changed the shape of the industry. This week, I’ll take a deeper dive into trends in the industry, innovations on the horizon and opportunities for improvement. Read on to discover the insights I absorbed from the front row of the New York Technology Council’s June 26th live panel event “The Big Data Explosion: How to Process, Analyze and Visualize in the Cloud”!
- Scott
Rose of Think
Big was an advocate of using big data to cultivate more innovation and increase the focus of
teams on innovative work, as opposed to day-to-day work keeping the lights
on:
- “[Benefit]
is only limited by the imagination of the organization…I’d be hard
pressed to look at an organization in any industry that can’t benefit.”
- Mr.
Rose added that in practice about 80-90% of information management
budgets go towards ETL while
only 10% go toward innovation – a ratio that should be flipped.
- Stuart
Sim of PlaceIQ agreed
that big data infrastructure has opened the door to creative business
model restructuring, but cautioned
that with big data comes big data processing:
- Imagine
an advertising campaign that can take real time consumer data and spit
out a custom-tailored advertisement.
- Though
he has experienced the incredible power of real time data collection,
he was quick to note that it isn’t always the most effective
solution, echoing Mr. Rose’s woes about implementation of big data
systems.
- “The
biggest challenge around big data is assembling meaningful data you can
do the science on. I spend an awfully large chunk of my time making sure
data is not mangled.”
- It’s
not simply a matter of collecting petabytes of consumer data – you have
to know what to look for to capitalize on this data.
- Edouard
Servan-Schreiber of 10gen,
who had extensive experience with predictive modeling before becoming
a MongoDB specialist,
agreed that data cleansing is an
unfortunate fact of life but agreed that despite this necessity big
data is a platform for innovation:
- “To
me it's just, ‘we won't resolve this, we'll just manage how much we want
to clean [the data] before we use it’.”
- “Everything
is so heavy to move, when these big organizations are trying to do
anything, there is an enormous amount of data processing to do what seems
like a trivial thing.”
- Cleaning
and sorting through data will always be a part of the equation – in some
ways there’s no way around that reality
- Despite the drawbacks, it may be worth the rigorous processing – Mr. Servan-Schreiber asserted that he could use data alone to predict if someone was falling in love
When refining the incredible potential stored in big data, it’s
important to think about what you need, where it comes from and what it will
take to turn that data into an asset – by keeping the end goal in mind, it’s
easier to go from data to dollars focusing on innovating solutions rather than
wading through an ocean of data.
Fitzgerald Analytics