Big Data Blog

Big Data Blog

It’s Not about being Data-Driven

Dec 18

Written by:
12/18/2012 6:28 AM  RssIcon

One side effect of the era of big data is that data has become big in the sense that everyone is talking about how becoming a successful organization in any industry is all about being data-driven.

A commonly cited example is the bestselling book Moneyball by Michael Lewis, which is often interpreted as a story of how Major League Baseball finally got data religion and embraced data-driven decision making. This interpretation was further hyped by the popular movie based on the book, which starred Brad Pitt in the role of Oakland Athletics general manager Billy Beane.

This paradigm shift in baseball is often referred to as pitting the intuition-driven decisions of scouts, managers, and players against the data-driven decisions of analysts, mathematicians, and computer geeks using the Sabermetrics created by Bill James and applied by Billy Beane — a technique that Baseball Hall of Famer Joe Morgan once famously derided as “a bunch of geeks trying to play video games.”

However, the reality is that baseball has always been data-driven because of its wealth of statistical data. The real paradigm shift in baseball was realizing that the predictive power of some statistics was not as reliable as has been historically believed.

Baseball Data Driven In his bestselling book The Signal and the Noise: Why Most Predictions Fail but Some Don't, Nate Silver explained that the real lessons of Moneyball were “not whether statistics should be used, but which ones should be taken into account. On-base percentage (OBP), for instance, as analysts like James had been pointing out for years, is more highly correlated with scoring runs (and winning games) than batting average, a finding which long went under-appreciated by traditionalists within the industry.”

As Silver explained, the essence of Beane’s philosophy is “collect as much information as possible, but then be as rigorous and disciplined as possible when analyzing it. Rigor and discipline is applied in the way the organization processes the information it collects, and not in declaring certain types of information off-limits.”

And this philosophy includes not declaring intuition off-limits since, as I blogged about in my post Data-Driven Intuition (a term coined by Jeffrey Ma), what we call intuition is often more data-driven than we give it credit for because it’s based on personal experience and professional expertise (e.g., such as the valuable information still provided by baseball scouts).

Although the era of big data is often heralded as the clarion call for innovative decision making, “good innovators,” Silver concluded, “typically think very big and they think very small. New ideas are sometimes found in the most granular details of a problem where few others bother to look. And they are sometimes found when you are doing your most abstract and philosophical thinking, considering why the world is the way that it is and whether there might be an alternative to the dominant paradigm. Rarely can they be found in the temperate latitudes between these two spaces, where we spend 99 percent of our lives. The categorizations and approximations we make in the normal course of our lives are usually good enough to get by, but sometimes we let information that might give us a competitive advantage slip through the cracks.”

Baseball Intuitive Big Picture Is your organization ignoring valuable information that could give it a competitive advantage? I don’t just mean the myriad of new data sources created by our increasingly data-constructed world. Is your organization also leveraging the intuition of your business leaders and subject matter experts?

In the era of big data, it’s not about being data-driven — because your organization has always been data-driven. It’s about what data your organization is being driven by — and whether that data is driving your organization to make better decisions.

Location: Blogs Parent Separator Jim Harris

7 comment(s) so far...


# nihao

By TrackBack on   8/18/2014 4:46 AM

I've been cooking them upside down and unstuffed for years. As you mention, flipping a hot turkey is hard, dangerous, and defeats the purpose of cooking it upside down. I cook it the entire time upside down. Ogling a turkey is overrated. But if you are going to stuff the turkey, you may as well cook it right side up. The stuffing will collect all the juices that might have dripped into the breast.

By TrackBack on   8/21/2014 4:19 AM

Longchamp le pliage large tote Grey

Christmas Cheer in Crocker Park in Westlake
# Longchamp le pliage large tote Grey

By TrackBack on   8/21/2014 9:46 PM

Covina 2, aufgeschlagen Mission Viejo 0 Mary Letourneau eine drei Schl盲ger mit 12 Strikeouts als die Colts gewann die letzten 3 A Division im Mayfair Park.

By TrackBack on   8/22/2014 8:25 PM

C'est drôle comme on peut être satisfait par la simple idée d'avoir fait une telle affaire, alors même que je n'avais pas franchement besoin dans l'immédiat d'une fringue que je ne pourrai mettre avant quelques mois.

By TrackBack on   8/26/2014 8:59 PM

Bien vos serviettes hygiéniques lavables.

By TrackBack on   8/27/2014 10:49 PM

Ray Ban Polarized lunette Outlet RB2077

Jellypop's Coyne in dark brown is the perfect combination of tough biker girl meets ready for action city girl. With crisscross straps get ready for winter, $60
# Ray Ban Polarized lunette Outlet RB2077

By TrackBack on   8/28/2014 10:40 PM

Search Big Data Blogs


Big Data (126)
Analytics (66)
Pervasive (50)
DataRush (33)
Hadoop (31)
Industry trends (22)
predictive analytics (20)
Scalability (20)
Multicore (15)
Data Mining (12)
Parallelism (10)
Java (9)
Jim Harris (9)
Cloud (8)
Cyber Security (8)
MapReduce (8)
big data analytics (7)
Data Volumes (7)
Data Warehouse (7)
RushAnalytics (7)
Volumes (7)
Actian (6)
Algorithms (6)
Cost-effective (6)
David Loshin (6)
Decision Support (6)
Julie Hunt (6)
RushAnalyzer (6)
analytics tools (5)
Dataflow (5)
machine learning (5)
Data Science (4)
Forrester (4)
Google (4)
Green IT (4)
Healthcare (4)
Phil Simon (4)
YARN (4)
analytics processes (3)
Big Data Science (3)
BigQuery (3)
Bloor (3)
data centers (3)
data integration (3)
Data Preparation (3)
data tools (3)
data-driven (3)
DataMatcher (3)
machine generated data (3)
Malstone B (3)
Mike Hoskins (3)
Opera Solutions (3)
Retail Analytics (3)
Security (3)
Smart Grid (3)
software (3)
Solutions (3)
telecommunications (3)
transportation analytics (3)
Age of Data (2)
analytics accuracy (2)
architecture (2)
Austin (2)
Bloor Research (2)
Business Intelligence (2)
data management (2)
Data Rush (2)
David Inbar (2)
David Norris (2)
fraud (2)
fraud detection (2)
Gartner (2)
GigaOM (2)
Hadoop Summit (2)
IntegrationWorld (2)
intelligent transportation systems (2)
internet of things (2)
McKinsey (2)
meetup (2)
ParAccel (2)
Pervasive DataRush (2)
Rexer Analytics (2)
smart meters (2)
#FollowFriday (1)
a (1)
Amazon (1)
analytics workflow (1)
Application Development (1)
automation (1)
Benchmarks (1)
best practices (1)
Cloud Analytics Summit (1)
cloud computing (1)
Cloudera (1)
contests (1)
cost (1)
cyber security issues (1)
data flow architecture (1)
Data Integrator - Hadoop Edition (1)
data quality (1)
data visualization (1)
digital marketing (1)
Door64 (1)
easy big data analytics (1)
Ericson (1)
Esri (1)
Facebook (1)
Fuzzy Matching (1)
Goverment (1)
Hadoop User Group (1)
Hadoop World (1)
hardware (1)
HBase (1)
HDFS (1)
industrial internet (1)
Jazoon (1)
Jim Falgout (1)
MalStoneB (1)
Mansour Raad (1)
Neil Raden (1)
Netflix (1)
NetFlow (1)
operational intelligence (1)
Paige Roberts (1)
para (1)
PIG (1)
pilot projects (1)
Predictive Analytics World (1)
psychohistory (1)
Public Sector (1)
Redshift (1)
Robin Bloor (1)
ROI (1)
Rosaria Silipo (1)
RushAccelerator (1)
RushLoader (1)
Sampling (1)
Signal and Noise (1)
SmartDataCollective (1)
spatial analytics (1)
speed (1)
sports (1)
Stephen Swoyer (1)
Steve Shine (1)
Strata (1)
SXSW (1)
Telecom Analytics (1)
Telecommunications Industry Association (1)
TIA (1)
Transportation (1)
TurboRush (1)
VectorWise (1)
Zementis (1)

Latest Posts

Actian Big Data & Analytics Blog has MOVED!
Big Data Phrenology
Big Data, Simpson's Paradox and Sufficient Tools
Data Science and the Art of Data Visualization

Big Data Blog Archives

<August 2014>

Accelerating Big Data 2.0™