Big Data Blog

Big Data Blog

It’s Not about being Data-Driven

Dec 18

Written by:
12/18/2012 6:28 AM  RssIcon

One side effect of the era of big data is that data has become big in the sense that everyone is talking about how becoming a successful organization in any industry is all about being data-driven.

A commonly cited example is the bestselling book Moneyball by Michael Lewis, which is often interpreted as a story of how Major League Baseball finally got data religion and embraced data-driven decision making. This interpretation was further hyped by the popular movie based on the book, which starred Brad Pitt in the role of Oakland Athletics general manager Billy Beane.

This paradigm shift in baseball is often referred to as pitting the intuition-driven decisions of scouts, managers, and players against the data-driven decisions of analysts, mathematicians, and computer geeks using the Sabermetrics created by Bill James and applied by Billy Beane — a technique that Baseball Hall of Famer Joe Morgan once famously derided as “a bunch of geeks trying to play video games.”

However, the reality is that baseball has always been data-driven because of its wealth of statistical data. The real paradigm shift in baseball was realizing that the predictive power of some statistics was not as reliable as has been historically believed.

Baseball Data Driven In his bestselling book The Signal and the Noise: Why Most Predictions Fail but Some Don't, Nate Silver explained that the real lessons of Moneyball were “not whether statistics should be used, but which ones should be taken into account. On-base percentage (OBP), for instance, as analysts like James had been pointing out for years, is more highly correlated with scoring runs (and winning games) than batting average, a finding which long went under-appreciated by traditionalists within the industry.”

As Silver explained, the essence of Beane’s philosophy is “collect as much information as possible, but then be as rigorous and disciplined as possible when analyzing it. Rigor and discipline is applied in the way the organization processes the information it collects, and not in declaring certain types of information off-limits.”

And this philosophy includes not declaring intuition off-limits since, as I blogged about in my post Data-Driven Intuition (a term coined by Jeffrey Ma), what we call intuition is often more data-driven than we give it credit for because it’s based on personal experience and professional expertise (e.g., such as the valuable information still provided by baseball scouts).

Although the era of big data is often heralded as the clarion call for innovative decision making, “good innovators,” Silver concluded, “typically think very big and they think very small. New ideas are sometimes found in the most granular details of a problem where few others bother to look. And they are sometimes found when you are doing your most abstract and philosophical thinking, considering why the world is the way that it is and whether there might be an alternative to the dominant paradigm. Rarely can they be found in the temperate latitudes between these two spaces, where we spend 99 percent of our lives. The categorizations and approximations we make in the normal course of our lives are usually good enough to get by, but sometimes we let information that might give us a competitive advantage slip through the cracks.”

Baseball Intuitive Big Picture Is your organization ignoring valuable information that could give it a competitive advantage? I don’t just mean the myriad of new data sources created by our increasingly data-constructed world. Is your organization also leveraging the intuition of your business leaders and subject matter experts?

In the era of big data, it’s not about being data-driven — because your organization has always been data-driven. It’s about what data your organization is being driven by — and whether that data is driving your organization to make better decisions.

Location: Blogs Parent Separator Jim Harris

21 comment(s) so far...


# nihao

By TrackBack on   8/18/2014 4:46 AM

I've been cooking them upside down and unstuffed for years. As you mention, flipping a hot turkey is hard, dangerous, and defeats the purpose of cooking it upside down. I cook it the entire time upside down. Ogling a turkey is overrated. But if you are going to stuff the turkey, you may as well cook it right side up. The stuffing will collect all the juices that might have dripped into the breast.

By TrackBack on   8/21/2014 4:19 AM

Longchamp le pliage large tote Grey

Christmas Cheer in Crocker Park in Westlake
# Longchamp le pliage large tote Grey

By TrackBack on   8/21/2014 9:46 PM

Covina 2, aufgeschlagen Mission Viejo 0 Mary Letourneau eine drei Schl盲ger mit 12 Strikeouts als die Colts gewann die letzten 3 A Division im Mayfair Park.

By TrackBack on   8/22/2014 8:25 PM

C'est drôle comme on peut être satisfait par la simple idée d'avoir fait une telle affaire, alors même que je n'avais pas franchement besoin dans l'immédiat d'une fringue que je ne pourrai mettre avant quelques mois.

By TrackBack on   8/26/2014 8:59 PM

Bien vos serviettes hygiéniques lavables.

By TrackBack on   8/27/2014 10:49 PM

Ray Ban Polarized lunette Outlet RB2077

Jellypop's Coyne in dark brown is the perfect combination of tough biker girl meets ready for action city girl. With crisscross straps get ready for winter, $60
# Ray Ban Polarized lunette Outlet RB2077

By TrackBack on   8/28/2014 10:40 PM

Karen is a working mom with two young children. She has a passion for writing and especially loves crafts. She focuses her crafting articles on simple and inexpensive ways to make the best homemade gifts. Ka. View profile

By TrackBack on   8/30/2014 10:48 PM

By TrackBack on   9/1/2014 12:16 AM

Unless you have the machine to handle it, I'd stick with the 3.0Gb/s drives. From everything I've read, 95% of the systems out there and that are being built, don't even use the full 3Gb/s of standard drives. Save your money and just look for a drive with the most cache you can get, that will help the most, IMO and in others I've read. Also, If your looking into a SSD, make sure you do your reading. Unlike a standard "Disk" drive; you don't Defrag and have a handful of settings you have to set differently. The speed of a SSD is impressive though!

By TrackBack on   9/1/2014 7:47 AM

Portafoglio Gucci

Elle promène sa grâce féline sur les podiums du monde entier et fait partie du top 20 des mannequins les plus demandés. L'égérie de notre spécial mode est aussi une grande fille toute simple. Rencontre en coulisses du shooting. [.] Tels furent les premiers pas d'Heather Marks dans le monde de la mode. un flop. J'étais avec ma mère. J'avais 14 ans. On cherchait je ne sais plus quoi, peut être une machine à laver ou un réfrigérateur. Et puis quelqu'un, dans un haut parleur, a invité les filles à venir se présenter au concours de beauté pour devenir mannequin. [.] Je ne savais pas marcher, je pensais à autre chose, j'étais en baskets alors que les autres filles étaient maquillées et en talons aiguilles. Cette différence fut sa chance. Elle eut droit à un deuxième passage. Là, j'ai fermé la bouche. On ne voyait plus mon appareil. [.]
# Portafoglio Gucci

By TrackBack on   9/2/2014 9:30 PM

Do you agree that my self diagnosis is likely? Am I doing the right things? What else should I be doing/not doing?

By TrackBack on   9/4/2014 2:15 AM

If you return your item to us and it is found not to be in a suitable condition to qualify, the item will be returned to you at your expense.Christian Louboutin replica

By TrackBack on   9/5/2014 2:17 AM

scarpe con tacco alto prada

Vous pouvez également pétrir au robot mixeur.
# scarpe con tacco alto prada

By TrackBack on   9/11/2014 8:34 PM

louboutin homme mariage

Thank you, Laura. Good morning, everyone, and welcome to our second quarter 2013 conference call. On the call today are Blake Krueger, our Chairman, CEO and President; and Don Grimes, our Senior Vice President and CFO.
# louboutin homme mariage

By TrackBack on   9/13/2014 7:01 AM

gucci outlet online new york

The first pair of ray ban sunglasses called ray ban aviator was born in America in 1937. It was particularly designed for American pilots to protect their eyes from strong sunlight why flying. Since that, ray ban has become one of the symbols of American army. It also rapidly became popular in the worldwide. Ray ban became extremely hot among those celebrities and movie stars. For example, Cary Grant and Audrey Hepburn were its big fans.
# gucci outlet online new york

By TrackBack on   9/13/2014 4:40 PM

hollister deutschlan

Walters a aussi appris l'importance de rester en contact avec vos amis, les associations d'anciens 茅l猫ves, et des relations d'affaires tout en cultivant de nouveaux contacts. Le mantra des demandeurs d'emploi est le r茅seau, r茅seau, r茅seau.
# hollister deutschlan

By TrackBack on   9/14/2014 11:12 AM

louboutin femme quel prix

William Berger, an adviser for the foundation, says it is important to see a board certified allergist for diagnosis and treatment. He says it's a misperception that kids will outgrow allergies:
# louboutin femme quel prix

By TrackBack on   9/14/2014 10:58 PM

yelp gucci san diego

The renewed stand off between Ballydoyle and Godolphin in the big race is replicated by the next generation, both at Ascot today and at the Curragh tomorrow. You'll Be Mine (3.05) is bred to be top class and has made a persuasive start in maidens, but both superpowers can be eclipsed by John Gosden, on precisely the same account, with High Twelve (2.30).
# yelp gucci san diego

By TrackBack on   9/15/2014 5:28 PM

jordan femme glamour vetement

Lucky KURVEE ladies will win giveaways courtesy of IGIGI Fashions, Serge of Beverly Hills Salon, Sonsi and Kate Spade with more to be announced.
# jordan femme glamour vetement

By TrackBack on   9/17/2014 9:51 PM

hermes bags

"Malgr茅 un environnement de vente difficile, nous avons r茅alis茅 une croissance de ligne de fond tout en restant fid猫le au positionnement ambitieux de nos marques", a d茅clar茅 Abercrombie Fitch pr茅sident et chef de la direction Mike Jeffries dans un communiqu茅.
# hermes bags

By TrackBack on   9/18/2014 6:13 PM

Search Big Data Blogs


Big Data (126)
Analytics (66)
Pervasive (50)
DataRush (33)
Hadoop (31)
Industry trends (22)
predictive analytics (20)
Scalability (20)
Multicore (15)
Data Mining (12)
Parallelism (10)
Java (9)
Jim Harris (9)
Cloud (8)
Cyber Security (8)
MapReduce (8)
big data analytics (7)
Data Volumes (7)
Data Warehouse (7)
RushAnalytics (7)
Volumes (7)
Actian (6)
Algorithms (6)
Cost-effective (6)
David Loshin (6)
Decision Support (6)
Julie Hunt (6)
RushAnalyzer (6)
analytics tools (5)
Dataflow (5)
machine learning (5)
Data Science (4)
Forrester (4)
Google (4)
Green IT (4)
Healthcare (4)
Phil Simon (4)
YARN (4)
analytics processes (3)
Big Data Science (3)
BigQuery (3)
Bloor (3)
data centers (3)
data integration (3)
Data Preparation (3)
data tools (3)
data-driven (3)
DataMatcher (3)
machine generated data (3)
Malstone B (3)
Mike Hoskins (3)
Opera Solutions (3)
Retail Analytics (3)
Security (3)
Smart Grid (3)
software (3)
Solutions (3)
telecommunications (3)
transportation analytics (3)
Age of Data (2)
analytics accuracy (2)
architecture (2)
Austin (2)
Bloor Research (2)
Business Intelligence (2)
data management (2)
Data Rush (2)
David Inbar (2)
David Norris (2)
fraud (2)
fraud detection (2)
Gartner (2)
GigaOM (2)
Hadoop Summit (2)
IntegrationWorld (2)
intelligent transportation systems (2)
internet of things (2)
McKinsey (2)
meetup (2)
ParAccel (2)
Pervasive DataRush (2)
Rexer Analytics (2)
smart meters (2)
#FollowFriday (1)
a (1)
Amazon (1)
analytics workflow (1)
Application Development (1)
automation (1)
Benchmarks (1)
best practices (1)
Cloud Analytics Summit (1)
cloud computing (1)
Cloudera (1)
contests (1)
cost (1)
cyber security issues (1)
data flow architecture (1)
Data Integrator - Hadoop Edition (1)
data quality (1)
data visualization (1)
digital marketing (1)
Door64 (1)
easy big data analytics (1)
Ericson (1)
Esri (1)
Facebook (1)
Fuzzy Matching (1)
Goverment (1)
Hadoop User Group (1)
Hadoop World (1)
hardware (1)
HBase (1)
HDFS (1)
industrial internet (1)
Jazoon (1)
Jim Falgout (1)
MalStoneB (1)
Mansour Raad (1)
Neil Raden (1)
Netflix (1)
NetFlow (1)
operational intelligence (1)
Paige Roberts (1)
para (1)
PIG (1)
pilot projects (1)
Predictive Analytics World (1)
psychohistory (1)
Public Sector (1)
Redshift (1)
Robin Bloor (1)
ROI (1)
Rosaria Silipo (1)
RushAccelerator (1)
RushLoader (1)
Sampling (1)
Signal and Noise (1)
SmartDataCollective (1)
spatial analytics (1)
speed (1)
sports (1)
Stephen Swoyer (1)
Steve Shine (1)
Strata (1)
SXSW (1)
Telecom Analytics (1)
Telecommunications Industry Association (1)
TIA (1)
Transportation (1)
TurboRush (1)
VectorWise (1)
Zementis (1)

Latest Posts

Actian Big Data & Analytics Blog has MOVED!
Big Data Phrenology
Big Data, Simpson's Paradox and Sufficient Tools
Data Science and the Art of Data Visualization

Big Data Blog Archives

<September 2014>

Accelerating Big Data 2.0™