Big Data Blog

Big Data Blog

By Jim Harris on 9/17/2013 7:10 AM

Although big data turbocharges information overload, a lot of what we relentlessly datafy and digitize also turbocharges information underwhelm.  So, while you are scratching your head trying to tell the difference between signal and noise, don’t try to understand every bump you will no doubt discover atop your head.

By Phil Simon on 8/29/2013 6:53 AM

Yes, Simpson's Paradox existed well before the advents of YouTube, Instagram, Twitter. With so much information available to us, what truths will it mask? What's more, an increasing percentage of this data is unstructured and, I would argue, subject to some level of interpretation. Big Data will lead to some big discoveries and insights, but also some big mistakes.

By Phil Simon on 8/12/2013 7:00 AM

Everyone is talking about Big Data, but relatively few organizations are actually doing anything with it. To that end, there’s tremendous upside for skilled organizations and first movers—and it won’t last forever. In other words, when the bar is low, the time to act is now.

By Paige Roberts on 8/8/2013 7:00 AM

I was a fly on the wall for an interesting conversation on Twitter between two UK centered BI / Analytics and Data Warehouse consultants, Jacqui Taylor of Flying Binary, and Joe Harris. Their conversation got me thinking about the big data gravitational giant that Hadoop has become, and where we fit in its orbit.

By Phil Simon on 7/22/2013 7:04 AM
What's the ROI on Big Data? It's the big question, isn't it? A recent InformationWeek survey manifests the same thing. There's a chasm between hope and reality. Brass tacks: there's no simple answer, but there's plenty to consider before going down that road. In no particular order, here are a few things to think about before taking the plunge.
By Phil Simon on 7/8/2013 7:15 AM

As any pro will tell you, over the long term, in poker you’ll do better when you play against loose, clueless, and irresponsible players. Of course, on any given hand, a much worse player may beat you because of dumb luck. If you play all day next to sharks like Doyle Brunson, Johnny Chan, and “Poker Brat” Phil Helmuth, you may win a hand or two. Over a few hours or days, though, you’ll almost certainly lose your shirt.

By Robin Bloor on 6/28/2013 6:55 AM

I have mentioned the idea of an event-driven architecture before, if not here on this blog, then elsewhere on the web. The idea is simple enough. The business is interested in events and nowadays it can capture many events. We never used to think in terms of events as a foundation for building a computer system for a business. We thought in terms of transactions. It's time to shift our thinking.

By Jim Harris on 6/24/2013 7:00 AM

In their new book Decisive: How to Make Better Choices in Life and Work, Chip Heath and Dan Heath shared the story of a scientific equipment manufacturer that was deciding whether to make a big bet on wireless sensors.  This was back in 2006 when the technology was still in the nascent phase.  Rather than jump headfirst into the wireless market with an “all or nothing” approach, the manufacturer chose a “little something” approach, realizing what they needed to do was ooch.

By Jim Harris on 6/24/2013 7:00 AM

In their new book Decisive: How to Make Better Choices in Life and Work, Chip Heath and Dan Heath shared the story of a scientific equipment manufacturer that was deciding whether to make a big bet on wireless sensors.  This was back in 2006 when the technology was still in the nascent phase.  Rather than jump headfirst into the wireless market with an “all or nothing” approach, the manufacturer chose a “little something” approach, realizing what they needed to do was ooch.

By Phil Simon on 6/17/2013 6:51 AM

On May 13th, I traveled from my home in Las Vegas, Nevada to Toronto, Ontario, Canada. The purpose of my trip: to keynote a conference. As I am wont to do, I visited the airport bookstore to peruse the latest releases. I was shocked to see Big Data: A Revolution That Will Transform How We Live, Work, and Think front and center at Hudson’s. Yes, literally right next to books from Michael Crichton, John Grisham, and other bestselling authors, I found a book about the massive amounts of unstructured data streaming at us faster than ever. 

By Jim Harris on 6/3/2013 7:00 AM

iStock12666083In his book The Victory Lab: The Secret Science of Winning Campaigns, journalist Sasha Issenberg explored the analytical revolution upending the way political campaigns are run in the 21st century, including cutting-edge persuasion experiments and innovative ways to mobilize voters, which is re-engineering a high-stakes industry previously run on little more than gut instinct and outdated assumptions...

By David Loshin on 5/23/2013 10:21 AM

There are a number of decisions to be considered when assembling your development, testing, and production environments for big data application development, mostly surrounding the selection of hardware, software, data management, and application development platform...

By Rosaria Silipo on 5/23/2013 7:00 AM

Rosaria Silipo discusses her use of Actian RushAnalytics to tackle a time-sensitive big data project. Using RushAnalytics, she found the results to be "amazing." The execution time of the whole workflow went down from circa 3 days to one hour and 16 minutes! Read more about this interesting KNIME big data project.

By David Loshin on 5/20/2013 7:00 AM

In my last blog post, I suggested that integrating a big data pilot involved developing a program plan consisting of multiple work streams focusing on architecture and design, hardware/software/data infrastructure, data management, and system configuration and management. At the same time, a critical change management task is to socialize and then develop skills for big data application development, and essentially make Big Data application development part of the information strategy...

By Richard Maddox on 5/17/2013 7:00 AM

Door64, Austin’s largest professional technology organization, recently held its first “Big Data & Analytics” Speaker Series and Exhibit. The event served as another proof point that Austin continues to become a leading big data city. A number of comments, in fact, were made about the high level of big data and Hadoop knowledge among attendees...

By Jim Harris on 5/13/2013 7:00 AM

“Bow your head: the hot buzzword big data has ascended to royalty,” declared Eric Siegel, in his book Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die.  “It’s in every news clip, every data science presentation, and every advertisement for analytics solutions.  It’s a crisis!  It’s an opportunity!  It’s a crisis of opportunity!” Siegel then shares a big secret about big data...

By Richard Maddox on 5/10/2013 8:07 AM

Don’t work with a monstrosity when the future of big data analytics is here now. Tame the Horror of Exploding Data Volumes with a lot less stress and expense – and get the answers to the questions you want to ask, fast enough to save your world...

By Julie Hunt on 5/6/2013 7:00 AM

These days the capabilities for processing and analyzing big data are not just high tech luxury for large companies – online retailers of all sizes can take advantage of big data insights due to many cost-effective options both for on-premises and in the cloud. “Big data” doesn’t simply mean large volumes of data – it’s more about data that is difficult to process and analyze through traditional methods. All businesses potentially have “big data” in one form or another that could yield significant or even game-changing insight. The starting point for any company is developing a strategy and the right questions that can be answered by big data analysis...

By Richard Maddox on 5/3/2013 11:50 AM

Actian Corporation’s recent acquisition activity grabbed TDWI’s (The Data Warehousing Institute) attention.  Following the acquisitions of Pervasive Software, ParAccel and VectorWise, TDWI’s Steve Swoyer interviewed Actian’s CTO Mike Hoskins (formerly Pervasive’s CTO) to get a bead on where the company is headed post acquisition...

By Jim Harris on 4/29/2013 7:00 AM

In their recent Harvard Business Review blog post, Jeff Bladt and Bob Filbin explained a data scientist’s real job is storytelling.  “Data gives you the what, but humans know the why.  Without a human frame, data will only confuse, and certainly won’t lead to smart organizational behavior.  The best business decisions come from intuitions and insights informed by data.  Data scientists want to believe that data has all the answers.  But the most important part of our job is qualitative: asking questions, creating directives from our data, and telling its story.” Read more...

By Richard Maddox on 4/26/2013 7:00 AM

Data reality doesn’t byte. It terabytes, and increasingly petabytes. Consequently, many organizations are running scared, overrun with data. Data warehouses in the cloud can be terrific options for both enterprises and small companies to deal with vast amounts of data, including unstructured data. The reasons: Data warehouses in the cloud generally are cost effective, secure, scalable, fault tolerant, and often less stressful than handling data challenges internally...

By David Loshin on 4/22/2013 8:00 AM

It is unfortunate when the enthusiasm over new technology overwhelms the establishment of grounded business justifications for that technology’s adoption. But often project plans focus on the delivery of the capability in ways that neglect solving specific business problems. In these situations, components of the technology are delivered and milestones are reached, but at some point people realize that the product does not address the end-user expectations, triggering a redesign in the best case or abandonment in the worst case...

By Richard Maddox on 4/19/2013 5:00 PM

IntegrationWorld 2013 (#IW13) – always an important Pervasive Software event – took on even more importance this year. Just a few days ahead of the event, Actian Corporation acquired Pervasive Software. With many customers, partners and analysts on hand, Steve Shine, Actian’s CEO, had the opportunity to outline the Actian/Pervasive vision moving ahead...

By Richard Maddox on 4/16/2013 3:01 PM

Bloor Research’s Analytics expert, David Norris, and Actian CTO Mike Hoskins, recently participated in a webinar to discuss the future of the date warehouse in the Age of Data. A number of timely topics were discussed...

By Jim Harris on 4/15/2013 7:30 AM

In his recent blog post Big Data and The Wizard of Oz Syndrome, Rick Sherman examined mistakes commonly made by enterprises when initiating their first big data projects, noting there is nothing new about these common mistakes, as they are the same ones enterprises have been making during every new technology wave...

By Richard Maddox on 4/12/2013 8:00 AM

Mansour Raad, senior software architect at Esri, recently used DataRush in a workflow demo at the DevSummit in Palm Springs, CA.  His work in spatial analytics shows heatmap data visualization for trend detection ...

By Richard Maddox on 4/12/2013 8:00 AM

Mansour Raad, senior software architect at Esri, recently used DataRush in a workflow demo at the DevSummit in Palm Springs, CA.  His work in spatial analytics shows heatmap data visualization for trend detection ...

By David Loshin on 4/8/2013 9:19 AM

Let’s presume that your big data analytics pilot has demonstrated value in a way that is reasonable from a management perspective and feasible from an execution perspective.  The next step is to take the blueprint and roadmap and transition them into a more comprehensive program plan, consisting of...

By Richard Maddox on 4/4/2013 1:34 PM

The Pervasive Big Data & Analytics team will be in the heart of the action at this year’s Pervasive IntegrationWorld event on April 14-16 in Austin. If you’re attending, stop by our booth. Our expert technical staff will be on hand to handle your specific big data questions, and show off our killer analytic technology. Bring us your analytic challenge, and we’ll take a shot at helping you solve it...

By Richard Maddox on 3/28/2013 9:07 AM

Bloor Research’s David Norris, Practice Leader in Analytics, in his excellent article “Is Data Warehousing Holding Back the Advance of Analytics?” faces the realization that “by extracting data from the operational environment, and loading it into a business intelligence environment, we introduce limitations that defeat the basis of why we set about doing it in the first place.” Norris reviews the traditional data warehouse environment and draws attention to the setbacks companies face attempting to analyze large volumes of data in enough time to take meaningful action...

By Robin Bloor on 3/25/2013 8:25 AM

Scale Up, Scale OutSUSO stands for “Scale Up Scale Out,” in that order. It suggests, correctly in my view, that in designing and implementing software architecture, the designer should first scale up to use all compute power available on each machine before scaling out to multiple machines.

By Richard Maddox on 3/22/2013 8:43 AM

The Hadoop platform’s MapReduce, with its rock-star status gained from the success it delivered to Google’s Web search, continues to be the driver behind many new applications across many business sectors, providing distributed pattern-based searching, distributed sorting, index building, machine learning and more. Both the strengths and weaknesses of MapReduce resonate through the real-world applications of this new technology...

By Jim Harris on 3/18/2013 7:50 AM

Data Driven Traffic JamIn his bestselling book The Signal and the Noise: Why Most Predictions Fail but Some Don’t, Nate Silver provided an excellent example of the potential downside of the widespread adoption of the data-driven decision proces inherent in automotive navigation systems. “A self-canceling prediction,” Silver explained, “is a case where a prediction tends to undermine itself. One interesting case is the GPS navigation systems that are coming into more and more common use.

By Richard Maddox on 3/15/2013 10:58 AM

The KNIME community is growing. The turnout at the recent KNIME User Workshop in Zurich attracted over 150 people from Europe, the US, Japan, Australia and elsewhere – a new record for the event, now in its 6th year. While traditionally known for managing complex chemical and bio-informatics data pipelines, KNIME is increasingly being adopted for data mining in other areas such as customer analytics, financial services, risk assessment and social networks where there are many opportunities for extracting more value from big data.

By David Loshin on 3/11/2013 5:58 AM

In my last post, I suggested that in many cases the stereotypical categories of big data application involved one of two aspects of performance improvement: either “doing things faster/cheaper” or “getting better results.” In either case, though, determining the degree to which big data addresses the business drivers is a function of casting the performance improvement in the context of specific organizational business drivers such as increased revenue, decreased costs, improved productivity, reduced risk, or improved customer experience, among others. As with any value justification, quantifying the success criteria for big data necessarily involves linking “bigger,” “faster,” “cheaper,” and/or “better” to measurable value.

By Taylor Bloom on 3/8/2013 10:22 AM

“Big Data,” a compilation of data sets so enormous and complex that they are difficult to decipher using traditional database management resources–is one of the latest trends to shake up the sports world. Creating a harmonious relationship between jocks and nerds, Big Data analysis is causing professional sports teams to utilize new technologies in order to make sense of mountains of previously untapped sports data.

By Richard Maddox on 3/1/2013 6:07 AM

Pervasive Big Data & Analytics Chief Technologist Jim Falgout recently had an opportunity to speak to the Bay Area Hadoop User Group (HUG), along with Mukund Madhugiri and Baljit Deot of Yahoo! and Hari Shreedharan of Cloudera. Jim discussed the major barriers to effective Hadoop deployments in the enterprise – complexity and the steep learning curve of MapReduce.

By Julie Hunt on 2/25/2013 5:57 AM

Cyber SecurityA lot has been happening to change how IT operates in enterprises with the entrance of new technologies, including the consumerization of IT, BYOD, and cloud computing. Unfortunately, many of the innovations and changes for corporate IT have opened the door for escalated cyber security challenges. Corporate security teams now have to address global venues for protecting the enterprise and can no longer view security as a silo’d function of “wall building” and defensive functions. Attacks by sophisticated cyber criminals and hackers call for proactive cyber security processes, where enterprises continuously hunt for current and potential threats.

By Richard Maddox on 2/21/2013 5:41 AM

Machine LearningWe’d like to share the ten Big Data & Analytics blog posts which we believe are the most insightful, entertaining and useful posts of 2012. These posts received–and still receive–a lot of reader interest and cover a range of big data and predictive analytics topics from specific business uses to the future of data warehouses in the Hadoop world to futuristic thinking of where predictive analytics may be leading us (the good and bad)–and more.

By David Loshin on 2/18/2013 5:57 AM

In my last post I began to look at the types and characteristics of business problems that are suited to big data analytics programming models, and one conclusion that we can draw from that discussion is that one of the main drivers of suitability is the desire to improve performance. Interestingly, though, I can break that desire for improved performance into two different categories: improved computational performance vs. improved business performance.

By Paige Roberts on 2/14/2013 6:00 AM

If you think that the overwhelming media focus on new big data analytics methods and technologies is just a Bad Obsession, well You Ain’t the First. But the thing about the trough of disillusionment is that it leads eventually to the plateau of productivity, and that’s where businesses get the real benefit from new technology. 

By Jim Harris on 2/11/2013 5:45 AM

As the dawn of a new year is ascending, the big data hype cycle is descending from the peak of inflated expectations, and with it a lot of recent blog posts have been written in an attempt to provide a more balanced perspective about the potential of big data analytics.

By Richard Maddox on 2/8/2013 2:35 PM

SmartDataCollectivePaige Roberts, Technology Adoption Manager for Pervasive Big Data & Analytics, found a recent SmartDataCollective blog post by Kathyrn Kelly, Data Scientists Not Required: Big Data Is About Business Users dead-on in her assessment that analytics results need to be more accessible to business users.

By David Loshin on 2/4/2013 5:45 AM

In my last post, we started to explore some of the characteristic types of applications that had been adapted using big data techniques and we considered the underlying drivers. What we saw was that in many cases, most of the applications listed by users as built using the big data platform Hadoop were not new from an algorithmic perspective, but were adaptations of familiar solutions targeted at a platform that would be expected to scale with the growth of the volumes and varieties of inputs.

By Richard Maddox on 2/1/2013 6:31 AM

A recent Twitter exchange centered on those big data cities that could be deemed “Super Bowl” cities. San Francisco and Boston, as expected, came up. There’s no doubt the Bay area, Boston-Cambridge – and increasingly New York – are major big data and analytics hubs. There are also others that are notable. Southern California, for one, is a growing presence, as is Chicago. And then there’s Austin, recently named by Forbes as the fastest growing city in the U.S. and home to Pervasive Software and our Pervasive Big Data & Analytics product team. Besides us, there are many growing big data and analytics organizations and companies in the area that are making their marks.

By Julie Hunt on 1/28/2013 5:29 AM

A popular observation these days is that big data democratizes decision-making and analytics – in other words, big data mining technologies can bring large enterprise capabilities to many more organizations at a reasonable cost, in part thanks to the Hadoop platform. For the Insurance industry this means the ability to proactively implement greatly improved methods and processes to speed up and expand fraud detection. The usual modus operandi for the Insurance industry when adopting new technologies has been that of a clear laggard. However the high price of fraud and the increasing availability of cost-effective technology solutions for working with big data should push many Insurers into taking advantage of big data insights.

By David Loshin on 1/21/2013 6:02 AM

The lure of a new technology often blinds people to a clear understanding of what the technology will or won’t do, and the continued hype around big data is no exception. We have seen a number of customers not just talking about tinkering with big data technologies, they have also begun to experiment and pilot different approaches, but with limited success. Interestingly the root cause of their disappointment is not the technology per se, but rather the fact that their expectations aren’t met because they have not adequately determined what they plan to achieve using the technology or they do not understand the benefits and drawbacks of the technology.

By Paige Roberts on 1/18/2013 5:41 AM

Recycle Hadoop PowerIn January, the birth of a new year, we think about how to make the new year better than the last. Now is the birth of a new technology revolution. Now is the time to resolve to learn from the costs associated with data centers already in production. Current data centers waste over 80% of compute power, waiting for peak loads and using single-threaded serial software designed for the machines of the last millennium. That’s the kind of bad habit we simply can’t afford if the new big data technology strategies like Hadoop are going to become widespread. That much waste is not only unforgivable in the microcosm of a company’s IT budget, but unsustainable in the macrocosm of the world economy.

By Jim Harris on 1/14/2013 5:50 AM

There are times when big data seems to resemble that big mountain of mashed potatoes formed by "Weird Al" Yankovic in the movie UHF, about which he mutters (mimicking Richard Dreyfuss in the movie Close Encounters of the Third Kind): "This means something. This is important." Big data analytics sometimes seems like we are digging through all those mashed potatoes with sporks, hoping to simultaneously scoop away all the noise and stab any signals we can find.

By Richard Maddox on 1/8/2013 10:04 AM

Big Data Contest OpportunitiesBig data contests offer a great opportunity for developers and startups to show their stuff, while adding workable solutions to organizations seeking to use big data to improve operations and relationships. Here are some important contest portals and a few of the interesting contests going on. If you know of a new contest or another contest portal, please put it in the comments. We’d like to help you get the word out.

By Richard Maddox on 12/20/2012 5:53 AM

According to comScore’s tracking of 2012 holiday retail sales, $33.8 billion has been spent online, marking a 13% increase over the same days of 2011. Reasons for the online sales increase are many, including the proliferation of handheld devices and weather conditions.

By Jim Harris on 12/18/2012 6:28 AM

Baseball Data Driven One side effect of the era of big data is that data has become big in the sense that everyone is talking about how becoming a successful organization in any industry is all about being data-driven. A commonly cited example is the bestselling book Moneyball by Michael Lewis, which is often interpreted as a story of how Major League Baseball finally got data religion and embraced data-driven decision making.

By Jim Harris on 12/10/2012 1:24 PM

Using the science fiction of Asimov’s Psychohistory, Paige Roberts recently blogged about whether or not data science can accurately predict human behavior using machine learning and predictive analytics.

By Julie Hunt on 12/3/2012 10:58 AM

Telecom Analytics Communication networks have become the center of the universe for most businesses and consumers on a global basis, particularly with the continual proliferation of smart devices that are now pounding telecom service providers 24/7 with intense demand for bandwidth and reliable connectivity.

By Liz Ivey on 11/29/2012 11:44 AM

It’s no secret that data, as a whole, is hard to analyze. If only there was an easy button.

Traditional data analysis and unstructured data are colliding. Organizations are seeing the value in mining their big data avalanche, filled with structured and unstructured data.  The rise of social media has been a gateway for unstructured data analysis, such as text-based or sentiment analysis of twitter feeds, emails, applications, call center logs, and so much more.

By Richard Maddox on 11/21/2012 5:33 AM

Retail Analytics There will be those retailers celebrating the holidays because they implemented big data strategies and gleaned insights to better compete in the challenging retail environment. Unfortunately, other retailers will head into 2013 facing sales shortfalls and lost customers. A primary reason: They likely failed to use big data to get a real-time bead on what their tech-savvy holiday shoppers want and then failed to deliver on those wishes.

By Paige Roberts on 11/19/2012 3:32 PM

Predictive Analytics Last week, I had an interesting conversation with James Urquhart of GigaOm and enStratus, with a comment or two from Reuven Cohen of Forbes and Virtustream. The conversation centered around the idea that current and planned future paths of big data predictive analytics are looking more and more like the imaginary science of psychohistory, invented by classic science fiction author, Isaac Asimov, in his Foundation novels.

By Richard Maddox on 11/16/2012 4:15 PM

Our partner Opera Solutions has been selected by the Centers for Medicare & Medicaid Services (CMS) to provide advanced analytics to enhance operational controls and prevent fraud in federally funded Health Insurance Exchange Operations (HIX) managed by the CMS Center for Consumer Information and Insurance Oversight (CCIIO).

By Jim Harris on 11/12/2012 6:13 AM

In my blog post Darth Vader, Big Data, and Predictive Analytics, I explained that analytics is not a forced choice between human cognition and computer automation.  Instead, humans, with their intuition-driven tacit understanding, and computers, with their data-driven statistical models, must work together to bring balance to the Analytical Force.

By Richard Maddox on 11/9/2012 4:09 PM
As big data grows, cyber security threats loom larger. This is a ‘given’ now facing any organization that collects and stores volumes of data. Even smaller and mid-size organizations are not immune....
By Richard Maddox on 11/2/2012 12:59 PM
October’s Strata + Hadoop World proved to be fertile ground for big announcements and important insights. Attendees, including Cloudera CEO Mike Olsen, pointed out that Hadoop and MapReduce, even with...
By Jim Harris on 10/29/2012 5:40 AM

Historically, standing between operations and analytics was the hulking amalgamation of extracted, transformed, and loaded data that anxiously awaited the queries of business users.  It was called the Data Warehouse (or sometimes the Data Outhouse). 

By Richard Maddox on 10/26/2012 5:48 AM
Pervasive Big Data & Analytics Chief Technologist Jim Falgout discusses the Hadoop platform and how it is evolving. His...
By Julie Hunt on 10/25/2012 6:14 AM

Many companies are finally recognizing that customer service and contact centers are important resources for competitive advantage and improving customer experiences. It’s easy to see that consistently high-quality customer experiences resulting from well-performing customer service organizations lead to successful business outcomes for the company. Satisfied customers are likely to come back for more and recommend the company’s products. The challenge for companies: customer expectations for quality interactions with companies are growing.

By Paige Roberts on 10/23/2012 12:25 PM
John Furrier and Dave Valente of Silicon Angle’s theCube caught up with David Inbar of Pervasive Big Data & Analytics at the IBM Information on Demand event in Las Vegas yesterday. David answered some...
By Neil Raden on 10/17/2012 8:57 AM
Can the ability to extract meaning and sentiment from previously unconventional data sources reorient the role of business rules?

In a typical customer application, scoring models are created...
By Richard Maddox on 10/4/2012 9:41 AM
This is exciting news! Opera Solutions is spotlighting its use of Pervasive DataRush™ on its blog....
By Jim Harris on 10/2/2012 6:15 AM

“He’s more machine now than man.  His mind is twisted and evil.”

I sense a similar sentiment in business leaders who resist the machine learning algorithms that are becoming more of a necessity in the era of big data.  Some business leaders fear that the future of big data analytics could twist decision making into becoming more machine than man — if more decisions are made by a computer algorithm, and fewer decisions are made by a human mind.

By Paige Roberts on 9/28/2012 6:23 AM
The goal of predictive analytic strategies in a business is to predict future trends that will affect the company’s bottom line, and use that information to make better decisions. The business that can do this better than their competitors wins cost reductions, revenue increases, and happy stock holders. So, in this race for knowledge, how does one company get the checkered flag?
By Richard Maddox on 9/27/2012 4:56 PM
Pervasive Big Data & Analytics very much looks forward to Predictive Analytics World in Boston on September 30-October 4, 2012....
By Julie Hunt on 9/26/2012 7:19 AM
Big data analytics are becoming an important source of intelligence for marketers – but it is also an overwhelming undertaking for most marketers, many of whom are quickly having to add data analyst...
By Richard Maddox on 9/20/2012 6:02 AM
Mike Hoskins, Pervasive Big Data & Analytics General Manager, has embarked on a series of short videos to drive home a few hard truths about big data. He’s candid about what businesses need to think...
By Jim Harris on 9/18/2012 6:24 AM
Because of concerns about its signal-to-noise ratio, big data analytics is sometimes compared to finding a golden needle in a haystack of data. In other words, you have to dig through a whole lot of hay (i.e., massive amounts of data) before you find a golden needle (i.e., data-driven business insight).
By Pervasive Big Data on 9/12/2012 4:41 PM
Pervasive Big Data and Analytics Technology Adoption Manager, Paige Roberts, has an article in SmartDataCollective called “A Different Strategy for Solvable Problems in Big Data Predictive Analytics.”...
By Pervasive Big Data on 9/7/2012 3:40 AM
Pervasive CTO Mike Hoskins and InformationWeek Reporter Jeff Bertolucci recently discussed the current and future state of big data...
By David Inbar on 8/30/2012 10:57 AM
A couple of weeks ago, we applauded how hardware vendors and data center designers have been assiduously working on reducing energy consumption in servers, networks and facilities. And we not-so-gently raised the question about the hardware and energy efficiency of software. This blog picks up on the theme because outdated, inefficient analytics software is getting in the way of the promise of big data. 
By Pervasive Big Data on 8/29/2012 4:10 AM
Bigger, Better, and Faster Data Science made a splash this past weekend in Fremont, California, at the Big Data Science Meetup....
By Pervasive Big Data on 8/23/2012 8:49 AM

Pervasive Big Data heads to Fremont, CA, this weekend to join the Big Minds of Google and Zementis at the Big Data Science Meetup.  We know your weekends are precious, but this special opportunity to meet and learn from Jim Falgout (Pervasive Big Data), Ryan Boyd (Google) and Michael Zeller (Zementis).

By Paige Roberts on 8/21/2012 4:12 AM
Historically, in a situation where there is a lot of data to analyze, particularly if that data is being generated very rapidly and has to be analyzed as it is produced, analytics software has fallen...
By Paige Roberts on 8/17/2012 4:05 AM
A look at the Rexer Analytics Data Miner Survey and the KDnuggets Analytics, Data Mining, Big Data Software Poll, with particular attention to the analytics tools rated highest in popularity and customer satisfaction: STATISTICA, R, Rapid Miner, and KNIME.
By Paige Roberts on 8/6/2012 4:00 AM
Green IT is a subject dear to my tree-hugging heart and soul. As it should be to anyone who likes breathing clean air, and flipping a switch and having lights come on. Most data centers are getting 15% utilization, which means 85% of the available hardware is sucking power and doing nothing. It's time to put the blame for that where it really belongs.
By Richard Maddox on 8/3/2012 1:07 PM
Gartner is offering a new report Market Trends: Big Data Opportunities in Vertical Industries which compares the business drivers and uses of big data across 11 vertical industries to help solution...
By Richard Maddox on 7/26/2012 4:18 AM
A recent InformationWeek Virtual Event highlighted sobering facts about the threat of fraudulent activity for businesses:

U.S. organizations lose 5% of annual revenues to fraud translating...
By Liz Ivey on 7/24/2012 11:18 AM
Imagine… running a query on your massive dataset and getting the answer before your coffee is ready! Google is helping you do just that with Google BigQuery. ...
By Paige Roberts on 7/20/2012 4:09 AM
Two big messages from Amazon at their first ever presentation in Texas: First, Amazon cloud computing services are changing, and second, Amazon is growing. AWS security guy, Max Ramsay presented to a rapt audience this week at the Austin Cloud User Group meeting.
By Paige Roberts on 7/16/2012 4:36 AM
When new products coming out claim integration with Hadoop, are they talking about simple string matching, or real data integration, genuine semantic interoperability? If you have a tremendous glop of unstructured junk, with no traditional data integrity processes in the data load, can you ever get useful, actionable predictive analytics that your CEO can trust out of that mess?
By Richard Maddox on 7/10/2012 1:28 PM
The ZDNet Hot Topics Web seminar “Big Data Technology: Are You in Over Your Head?” featuring Pervasive CTO Mike Hoskins and big data blogger Andrew Brust, provided an informative background on the emergence...
By Richard Maddox on 6/28/2012 7:44 AM
The news coming out of this year's attendance-record-setting Hadoop Summit was fast and furious as vendors used the lead up to the event to announce major releases, many bringing big data closer to...
By Richard Maddox on 6/12/2012 9:58 AM
Successful big data projects in the cloud – in which terabytes of data flow daily between operational and analytical data stores – require scalable data integration that not only handles these large...
By Julie Hunt on 5/10/2012 8:13 AM
Whether the focus of an enterprise is on “big data” or not, analytics-driven companies are faced with processing many new data sources from inside and outside the firewall. These data sources may be complex...
By Pervasive Big Data on 5/1/2012 1:43 PM
Many consider the Google BigQuery launch to be a major leap forward in bringing real-time analytics in the cloud to businesses and developers. Since its limited preview in November 2011, BigQuery has...
By Jeff Kaplan on 4/24/2012 9:54 AM
THINKstrategies’ views in a number of online publications about the unprecedented opportunities and challenges posed by today’s rapidly evolving Cloud-based business intelligence (BI), Big Data and data integration developments.
By Jeff Kaplan on 4/20/2012 2:39 PM
A recent IDC study predicted the market for big data technology and services will grow from $3.2 billion in 2010 to $16.9 billion in 2015. Yet, Gartner says more than 85% of Fortune 500 organizations will...
By David Inbar on 11/11/2011 2:21 PM
Organizations are increasingly looking towards big data to support their business decisions through analysis. Currently, Hadoop seems to be a leading choice for businesses to implement for big data management,...
By David Inbar on 11/11/2011 12:19 PM

Last month, Forrester Research released a report, “ Expand Your Digital Horizon With Big Data,” for CIOs that focused on how they should approach big data in order to take full advantage of it for their businesses. It addressed how big data is influencing markets across industries and is prevalent in various business sectors, such as healthcare, web marketing and telecommunications.

By Pervasive DataRush on 8/31/2011 9:27 AM

To help enterprises learn more about big data and how it fits with their traditional data warehouse and data mart, Pervasive DataRush and Karmasphere are hosting a webinar titled, "Big Data: The Role, Value and Best Practices of Hadoop."

By Pervasive DataRush on 8/3/2011 9:59 AM

We had the pleasure of meeting with David Linthicum last month to get his thoughts on Big Data.

By Liz Ivey on 7/19/2011 3:44 PM
We are in Washington D.C. this week exhibiting at the FOSE show and we're showing this awesome demo that we've been working on. Pervasive has taken Pervasive DataRush to new levels in order to...
By Richard Maddox on 6/22/2011 3:46 AM
Pervasive Software’s Chief Integration Technologist Paul Dingman will be a presenter at Hadoop Summit 2011 on June 29,...
By Richard Maddox on 6/17/2011 11:12 AM
If you’ve explored our website lately, you may have noticed that we now offer a daily news aggregator called Big Data Analytics Digest....
By Richard Maddox on 6/3/2011 6:20 AM
Jim Discusses Leveraging Multicore Systems for Hadoop and HPC Workloads

Check out Pervasive DataRush Chief Technologist Jim Falgout at the AMD Fusion Developer Summit...
By Richard Maddox on 6/2/2011 6:14 AM
We wanted to share a detailed summary of the report on Big Data by McKinsey Global Institute (MGI) that was...
By Liz Ivey on 5/10/2011 3:37 AM
Tackling big data is not a job that is only going to be solved by programmers alone.  It’s going to be solved in concert with data scientists and analysts.  What tools exist for both programmers and non-programmers?  ...
By Liz Ivey on 3/21/2011 4:12 AM
Pervasive Software is a proud sponsor and exhibitor of GigaOM’s Structure Big Data 2011 taking place on March 23 in New York City. ...
By Richard Maddox on 2/18/2011 9:09 AM
Attending the O'Reilly Strata Conference, I received lots of food for thought about the future of Big Data, as well as further validation that Pervasive DataRushTM is a good framework to respond to many...
By Richard Maddox on 1/28/2011 11:07 AM
January, February and March will be busy months for our technology evangelists. Pervasive DataRush Director of Product Management Davin Potts, Pervasive DataRush Chief Technologist Jim Falgout and Pervasive...
By Richard Maddox on 1/17/2011 11:17 AM
The Intel Concurrency Checker can be used to evaluate the performance scaling of applications on multi-core systems and to help further optimize applications. It’s a tool that is used to check application...
By Richard Maddox on 12/6/2010 7:04 AM
We’ll show you how on Dec. 8 Like most software organizations yours probably needs a cost-effective approach to deliver analytics or other data-intensive solutions amid increasing data volumes and growing...
By Azeem Jiva on 10/6/2010 2:00 AM
We've been working hard here at Pervasive and DataRush 4.4.1 is now available for download. Don't let the small version number increment fool you, there are many worthwhile changes to DataRush. Including...
By Richard Maddox on 10/1/2010 4:14 AM
Did you notice the sea change taking place in big data computing?

Multicore processors are redefining the compute density of a single server. When fully leveraged with the right software, multicore...
By Liz Ivey on 8/6/2010 10:04 AM
Jim Ericson’s blog on (Really) Big Data perked our interests. In the industry of big data, we’re always curious...
By Richard Maddox on 7/7/2010 11:00 AM
Security spending in a downturn is under tight scrutiny. PricewaterhouseCoopers found this to be the case when it surveyed 7,200 executives in over 130 countries for its 2010 “Trial by Fire” report. One...
By Liz Ivey on 6/23/2010 10:27 AM
You probably understand that Pervasive DataRush is a fast processing engine and is great for data preparation, but did you know that the just released version 4.4 also includes an analytics platform? Pervasive DataRush for Analytics now allows users to analyze their entire gigantic data set (instead of just a sample) and make use of the newly-available core analytics library to perform data mining operations leveraging the high throughput and scalability of the Pervasive DataRush engine.

By Jim Falgout on 6/9/2010 10:09 AM
Gordon Brown posted a blog on Redfin commenting on a presentation by Jeff Hammerbacher discussing the use of Hadoop at Facebook.Having solved many big data problems at Facebook, Jeff has great credibility...
By Richard Maddox on 4/13/2010 4:51 AM
And we’re not done yet—Pervasive DataRush is now approaching 2 Terabytes/Hour on a single server!!

Pervasive® DataRush™...
By Richard Maddox on 4/13/2010 4:51 AM
And we’re not done yet—Pervasive DataRush is now approaching 2 Terabytes/Hour on a single server!!

Pervasive® DataRush™...
By Azeem Jiva on 4/1/2010 1:46 AM
As Intel and Advanced Micro Devices battle it out for processor dominance, we as consumers are finding faster and faster computers for increasingly lower prices. Quad core systems are now common...
By Jim Falgout on 3/15/2010 5:34 AM
I mentioned in an earlier blog, the contest...
By Jim Falgout on 3/11/2010 10:05 AM
In a recent blog, Robin Bloor discusses how parallelism is required for software to go really fast on todays multicore computers. He brings up this point about MapReduce: using MapReduce on problems...
By Jim Falgout on 3/5/2010 2:29 AM
Processors continue to become more "core dense" as we approach a watershed mark in multicore development. We've approached the point where a 4 processor system is taking on the characteristics of a "cluster...
By Azeem Jiva on 2/9/2010 1:46 AM
Traditionally Java Performance has always been a bit of a misnomer.  In the early days of Java 1.1, performance was a secondary consideration after ease of programming and usability.  But the last few years have seen some amazing performance enhancements in the JVM.  Escape Analysis, Compressed References, and other JDK7 performance enhancements.  I'm not even including the myriad of smaller changes that the Sun JVM engineers are working on.

By Richard Maddox on 12/14/2009 9:17 AM
Government data, when made available and presented effectively, can provide useful information to organizations and citizens alike. Star News Online presents compelling possibilities in the article, “Local Governments Offer Data to Miners.”...
By Jim Falgout on 11/5/2009 4:36 AM
This article by Gordon Haff provides some interesting insights into Intel's thinking around software parallelization. At Pervasive, we talk about the trade-off of using Java to build a framework like...
By Azeem Jiva on 7/7/2009 5:57 AM
I was lucky enough to present and attend Jazoon 2009 in Zurich, Switzerland.  My talk How badly written optimizations can undo automatic JVM benefits was...
By Azeem Jiva on 6/10/2009 2:59 AM
This year's JavaOne was a great success.  It was vibrant, and filled...

Search Big Data Blogs


Big Data (126)
Analytics (66)
Pervasive (50)
DataRush (33)
Hadoop (31)
Industry trends (22)
predictive analytics (20)
Scalability (20)
Multicore (15)
Data Mining (12)
Parallelism (10)
Java (9)
Jim Harris (9)
Cloud (8)
Cyber Security (8)
MapReduce (8)
big data analytics (7)
Data Volumes (7)
Data Warehouse (7)
RushAnalytics (7)
Volumes (7)
Actian (6)
Algorithms (6)
Cost-effective (6)
David Loshin (6)
Decision Support (6)
Julie Hunt (6)
RushAnalyzer (6)
analytics tools (5)
Dataflow (5)
machine learning (5)
Data Science (4)
Forrester (4)
Google (4)
Green IT (4)
Healthcare (4)
Phil Simon (4)
YARN (4)
analytics processes (3)
Big Data Science (3)
BigQuery (3)
Bloor (3)
data centers (3)
data integration (3)
Data Preparation (3)
data tools (3)
data-driven (3)
DataMatcher (3)
machine generated data (3)
Malstone B (3)
Mike Hoskins (3)
Opera Solutions (3)
Retail Analytics (3)
Security (3)
Smart Grid (3)
software (3)
Solutions (3)
telecommunications (3)
transportation analytics (3)
Age of Data (2)
analytics accuracy (2)
architecture (2)
Austin (2)
Bloor Research (2)
Business Intelligence (2)
data management (2)
Data Rush (2)
David Inbar (2)
David Norris (2)
fraud (2)
fraud detection (2)
Gartner (2)
GigaOM (2)
Hadoop Summit (2)
IntegrationWorld (2)
intelligent transportation systems (2)
internet of things (2)
McKinsey (2)
meetup (2)
ParAccel (2)
Pervasive DataRush (2)
Rexer Analytics (2)
smart meters (2)
#FollowFriday (1)
a (1)
Amazon (1)
analytics workflow (1)
Application Development (1)
automation (1)
Benchmarks (1)
best practices (1)
Cloud Analytics Summit (1)
cloud computing (1)
Cloudera (1)
contests (1)
cost (1)
cyber security issues (1)
data flow architecture (1)
Data Integrator - Hadoop Edition (1)
data quality (1)
data visualization (1)
digital marketing (1)
Door64 (1)
easy big data analytics (1)
Ericson (1)
Esri (1)
Facebook (1)
Fuzzy Matching (1)
Goverment (1)
Hadoop User Group (1)
Hadoop World (1)
hardware (1)
HBase (1)
HDFS (1)
industrial internet (1)
Jazoon (1)
Jim Falgout (1)
MalStoneB (1)
Mansour Raad (1)
Neil Raden (1)
Netflix (1)
NetFlow (1)
operational intelligence (1)
Paige Roberts (1)
para (1)
PIG (1)
pilot projects (1)
Predictive Analytics World (1)
psychohistory (1)
Public Sector (1)
Redshift (1)
Robin Bloor (1)
ROI (1)
Rosaria Silipo (1)
RushAccelerator (1)
RushLoader (1)
Sampling (1)
Signal and Noise (1)
SmartDataCollective (1)
spatial analytics (1)
speed (1)
sports (1)
Stephen Swoyer (1)
Steve Shine (1)
Strata (1)
SXSW (1)
Telecom Analytics (1)
Telecommunications Industry Association (1)
TIA (1)
Transportation (1)
TurboRush (1)
VectorWise (1)
Zementis (1)

Latest Posts

Actian Big Data & Analytics Blog has MOVED!
Big Data Phrenology
Big Data, Simpson's Paradox and Sufficient Tools
Data Science and the Art of Data Visualization

Big Data Blog Archives

<October 2014>

Accelerating Big Data 2.0™