Big Data Blog

Big Data Blog

Cyber Security Issues: Ignorance is Not Bliss

Feb 25

Written by:
2/25/2013 5:57 AM  RssIcon

A lot has been happening to change how IT operates in enterprises with the entrance of new technologies, including the consumerization of IT, BYOD, and cloud computing. Unfortunately, many of the innovations and changes for corporate IT have opened the door for escalated cyber security challenges. Corporate security teams now have to address global venues for protecting the enterprise and can no longer view security as a silo’d function of “wall building” and defensive functions. Attacks by sophisticated cyber criminals and hackers call for proactive cyber security processes, where enterprises continuously hunt for current and potential threats.

Cyber Security Today’s cyber security threats frequently operate as patterns that usually deviate from expected behavior for  most authorized users or for activity on particular devices or IP addresses. To identify and fight such attacks, security teams need technology that can find and analyze deviant trends. This is a natural fit for big data mining and analytics (predictive analytics in particular). Enterprises can now take the necessary actions to block such cyber threats and make improvements to prevent future onslaughts.

Machine-generated data provides fertile ground for using big data analytics to root out cyber security threats. But machine-generated data can be quite challenging for aggregation, data mining, and analytics. Such data must be processed very quickly, frequently in real-time, and usually exists in large volumes that are continuously proliferating. Machine data sources are quite disparate; many of them are multi-structured formats. Machine data lives in the IT infrastructure: network logs, event logs, firewall and security system data, web logs, email logs—anything and everything operating in the infrastructure.  

Once data has been extracted from machine-generated sources, it can be enriched with other kinds of data to elicit patterns and trends related to cyber attacks. Analytics take on a forensics quality while searching through data for patterns of irregular or unexpected activity. Big data analytics require complex data modeling and queries with continuous refinement and testing to scour through all of the data. Based on the understanding derived from analytics results, machine learning algorithms can be developed for ongoing monitoring of systems to detect new threats. Analytics and monitoring approaches will have to be constantly altered and fine-tuned to anticipate ever-changing cyber-threat tactics. Situational awareness is another important aspect of fighting cyber attacks. Big data analytics strengthen situational awareness primarily through fast real-time assessments that can reduce time to decisions and actions in response to potential threats and certain anomalies.

Cybersecurity Lock The advent of Hadoop processing infrastructures is making a significant impact on more successful outcomes for big data analytics and various applications such as cyber security. Middleware offerings are becoming available to reduce processing time and bottlenecks, and to provide better tools to quickly develop data mining and analytics processes. With cost-effective and efficient tools, enterprises are able to widen the ability to ask many kinds of questions, to test—and to fail—many ways, and explore more of what might be found in large volumes of machine-generated data.

For all of these techniques, it is essential that a variety of domain experts are involved to prescribe the right methodologies, and to validate results and approaches. Expertise in cyber threats combined with human experience and insight will bring vital perspectives to solving security problems. There is a lot of work to do to create and maintain vigilant processes to fight cyber attacks as is shown in a recent ESG research report:

While big data security analytics will roll out faster than most people think, there are bound to be some speed bumps along the way. In fact, some of the more annoying short-term issues will be around basic operational tasks like collecting, normalizing, and sharing security data in a multitude of formats, schemas, and syntaxes.

  • 54 percent are experiencing “significant difficulties” or “some difficulties” with security data normalization
  • 54 percent are experiencing “significant difficulties” or “some difficulties” with security data capture
  • 52 percent are experiencing “significant difficulties” or “some difficulties” with security data sharing

Search Big Data Blogs


Big Data (126)
Analytics (66)
Pervasive (50)
DataRush (33)
Hadoop (31)
Industry trends (22)
predictive analytics (20)
Scalability (20)
Multicore (15)
Data Mining (12)
Parallelism (10)
Java (9)
Jim Harris (9)
Cloud (8)
Cyber Security (8)
MapReduce (8)
big data analytics (7)
Data Volumes (7)
Data Warehouse (7)
RushAnalytics (7)
Volumes (7)
Actian (6)
Algorithms (6)
Cost-effective (6)
David Loshin (6)
Decision Support (6)
Julie Hunt (6)
RushAnalyzer (6)
analytics tools (5)
Dataflow (5)
machine learning (5)
Data Science (4)
Forrester (4)
Google (4)
Green IT (4)
Healthcare (4)
Phil Simon (4)
YARN (4)
analytics processes (3)
Big Data Science (3)
BigQuery (3)
Bloor (3)
data centers (3)
data integration (3)
Data Preparation (3)
data tools (3)
data-driven (3)
DataMatcher (3)
machine generated data (3)
Malstone B (3)
Mike Hoskins (3)
Opera Solutions (3)
Retail Analytics (3)
Security (3)
Smart Grid (3)
software (3)
Solutions (3)
telecommunications (3)
transportation analytics (3)
Age of Data (2)
analytics accuracy (2)
architecture (2)
Austin (2)
Bloor Research (2)
Business Intelligence (2)
data management (2)
Data Rush (2)
David Inbar (2)
David Norris (2)
fraud (2)
fraud detection (2)
Gartner (2)
GigaOM (2)
Hadoop Summit (2)
IntegrationWorld (2)
intelligent transportation systems (2)
internet of things (2)
McKinsey (2)
meetup (2)
ParAccel (2)
Pervasive DataRush (2)
Rexer Analytics (2)
smart meters (2)
#FollowFriday (1)
a (1)
Amazon (1)
analytics workflow (1)
Application Development (1)
automation (1)
Benchmarks (1)
best practices (1)
Cloud Analytics Summit (1)
cloud computing (1)
Cloudera (1)
contests (1)
cost (1)
cyber security issues (1)
data flow architecture (1)
Data Integrator - Hadoop Edition (1)
data quality (1)
data visualization (1)
digital marketing (1)
Door64 (1)
easy big data analytics (1)
Ericson (1)
Esri (1)
Facebook (1)
Fuzzy Matching (1)
Goverment (1)
Hadoop User Group (1)
Hadoop World (1)
hardware (1)
HBase (1)
HDFS (1)
industrial internet (1)
Jazoon (1)
Jim Falgout (1)
MalStoneB (1)
Mansour Raad (1)
Neil Raden (1)
Netflix (1)
NetFlow (1)
operational intelligence (1)
Paige Roberts (1)
para (1)
PIG (1)
pilot projects (1)
Predictive Analytics World (1)
psychohistory (1)
Public Sector (1)
Redshift (1)
Robin Bloor (1)
ROI (1)
Rosaria Silipo (1)
RushAccelerator (1)
RushLoader (1)
Sampling (1)
Signal and Noise (1)
SmartDataCollective (1)
spatial analytics (1)
speed (1)
sports (1)
Stephen Swoyer (1)
Steve Shine (1)
Strata (1)
SXSW (1)
Telecom Analytics (1)
Telecommunications Industry Association (1)
TIA (1)
Transportation (1)
TurboRush (1)
VectorWise (1)
Zementis (1)

Latest Posts

Actian Big Data & Analytics Blog has MOVED!
Big Data Phrenology
Big Data, Simpson's Paradox and Sufficient Tools
Data Science and the Art of Data Visualization

Big Data Blog Archives

<October 2014>

Accelerating Big Data 2.0™