Why you should care about the democratization of Network Data

Any conversation about the democratization of data has to include network data. Donal O’Sullivan, VP Product Management, tells us why.

Why you should care about the democratization of Network DataBy Donal O'Sullivan    October 15, 2015      Thinking

Why you should care about the democratization of Network Data

Forbes recently published an article on the Age of Data Democratization, a piece about the inexorable spread of easily consumable data tools throughout organisations. One thing stood out to me - or rather, the absence of one thing: there was no mention of Network Data. This was more than a little surprising. Though companies are using many data sources for business insight, Network Data has arguably the highest intrinsic value in today’s real-time business, and is also the data source most in need of democratization.

Firstly, let’s agree what we mean by Network Data. We mean the actual packets that are travelling around your network, using various encapsulations and protocols like IP, Ethernet, MPLS, HTTP etc. We don’t mean stats gathered from routers and switches, we don’t mean NetFlow records, we don’t mean any other derived metrics, we mean all of the actual data packets and the entire L2-L7 payload.

This distinction is really important, because this packet data contains the full record of every application request and response, every customer interaction, every business transaction, every performance outage and protocol error, every security breach and every overloaded system. Flow data and network stats have their uses but simply don’t contain this level of richness.

Today, there is a tacit acceptance of the value of network data.

  1. Most large corporations invest in packet capture devices – when there’s a serious outage or high-profile IT problem, engineers crank up the packet capture devices and invest hours or days poring through terabytes of data looking for that smoking needle in the haystack, if you’ll pardon the mixed metaphor.
  2. One year ago Gartner debuted a Magic Quadrant and Critical Capabilities Report for Network Performance Monitoring and Diagnostics, which is dominated by packet capture companies, with total spend over $1B per annum. You can get copies of the documents here.
  3. Security organizations realize that network data is the ultimate, unbiased source of intelligence about breaches and threats. They too are investing billions in devices to capture packets in anticipation of the need to perform post-hoc analysis when other indicators point to evidence of a breach.

But network data today has some pretty significant problems: it’s mainly usable only by the network team, it’s expensive to deploy, very time-consuming to extract value from, and really only works for episodic after-the-fact investigations.

The rich content of network data makes it compelling, but it’s so difficult to use that it becomes the “data-of-last-resort,” so to speak. The vast majority of captured network data today is archived or deleted without ever being inspected by a human or a machine algorithm. Billions are invested and the data is just thrown away. What a waste.

The movement to democratize network data aims to change all that. It aims to bring network data out of the back room and finally deliver value for those billions of dollars invested.

What if every packet traversing your infrastructure were inspected, decapsulated, decrypted, decoded, reassembled, analyzed, enriched, indexed, presented, published and stored, in real time, all the time? What if it was simple to use, for anyone in the organization? What if each application and business group could see just the specific data that was important to them?

Network data would no longer be confined within the network group – there would be a continuous stream of value and insights being shared through the organization. You would gain real-time insights into all the richness contained in the network data – customer transactions, security breaches, IT performance. Easily. Automatically.

This democratization is happening. We at Corvil are part of it, but the movement goes well beyond us. The value in network data is too compelling to be ignored, and over the next 12 months we believe you’ll see more and more companies join up with this movement and tap into that value.

Why you should care about the democratization of Network Data

Donal O'Sullivan, Vice President, Product Management, Corvil
Corvil is the leader in performance monitoring and analytics for electronic financial markets. The world’s financial markets companies turn to Corvil analytics for the unique visibility and intelligence we provide to assure the speed, transparency, and compliance of their businesses globally. Corvil watches over and assures the outcome of electronic transactions with a value in excess of $1 trillion, every day.

You might also be interested in...