I am going to take up a commonly asked question around Data Auditing & Protection - how and why is data auditing different from SIM/SEM products?
A common mistake I have seen is that as enterprises worry about auditing their stored data applications for compliance or security, their first instinct is to turn to their existing SIM investments. This is usually a waste of time and resources. While SIM’s may be marketed as general purpose loggers, they are in general a poor fit for the data auditing & protection problem. Why? SIM technologies are broad-based collectors and repositories of security event information. A SIM collects small security logs from hundreds of devices such as Firewalls, IPS, application security logs to provide a broad dashboard of enterprise risk. This begs the question: Why can’t a SIM be used to offer data-level auditing? There are four key reasons:
o Data-level logs typically do not exist. A data auditing system is needed to “create” them without impacting data servers. A SIM in contrast always assumes that source or device security logs are available.
o Data-level logs are *very* verbose, containing millions of transactions. Feeding this to a SIM is like hooking up a fire-hose to a tap. A data auditing system filters the data-level logs with policies down to critical data events, and also analyzes them down to a few rare high-risk alerts. These alerts can then be escalated to a SIM.
o Data-level activity logs are deeply tied to the semantics of the individual application. For example, Oracle transactions behave very differently from SQL Server transactions, which in turn are different from data access in Windows fileserver. Feeding such logs directly to SIM would be meaningless since the SIM has no awareness of the consistent semantic model between these activities. A data auditing system understands the deep “context” of each data server log and can decode and normalize all data server logs into a single object model. This model can then be filtered and be piped to a SIM for further escalation.
o Deep intelligence is required to detect data theft and data fraud in real-time, an intelligence that SIM lacks. Data auditing adds this real-time intelligence to data-level monitoring.
In summary, a SIM is a broad but shallow collector of risk information. In contrast, the Data auditing and protection problem is a narrower but much deeper problem. Data auditing systems have to collect and parse a deep application-level data activity log, they have to analyze them for fraud, theft, or other data-level non-compliance and escalate alerts to help mitigate risks.
As always, I have been racking my head trying to find a real-life analogy to describe data auditing vs SIM. The simplest one I have come up with so far is the following – imagine you live in a neighborhood. A SIM is like the “neighborhood watch” patrol car that keeps track of on-going activity across several blocks, trying to detect risks, ex. who is walking around at late night? Data auditing systems are like the motion sensors and alarm systems inside each house. They are meant to track activity inside the house and detect theft and sound an alarm. You can see that the patrol car can sweep across a much larger region, but has limited insight into each house. On the other hand, a house alarm system has a deep view of the house, but is restricted to the house. If this analogy makes sense, it is clear why one should never use a SIM as a data auditing system – do you really want to have the patrol car constantly snoop inside your house, trying to figure out if everything is normal? Not only is this unscalable, but it poses an inherent mismatch of “intelligence” and context. In the best case, you may get false positives (such as the cop barging into the house); in the worst case the cop has a limited view and no idea whats going on in the house while a full-scale theft is in action.
Can you think of another analogy? Please drop me a note.