System information and event management (SIEM) systems and intrusion detection and prevention systems (IDS/IPS) are our frontline defences in an increasingly hostile world. As effective as they seem, if they're not properly configured and maintained, they can become essentially useless.
It is essentially digital sanitation which may not seem like a particularly interesting subject, but it is extremely important. Imagine what would happen if your company's rubbish collection stopped being collected. While SIEM/IDS/IPS maintenance isn't quite as smelly, it's very similar. Networks and threat landscapes change. Most IDS rule sets don't change (other than getting bigger). Software or services stop being used. And unfortunately, somebody needs to wade through your SIEMS's fine details to find these things.
There are a few ways to mitigate the tedious work to make the task more manageable.
Roll Out Gradually
If you are just rolling out your SIEM/IDS/IPS system, avoid turning on everything at once — this could be a big mistake. Turn on subnets or VLANs one by one — tuning as you go. A SIEM throwing out 100 alerts in a day is a sure bet of burning out your security analysts quickly, and pretty much ensures overly liberal whitelists will quickly become commonplace.
Also see: Are free VPNs safe to use?
Step by step Documentation
Make sure every rule set added is documented (and reviewed by someone) when it is added. And make sure the documentation is complete. The "magic whitelist" is routinely encountered when doing audits. Such an incomplete entry usually happens at 4:59 p.m. on a Friday afternoon filled with false positives. Unfortunately, Monday rolls around and the whitelist isn't updated, and months later a similar alert is triggered. Make sure your documentation is precise and helpful. Don't allow any ambiguity. If something is a false positive, explain why.
Don't hurry through the configuring of your baseline — no matter how long it takes. Tuning out the legitimate false positives while taking the time to thoroughly investigate each alert is important. The more granular you can get your rule sets, the better. If you can pinpoint false positives from specific machines in a subnet, exclude the machines and not the subnet.
Remember that you are doing this not only to find stale or overly liberal policies but to improve performance. Your SIEM/IDS needs to process every single rule it encounters, so the more rules you have, the slower your device will be. Look for ways you can consolidate rules wherever possible — and remove the unnecessary.
Make auditing a regular exercise. Consider doing this task monthly or bimonthly, rather than leaving a huge amount to do at the end of the year. Tie changes into a change control process wherever possible. If changes are made on the fly, make sure they are still entered into the change registry as an emergency modification. A robust change control framework is imperative; not only will it help drive security, your entire IT infrastructure will benefit.
Wherever possible, use this audit phase to help build or validate your configuration management database/asset tracking process. When you encounter a system, you should be able to identify what OS/apps/data are on this box, who is the point of contact, and ideally who the data custodian for the box is.
Also see: Steps To Combat Insider Threats
Validate Log Flows
Make sure you routinely validate that traffic from all devices is still inbound. Often, outages of a significant period get silenced — only to not be turned back on when the outage is resolved. I've seen a company go well over a year before realising there were no alerts triggering in their SIEM from an entire branch location.
Check Your Capacity
When reviewing your SIEM, be sure that your storage capacity is appropriate for your logs. Many SIEM solutions base their pricing on consumption. Sometimes, companies decide to limit what they are logging in an event to save money. This can be a crucial mistake. Many — probably most — penetrations of networks initially occur on individual user machines rather than on critical systems. If you are monitoring only critical or major systems to cut costs, you might very well be missing all the initial warning signals of the initial intrusion. Your first sign of trouble may be way down the line and too late to avoid exposure of your company information.
Proper capacity doesn't just apply to the size of the SIEM's hard drives. It is critical to ensure that you have adequate staffing for deploying and maintaining a SIEM. Don't underestimate the manpower required when you budget for operations.
A properly configured and maintained SIEM/IDS can save you from a great deal of unexpected trouble. But as I've explained, they aren't devices you can simply set and forget. They can be a big financial and logistical burden, but if you follow this advice it will go along way to mitigating the risk.