Log Management: How to collect and use logs proactively?
When collecting and using logs, do you use the ’ostrich method‘? Why not use your logs proactively so you are able to search all your logs in one place and also automatically be notified when your logs tell you something you need to be aware of?
How do we use our logs actively to get better IT operations?
We define the ’ostrich method‘ as the usage of logs reactively, which is why problems are discovered late in the process.
In this method, we see that it is common to manually go back to a server or a switch to search through the log to see if you can find a problem. Some even let their switches (and maybe even servers) send all syslogs into a common base, and then forget all about them. This ’log trash can‘ make it very cumbersome to search logs afterwards, and it is very hard to be proactive and arm your cyber defence against reoccurring problems. In other words. You are not using logs actively.
We unfortunately see that a lot of IT departments are using their logs according to this method. This means no collection of logs, no proactive use of logs, only a reactive, by diving into device logs to find the needle in the haystack, which may give you a clue about a malfunction or a security breach. Alternatively, you might use logs by pointing all your devices to a single base, where you store logs in what can best be described as a trash can. The easiest way is to press ’delete all‘ - much easier than using these logs actively.
But your logs are the ultimate source of information about the condition, errors and security flaws of all your IT systems. Achieve better IT operations by using logs actively:
- Active collection of logs from your servers, routers, switches, firewalls, access points, applications, etc. to one common database. Logs are usually forwarded as syslogs and typically your Windows event logs are converted to syslogs using a local agent on the individual server.
- Log correlation is a discipline in which your log solution intelligently screens and compares the incoming logs in terms of frequency and coherence. It allows the system to automatically alert you when there is something in your logs that you NEED to be aware of. In so doing, you are using your logs proactively.
- Log search is performed via your log system in an interface that makes it possible (and easy) to search across all your logs for a certain type of information, during a certain period of time (e.g. what did a certain IP address do on the web last Wednesday between 12 and 14?).
- Compliance and reporting are other great functions in a good log solution, because with the built-in functions in your log solution, you make it easy to keep both Audit and Management satisfied with demonstrable compliance and built-in reports on inventory, security, operational errors, application errors, etc.
We have extensive experience in consulting and implementing how to secure and streamline IT operations to harden the defence against cyber threats. We help our clients determine and implement the right controls, processes and tools to increase cybersecurity within the organisation.