Almost every single computing device, laptop, desktop, router, firewall, keeps a record of things that have changed over time. Borrowing from maritime parlance, we call these “logs”. Logs are ledgers of changes in state, recorded and stored in sequence, and can tell you a lot about what that computing device has been doing, why it has been doing it, and when.
Leaving A Trail
By browsing the Internet at work, for example, I create a potentially massive digital contrail which can tell anyone willing to piece it together a complete story of where I went, what I did, and when. This is not just true of legitimate users. Malicious traffic leaves the same traces, giving you the opportunity to investigate potentially malicious activity on the network.
Reading logs and piecing together the stories of the thousands of digital ‘flights’ that business users are taking every day can be profoundly dull. Based on numbers from the SANS Institute (albeit from 2009) a network with 750 devices might generate roughly 1GB of log data every 24 hours on a normal day. That’s 700,000 pages of text or 36 months of solid reading material for an average person. Roll your sleeves up!
Missing An Opportunity
Given this ‘wealth’ of material, it’s no surprise that many businesses have not made the best use of the logs they have access to. Computing devices are dutifully creating them, but without management, the memory or disk space they are using tends to get recycled without the logs being used. This can frustrate digital forensics after an attack and misses an opportunity to reduce the risk of persistent and undetected data breaches.
Creating Business Value
The first step to creating business value from your logging capability is a Log Management System. This is simply a combination of configuration and software that centralises logs generated from selected devices and puts them in a system that offers tools for analysing, reporting, or making them available for audit. A company can then set acceptable and useful parameters for keeping their logs.
ALogs offer lasting value to a company’s information security. Useful not just for spotting rogue devices, malware, misconfigurations, shadow IT applications and legacy systems, but also for possibly detecting hacking activity, insider threats and bots behaving badly.
What IT Managers, responsible for a medium-sized wide-area network, seem to want is a system to collect all of their logs, read them, and let them know if there’s something unusual or dodgy.
The Usual Response
The traditional response to this requirement is a Security Incident or Event Management (SIEM) system. These are combinations of Log Management with advanced correlation capabilities which identify potential threats in the stream of log data. They sound great but they can suffer with some common difficulties.
- SIEMs are most useful for companies large enough to have a Security Operations Centre (SOC). Smaller companies need staff dedicated to information security before investing in a tool to make their jobs easier.
- The rules of correlation tend to have to be built manually – which means professional services costs after installation or a lot of work for someone internally.
- They offer most value when contextualised to a business network environment. It takes work to find the value they can offer.
- The pricing strategies seem to be commonly based on log volumes and, as we know, these are unknown at the point of purchase, and vary depending on if the network is under attack.
For these and many other reasons, the most common place to find a copy of SIEM software (trial or otherwise) in medium-sized businesses is on an IT Manager’s shelf, in a drawer, or on a ‘to do’ list. That’s a shame, but not a surprise.
Unlocking The Power of SIEM
Machine learning techniques offer a powerful potential method of unlocking the power of a SIEM for the smaller network. Instead of manually intensive correlation rules created by hand, or financially intensive professional services provided by a SIEM provider, analysing and spotting patterns in common log formats is perfect for automation through Artificial Intelligence.
AIs are a fraction of the price of hiring a dedicated security analyst. They can digest volumes of logs quicker and more diligently than any human, take no breaks, and can operate 24×7. They take time to learn a network, and to be able to ignore some of the dubious traffic flows that many smaller networks have grown up with, but this is no different to the time it takes to contextualise a traditional SIEM.
Size Doesn’t Matter
So whilst larger organizations can make use of a SIEM, smaller companies need to look for a security analysis AI that it is licensed on network size, not log volume. It should also have a natural language artificial intelligence engine that can build correlations and report on suspicious flows that might constitute a security threat from across your network with a high degree of confidence. You can also probably do without a SOC.
So whatever your business size, you can still utilize all those gigabytes of logs you are accumulating.