Analyzing network flow records
Data Security
The method of detecting downloads directly from the metadata presented here obviously has a considerable effect on the general security of the devices on your network. Many updates are not intended for all versions of an operating system, so you can relatively quickly find out exactly what version is in use. Various manufacturers of security software offer countermeasures that prevent complete scanning of the Internet or your own subnet.
But because metadata is passive by nature, it cannot be filtered by these protection systems – and it is also impossible to get rid of metadata. This method is far less suitable for use on large networks because, on the one hand, the detection rate drops severely the farther away you are from the download server. On the other hand, the alternative approaches that many providers now use to deliver updates also takes its toll.
In particular, the peer-to-peer update function in Windows 10, as well as Content Delivery Networks (CDNs), which manage a globally distributed network with a correspondingly large number of different IP addresses, should be mentioned. These large providers in particular offer so many files of different types that identification based solely on the size of the file seems to be fairly meaningless. Without a knowledge of the DNS requests, it is impossible – especially in the case of CDNs – to identify the domain that was originally the target of the request.
That said, it is important to note that the general lack of attention paid to metadata can become problematic if worst comes to worst and an attacker is able to exploit an unpatched vulnerability on a system footprinted using this method.
Reference Downloads
To be able to assign flows to downloads, it is important to know what possible downloads exist. This task is impossible to handle manually because of the sheer volume of possibilities, so you need to think about an automation strategy. Two methods turn out to be very useful here.
The first method is based on grabbers, which work much like the classical search engine grabbers that index the Internet for fast searching. This involves searching the Internet or a suitable selection of websites for download links. Because the detection rate is very poor in the lower kilobyte range of numbers, administrators will only want to consider downloads whose size exceeds a certain threshold for indexing.
However, this method has the massive disadvantage that it prevents the detection of incremental updates because the download links typically offer the full download. Additionally, the collection can become unmanageably large even after a very short time.
The second method is based on honeypots equipped with a software configuration similar to those used on enterprise networks. By monitoring the network traffic to these honeypots, administrators can now directly observe update sequences. Additionally, it is possible to start downloads directly from the systems, making it easy to map the flows because the honeypot systems are not used for any other purpose.
The major advantage offered by this method is that the recorded packet sizes lead to good detection rates, especially if the honeypots are located on the same subnet as the systems you want to protect. Moreover, it is easier to emulate and analyze special update mechanisms. These benefits come at a price, in that you can only monitor known software versions and combinations and you are relying on honeypot systems that need to work with full, licensed versions of the software you deploy.
Conclusions
For IT staff who want to keep track of their own IT infrastructure and do not have, or are not allowed to have, access to all of the systems, the method introduced here is an additional option that supplements classical penetration tests to provide better asset protection. It also draws attention to the value of metadata. If you log flow records directly on the switches and backbone routers on your network, you will also ensure that the distance to the systems you are monitoring is not too large, which means that the variance in the monitored download sizes remains manageable.
Infos
- Shadow IT: https://en.wikipedia.org/wiki/Shadow_IT
- "Managing port scan results with Dr. Portscan" by Wolfgang Hommel, Stefan Metzger, Michael Grabatin, and Felix von Eye, Linux Pro Magazine, issue 155, October 2013, pg. 20, http://www.linuxpromagazine.com/Issues/2013/155/Dr.-Portscan
- Bernhard, Andreas, Netzbasierte Erkennung von Systemen und Diensten zur Verbesserung der IT-Sicherheit [Network-Based Detection of Systems and Services to Improve IT Security], Bachelor thesis, Ludwig-Maximilians-University, Munich, March 2014, http://www.mnm-team.org/pub/Fopras/bern14/PDF-Version/bern14.pdf [in German]
- Softflowd: http://www.mindrot.org/projects/softflowd
- Flow-tools: https://code.google.com/p/flow-tools
- Data retention laws: https://en.wikipedia.org/wiki/Telecommunications_data_retention
- Pandas: http://pandas.pydata.org
- Sklearn: http://scikit-learn.org
« Previous 1 2
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
So Long Neofetch and Thanks for the Info
Today is a day that every Linux user who enjoys bragging about their system(s) will mourn, as Neofetch has come to an end.
-
Ubuntu 24.04 Comes with a “Flaw"
If you're thinking you might want to upgrade from your current Ubuntu release to the latest, there's something you might want to consider before doing so.
-
Canonical Releases Ubuntu 24.04
After a brief pause because of the XZ vulnerability, Ubuntu 24.04 is now available for install.
-
Linux Servers Targeted by Akira Ransomware
A group of bad actors who have already extorted $42 million have their sights set on the Linux platform.
-
TUXEDO Computers Unveils Linux Laptop Featuring AMD Ryzen CPU
This latest release is the first laptop to include the new CPU from Ryzen and Linux preinstalled.
-
XZ Gets the All-Clear
The back door xz vulnerability has been officially reverted for Fedora 40 and versions 38 and 39 were never affected.
-
Canonical Collaborates with Qualcomm on New Venture
This new joint effort is geared toward bringing Ubuntu and Ubuntu Core to Qualcomm-powered devices.
-
Kodi 21.0 Open-Source Entertainment Hub Released
After a year of development, the award-winning Kodi cross-platform, media center software is now available with many new additions and improvements.
-
Linux Usage Increases in Two Key Areas
If market share is your thing, you'll be happy to know that Linux is on the rise in two areas that, if they keep climbing, could have serious meaning for Linux's future.
-
Vulnerability Discovered in xz Libraries
An urgent alert for Fedora 40 has been posted and users should pay attention.