Archive for the ‘comp’ tag

Beneficial side effects of running a honeypot  

Posted at 11:42 pm in Uncategorized

Spam (the non-electronic version)

I’ve been running a honeypot for quite a while now, it started out as a pure SSH honeypot – first with Kippo and then I migrated to Cowrie. Some time later I added more honeypot services to the unit in the form of InetSim. The InetSim software provides multiple plaintext services like HTTP, FTP, and SMTP, as well as the encrypted versions.

HTTP and FTP are services where the intruders will try to download something from the honeypot, and InetSim will serve them a predefined set of standard sample documents. The HTTP and FTP also allow uploads, in which case any submitted content will be saved for future analysis by the honeypot administrator.

However, the funniest side effect of running InetSim – at least so far – is with its SMTP service. Spammers will happily use this, what they will think is a newly discovered “open relay”, for distributing annoying spam and/or more malicious phishing mail. All the spam they push through the service acting like an MTA will of course be sinkholed (and saved locally), while they most likely believe that they have distributed their content.

As the below table listing the last two weeks’ top 10 most active spammer IPs shows, the most active spammer “successfully delivered” no less than 300 000 spam messages through (or rather to) the honeypot SMTP. The honeypot itself will obviously drop those mails to the ground, and if the software hadn’t done it (or if the attacker had found a way to break out of the honeypot), the honeypot resides in a very strictly controlled environment ensuring that no spam would’ve found its way out anyway.

IP addressNumber of spam mails

While neither spam, phishing mails nor open mail relays are normally laughing matters, I truly enjoy knowing that the spammers have wasted their time with a non-functional mail server believing that they got their job done. One can also hope that the people behind the spam/scam pays for their service.

Written by admin on July 13th, 2016

Tagged with , , , , ,

Near-realtime blacklist warnings with NetFlow, Perl and OTX  

Posted at 7:43 pm in Uncategorized

Installing IDS sensors in your network for monitoring traffic is not always feasible, for several possible reasons. Perhaps the network infrastructure is too complex, leading to blind spots. Maybe the affected network links have higher capacity than your ad hoc IDS sensor, causing packet loss on the sensor. Or your company may be organized in such a way that installing “foreign” hardware in the network infrastructure is not easily done.

Still, without going “all in” on a potentially expensive IDS project, it could be useful with some insight into what’s going in to and out from your network, keeping an eye on known malicious IP addresses and networks. Setting up a NetFlow feed from the company’s routers will usually not incur any significant loads, nor does it interfere with the network traffic, so that could be a possible approach. I’ve previously covered NetFlow and SiLK for rear-view mirror analysis of whether any blacklisted IP resources have been communicating with your system and users in the past. What if we could do the same, just in (almost) real-time? With the help of Perl and the Net::Flow module, we can.

Bill of material

  • Router(s) that support(s) NetFlow (I’ve used version 9 but the Perl module seems to support v5 and IPFix as well).
  • Perl, and the Net::Flow module for parsing the NetFlow data.
  • One or more IP blacklists of your choice. For the purpose of this test I’m using my subscribed lists from AlienVault’s Open Threat Exchange, but the list of IP addresses to compare against can easily be extended with – or replaced by – other lists like the SANS blocklist or any DNSBL/RBL.

The Perl script I’ve set up for this purpose is crudely derived from the Net::Flow sample code, and after my tweaks it’s currently not something that should see the light of day. Its output, however, is pretty enough for a modest presentation. The IP addresses (IPv4 as well as IPv6) and other info are extracted from the different flow fields, detailed in this Cisco document.  In my script, each offending IP is associated with URLs linking to OTX pulses where further information can be found.

Some sample entries from the Perl script’s output:

2016-06-07 12:38:20 : -> (TCP)

2016-06-07 13:37:46 : -> (TCP)

2016-06-07 13:51:34 : -> (TCP)

2016-06-07 17:51:13 : -> (TCP)

2016-06-07 18:00:52 : -> (UDP)


Some unsolicited questions and answers

  • What can this be used for? It can be a proof-of-concept, in cases where you might need to argue why you want to install an IDS. It can also be used for statistical purposes, to get a grasp of how often your network is communicating with malicious systems on the Internet.
  • Will I be missing information with this simplified setup? Yes, most likely. This implementation is not intended as an IDS replacement, but it will give an indication of unwanted activity to and from your network. Also, your router may provide sampled NetFlow data, e.g. only a portion of the traffic will be selected for NetFlow analysis. At times you might see only the response traffic, in cases where a remote node contacting a non-responsive port will not always be classified as an established flow but a related ICMP response might be.
  • Why isn’t it real-time? A flow won’t be registered by the router until a connection is completed or has timed out. Depending on your router’s configuration, it could also be batching up the NetFlow feeds for regular transfers. I’ve seen 20 to 30 seconds delay between the actual connection and the NetFlow push from the router.
  • Can I use the output somewhere else? Sure, you can make the Perl script log to syslog or to a file that OSSEC or something similar can read from.


Written by bjorn on June 7th, 2016

Tagged with , , , , , , ,

The inherent risks of visualizing firewall probes  

Posted at 8:18 am in Uncategorized

For some time now, I’ve been graphing all unsolicited network traffic destined for my network. For instance, it’s quite useful for detecting slow scans, which will show up as the diagonally aligned green scatter points in this plot (click to enlarge).


Slow portscan, from high ports to low ports

Other scans and probes often happen faster, when the attacker isn’t much concerned about being detected. These will appear in the plot as a lot of vertically aligned scatter points. In the plot shown below, the attackers have scanned a limited set of ports for about 30 minutes.


After writing a previous blog article about the plots as well as discussing the setup with my colleagues, and even showing what can happen with such a feature, there was really no reason to act surprised when weird patterns started to appear in the firewall plots.

The first synchronized portscan resulted in a chicken. Because of the logarithmic scale of the plot, the attacksdrawings will have higher precision when aiming for the high ports.


Then after a few weeks of just the normal hostile activity and a few not-so-successful creative port scans, a very well defined ant suddenly appeared.


In the firewall plot, TCP connections will be plotted as green and UDP connections will be plotted as light blue. After a few poorly disguised questions regarding whether I was plotting other protocols and, if so, which colors they would be, it became evident that some new plan was being hatched. And, lo and behold:


I’m already considering implementing additional colour schemes to separate IPv4 from IPv6, and I can probably just throw in the towel and ask my colleagues which colours they will need for their next piece of firewall art 🙂

Written by bjorn on April 27th, 2016

Tagged with , ,

SSH outbound connections – what are they trying?  

Posted at 10:38 pm in Uncategorized

Still fascinated by the outbound connection attempts from my Cowrie honeypot, I’ve been looking into what the intruders are trying to obtain with the outbound connections. As previously mentioned, there are bots actively attempting outbound connections towards a lot of remote services. Most are simply TCP socket connection attempts, but now and again the connection attempts hold payload data. Payload for encrypted services (SMTPS, HTTPS etc) is already encrypted. That leaves the plaintext services, mostly SMTP and HTTP.

The following Munin graph shows today’s activity. At their busiest, the Russian bots performed outbound connection attempts at a rate of 17 attempts per minute (one per 3-4 seconds).

There are a few attempts to connect to mail servers. The following EHLO greetings, i.e. how the intruders try to introduce the honeypot when connecting, are among the ones observed:









The remaining attempts described here are HTTP requests. The requests are for the web root (GET /) unless otherwise noted. All requests have more headers than what’s shown here, I’ve pruned the less interesting ones for readability.

The bots attempt several requests towards “check my IP” sites, perhaps to check connectivity and/or to detect the outside IP in a NATed environment:







GET /ip.php?i= HTTP/1.1
Accept-Language: ru-RU,ru


GET /showmyip.php HTTP/1.1


Then there are some attempts to reach URL shorteners. Ignoring the fact that these headers are crafted, the Google referers are obviously fake since a Google HTTPS search will not pass the referer to an HTTP site.

GET /make_url.php HTTP/1.1










POST /mod_perl/ HTTP/1.1
X-Requested-With: XMLHttpRequest
Cookie: permasession=145xxxx290|pscxxxxzxq


They’re also trying to connect to Craigslist. These attempts have started to appear the last few days. Note: Parts of the URLs are obfuscated.

GET /reply/eau/m4w/548xxxx413 HTTP/1.1
X-Requested-With: XMLHttpRequest


GET /reply/evv/m4w/550xxxx297 HTTP/1.1


GET /reply/hez/m4w/546xxxx191 HTTP/1.1
User-Agent: Mozilla/5.0 (Nintendo WiiU) AppleWebKit/534.52 (KHTML, like Gecko) NX/ NintendoBrowser/
X-Requested-With: XMLHttpRequest


The function of the below connection attempts are still unexplored:

POST /GetSignedKey_new1.php HTTP/1.0
Connection: keep-alive
User-Agent: Instagram 5.0.0 Windows Phone (8.10.14147.180; 480x320; NOKIA; tAbd_apiM; uk_UA)


User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)



Some statistics describing the honeypot activity the last few weeks, only counting the intruders that are also attempting outbound connections:

Attempts Originating IP


Written by bjorn on March 22nd, 2016

Tagged with , ,

Threat intelligence: OTX, Bro, SiLK, BIND RPZ, OSSEC  

Posted at 8:15 am in Uncategorized

Building a toolbox around threat intelligence can be done with freely available tools. Shared information about malicious behaviour allows you to detect and sometimes prevent activity from – and to – Internet resources that could compromise your systems’ security.

I’ve already described how to use lists of malicious domain names in a BIND RPZ (Response Policy Zone). Adding an information feed like AlienVault OTX (Open Threat Exchange) to the mix further extends the awareness and detection capabilities.

AlienVault is probably most known for their SIEM (Security Information and Event Management) named Unified Security Management™, with a scaled-down open source version named Open Source Security Information and Event Management (OSSIM). They also provide a platform for sharing threat intelligence, namely Open Threat Exchange (OTX). OTX is based on registered users sharing security information, for instance domains and hostnames involved in phishing scams, IP addresses performing brute force SSH login attempts, etc. The information is divided into so-called pulses, each pulse a set of information items considered part of the same malicious activity. For example, a pulse can contain URLs to a site spreading drive-by malware, the IP addresses of their C&C, along with hashes of the files. By selecting which pulses and/or users to subscribe to, the registered information in each pulse will be available through a feed from their API.

Carefully reviewing which users/pulses to subscribe to – there’s always a risk of false positives – I’m now regularly receiving an updated feed. This feed is parsed and currently split into two files: One RPZ file containing hostnames and domains for use with BIND, and one file containing IP addresses for use with SiLK.

As explained in an earlier post, OSSEC will let me know if someone (or something) makes DNS requests for a domain or hostname registered as malicious. Extending this to include the DNS records obtained from OTX was simply a matter of defining a new RPZ in BIND. Depending on how this is used (block? redirect? alert?), a whitelist should be in place to prevent accidental blocking of known good domains. One pulse describes all the Internet resources a client infected by a certain exploit will contact, including some certificate authorities which are not necessarily considered evil.

The file with IP addresses can be used directly with a firewall, by logging or even blocking or throttling traffic to/from the IP addresses in question. For rear-view mirror analysis it can be used with SiLK, to find out if there has been any network traffic to or from any of these addresses. To do this, you will first have to create an IP set with the command rwsetbuild:

# rwsetbuild /some/path/ip-otx.txt /some/path/ip-otx.set


Now we can use this set file in our queries. For this query I’ve manually selected just a few inbound matches:

# rwfilter --proto=0-255 --start-date=2016/01/01 \
  --sipset=/some/path/ip-otx.set --type=all \
  --pass=stdout | rwcut -f 1-5
            sIP|            dIP|sPort|dPort|pro|||60264|   53| 17|||33091|   80|  6|||63604|  993|  6|||60633|  993|  6|||60888|  993|  6|||32985|  993|  6|||33060|  993|  6|||33089|  993|  6|||33103|  993|  6|||33165|  993|  6|||33185|  993|  6|||33614|  993|  6|||33750|  993|  6|||60330|  993|  6|||60000|   80|  6|||60000|   80|  6|||    0|    0|  1|||43176|   53| 17|||    0|    0|  1|||60000|   80|  6|||60000|   80|  6|


When you need more details about the listed address or other indicators, OTX provides a search form to find the pulse(s) in which the indicator was registered.

OTX can be used with Bro as well, and there are at least two Bro scripts for updating the feeds from the OTX API. The one that works for me is The script will make Bro register activity that matches indicators from an OTX pulse.

Sample log entries, modified for readability: 59541 some.dns.ip    53 - - -
                                            Intel::DOMAIN DNS::IN_REQUEST 40453 80 - - -
                                            Intel::DOMAIN HTTP::IN_HOST_HEADER   47235  80 - - -


This article mentions just a few components that can be combined. Obviously there’s a lot of possibilities for integrating and interfacing between different systems. There are several companies that provide threat intelligence feeds, some for free and some for paying customers. Depending on the product(s), a SIEM would be able to combine and correlate the different kinds of threat intelligence to detected events.

Written by bjorn on March 9th, 2016

Tagged with , , , , , , , , , ,

ClamAV client/server setup  

Posted at 8:37 pm in Uncategorized

Note: This may very well be well-known information, but I found it difficult to get exact answers from the official ClamAV documentation, available man pages, and other kinds of documentation. The most useful hint originated from a mailing list thread considering ClamAV version 0.70, which is getting rather outdated.

My original issue was getting antivirus functionality with mod_security and Apache on a Raspberry Pi server. Due to memory constraints it seems Apache and ClamAV (my version at the time of writing: 0.99) do not coexist happily on the same RPi unit. The obvious solution: Run the ClamAV daemon on a separate device, and set up mod_security with client-side scanning.

The command-line client for antivirus scanning with the ClamAV daemon is named clamdscan. In older Debian releases like Squeeze and Wheezy, clamdscan is included in Debian’s clamav-daemon package, so the daemon will be installed even though you only need the client. This has been fixed in Debian Jessie and above, where clamdscan has become a separate package.

Both the ClamAV daemon (clamd) and the scanner client (clamdscan) have the same configuration file, unless otherwise specified. In Debian this is /etc/clamav/clamd.conf. Getting the client/server relationship configured is a matter of defining the socket on which they communicate. If the client and daemon (server) is running on the same system, the most efficient communication happens over a Unix socket (clamd.conf setting: LocalSocket). On different systems, however, you will need to use the settings TCPAddr and TCPSocket:

TCPAddr defines the IP address (and not TCP address, which would be a port number) on which the server should listen and/or which the client should make contact. Note that the documentation states that TCPAddr is used to define the IP address(es) clamd should listen on, and that it’s by default disabled. However, when setting TCPSocket and leaving TCPAddr unconfigured, clamd will listen on all IP addresses ( The documentation also makes no mention that the setting is used by clamdscan.

TCPSocket is the TCP port on which the communication takes place.

The following diagram illustrates the relationship:


Note: On a Squeeze/Wheezy Debian system, setting TCPAddr to a non-local IP address in clamd.conf will naturally make clamd (clamav-daemon) complain. You should disable clamav-daemon and clamav-freshclam on a client-only system:

# update-rc.d -f clamav-daemon remove
update-rc.d: using dependency based boot sequencing
# update-rc.d -f clamav-freshclam remove
update-rc.d: using dependency based boot sequencing


After configuring as specified above, antivirus functionality should be tested with clamdscan. On the client node:

# clamdscan -v klez.exe 
klez.exe: W32.Elkern.C FOUND
----------- SCAN SUMMARY -----------
Infected files: 1


On the server node, from the ClamAV log:

Sat Mar 5 20:31:59 2016 -> instream(local):
W32.Elkern.C(16bc8fcec023b05b38af3580607bb728:92499) FOUND


Finally, I reconfigured the “” file in mod_security by changing from clamscan to clamdscan.


Written by bjorn on March 5th, 2016

Tagged with , , , , ,

Visualizing honeypot activity  

Posted at 5:02 pm in Uncategorized

Certain honeypot intruders are quite persistently trying to open outbound SSH tunnels, as described in an earlier article. So far I’ve seen a lot of attempts to open tunnels towards mail server TCP ports 25 (SMTP), 465 (SMTPS) and 587 (submission); web servers on TCP ports 80 (HTTP) and 443 (HTTPS); but also several other TCP ports.

For visualizing the activity, I fed the logs to AfterGlow. Below is shown a diagram of attempted SSH tunnels, where the intruders’ IP addresses are shown as red circles, the ports to which they attempt to connect are are light blue, and the targets are yellow triangles.

As the diagram shows, certain targets are attacked from different intruders (although with adjacent IP addresses). The objects’ sizes indicate frequency.


Simple honeypot map (one day only)

Simple honeypot map (one day only, low activity)


Another diagram, illustrating the frequency of attacks:

Medium complexity, still mostly readable.

One day of activity, two attackers. Still mostly readable.


Feeding two weeks of logs to AfterGlow was less informative. The graph clearly shows that certain sources are very busy, and certain destinations are frequently attacked – but that’s about where the diagram stops being useful.

Complex honeypot mapping (two weeks)

Complex honeypot map (two weeks)


In combination with some drill-down details, AfterGlow could be quite useful for analyzing details. I’ve got two items on my AfterGlow wishlist: 1) that labels go on top of objects, and 2) better avoidance logic so that objects do not cover other objects.

The corresponding Munin graph is also registering SSH tunneling attempts.

Honeypot activity

Written by bjorn on February 26th, 2016

Tagged with , , , ,

Honeynet outbound probes  

Posted at 8:42 am in Uncategorized

My Cowrie honeypot is now seeing a surge of outbound SSH tunnel probes, both towards different mail servers but also towards a specific web server, probably with the purpose of informing about a successful intrusion. The honeypot has seen outbound attempts before, but not as persistent as with this bot from .ru.

Cowrie fakes successful SSH tunneling, so the bot is at least kept somewhat busy. The honeypot is also in a very tight network environment with limited possibilities for outbound connections.

Here are some examples, formatted for readability:

2016-02-22 01:43:00+0100 [SSHService ssh-connection on
  HoneyPotTransport,1580,] direct-tcp connection
  request to
2016-02-22 01:43:01+0100 [SSHService ssh-connection on
  HoneyPotTransport,1580,] direct-tcp connection
  request to
2016-02-22 01:43:54+0100 [SSHChannel None (883) on
  SSHService ssh-connection on HoneyPotTransport,1580,]
  direct-tcp forward to with data
2016-02-22 02:00:28+0100 [SSHChannel None (979) on
  SSHService ssh-connection on HoneyPotTransport,1580,]
  direct-tcp forward to with data
  'POST / HTTP/1.1\r\nUser-Agent: Mozilla/4.0 (compatible; MSIE 6.0;
   Windows NT 5.1; SV1)\r\nContent-Type: application/x-www-form-urlencoded\r\n
   Connection: close\r\nContent-Length: 21\r\nHost:\r\n\r\n'
2016-02-22 03:39:18+0100 [SSHChannel None (0) on
  SSHService ssh-connection on HoneyPotTransport,1589,]
  direct-tcp forward to with data
  'POST / HTTP/1.1\r\nUser-Agent: Mozilla/4.0 (compatible; MSIE 6.0;
   Windows NT 5.1; SV1)\r\nContent-Type: application/x-www-form-urlencoded\r\n
   Connection: close\r\nContent-Length: 21\r\nHost:\r\n\r\n


Written by bjorn on February 22nd, 2016

Tagged with , , ,

More Logstalgia fun: Honeypot visualization  

Posted at 7:37 pm in Uncategorized

As the saying goes, when all you have is a hammer, everything looks like a nail. Well, it’s not that bad, but with a tool like Logstalgia available there’s a pretty low threshold for looking for other ways to use it. So why not try visualizing honeypot login activity?

I’ve been running a honeypot for some time, first using Kippo and later switching to Cowrie. Among Cowrie’s useful improvements is the ability to log to syslog. Already having a parser in place for converting syslog activity to a feed that Logstalgia accepts, adding Cowrie-to-Logstalgia support didn’t take much effort.

An additional parameter is added to indicate successful logins (at least from the intruder’s point of view), Logstalgia intuitively shows this by making the paddle not block the attempt. Also, instead of faking some status code, I set up the converter to assign the login name to the “URL” field and the password to the “status code” field. That way Logstalgia shows consecutive attempts with the same login name as a series of attacks on the same resource, while the different attempted passwords bounce off the paddle.

Note that the short video is running at 4x normal speed. You’ll have to click to make it start.

Sample syslog input (slightly redacted for readability):

cowrie: [SSHService ssh-userauth on HoneyPotTransport,446,] login attempt [ts/ts] failed
cowrie: [SSHService ssh-userauth on HoneyPotTransport,447,] login attempt [apache/apache] failed
cowrie: [SSHService ssh-userauth on HoneyPotTransport,448,] login attempt [games/games] failed
cowrie: [SSHService ssh-userauth on HoneyPotTransport,449,] login attempt [minecraft/minecraft] failed


The corresponding Logstalgia feed:



The output was fed to Logstalgia like this:

cat output.txt | logstalgia -600x200 -g "Login,URI=^[a-zA-Z0-9],100" -x -


With live visualization via syslog, the data is fed to Logstalgia directly and not from a file like shown above.

For a nice final touch, I’ve also added a Munin graph showing honeypot login attempts. The graph was made with the “loggrep” plugin, looking for corresponding values.

Written by bjorn on February 9th, 2016

Tagged with , , , , , ,

Live visualizing Mikrotik firewall traffic with Logstalgia  

Posted at 9:54 pm in Uncategorized

Previously I’ve written about visualizing firewall activity. Revitalizing a fireplot graphing tool gives a nice day-to-day overview, but after being reminded of Logstalgia in this Imgur post I wanted to give live visualization a shot.

Logstalgia is a neat tool for visualizing activity, by feeding it log files or live feeds. It’s originally designed for parsing web server logs, but it also accepts a generic format that allows for other purposes as well. By writing a short Perl script that acts like a syslog server (receiver) and converting the input to a format that Logstalgia accepts, my Mikrotik router is now live reporting any connection attempt to or through the firewall. For the visualization below I triggered a portscan to create some activity, or the video would be rather boring.

To make this work, the firewall must somehow identify traffic that’s being denied (unless you only log blocked traffic). The script will then pass only these log records to Logstalgia. I’ve been testing this with a Mikrotik device, but any firewall able to log to or through syslog will work fine.

Original syslog input

Feb 6 21:42:36 BLOCK: in:ether1 out:(none), src-mac 00:00:00:6a:f3:9c,
proto ICMP (type 8, code 0),>, len 28


Logstalgia formatted output:



I’m starting the perl script and Logstalgia like this:

./syslog2logstalgia | logstalgia -800x640 --disable-progress -x \
--no-bounce --hide-response-code --sync \
-g "TCP,URI=TCP,45" -g "UDP,URI=UDP,45" -g "ICMP,URI=ICMP,10" \


Note that visualizing firewall logs with Logstalgia has been done by a lot of other people. Howtos for other firewall products may be available via your favourite search engine.

Written by bjorn on February 6th, 2016

Tagged with , , , , ,