Archive for the ‘ossec’ tag

Near-realtime blacklist warnings with NetFlow, Perl and OTX  

Posted at 7:43 pm in Uncategorized

Installing IDS sensors in your network for monitoring traffic is not always feasible, for several possible reasons. Perhaps the network infrastructure is too complex, leading to blind spots. Maybe the affected network links have higher capacity than your ad hoc IDS sensor, causing packet loss on the sensor. Or your company may be organized in such a way that installing “foreign” hardware in the network infrastructure is not easily done.

Still, without going “all in” on a potentially expensive IDS project, it could be useful with some insight into what’s going in to and out from your network, keeping an eye on known malicious IP addresses and networks. Setting up a NetFlow feed from the company’s routers will usually not incur any significant loads, nor does it interfere with the network traffic, so that could be a possible approach. I’ve previously covered NetFlow and SiLK for rear-view mirror analysis of whether any blacklisted IP resources have been communicating with your system and users in the past. What if we could do the same, just in (almost) real-time? With the help of Perl and the Net::Flow module, we can.

Bill of material

  • Router(s) that support(s) NetFlow (I’ve used version 9 but the Perl module seems to support v5 and IPFix as well).
  • Perl, and the Net::Flow module for parsing the NetFlow data.
  • One or more IP blacklists of your choice. For the purpose of this test I’m using my subscribed lists from AlienVault’s Open Threat Exchange, but the list of IP addresses to compare against can easily be extended with – or replaced by – other lists like the SANS blocklist or any DNSBL/RBL.

The Perl script I’ve set up for this purpose is crudely derived from the Net::Flow sample code, and after my tweaks it’s currently not something that should see the light of day. Its output, however, is pretty enough for a modest presentation. The IP addresses (IPv4 as well as IPv6) and other info are extracted from the different flow fields, detailed in this Cisco document.  In my script, each offending IP is associated with URLs linking to OTX pulses where further information can be found.

Some sample entries from the Perl script’s output:

2016-06-07 12:38:20 : 93.174.93.94:48928 -> aa.bb.cc.dd:53 (TCP)
 https://otx.alienvault.com/pulse/56fdd27e4637f207cbccfda7/
 https://otx.alienvault.com/pulse/5711d7740ebaa4015af20592/

2016-06-07 13:37:46 : aa.bb.cc.dd:5901 -> 183.60.48.25:12207 (TCP)
 https://otx.alienvault.com/pulse/56bbe5ba4637f25d9365dcab/
 https://otx.alienvault.com/pulse/568a9c1f67db8c057c6fc09f/

2016-06-07 13:51:34 : aa.bb.cc.dd:443 -> 184.105.139.67:58879 (TCP)
 https://otx.alienvault.com/pulse/56c3ab564637f26ad04e3dc3/

2016-06-07 17:51:13 : 216.243.31.2:43140 -> aa.bb.cc.dd:443 (TCP)
 https://otx.alienvault.com/pulse/5709fcb267db8c4b471bdc3c/
 https://otx.alienvault.com/pulse/568a99df4637f2624bcdbcb8/

2016-06-07 18:00:52 : 93.174.93.50:46521 -> aa.bb.cc.dd:53 (UDP)
 https://otx.alienvault.com/pulse/5709fcb267db8c4b471bdc3c/
 https://otx.alienvault.com/pulse/571c4147c1492d015c14c214/

 

Some unsolicited questions and answers

  • What can this be used for? It can be a proof-of-concept, in cases where you might need to argue why you want to install an IDS. It can also be used for statistical purposes, to get a grasp of how often your network is communicating with malicious systems on the Internet.
  • Will I be missing information with this simplified setup? Yes, most likely. This implementation is not intended as an IDS replacement, but it will give an indication of unwanted activity to and from your network. Also, your router may provide sampled NetFlow data, e.g. only a portion of the traffic will be selected for NetFlow analysis. At times you might see only the response traffic, in cases where a remote node contacting a non-responsive port will not always be classified as an established flow but a related ICMP response might be.
  • Why isn’t it real-time? A flow won’t be registered by the router until a connection is completed or has timed out. Depending on your router’s configuration, it could also be batching up the NetFlow feeds for regular transfers. I’ve seen 20 to 30 seconds delay between the actual connection and the NetFlow push from the router.
  • Can I use the output somewhere else? Sure, you can make the Perl script log to syslog or to a file that OSSEC or something similar can read from.

 

Written by bjorn on June 7th, 2016

Tagged with , , , , , , ,

Threat intelligence: OTX, Bro, SiLK, BIND RPZ, OSSEC  

Posted at 8:15 am in Uncategorized

Building a toolbox around threat intelligence can be done with freely available tools. Shared information about malicious behaviour allows you to detect and sometimes prevent activity from – and to – Internet resources that could compromise your systems’ security.

I’ve already described how to use lists of malicious domain names in a BIND RPZ (Response Policy Zone). Adding an information feed like AlienVault OTX (Open Threat Exchange) to the mix further extends the awareness and detection capabilities.

AlienVault is probably most known for their SIEM (Security Information and Event Management) named Unified Security Management™, with a scaled-down open source version named Open Source Security Information and Event Management (OSSIM). They also provide a platform for sharing threat intelligence, namely Open Threat Exchange (OTX). OTX is based on registered users sharing security information, for instance domains and hostnames involved in phishing scams, IP addresses performing brute force SSH login attempts, etc. The information is divided into so-called pulses, each pulse a set of information items considered part of the same malicious activity. For example, a pulse can contain URLs to a site spreading drive-by malware, the IP addresses of their C&C, along with hashes of the files. By selecting which pulses and/or users to subscribe to, the registered information in each pulse will be available through a feed from their API.

Carefully reviewing which users/pulses to subscribe to – there’s always a risk of false positives – I’m now regularly receiving an updated feed. This feed is parsed and currently split into two files: One RPZ file containing hostnames and domains for use with BIND, and one file containing IP addresses for use with SiLK.

As explained in an earlier post, OSSEC will let me know if someone (or something) makes DNS requests for a domain or hostname registered as malicious. Extending this to include the DNS records obtained from OTX was simply a matter of defining a new RPZ in BIND. Depending on how this is used (block? redirect? alert?), a whitelist should be in place to prevent accidental blocking of known good domains. One pulse describes all the Internet resources a client infected by a certain exploit will contact, including some certificate authorities which are not necessarily considered evil.

The file with IP addresses can be used directly with a firewall, by logging or even blocking or throttling traffic to/from the IP addresses in question. For rear-view mirror analysis it can be used with SiLK, to find out if there has been any network traffic to or from any of these addresses. To do this, you will first have to create an IP set with the command rwsetbuild:

# rwsetbuild /some/path/ip-otx.txt /some/path/ip-otx.set

 

Now we can use this set file in our queries. For this query I’ve manually selected just a few inbound matches:

# rwfilter --proto=0-255 --start-date=2016/01/01 \
  --sipset=/some/path/ip-otx.set --type=all \
  --pass=stdout | rwcut -f 1-5
            sIP|            dIP|sPort|dPort|pro|
   185.94.111.1|  my.ip.network|60264|   53| 17|
   216.243.31.2|  my.ip.network|33091|   80|  6|
   71.6.135.131|  my.ip.network|63604|  993|  6|
   71.6.135.131|  my.ip.network|60633|  993|  6|
   71.6.135.131|  my.ip.network|60888|  993|  6|
   71.6.135.131|  my.ip.network|32985|  993|  6|
   71.6.135.131|  my.ip.network|33060|  993|  6|
   71.6.135.131|  my.ip.network|33089|  993|  6|
   71.6.135.131|  my.ip.network|33103|  993|  6|
   71.6.135.131|  my.ip.network|33165|  993|  6|
   71.6.135.131|  my.ip.network|33185|  993|  6|
   71.6.135.131|  my.ip.network|33614|  993|  6|
   71.6.135.131|  my.ip.network|33750|  993|  6|
   71.6.135.131|  my.ip.network|60330|  993|  6|
  185.130.5.224|  my.ip.network|60000|   80|  6|
  185.130.5.224|  my.ip.network|60000|   80|  6|
  198.20.99.130|  my.ip.network|    0|    0|  1|
  185.130.5.201|  my.ip.network|43176|   53| 17|
  129.82.138.44|  my.ip.network|    0|    0|  1|
  185.130.5.224|  my.ip.network|60000|   80|  6|
  185.130.5.224|  my.ip.network|60000|   80|  6|

 

When you need more details about the listed address or other indicators, OTX provides a search form to find the pulse(s) in which the indicator was registered.

OTX can be used with Bro as well, and there are at least two Bro scripts for updating the feeds from the OTX API. The one that works for me is https://github.com/hosom/bro-otx. The script will make Bro register activity that matches indicators from an OTX pulse.

Sample log entries, modified for readability:

my.ip.network 59541 some.dns.ip    53 - - - union83939k.wordpress.com
                                            Intel::DOMAIN DNS::IN_REQUEST
my.ip.network 40453 54.183.130.144 80 - - - ow.ly
                                            Intel::DOMAIN HTTP::IN_HOST_HEADER
74.82.47.54   47235 my.ip.network  80 - - - 74.82.47.54
                                            Intel::ADDRConn::IN_ORIG

 

This article mentions just a few components that can be combined. Obviously there’s a lot of possibilities for integrating and interfacing between different systems. There are several companies that provide threat intelligence feeds, some for free and some for paying customers. Depending on the product(s), a SIEM would be able to combine and correlate the different kinds of threat intelligence to detected events.

Written by bjorn on March 9th, 2016

Tagged with , , , , , , , , , ,

Logging WordPress activity to OSSEC  

Posted at 11:53 pm in Uncategorized

Today I came across this blog article, explaining how to make WordPress log suspicious activity to an audit log file, which in turn can be monitored by OSSEC. Everything mentioned in the article was all fine and dandy, until I read the last paragraph: “Note that for this feature to work, you have to use our fork of OSSEC […]“.

Being less than enthusiastic about replacing my existing OSSEC (version 2.8.3) installation with a fork (even if the fork happens to originate from OSSEC’s founder), I wanted to make this work with what I’ve already got. Following the main instructions from the blog article, I installed the sucuri-scanner plugin but did not request an API key – at this point, at least. Then, by providing an absolute path to an existing log file, to which the web server has write access, I activated the plugin’s audit log exporter.

The same absolute path was added to the OSSEC agent‘s ossec.conf (/var/ossec/etc/ossec.conf) file as a syslog file:

 <localfile>
   <log_format>syslog</log_format>
   <location>/var/log/wp/site/audit.log</location>
 </localfile>

 

So far, so good – now to make the OSSEC manager correctly decode and parse the log events.

First, I replaced the whole wordpress_rules.xml file with the one provided by the OSSEC fork. I found the updated wordpress_rules.xml file on https://bitbucket.org/dcid/ossec-hids/, and from there by navigating the source tree (source → ossec-hids/etc/rules/). The exact file location in the repository could change with future versions and commits, so there’s not much of a point in providing an exact URL. Apart from a signature ID collision (two signatures had sid 9507) this updated file was an improvement over the wordpress_rules.xml that came with OSSEC 2.8.3. The file is too large to inline here.

The final piece of the puzzle was to provide a useful decoder. I’ve added the following to the manager‘s /var/ossec/rules/local_decoder.xml:

<decoder name="wordpressaudit">
 <parent>windows-date-format</parent>
 <use_own_name>true</use_own_name>
 <prematch offset="after_parent">WordPressAudit </prematch>
 <regex offset="after_prematch">^\S+ \S+ : (\w+): (\S+); </regex>
 <order>action, srcip</order>
</decoder>

 

And bingo, it works. Failed WordPress logins, along with some other weird activities – and normal as well, if you so wish – will now be identified by OSSEC and you can set the severity levels accordingly. Here’s an example of the OSSEC alert log’s detection of a failed login (formatted for readability):

** Alert 1454537692.8654637: mail - syslog,wordpress,authentication_failed,
2016 Feb 03 23:14:52 (webserver.example.com) 10.0.0.10->/var/log/wp/site/audit.log
Rule: 9501 (level 7) -> 'WordPress authentication failed.'
Src IP: 192.168.0.42
2016-02-03 22:14:51 WordPressAudit www.example.com bjorn@ruberg.no : Error:
  192.168.0.42; User authentication failed: EvilUser

 

and even

** Alert 1454541645.146379: mail - syslog,wordpress,syscheck,
2016 Feb 04 00:20:45 (webserver.example.com) 10.0.0.10->/var/log/wp/site/audit.log
Rule: 9508 (level 7) -> 'WordPress post updated.'
2016-02-03 23:20:44 WordPressAudit www.example.com bjorn@ruberg.no : Notice:
  bjorn, 192.168.100.100; Post was updated; identifier: 4096; name: Some Article Name

 

Written by bjorn on February 3rd, 2016

Tagged with , , , , ,

Malware detection with DNS RPZ and OSSEC  

Posted at 2:06 pm in Uncategorized

Building upon a sysadvent article I wrote at work, I’ve set up a dedicated Response Policy Zone using the freely available data files from the Malware Domain Blocklist. There are different ways to do this, but for this particular purpose I’ve imported the text file and generated a single zone file locally. BIND supports up to 32 RPZs, so in my config I’ve set this up as a separate zone, referenced as “malware”.

Below is the zone definition:

zone "malware" {
  type master;
  file "/etc/bind/db.malwaredomains";
};

 
Defining the “malware” zone as an RPZ (I have two response policy zones, one simply named rpz and now this one named malware):

options {
  response-policy { zone "rpz"; zone "malware"; };
};

 
Configure logging. The zones defined in the above response-policy {} setting fall under the rpz logging category.

logging {
  channel named-rpz {
    file "/var/log/bind/rpz.log" versions 3 size 250k;
    severity info;
  };
  category rpz {
  named-rpz;
  };
};

 
In the BIND log files, requests for domains in the malware zone are logged in the RPZ log file, suffixed with the zone reference, namely “malware”.

client 127.0.0.1#53547 (czgtgj.com): rpz QNAME Local-Data rewrite czgtgj.com via czgtgj.com.malware

 
After testing that attempts to reach malware sites are indeed logged by the DNS server, I configured OSSEC to tail BIND’s malware query log. For this I had to write a decoder and define logging rules in OSSEC, shown below. These could probably be drastically improved.

The end result is exactly as I wanted: If someone (or something) on my network is trying to reach a resource within a domain registered as affiliated with malware, OSSEC will react and alert by email, raise an alarm in your SIEM, or whatever else you want OSSEC to do.

From /var/ossec/etc/local_decoder.xml:

<decoder name="malware-dns">
  <prematch>^client </prematch>
</decoder>
<decoder name="malware-dns-lookup">
  <parent>malware-dns</parent>
  <regex offset="after_parent">^(\.+)#\d+ \((\.+)\): \.+.malware$</regex>
  <order>srcip, extra_data</order>
</decoder>

 
From /var/ossec/rules/malware_dns_rules.xml:

<group name="syslog,bind">
  <rule id="110201" level="0">
    <decoded_as>malware-dns</decoded_as>
    <description>Malware DNS group</description>
  </rule>
  
  <rule id="110202" level="8">
    <if_sid>110201</if_sid>
    <match>malware$</match>
    <description>Malware DNS lookup</description>
  </rule>
</group>

 
From /var/ossec/etc/ossec.conf:

<rules>
  [...]
  <include>malware_dns_rules.xml</include>
</rules>

 
Now, if something should reach out to a malware domain, I will get an email from my OSSEC server:

Received From: server->/var/log/bind/rpz.log
Rule: 110202 fired (level 8) -> "Malware DNS lookup"
Portion of the log(s):

client 127.0.0.1#52162 (czgtgj.com):
rpz QNAME Local-Data rewrite czgtgj.com via czgtgj.com.malware

 

Written by bjorn on December 8th, 2015

Tagged with , , , , , ,