While tinkering with the settings for my site, I discovered an Apache module called mod_proxy. I was interested in it because I am running two webservers – one for www.samhobbs.co.uk and one for webmail, and I wanted to redirect traffic from one part of the site to the webmail server using ProxyPass. Unfortunately, I was over-enthusiastic in my explorations and made an error: I enabled my server to be used as an open proxy, and attracted thousands and thousands of dodgy requests from around the world. What this meant is that anyone could connect to my server and use it to visit web pages whilst concealing their true identity: the pages visited would only see my IP, not theirs.

What I did wrong.

What I did was stupid, embarrassingly so. It wouldn’t have happened if I had read the manual, or even just paid attention to the useful information in the config files. On Debian and Ubuntu systems, the settings for mod_proxy are located here: /etc/apache2/mods-enabled/proxy.conf, and look something like this:

<IfModule mod_proxy.c>

# If you want to use apache2 as a forward proxy, uncomment the
# 'ProxyRequests On' line and the <Proxy *> block below.
# WARNING: Be careful to restrict access inside the <Proxy *> block.
# Open proxy servers are dangerous both to your network and to the
# Internet at large.
#
# If you only want to use apache2 as a reverse proxy/gateway in
# front of some web application server, you DON'T need
# 'ProxyRequests On'.

ProxyRequests off
# <Proxy *>
#        AddDefaultCharset off
#        Order deny,allow
#        Deny from all
#        #Allow from .example.com
#</Proxy>

# Enable/disable the handling of HTTP/1.1 "Via:" headers.
# ("Full" adds the server version; "Block" removes all outgoing Via: headers)
# Set to one of: Off | On | Full | Block
#ProxyVia Off

</IfModule>

Line 13 was the unlucky one here. You don’t need to enable ProxyRequests to use ProxyPass, but I naively changed line 13 to ProxyRequests on, and didn’t even restrict its usage. Note that the config file actually warns against doing this – it was really stupid. Anddd… hey presto: my RasPi was now an open proxy ready to be exploited by the world’s tech savvy miscreants and script kiddies.

Some Signs that your server is being used as an open proxy:

The time it took me from changing the setting to realising something was wrong was about a week. The number of requests was small at first, and at some point someone must have realised the server could be used as a proxy because the number of requests increased dramatically, which is when I noticed. There were a few things that tipped me off. The first was that my router slowed down dramatically. A quick look at the admin page showed that the number of active IP connections was maxed out. You can check the number of active IP connections on a DD-WRT router on the web interface under Status --> Router. The screenshot below is my router during normal activity. When it was an open proxy, the number of IP connections was about 4090. The image above shows the number of active connections during normal use. Other routers should have a similar function to this. If you are renting hosting space from somewhere else then you won’t be able to use this an indicator because your server isn’t behind your router. The second thing that tipped me off was that when I did a google search I got this message: “Our systems have detected unusual traffic from your computer network” and a CAPTCHA box to confirm that I was a human. Presumably, this is because some kind of automated service was using the open proxy to perform google searches. By this time I was very concerned, and had shut off port forwarding on my router while I investigated. I took a look at the apache access logs confirmed the worst: thousands and thousands of requests had been made to the server in the last few days, some of them to really dodgy porn sites and other pages you really wouldn’t want people to think you had visited. Just look at the size of the logs and the dates – the requests to the server for hosts that weren’t defined as virtual hosts in /etc/apache2/sites-enabled/000-default had exploded over the last couple of days. The content of the log files sheds more light on the requests the server was receiving: Notice the two numbers after the GET address. The first one is a code that will tell you whether or not the request succeeded. If the number is 200, the request was successful. Server error is 500. Other codes can be checked here: http://www.w3.org/Protocols/rfc2616/rfc2616.txt.

What to do if you’ve made the same mistake I did

While you sort out the server configuration, turn off port forwarding on your router’s admin page. While there were 4000 active IP connections to the Pi I couldn’t even open a SSH session, either to the local IP address or the global one. I’m not sure whether this is because the active IP sessions were maxed out, or because the server was overloaded, but either way it’ll stop those unwanted connections until you can sort out the configuration. First, change the setting “ProxyRequests” to “off” in /etc/apache2/mods-enabled/proxy.conf. This will stop your server being used as an open proxy, but when someone requests a page on some other site (www.someotherdomain.com/home) your site will look for a corresponding page on your website, and serve www.yourdomain.com/home. This is obviously a waste of your resources, as whoever was trying to use your server as a proxy has no interest in your site at all. The way to get around this is to add to your VirtualHosts in /etc/apache2/sites-enabled/000-default.

<VirtualHost *:80>
        ServerName default.only
        <Location />
                Order allow,deny
                Deny from all
        </Location>
</VirtualHost>

<VirtualHost *:80>
        ServerAdmin webmaster@localhost
        ServerName www.samhobbs.co.uk:80

        DocumentRoot /var/www
</VirtualHost>

The first virtual host (default.only) must go before your other VirtualHosts, and means that if anyone requests a domain that is not defined as a VirtualHost, the request will be rejected instead of sending the corresponding page for your actual site. Now reload the apache configuration:

sudo service apache2 reload

…and enable port forwarding on your router again. Because proxy requests now return an error code, requests from automated services soon drop off, and anyone trying to connect manually will also lose interest. The number of active IP connections started out around 300, and have now dropped off to below 100. This should solve your problem. I feel really stupid for making the mistake in the first place, but at least I’ve learned something about Apache. Next time I’ll RTFM!