Recently, I've spent a lot of time tweaking my ModSecurity configuration to remove some false positives. This tutorial will:

• Explain the the various methods of altering ModSecurity rules starting with the crudest and working up to the more specific techniques
• Give some varied examples of custom rules written for exception handling, with a particular focus on the rules distributed by the OWASP Core Rule Set team.

I am calling the process of removing false positives "whitelisting", but technically I should be calling it "exception handling". However, I think more people looking for this information will find it by searching for "whitelisting".

Basic Whitelisting

My previous articles about ModSecurity have discussed some of the cruder methods of removing false positives. When I was new to ModSecurity, I even wrote a BASH script to create a simple whitelist based on ModSecurity messages in the apache error log. The script made use of the crudest (and easiest) whitelisting technique: removing rules entirely with SecRuleRemoveById, for example this configuration disables rule 981143:

SecRuleRemoveById 981143

Something that wasn't immediately obvious to me is that these are Apache directives and can go in any Apache configuration file. You can put them inside the main Apache config file or a specific VirtualHost file. You can also put them in a separate file entirely, and Include them, like this:

# Include personalised whitelist file for ModSecurity
Include /etc/modsecurity/whitelists/samhobbs.co.uk.conf

One of the features the BASH script I wrote made use of was the ability to combine SecRuleRemoveById with Location and LocationMatch statements to remove rules at specific locations, e.g.:

<LocationMatch "^/comment/[0-9]+/approve$">
SecRuleRemoveById 981143
</LocationMatch>

The difference between Location and LocationMatch is that the latter supports regular expressions, like in the example above. In addition to SecRuleRemoveById, there are two other directives available, SecRuleRemoveByMsg and SecRuleRemoveByTag . These do the same thing, but match messages and tags instead of rule IDs, which can be especially handy when you want to do something like remove all SQLi rules in the Core Rule Set.

More Specific Whitelisting

The methods of whitelisting above work well enough for simple cases, but if you over-use them you'll soon end up with the whole of your CRS turned off, or broken! The next level of complexity involves some new directives, discussed below.

SecRuleUpdateActionById

The first of these directives is SecRuleUpdateActionById, which allows us to update the action performed by a rule. The Core Rule Set is best used in anomaly scoring mode, where the complete chain of rules is evaluated during each phase of request processing, and an overall score is generated to decide whether to block the request or not. However, if you are running ModSecurity in traditional mode then the first rule that matches with the block action will execute the default action, which is normally deny. In this situation, if you wanted to prevent a rule from causing a request to be denied, but still wanted it to be logged, you could do so like this:

SecRuleUpdateActionById 981143 "pass"

Any of the parameters that are part of the action list in a SecRule statement can be updated here. If you want to specify a list, you can do it inside a set of quotation marks. The old list is inherited and overwritten by any new parameters you specify, so you don't need to re-type the whole thing. The one exception I have found is chain - if you are updating a rule that is starting or continuing a chain, you must specify chain again or the subsequent rules will not be treated as part of the chain.

SecRuleUpdateTargetById, SecRuleUpdateTargetByMsg and SecRuleUpdateTargetByTag

Each ModSecurity rule specifies a list of variables to be tested against the operator (e.g. @rx, @beginsWith, @streq etc.). If you want to add additional variables to the list to be inspected, or remove a particular one that is causing a problem, you can use one of these parameters. The following rule would remove the comment_body argument from the list inspected by rule 981143:

SecRuleUpdateTargetById 981143 !ARGS:comment_body

Note that the arguments you add will be processed on top of the existing list, so if the original list contained ARGS and it was updated by the modification above, the complete list would be ARGS !ARGS:comment_body or in words "all arguments apart from comment body".

Chaining SecRule statements

If you want to use one of the directives above, but only apply it in specific situations, you can create chains of statements. This (made up example) chain removes request cookies from the target list of 981231 at yourdomain.com/admin :

SecRule REQUEST_URI "@beginsWith /admin" \
  "chain, \
  id:'000001', \
  phase:1, \
  t:none, \
  nolog, \
  pass"
  SecRuleUpdateTargetById 981231 !REQUEST_COOKIES

To create a logical "and", just increase the length of the chain - this extended example only updates the target list if the URL begins with /admin and the IP address of the client is 192.168.1.1:

SecRule REQUEST_URI "@beginsWith /admin" \
  "chain, \
  id:'000001', \
  phase:1, \
  t:none, \
  nolog, \
  pass"
  SecRule REMOTE_ADDR "@ipMatch 192.168.1.1" chain
  SecRuleUpdateTargetById 981231 !ARGS:REQUEST_COOKIES

Whitelisting with "ctl" actions

Although it is possible to amend the rule set selectively by creating chains with SecRuleUpdateTargetById and similar parameters, these chains can quickly become difficult to read. My preferred method of mitigating false positives is to use the newer ctl versions of those actions:

• ctl:ruleRemoveById (or Msg or Tag)
• ctl:ruleRemoveTargetById (or Msg or Tag)
• ctl:ruleEngine On|Off|DetectionOnly

I find this method easier on the eye, since everything that starts with SecRule is a condition, and the actions (updating target lists, removing rules etc.) are placed in the action list. If you have two conditions (logical "and") then you have two SecRule statements in the chain, and the ctl action goes in the action list of the last one. The rule below is equivalent to the previous example:

SecRule REQUEST_URI "@beginsWith /admin" \
  "chain, \
  id:'000001', \
  phase:1, \
  t:none, \
  nolog, \
  pass"
  SecRule REMOTE_ADDR "@ipMatch 192.168.1.1" \
    ctl:ruleRemoveTargetById=981173;ARGS:REQUEST_COOKIES

Note that you don't need the ! in front of the argument you want to remove when using ctl:ruleRemoveById. Remember to put the ctl actions in the last rule in a chain if you want them to be executed when the whole chain matches - the example code below would result in the target list being updated for all IP addresses, because the action is fired before the @ipMatch condition is evaluated:

SecRule REQUEST_URI "@beginsWith /admin" \
  "chain, \
  id:'000001', \
  phase:1, \
  t:none, \
  nolog, \
  pass, \
  ctl:ruleRemoveTargetById=981173;ARGS:REQUEST_COOKIES"
  SecRule REMOTE_ADDR "@ipMatch 192.168.1.1"

Since ctl: actions are evaluated at runtime, these rules should generally go in modsecurity_crs_15_customrules.conf so they are processed before the rule they are amending.

Examples

This section lists some real world examples of rules I wrote for my setup.

Use a custom error file

No matter how careful and thorough you are when creating your whitelist, it is inevitable that some users will be blocked when trying to do legitimate things. Luckily, it is possible to serve clients a custom error document when they are blocked, which is much less frustrating for them than seeing a standard "500: Internal Server Error" with no explanation of what happened. There are three rules in the CRS that do all the blocking when ModSecurity is deployed in anomaly scoring mode. First, update the actions of these rules in modsecurity_crs_60_customrules.conf:

# update action for inbound blocking rules in modsecurity_crs_49_inbound_blocking.conf
# so that they use the Security Error script
SecRuleUpdateActionById 981175 "chain,deny,status:501"
SecRuleUpdateActionById 981176 "chain,deny,status:501"

# also update the outbound blocking rules
SecRuleUpdateActionById 981200 "chain,deny,status:501"

Next, tell Apache to serve a custom error page for error 501:

ErrorDocument 501 /security-error.php

This directive could go in the same rule file, or if you want to serve different pages for different domains (e.g. with different contact details), you could place it in your VirtualHost file instead. The path is relative to the DocumentRoot defined in the VirtualHost; my site files are stored in /var/www/samhobbs so I created the custom error page at /var/www/samhobbs/security-error.php. Here's my file:

<?php header("HTTP/1.0 403 Forbidden"); ?>

<html>

<head>
<title>Security Error</title>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
</head>

<body>
<h1>Security Error</h1>
<pre>
Your request has been blocked for security reasons. Please try again in a few minutes.
Should the problem persist, please email me (see the contact page for details) with the
following unique ID, which will help to identify your request:

<?php echo htmlspecialchars($_SERVER["REDIRECT_UNIQUE_ID"], ENT_QUOTES, 'UTF-8'); ?>


Sorry for any inconvenience.

<!--
This comment is here to increase the page size and prevent Internet Explorer from masking
the message. More information is available at the following address:

        http://support.microsoft.com/default.aspx?scid=kb;en-us;Q294807
-->

</body>
</html>

You'll notice that the start of the script sets the status back to 403 - the 501 status is just for internal use/convenience (you only want to serve this page for security errors, not for actual internal server errors as well). This file has to be readable by Apache, but it does not have to be executable. Mine has ownership root:root and permissions 644. You will notice that the variables ModSecurity uses for processing rules are made available to the PHP script, prefixed with REDIRECT_, so REDIRECT_UNIQUE_ID is the unique ID that appears in the Apache logs. If someone sends you that ID, it's simple to look it up in the audit log and determine why the request was blocked. You can see what this page looks like to users by clicking this link: https://samhobbs.co.uk/?test=test-attack The rule causing the request to be blocked when you click that URL is this:

SecRule ARGS "test-attack"\
  "id:'000001', \
  phase:1, \
  log, \
  deny, \
  status:501"

This is one of the many cool tips I discovered by reading the ModSecurity handbook, although I had to modify the example slightly as the HTTP response codes allowed by default in apache2 seem to have changed since the book was written (the book suggests using 509, but the configuration checking tool throws errors if you try and use that code). I also had to add php tags to the script.

Turn off denial of service counter for a specific URI or path

The denial of service rules in modsecurity_crs_11_dos_protection.conf count the number of requests from each IP address and block new requests when the limit has been exceeded. The rule that does the blocking is 981045:

# Block and track # of requests but don't log
SecRule IP:DOS_BLOCK "@eq 1" "phase:1,id:'981045',t:none,drop,nolog,setvar:ip.dos_block_counter=+1"

This rule checks to see if the ip.dos_block is set, but since it doesn't actually set it itself, removing the rule at a specific URL will prevent the client from being blocked for part of the site, but it will be blocked everywhere else. Not a good solution. Here is the rule that counts the number of requests:

#
# DOS Counter
# Count the number of requests to non-static resoures
# 
SecRule REQUEST_BASENAME "!\.(jpe?g|png|gif|js|css|ico)$" "phase:5,id:'981047',t:none,nolog,pass,setvar:ip.dos_counter=+1"

Removing this rule on parts of the site where you expect lots of requests (like ownCloud) prevents the IP block from being created in the first place:

SecRule REQUEST_URI "@beginsWith /owncloud" \
  "id:'000013', \
  phase:5, \
  t:none, \
  nolog, \
  pass, \
  ctl:ruleRemoveById=981047"

This rule needs to be processed before the file modsecurity_crs_11_dos_protection.conf, so place it in a new file named modsecurity_crs_10_customrules.conf.

Enable extra HTTP request methods and content types for specific locations

The CRS sets the allowed HTTP request methods in modsecurity_crs_10_setup.conf rule 900012, and enforces them in file modsecurity_crs_30_http_policy.conf, rule 960032. Here is the relevant section from the setup file:

SecAction \
  "id:'900012', \
  phase:1, \
  t:none, \
  setvar:'tx.allowed_methods=GET HEAD POST OPTIONS', \
  setvar:'tx.allowed_request_content_type=application/x-www-form-urlencoded|multipart/form-data|text/xml|application/xml|application/x-amf|application/json', \
  setvar:'tx.allowed_http_versions=HTTP/0.9 HTTP/1.0 HTTP/1.1', \
  setvar:'tx.restricted_extensions=.asa/ .asax/ .ascx/ .axd/ .backup/ .bak/ .bat/ .cdx/ .cer/ .cfg/ .cmd/ .com/ .config/ .conf/ .cs/ .csproj/ .csr/ .dat/ .db/ .dbf/ .dll/ .dos/ .htr/ .htw/ .ida/ .idc/ .idq/ .inc/ .ini/ .key/ .licx/ .lnk/ .log/ .mdb/ .old/ .pass/ .pdb/ .pol/ .printer/ .pwd/ .resources/ .resx/ .sql/ .sys/ .vb/ .vbs/ .vbproj/ .vsdisco/ .webinfo/ .xsd/ .xsx/', \
  setvar:'tx.restricted_headers=/Proxy-Connection/ /Lock-Token/ /Content-Range/ /Translate/ /via/ /if/', \
  nolog, \
  pass"

The methods GET, HEAD, POST, and OPTIONS are usually enough for something like a simple blog, and disabling the less well-known methods is probably quite a good idea, because legitimate users won't be using them (bots might, in an attempt to get your web app to react in a non-standard way and expose some vulnerability or sensitive information). However, if you run ownCloud or a similar service, you will find that calDAV, cardDAV and webDAV together use the methods PROPFIND, REPORT, PUT and MKCOL, which you need to enable to avoid these requests from being blocked by ModSecurity. The same is true for standard content types: most requests will be fine with the defaults, but I've noticed ModSecurity uses text/calendar for calDAV and application/octet-stream for webDAV. The following rule will enable these requests by overriding the allowed methods and content types for ownCloud transactions, without allowing them globally:

SecRule REQUEST_URI "@beginsWith /owncloud/remote.php" \
  "id:'000002', \
  phase:1, \
  t:none, \
  setvar:'tx.allowed_methods=GET HEAD POST OPTIONS PROPFIND REPORT PUT MKCOL', \
  setvar:'tx.allowed_request_content_type=application/x-www-form-urlencoded|multipart/form-data|text/xml|application/xml|application/x-amf|application/json|application/octet-stream|text/calendar', \
  nolog, \
  pass"

I put this rule in modsecurity_crs_15_customrules.conf.

Allow missing accept header for a specific location

Pretty much every browser will send an "accept" header to the server to communicate which content types it is happy to receive. For this reason, requests without accept headers are protocol anomalies (not strictly a violation, but a good sign it's not a genuine request) and are flagged by the rules in modsecurity_crs_21_protocol_anomalies.conf. However, you may find that certain programs (particularly ones that send lots of requests like sync clients) omit the accept header. This rule will remove the rule check for ownCloud:

SecRule REQUEST_URI "@beginsWith /owncloud" \
  "id:'000005', \
  phase:2, \
  t:none, \
  ctl:ruleRemoveById=960015, \
  nolog, \
  pass"

Fight comment spam with hidden fields

There are modules that achieve something similar to this for Drupal and probably WordPress: a field is added to comment forms and hidden with CSS. Humans don't see it but bots do, and the dumb ones fill it in causing the comment to be blocked. This rule will block those requests before they are even seen by the web app, which is preferable (some of these bots are carrying out SQLi probing attacks).

SecRule ARGS:feed_me "!@rx ^$" \
  "id:'000023', \
  phase:2, \
  log, \
  t:none, \
  block, \
  logdata:'SPAM comment detected - hidden filled in comment form', \
  msg:'SPAM comment detected - hidden field filled in comment form', \
  severity:'2', \
  setvar:'tx.msg=%{rule.msg}', \
  setvar:tx.anomaly_score=+20, \
  setvar:tx.%{rule.id}-CUSTOM_RULE/COMMENT_SPAM"

The regex matches "not empty". This one goes in modsecurity_crs_15_customrules.conf.

Turn on the XML processor for CalDAV/CardDAV/WebDAV requests

The CRS has some rules that enable the XML processor for standard XML content types, but many *DAV requests have custom content types and therefore don't trigger these rules. If ModSecurity inspects an XML request body without processing it with the XML processor, it will spew out a bunch of false positives due to all of the < and > characters (normally you get a "number of special characters exceeded" message), and there's a good chance the request will be blocked. Turning on the XML request body processor will ensure the XML is valid (this check is done in both the CRS and the ModSecurity setup file, so you don't need to add anything), and it will also strip the XML tags leaving just the data, which is run against the other rules.

# turn on the XML processor for webdav and caldav requests
SecRule REQUEST_URI "@rx ^/owncloud/remote.php/(webdav|caldav|carddav)" \
  "chain,id:'000090',phase:1,t:none,t:lowercase,pass,nolog"
    SecRule REQUEST_METHOD "@rx (PROPFIND|REPORT)" \
      "ctl:requestBodyProcessor=XML"

That rule should be placed in modsecurity_crs_15_customrules.conf.

Log all comments, not just ones with high anomaly scores

If there's a specific type of request that you always want to log regardless of whether the request matched any rules or not, you can force ModSecurity to log it. In my case, I wanted a record of all comments posted on the site:

# log all comments to auditlog
SecRule REQUEST_METHOD "@streq post" \
  "chain, \
  id:'000081', \
  phase:1, \
  t:none, \
  t:lowercase, \
  t:normalisePath, \
  msg:'All comments logged to auditlog'"
    SecRule REQUEST_URI "@rx ^(/comment/reply(/\d+)*|/comment/\d+/edit)$" \
      "setvar:'tx.msg=%{rule.msg}', \
      nolog,auditlog

A couple of things worth noting here: be careful when using transformation functions. At first when I wrote this rule, it started SecRule REQUEST_METHOD "@streq POST", but after transforming the data to lowercase the request method is no longer POST, it's post, so the rule didn't match. Now I have a log of all the legitimate comments on the site as well as the ones that tripped the CRS rules.

Missing HTTP only flag in cookies

You will find that you get lots of misconfiguration messages that fill up your audit log if you haven't configured your site to set the httponly flag in your cookies. This isn't a modsecurity rule, but it's relevant so I decided to include it. You can solve this issue by changing a parameter in /etc/php5/apache2/php.ini:

session.cookie_httponly = 1

And you should no longer see the messages.

Inspect specific requests with the debug log

If your rules aren't working in the way you expected them to, you might want to see detailed information about how modsecurity is processing the relevant requests in the debug log. However, I quickly learned that you don't want to enable debug logging globally because the amount of output produced is huge! Instead, you can turn it on for specific requests (e.g. comments):

# turn on debug logging for comments
SecRule REQUEST_METHOD "@eq POST" id:'000025',phase:1,chain,t:none,nolog
  SecRule REQUEST_URI "@rx ^(/comment/reply(/\d+)*|/comment/\d+/edit)$" \
    log,ctl:debugLogLevel=5

This rule goes in modsecurity_crs_15_customrules.conf.

Remove parameter names from the target list for a specific rule

I wrote this rule for my brother's WordPress site, which is set up to make changes using FTP. The argument names for the login data was triggering the special character limit, so I made an exception for them:

SecRule REQUEST_URI "@beginsWith /authorize.php" \
  "id:'000028', \
  phase:2, \
  t:none, \
  ctl:ruleRemoveTargetById=981173;ARGS_NAMES:connection_settings[ftp][username], \
  ctl:ruleRemoveTargetById=981173;ARGS_NAMES:connection_settings[ftp][password], \
  ctl:ruleRemoveTargetById=981173;ARGS_NAMES:connection_settings[ftp][advanced][hostname], \
  ctl:ruleRemoveTargetById=981173;ARGS_NAMES:connection_settings[ftp][advanced][port], \
  nolog, \
  pass"

Again, this rule goes in modsecurity_crs_15_customrules.conf. I could also have used a regular expression to match all of the argument names.

Remove a group of rules for a specific field (e.g. free-form fields like comment boxes)

The place you're likely to get most false positives is free-form text fields, for example comment bodies. This rule makes use of the fact that the CRS tags rules by type (e.g. SQL injection), which allows us to update the target list for all SQLi and XSS rules so that it excludes the comment_body field:

SecRule REQUEST_URI "@rx ^(/comment/reply(/\d+)*|/comment/\d+/edit)$" \
  "id:'000029', \
  phase:2, \
  t:none, \
  ctl:ruleRemoveTargetByTag=OWASP_CRS/WEB_ATTACK/SQL_INJECTION;ARGS:comment_body[und][0][value], \
  ctl:ruleRemoveTargetByTag=OWASP_CRS/WEB_ATTACK/XSS;ARGS:comment_body[und][0][value], \
  nolog, \
  pass"

This rule goes in modsecurity_crs_15_customrules.conf.

Put the rule engine in DetectionOnly mode for new apps or a specific path

Initially, your whole ModSecurity installation is likely to be in DetectionOnly mode while you remove false positives. However, you may want to add a new web app to Apache after you have done your initial whitelisting exercise and turned the engine On. In this case it's much better to put the engine in DetectionOnly mode for just the new web app, leaving the engine On for the rest of your site while you do initial testing. You can then comment the rule when you're done.

SecRule REQUEST_URI "@beginsWith /webapp" \
  "id:'000080', \
  phase:1, \
  t:none, \
  ctl:ruleEngine=DetectionOnly, \
  nolog, \
  pass"

Likewise, you may want to turn the rule engine off for the admin backend for your site:

SecRule REQUEST_URI "@beginsWith /admin" \
  "id:'000003', \
  phase:1, \
  t:none, \
  ctl:RuleEngine=Off, \
  nolog, \
  pass"

These rules go in modsecurity_crs_15_customrules.conf.

Allow multiple URL encoding in comments

One of the protocol violation rules for the CRS was being triggered for URLs submitted in the body of node edits for Drupal, especially when the text was different to the URL like this:

<a href="https://samhobbs.co.uk">my website</a>

...which is something I do quite a lot, so I removed it for the specific field:

SecRule REQUEST_URI "@beginsWith /node" \
  "id:'000006', \
  phase:2, \
  t:none, \
  ctl:ruleRemoveTargetById=950109;ARGS:body, \
  nolog, \
  pass"

I put that rule in modsecurity_crs_60_customrules.conf.

Increase the anomaly score threshold for blocking specific requests

The default inbound_anomaly_score_level (the value compared to the transaction score to decide whether to block a request) is set in modsecurity_crs_10_setup.conf with a value of 5. I wanted to raise the threshold for comments to reduce the chances of false positives blocking people from commenting:

SecRule REQUEST_URI "@rx ^(/comment/reply(/\d+)*|/comment/\d+/edit)$" \
  "id:'000014', \
  phase:1, \
  t:none, \
  nolog, \
  pass, \
  setvar:'tx.inbound_anomaly_score_level=10'"

This rule goes in modsecurity_crs_60_customrules.conf.

Remove rules matching common shell commands

The file modsecurity_crs_40_generic_attacks.conf contains rules that look for common shell commands embedded in requests. For most blogs these rules are useful, but since I'm expecting users to post commands in comments on this site, I removed the comment body from the target list. For example, rule 950907 matches wget, curl and cc:

SecRuleUpdateTargetById 950907 !ARGS:/^comment_body/

The regex matches any argument beginning with comment_body, e.g. comment_body[und][0][value]. This rule goes in modsecurity_crs_15_customrules.conf.

Allow multiple linefeeds in comments

Rule 960024 looks for four (4) non-word characters in a row. Two linefeeds (\r\n) in a row looks like (\x0d\x0a\x0d\x0a) and triggers the rule. For this reason you may want to remove it from comment fields:

SecRuleUpdateTargetById 960024 !ARGS:/^comment_body/

Argument names tripping special character limits

Rule 981173 checks for too many special characters and on Drupal sites may be triggered by the names of comment fields, e.g. ARGS_NAMES:comment_body[und][0][value] and ARGS_NAMES:comment_body[und][0][format]. This rule will remove the names of arguments starting with comment_body from that specific rule:

SecRuleUpdateTargetById 981173 !ARGS_NAMES:/^comment_body/

This rule goes in modsecurity_crs_60_customrules.conf.

Notes, References and Tools

If you want to review a large volume of data at once, you might find my commandline utility for reading a modsecurity audit log file into a sqlite database useful. I started out by running ModSecurity in DetectionOnly mode for a few months, which produced a huge amount of data in the audit logs. Reading it all into a database made it vastly easier to sort and get a feel for which false positives to prioritise. The ModSecurity reference manual is hosted on the project's github page. If you don't have the ModSecurity Handbook, I would highly recommend it. It is available directly from the publisher, in paper and ebook formats often cheaper than Amazon. Quite a cool company really, if you buy a paper copy you get the ebook thrown in (including subsequent revisions if the book is updated). In the variables list, &VARIABLE means the number of that variable, e.g. &REQUEST_HEADERS is the number of request headers. When writing rules for ModSecurity that are intended to increase an anomaly score, be aware that rule blocking is part of a complicated chain. One of the tests requires there to be a transaction variable that starts with a rule ID number, so you need to add something similar to this snippet to your rules if you want them to match:

setvar:tx.%{rule.id}-CUSTOM_RULE/COMMENT_SPAM

Hopefully you will find these rules useful. If you have any questions or spot a mistake, please leave a comment. Hopefully you won't be blocked :p