FutureQuest, Inc. FutureQuest, Inc. FutureQuest, Inc.
Knowledgebase: Spam/Email Filters
Custom Email Filter Recipes
Posted on 15 November 2003 10:43 PM

This tutorial is a CookBook of Copy-n-Paste Filter Recipes to help FutureQuest® Site Owners set up some of the more commonly requested one-liner shell script filters. Some of the included examples show how to delete, forward, or redirect an incoming email, based on a variety of test conditions.

Note: Although the copy-n-paste approach here is designed to make it somewhat simpler, setting up custom filters of this type is considered an advanced feature. For more information on the built-in filtering services offered by FutureQuest®, visit the "Email Filters - An Overview" tutorial. The built-in filters are designed to be simple to use in that they require no script at all. For those who require even more advanced filtering options visit the "Custom Filters: Advanced Email Scripting" tutorial.

The filter recipes provided on this page are actually custom filters that are simply very short, one-line shell scripts. A list of common commands used by many of these filter recipes is located below in the glossary of common programs, complete with explanations. Each entry in the Recipe List contains a summary with a brief explanation, followed by the filter recipe.

Important Information: You should thoroughly test all filters on a test mailbox before implementing, as FutureQuest® cannot recover email lost due to incorrectly configured filters. When using these recipes, pay special attention to the exact punctuation characters used and the spacing. Even minor changes, such as changing a double quote to a single quote, or inserting an extra space, can cause the filter to not work as desired. Advanced email filtering of this type is not directly supported by FutureQuest®. For additional assistance however, you may visit the FutureQuest Community Forums.

Email Filter Usage Is OPTIONAL and usage is AT YOUR OWN RISK.

To use one of these filters, go to the Email Manager in your CNC and click on the name of the email account or email alias for which you wish to implement the filter. This takes you to the Mailbox Properties for that account or alias. Now click on the "Edit" link next to "Custom Filters". Then scroll down the Filters page, select the "Simple Filter" type from the drop-down menu and paste the code for the recipe into the adjacent box on the same row, after first editing the filter to match your particular situation.

Tip: For information about the order in which filters run, visit the "Order in Which Email Filters Run" tutorial.

THE RECIPE LIST
FORWARDING EMAIL COPIES
Forwarding Email to One Address - forwards a copy of the incoming email to the address emailaddy@example.com. Standard delivery is not affected, so the email will still be delivered to the original recipient as well.

Note: The "forward" command is handled specially on the POP toasters, and can only be followed by a single email address or a list of email addresses (no other commands may follow).


forward emailaddy@example.com
Forwarding Email to a List of Addresses - forwards the incoming email to each of the addresses in the list (in this case, 2 addresses). Standard delivery is not affected, so the email will still be delivered to the original recipient as well.

Note: The "forward" command is handled specially on the POP toasters, and can only be followed by a single email address or a list of email addresses (no other commands may follow).


forward emailaddy@example.com email2@example.net
Conditional Forwarding - This is an example of forwarding only if the email is sent from a particular address. In any case, the email is still delivered to the original mailbox. If the "From" address matches match@example.com, a copy of the email is also forwarded to forward@example.com.

if 822field from | grep -iq "match@example.com"; then forward forward@example.com; fi
REDIRECTING EMAIL
Redirecting an Email: General Format - redirects incoming email to forward@example.com if the test program is "true". Otherwise, delivery of the email continues as normal. Examples of various test programs are located below.

condredirect forward@example.com testprogram
Redirecting an Email Based on the To or Cc Address - redirects incoming email to forward@example.com if either the To or Cc fields in the message headers contain the address match@example.com

condredirect forward@example.com iftocc match@example.com
Redirecting an Email From a Particular Address - redirects incoming email to forward@example.com if the From field in the message headers contains the address match@example.com

Note: The following filter should be on a single line.


condredirect forward@example.com sh -c '822field from | grep -iq match@example.com'
Redirecting an Email Based on Subject - searches the Subject field in the message headers for a particular word or phrase from a list in a textfile. If there is a match, the email is redirected to forward@example.com. Otherwise, the message is delivered normally.

Sample file is saved as /big/dom/xdomain/badsubj.txt with the following contents:
dvd
cell phone 
viagra 
low rates 
increase your hits
mortgage
then the filter would be (on a single line):

condredirect forward@example.com sh -c '822field subject | grep -qiFf "/big/dom/xdomain/badsubj.txt"'

Note:
The -f switch let's you read from file.
The -F takes each phrase, one on a line, as a literal string to match, not a regex.
The -i switch makes the match case insensitive.
DELETING EMAIL
Deleting an Email: General Format - Runs a test program and if this evaluates to "true", then deletes the email. Sample test programs are given in the examples that follow. This is simply a Bash shell script using an "if" test.

CAUTION: Deleted email cannot be recovered. If you cannot risk losing email, then we suggest using the Redirect options described above to redirect email to a local POP mailbox and then review it manually for false positives.


if test program; then exit 99; fi
Deleting Email From a Particular Sender - checks the From header of an incoming email for the email address sender@example.com. If it matches, the email is deleted and not delivered to the mailbox.

if 822field from | grep -iq "sender@example.com"; then exit 99; fi
Deleting Email From a Sender in a List - checks the From header of an incoming email to see if the email address matches any addresses or domains in a textfile. If a match is found, delete the email and do not deliver it to the mailbox.

Sample file is saved as /big/dom/xdomain/badfrom.txt with the following contents:
salesperson@example.net 
example.org
trouble@example.com 

@mail.baddomain.tld
then the following filter would be (all on one line):

if 822field from | grep -qiFf "/big/dom/xdomain/badfrom.txt"; then exit 99; fi

Note:
The -f says to read the patterns from the file.
The -F says to treat the patterns as text, not as regular expressions.

 


Deleting Email Sent Bcc - checks whether any email addresses of the email header fields such as To: or Cc: match one in a list of addresses. If not, deletes the email and doesn't deliver it to the email address.

if ! iftocc me@domain.tld me@example.com me@hotmail.com; then exit 99; fi
TEST PROGRAMS
Test Message Header Field for Content - checks whether the message header called fieldname contains the word matchword. If there is a match, returns "true" as an exit code of zero.

sh -c '822field fieldname | grep -qi matchword'
Test Message Header Field for Content from a List - checks whether the message header called fieldname contains any word or phrase from the file matchfile.txt where each word or phrase appears on a separate line in the file. If there is a match, returns "true" as an exit code of zero.

Sample file is saved as /big/dom/xdomain/matchfile.txt with the following contents:
dvd
cell phone 
viagra 
low rates 
increase your hits
mortgage


Note: The following filter should be on a single line.

sh -c '822field fieldname | grep -qiFf "/big/dom/xdomain/matchfile.txt"'
Test the Size of a Message - checks whether a message exceeds a certain size in bytes. If so, returns "true" as an exit code of zero, otherwise returns a non-zero exit code (i.e. "false"). In the below example, tests for a message greater than 1,000,000 bytes. This filter is also available as a Built-in filter in the CNC.
usage: /usr/local/fqdeliver/tests/msg-size bytes


Example usage:
/usr/local/fqdeliver/tests/msg-size 1000000
Test for Particular File Attachments - checks for file attachments that end in a particular extension. If the incoming email has an attached file with one of the listed extensions, this test returns "true" as an exit code of zero, otherwise it returns a non-zero exit code (i.e. "false"). File extensions are passed to the command as arguments. This filter is also available as a Built-in filter in the CNC.

Example usage:
/usr/local/fqdeliver/tests/no-ext mpg gif
Additional Tests in /usr/local/fqdeliver/tests - a number of tests are available in this directory. Here is a list with brief descriptions. They can be used similarly to the test for Message Size and the test for Particular File Attachments, as shown above.
  • no-attachment - exits true if the message has an attachment.

  • no-exe - exits true if the message has an attachment with an executable filename extension. This is also available as a Built-in filter in the CNC. For more information, see the Built-in filter section of the "Email Filters - An Overview" tutorial.

  • received-ip - exits true if the message arrived from one of the listed IPs. IPs are listed as arguments of the command. Also available as a Built-in filter in the CNC.

  • recipient - exits true if the envelope recipient matches a specified email address, passed to the command as an argument. The envelope recipient address is the email address in a Delivered-To header, not in the To or CC header. Also available as a Built-in filter in the CNC.

  • sender - exits true if the SMTP envelope sender matches one of the listed email addresses, passed to the command as an argument. Also available as a Built-in filter in the CNC.

Glossary of Commands Used in Filters

forward
forwards email to a list of specified email addresses
Note: The "forward" command is handled specially on the POP toasters, and can only be followed by a single email address or a list of email addresses (no other commands may follow).

bouncesaying
bounces email back to the sender with an optional error message, depending on the exit code of a test program (NOTE: Suspected spam email should never be "bounced". The sender address is usually forged and bounces go to an innocent party who may submit complaints against the FutureQuest network and your domain and may block email from the FutureQuest network.)

condredirect
forwards email to a new address, depending on the exit code of a test program

iftocc
checks whether the SMTP envelope recipient matches an address in either the To or Cc fields

except
reverses the Qmail exit code

grep
checks for pattern matches

sh
invokes the bash shell

test
a bash command for comparing two strings

822field
returns the contents of a message header field

More details on each of the above commands can be found by viewing the "man pages" in an SSH session. For example, to view the man page for iftocc, type the following at the command prompt:

[username@FQuest-Server]$ man iftocc

Additional related commands:

Here are some additional commands which could be useful. They are not demonstrated in this tutorial, however you may find out more about them by consulting their man pages accordingly.
  • 822date
  • 822header
  • 822received
  • 822print