Spam Filtering

Every email that is addressed to a cs.stanford.edu email address passes through our spam filtering software, SpamAssassin. This software tests each email message and, based on several hundred heuristics, assigned the message a score. Based on the score, SpamAssassin decides if an email is probably spam or not. If SpamAssassin thinks the email is probably spam, the message is modified in two ways to allow easy filtering:

  • The subject line is prepended with "*****SPAM*****"
  • A new header, "X-Spam-Flag: YES" is added.

Email that scores more than 30 points is considered far too spam-like to be delivered and is rejected. For scores lower than 30, the email message will still be delivered normally with the tags above.

Using the modifications that SpamAssassin makes, it is relatively simple to filter out possible spam should you wish. Here are some pointers to help you filter out the messages marked as spam, depending on how you read your mail.

Procmail for UNIX clients

(for users of UNIX-based email clients, such as pine, elm, mutt, mh, xhm, exmh, etc., for example on xenon)

These instructions will work with most mailers that store multiple email messages in a single file, known as mbox format. You may wish to check your settings with regard to MAILDIR below and customize it to fit your needs.

To start with, create a .procmailrc file using the following as a template:

 SHELL=/bin/sh                   # Use the Bourne shell (check your path!)
 MAILDIR=${HOME}/Mail            # First check what your mail directory is!
 LOGFILE=${MAILDIR}/procmail.log
 LOG="--- Logging ${LOGFILE} for ${LOGNAME}, "

 :0:
 * ^X-Spam-Flag: YES
 spam

Then, create a .forward file with the following line:

 "| /usr/local/bin/procmail -f-"

This will save all of the marked spam you get into ~/Mail/spam. For more procmail information, see www.procmail.org.

You can also use procmail to filter based on the X-Spam-Level header. For example, the rule

 :0:
 * ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
 /dev/null

sends all messages with a score of 15 or higher straight to the bit bucket.

To forward any remaining mail after you've dealt with spam, just add a forwarding rule (using "!") at the end of your .procmailrc file, after your spam rules have filtered out the spam (this way the spam you've filtered out will not be forwarded):

 :0:
 ! [email protected]

Be sure to replace [email protected] with the email address you want your email to be forwarded to!

Note for AFS systems (xenon, ai, robotics, theory):

The user 'mail' must have read permission on your home directory (this is on by default) and write permissions for any directories where procmail will write to mail boxes or log files. For the above example, since procmail will need to write to files in ~/Mail, run the following command:

fs sa ~/Mail mail write

Mozilla / Netscape

These instructions should work on Netscape 6.0 and higher, and all versions of Mozilla

  1. From the Tools menu, select Message Filters.
  2. Click on the New button, which should bring up the Filter Rules window:
  3. Give the filter some name in the Filter name box.
  4. Under "For incoming messages that", check "Match any of the following".
  5. From the first drop-down box select Customize... to bing up the Customize Headers window.
  6. Type X-Spam-Flag into the text box and click Add then OK.
  7. Now you should be able to select X-Spam-Flag from the first drop-down box in the Filter Rules window. Change the contains to read is, and type YES in the text box, as shown here:
  8. Under "Perform these actions", select what you want done with the matching message. We suggest moving it to a separate folder, at least until you are comfortable that no legitimate email is being tagged as spam.
  9. Click on OK. The new rule should be in place, as shown here:

Outlook

This is specifically for Outlook 2000. Newer versions, such as Outlook XP, should be similar.

  1. From the Tools menu, select Rules Wizard.
  2. Select New, then Start from a blank rule, and click on Next.
  3. Select Check messages after they arrive and click on Next.
  4. Under Which condition(s) do you want to check?, select With specific words in the message header.
  5. Under Rule Description, click on specific words.
  6. Enter this text exactly, without the quotes: "X-Spam-Flag: YES". Click Add, then OK, then Next.
  7. Now choose the what you want done with the messages marked as spam. Currently, we suggest you save them in a separate folder rather than deleting them. To do this, choose Move to the specified folder. Then select an existing folder, or click New to create a new one.
  8. After clicking Next, you can choose to make exceptions. At this time, we would suggest you make no exceptions, and forward us emails with all of the headers that should never be marked as spam.
  9. After clicking Next one last time, give the rule a name and click Finish.

Remember to check your spam folder periodically for false positives and to clean out the true spam.

CS WebMail (SquirrelMail), for Faculty and Staff only

These instructions are for CS WebMail for faculty and staff only (http://cs.stanford.edu/webmail). If you're using xenon WebMail for students (http://xenon.stanford.edu/webmail), see the instructions for procmail.

  1. Select Filters from the menu at the top of the WebMail page.
  2. Click on the Add a New Rule button at the bottom of the list of filters.
  3. Choose Header Match and click on Move on to step 2.
  4. From the condition drop-down box, choose AND (Every item must match.
  5. In the first header match line, choose the header X-Spam-Flag from the dropdown box, and is from the next dropdown box.
  6. In the first text box, type YES. Click on Move on to step 3.
  7. Choose the action to take on matching messages. We recommend you choose Move message into and create a new folder named spam or junk.
  8. Check the box next to the stop sign, to end the filter processing, and then click on Move on to step 4.
  9. Verify that the rule states what you want it to. It should say something like this:

    If the header "X-Spam-Flag" is "YES", then file it into the folder INBOX.spam. Then STOP processing rules.
    If you're satisfied with the rule, click Finished.

  10. When you're done adding rules, click on Save Changes at the bottom of the Filters page.

Reporting Mistakes

Email is by nature so varied that the anti-spam system will occasionally make mistakes. You can help reduce the number of times this happens by forwarding the mistakes to CSD-CF. The examples you send will be used to fine tune the anti-spam system so it doesn't repeat its error. Here's how:

  • False positives: The system will occasionally count something as spam that is not spam. Forward a copy of this email, with full headers, to [email protected]. Read Email Headers to learn how to forward with full headers.
  • False negatives: Some spam will occasionally slip through the system unmarked. Forward a copy of this email, with full headers, to [email protected]. Read Email Headers to learn how to forward with full headers.

Adjusting your Filter Settings

You are able to control some of the rules that SpamAssassin uses to decide whether a message is spam or not. Login to https://cs.stanford.edu/spam to adjust your settings. Detailed instructions are provided on that page.  Changing your spam settings on that page is current broken and the page is disabled.  If you need to change how your mail is checked for spam, create a CS support ticket at https://support.cs.stanford.edu.