Content Filtering with SpamAssassin


General information on spam filtering

SpamAssassin augments the headers of incoming email by adding several additional lines. It does not change the way that email is delivered in any way, but the addition of these new headers gives users the opportunity to filter their incoming mail according to their own standards. These additional headers provide a score for each message, which estimate of the likelihood that this particular piece of email is spam. Below is an example of what the augmented headers look like:

        X-Spam-Flag: YES
X-Spam-Status: Yes, hits=22.6 required=5.9
X-Spam-Level: **********************

SpamAssassin uses several heuristics to determine if a piece of mail is spam. Following is the report associated with the sample email from which the above header lines were taken:

  X-Spam-Report: Detailed Report
SPAM: -------------------- Start SpamAssassin results ----------------------
SPAM: This mail is probably spam. The original message has been altered
SPAM: so you can recognise or block similar unwanted mail in future.
SPAM: See http://spamassassin.org/tag/ for more details.
SPAM:
SPAM: Content analysis details: (22.60 hits, 5.9 required)
SPAM: INVALID_DATE (1.5 points) Invalid Date: header (not RFC 2822)
SPAM: UNDISC_RECIPS (1.5 points) Valid-looking To "undisclosed-recipients"
SPAM: NO_REAL_NAME (1.3 points) From: does not include a real name
SPAM: SMTPD_IN_RCVD (1.2 points) Received via SMTPD32 server (SMTPD32-n.n)
SPAM: MSGID_HAS_NO_AT (0.3 points) Message-Id has no @ sign
SPAM: FROM_HAS_MIXED_NUMS (0.3 points) From: contains numbers mixed in with letters
SPAM: ALL_CAPS_HEADER (0.2 points) Header with all capitals found
SPAM: INVALID_MSGID (0.0 points) Message-Id is not valid, according to RFC 2822
SPAM: DRASTIC_REDUCED (1.9 points) BODY: Drastically Reduced
SPAM: ONCE_IN_LIFETIME (1.8 points) BODY: Once in a lifetime, apparently
SPAM: REMOVE_SUBJ (0.8 points) BODY: List removal information
SPAM: HOME_EMPLOYMENT (0.6 points) BODY: Information on how to work at home (2)
SPAM: CALL_FREE (0.2 points) BODY: Contains a tollfree number
SPAM: SPAM_PHRASE_21_34 (1.9 points) BODY: Spam phrases score is 21 to 34 (high)
SPAM: [score: 22]
SPAM: LINES_OF_YELLING (0.2 points) BODY: A WHOLE LINE OF YELLING DETECTED
SPAM: RAZOR2_CHECK (3.9 points) Listed in Razor2, see http://razor.sf.net/
SPAM: RAZOR_CHECK (2.6 points) Listed in Razor1, see http://razor.sf.net/
SPAM: DATE_IN_PAST_24_48 (1.0 points) Date: is 24 to 48 hours before Received: date
SPAM: RCVD_IN_OSIRUSOFT_COM (0.4 points) RBL: Received via a relay in relays.osirusoft.com
SPAM: [RBL check: found 142.249.10.63.relays.osirusoft.com., type: 127.0.0.3]
SPAM: X_OSIRU_DUL (0.6 points) RBL: DNSBL: sender ip address in in a dialup block
SPAM: X_OSIRU_DUL_FH (0.4 points) RBL: Received from first hop dialup listed in relays.osirusoft.com
SPAM: [RBL check: found 142.249.10.63.relays.osirusoft.com., type: 127.0.0.3]
SPAM:
SPAM: -------------------- End of SpamAssassin results ---------------------

How to filter spam

You can use your mail client to filter, based on the modified headers. Many modern mail clients, such as Netscape, Mozilla, Pine, Eudora, Outlook, etc... will support this functionality. The header to use for these mail clients is: 
X-Spam-Flag: YES
For more information on this see the section on setting up mail clients to filter email.

Setting up mail clients to filter

Netscape 4

  1. Access the Edit -> Message Filters -> New menu and choose a name for the new filter.

  2. Access Filter Criteria -> Customize Headers -> New (this is usually auto-selected to Subject), enter X-Spam-Flag and click Ok.

  3. Return to the Filter Rules window, select the following Filter Criteria:

    	X-Spam-Flag contains YES
  4. Under Filter Action, select a destination folder where you would like to move the likely spam.

Netscape 6/Mozilla

  1. In the Mail window, select: Tools -> Message Filters -> New

  2. In the new filter pane, name the filter.

  3. Under the menu Filter Criteria -> Customize -> New Message Header enter X-Spam-Flag click Add, then OK.

  4. Select as the Filter Criteria: X-Spam-Flag contains YES

  5. Under Perform this action, select a destination folder where you would like move the likely spam.

Pine

As of Pine 4.44 (and possibly some earlier versions), you can automatically filter spam emails from your INBOX. You will need to add a filter rule to look for the X-Spam-Flag header; with this rule you can delete, mark or move the spam to a separate folder.

  1. Start by navigating to the Add Filter screen:

            (M)ain menu -> (S)etup -> (R)ules -> (F)ilters -> (A)dd
  2. By default, the rule will look in your INBOX. Leave this as the default.

  3. Under FILTERED MESSAGE CONDITIONS, navigate to the e(X)traHdr command to add a new header filter.

  4. Enter X-Spam-Flag, then use the (C)hange command to set the value to YES.

  5. In the ACTIONS section, you can set the Filter Action to delete, mark or move the message to a different folder. Enter your choice and save the changes.

Now when you start pine, your INBOX should have all messages marked as spam automatically filtered.

Eudora

  1. Go to the Tools menu and select Filters to open the Filters window.
  2. To add a new filter, click NEW.
  3. Select the option to Match Incoming messages.
  4. In the Header: field type in X-Spam-Flag.
  5. The next drop-down field should have the word 'contains'.
  6. In the field to the right of the word 'contains' type in YES.
  7. Under Action, move the mouse over the arrow next to None and click. (Not all actions are available in the free version of Eudora).
  8. When you do this a field with the word In will appear.
  9. Click on In and select New...
  10. Enter a name for the new mailbox to which your messages will be filtered.

For more information on setting up Eudora see the following web page located at Qualcomm:

http://www.eudora.com/techsupport/tutorials/win_filters.html

Outlook 2002

  1. Select Rules Wizard from the Tools menu.
  2. Select your Inbox folder for the "Apply changes to this folder" field.
  3. Click the New button.
  4. Select "Start from a blank rule".
  5. Select "Check messages when they arrive".
  6. Click "Next."
  7. Checkmark the Condition "with specific words in the subject".
  8. Click "specific words" in the Rule description field to edit it. Type [SPAM] in the "Specify words or phrases..." field.
  9. Click "Add" and then click "OK".
  10. Click "Next."
  11. Checkmark "move it to the specified folder" from the "What do you want to do..." list.
  12. Click "Specified" in the Rule description field to edit it.
  13. Click "New" to create a new folder.
  14. Type 'Spam' in the Name field
  15. Click in the "Select where to place the folder" field and click INBOX then "OK."
  16. Click "Finish" to create the rule.
  17. Click "OK" to exit.

Outlook Express 6

  1. Click on Tools -> Message Rules -> Mail
  2. Select "Where the subject line contains specific words" in section 1.
  3. Select "Move it to the specified folder"
  4. Click on "contains specific words" in the bottom box
  5. Enter [SPAM] into the box and click on add.
  6. Click on specified in the bottom box.
  7. Click on New Folder, enter a name for the folder (ie. spam) and click OK and then OK again.
  8. Click on OK once more and the rule will be complete.

NOTE: Outlook Express 6 can only filter local folders used by a POP3 protocol connection. If you are using an IMAP protocol connection, you must use another program to filter mail identified as spam.