[Mimedefang] Spamassassin and MimeDefang custom rules

Paul Murphy Paul.Murphy at argentadiscovery.com
Wed Oct 15 05:30:04 EDT 2008


Jon,

>/etc/mail/sa-mimedefang.cf
>In this config file I have the following custom test
>header BAD_TEST             Subject =~ /xyzzy/i
>score BAD_TEST              6.00
>describe BAD_TEST           Local Rule

This file is NOT parsed for rulesets - only for the main SA configuration options.

>Is there some way to capture a log of SA activity so I can perhaps see
>where I am going wrong?

Change your MD configuration to leave the /var/spool/MIMEDefang/mdefang-<msgid> directory in place (start with -d option), restart MD, send a test message, then take out the -d, find the directory (grep the mal log to get the message ID), become your filter user, and then in that directory, run:

spamassassin -p /etc/mail/sa-mimedefang.cf -t -d < INPUTMSG

This will run SA with your settings, and in debug mode - it won't produce exactly the same scores as MD due to several issues, but it will show whether your local rules are being considered, and whether they are being matched.

The output will be something like this: (I've added line numbers to assist in the commentary)

     1  [27281] dbg: logger: adding facilities: all
     2  [27281] dbg: logger: logging level is DBG
     3  [27281] dbg: generic: SpamAssassin version 3.1.7
     4  [27281] dbg: config: score set 0 chosen.
     5  [27281] dbg: util: running in taint mode? yes
     6  [27281] dbg: util: taint mode: deleting unsafe environment variables, resetting PATH
     7  [27281] dbg: util: PATH included '/opt/kde3/bin', keeping
     8  [27281] dbg: util: PATH included '/opt/gnome/bin', keeping
     9  [27281] dbg: util: PATH included '/usr/games', keeping
    10  [27281] dbg: util: PATH included '/home/defang/bin', keeping
    11  [27281] dbg: util: PATH included '/usr/bin/X11', keeping
    12  [27281] dbg: util: PATH included '/bin', keeping
    13  [27281] dbg: util: PATH included '/usr/bin', keeping
    14  [27281] dbg: util: PATH included '/usr/local/bin', keeping
    15  [27281] dbg: util: PATH included '/packages/bin', which doesn't exist, dropping
    16  [27281] dbg: util: PATH included '/usr/sbin', keeping
    17  [27281] dbg: util: PATH included '/sbin', keeping
    18  [27281] dbg: util: PATH included '.', which is not absolute, dropping
    19  [27281] dbg: util: final PATH set to: /opt/kde3/bin:/opt/gnome/bin:/usr/games:/home/defang/bin:/usr/bin/X11:/bin:/usr/bin:/usr/local/bin:/usr/sbin:/sbin
    20  [27281] dbg: message: ---- MIME PARSER START ----
    21  [27281] dbg: message: main message type: text/plain
    22  [27281] dbg: message: parsing normal part
    23  [27281] dbg: message: added part, type: text/plain
    24  [27281] dbg: message: ---- MIME PARSER END ----
    25  [27281] dbg: dns: is Net::DNS::Resolver available? yes
    26  [27281] dbg: dns: Net::DNS version: 0.46
    27  [27281] dbg: config: using "/etc/mail/spamassassin" for site rules pre files
    28  [27281] dbg: config: read file /etc/mail/spamassassin/init.pre
    29  [27281] dbg: config: read file /etc/mail/spamassassin/v310.pre
    30  [27281] dbg: config: read file /etc/mail/spamassassin/v312.pre
    31  [27281] dbg: config: using "/var/lib/spamassassin/3.001007" for sys rules pre files
    32  [27281] dbg: config: read file /var/lib/spamassassin/3.001007/updates_spamassassin_org.pre
    33  [27281] dbg: config: using "/var/lib/spamassassin/3.001007" for default rules dir
    34  [27281] dbg: config: read file /var/lib/spamassassin/3.001007/updates_spamassassin_org.cf
    35  [27281] dbg: config: using "/etc/mail/spamassassin" for site rules dir
    36  [27281] dbg: config: read file /etc/mail/spamassassin/10_report_template.cf
    37  [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_adult.cf
    38  [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_bayes_poison_nxm.cf
    39  [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_evilnum0.cf
    40  [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_evilnum1.cf
    41  [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_evilnum2.cf
    42  [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_genlsubj.cf
    43  [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_header.cf
    44  [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_html.cf
    45  [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_obfu.cf
    46  [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_oem.cf
    47  [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_random.cf
    48  [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_spoof.cf
    49  [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_stocks.cf
    50  [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_unsub.cf
    51  [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_uri0.cf
    52  [27281] dbg: config: read file /etc/mail/spamassassin/70_sare_whitelist.cf
    53  [27281] dbg: config: read file /etc/mail/spamassassin/72_sare_bml_post25x.cf
    54  [27281] dbg: config: read file /etc/mail/spamassassin/72_sare_redirect_post3.0.0.cf
    55  [27281] dbg: config: read file /etc/mail/spamassassin/99_local.cf
    56  [27281] dbg: config: read file /etc/mail/spamassassin/99_sare_fraud_post25x.cf
    57  [27281] dbg: config: read file /etc/mail/spamassassin/FuzzyOcr.cf
    58  [27281] dbg: config: read file /etc/mail/spamassassin/ImageInfo.cf
    59  [27281] dbg: config: read file /etc/mail/spamassassin/bogus-virus-warnings.cf
    60  [27281] dbg: config: read file /etc/mail/spamassassin/local.cf
    61  [27281] dbg: config: read file /etc/mail/spamassassin/sa-mimedefang.cf
    62  [27281] dbg: config: read file /etc/mail/spamassassin/tripwire.cf
    63  [27281] dbg: config: using "/home/defang/.spamassassin" for user state dir
    64  [27281] dbg: config: using "/etc/mail/sa-mimedefang.cf" for user prefs file
    65  [27281] dbg: config: read file /etc/mail/sa-mimedefang.cf
...
Update files processed:
    96  [27281] dbg: config: using "/var/lib/spamassassin/3.001007/updates_spamassassin_org/empty.pre" for included file
    97  [27281] dbg: plugin: fixed relative path: /var/lib/spamassassin/3.001007/updates_spamassassin_org/25_uribl.cf
    98  [27281] dbg: config: using "/var/lib/spamassassin/3.001007/updates_spamassassin_org/25_uribl.cf" for included file
    99  [27281] dbg: config: read file /var/lib/spamassassin/3.001007/updates_spamassassin_org/25_uribl.cf
   100  [27281] dbg: plugin: fixed relative path: /var/lib/spamassassin/3.001007/updates_spamassassin_org/20_anti_ratware.cf
   101  [27281] dbg: config: using "/var/lib/spamassassin/3.001007/updates_spamassassin_org/20_anti_ratware.cf" for included file
   102  [27281] dbg: config: read file /var/lib/spamassassin/3.001007/updates_spamassassin_org/20_anti_ratware.cf
   103  [27281] dbg: plugin: fixed relative path: /var/lib/spamassassin/3.001007/updates_spamassassin_org/25_razor2.cf
...
Blacklist checks:
   318  [27281] dbg: uridnsbl: domains to query: untiesticks.com questiontrains.com
   319  [27281] dbg: dns: checking RBL sa-other.bondedsender.org., set bsp-untrusted
   320  [27281] dbg: dns: checking RBL combined.njabl.org., set njabl-lastexternal
   321  [27281] dbg: dns: checking RBL combined.njabl.org., set njabl
...
Header and body regular expression checks:
   333  [27281] dbg: check: running tests for priority: 0
   334  [27281] dbg: rules: running header regexp tests; score so far=0
   335  [27281] dbg: rules: ran header rule __HAS_MSGID ======> got hit: "<"
   336  [27281] dbg: rules: ran header rule __SANE_MSGID ======> got hit: "<MUjhdgbjdmcacWS at UntieSticks.com>
   337  [27281] dbg: rules: "
   338  [27281] dbg: rules: ran header rule __MSGID_OK_HOST ======> got hit: "@UntieSticks.com>"
   339  [27281] dbg: rules: ran header rule __SUBJECT_ENCODED_QP ======> got hit: "=?ISO-8859-1?Q?"
   340  [27281] dbg: rules: ran header rule __SARE_HEAD_MIME_VALID ======> got hit: "1.0"
   341  [27281] dbg: rules: ran header rule __CT ======> got hit: "m"
   342  [27281] dbg: rules: ran header rule __TOCC_EXISTS ======> got hit: "<"
   343  [27281] dbg: rules: ran header rule __HAS_SUBJECT ======> got hit: "Y"
   344  [27281] dbg: rules: ran header rule __SARE_WHITELIST_FLAG ======> got hit: "R"
   345  [27281] dbg: rules: ran header rule __CTYPE_MULTIPART_ALT ======> got hit: "multipart/alternative"
   346  [27281] dbg: rules: ran header rule __CTYPE_HAS_BOUNDARY ======> got hit: "boundary"
   347  [27281] dbg: rules: ran header rule __MIME_VERSION ======> got hit: "1"
   348  [27281] dbg: rules: ran header rule BAD_ENC_HEADER ======> got hit: "=?ISO-8859-1?Q?You could get up to 750 GBP =2D Instantly with a 1-minute "
 ...
Summarising SA tests:
   485  [27281] dbg: rules: running header regexp tests; score so far=32.795
   486  [27281] dbg: rules: running body-text per-line regexp tests; score so far=32.795
   487  [27281] dbg: uri: running uri tests; score so far=32.795
   488  [27281] dbg: rules: running raw-body-text per-line regexp tests; score so far=32.795
   489  [27281] dbg: rules: running full-text regexp tests; score so far=32.795
   490  [27281] dbg: check: running tests for priority: 900
   491  [27281] dbg: rules: running meta tests; score so far=32.795
   492  [27281] dbg: rules: running header regexp tests; score so far=32.795
   493  [27281] dbg: rules: running body-text per-line regexp tests; score so far=32.795
   494  [27281] dbg: uri: running uri tests; score so far=32.795
   495  [27281] dbg: rules: running raw-body-text per-line regexp tests; score so far=32.795
   496  [27281] dbg: rules: running full-text regexp tests; score so far=32.795
   497  [27281] dbg: check: running tests for priority: 1000
   498  [27281] dbg: rules: running meta tests; score so far=32.795
   499  [27281] dbg: rules: running header regexp tests; score so far=32.795
...
AWL update:
   512  [27281] dbg: plugin: Mail::SpamAssassin::Plugin::AutoLearnThreshold=HASH(0x9514bb0) implements 'autolearn_discriminator'
   513  [27281] dbg: learn: auto-learn: currently using scoreset 3, recomputing score based on scoreset 1
   514  [27281] dbg: learn: auto-learn: message score: 32.795, computed score for autolearn: 25.953
   515  [27281] dbg: learn: auto-learn? ham=1, spam=7, body-points=23.698, head-points=6.255, learned-points=0
   516  [27281] dbg: learn: auto-learn? yes, spam (25.953 > 7)
   517  [27281] dbg: learn: initializing learner
   518  [27281] dbg: learn: learning spam
...
Bayes learn:
   556  [27281] dbg: bayes: database connection established
   557  [27281] dbg: bayes: found bayes db version 3
   558  [27281] dbg: bayes: Using userid: 2
   559  [27281] dbg: bayes: 90e28644e964898dcc6015cc5f5f318aa876d4a1 at sa_generated already learnt correctly, not learning twice
   560  [27281] dbg: learn: initializing learner
...
And finally, a decision:
   561  [27281] dbg: check: is spam? score=32.795 required=5
   562  [27281] dbg: check: tests=BAD_ENC_HEADER,EXCUSE_REMOVE,HTML_MESSAGE,NO_RECEIVED,NO_RELAYS,RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E4_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,SUBJECT_EXCESS_QP,URIBL_BLACK,URIBL_OB_SURBL,URI_NOVOWEL
   563  [27281] dbg: check: subtests=__CT,__CTYPE_HAS_BOUNDARY,__CTYPE_MULTIPART_ALT,__HAS_MSGID,__HAS_SUBJECT,__HTML_LINK_IMAGE,__LOCAL_PP_NONPPURL,__MIME_HTML,__MIME_QP,__MIME_VERSION,__MSGID_OK_HOST,__NONEMPTY_BODY,__SANE_MSGID,__SARE_BLACK_FG_COLOR,__SARE_BODY_BLNK_5_100,__SARE_HAS_BG_COLOR,__SARE_HAS_FG_COLOR,__SARE_HEAD_MIME_VALID,__SARE_HTML_HAS_A,__SARE_HTML_HAS_BR,__SARE_HTML_HAS_FONT,__SARE_HTML_HAS_IMG,__SARE_HTML_HAS_P,__SARE_HTML_HAS_TITLE,__SARE_META_MURTY3,__SARE_URI_ANY,__SARE_WHITELIST_FLAG,__SARE_WHITE_BG_COLOR,__SARE_WHITE_FG_COLOR,__SUBJECT_ENCODED_QP,__TAG_EXISTS_BODY,__TAG_EXISTS_CENTER,__TAG_EXISTS_HEAD,__TAG_EXISTS_HTML,__TAG_EXISTS_META,__TOCC_EXISTS,__TVD_BODY,__TVD_MIME_ATT_TP,__UNUSABLE_MSGID

The key part is on line 64 - sa-mimedefang.cf is read as a user preferences file, so it is not parsed for rulesets.

Add your rules to /etc/spamassassin/local.cf instead.

Best Wishes,

Paul.

_______________________________________________________________________
Argenta Discovery Ltd, 8-9 Spire Green Centre, Harlow, Essex, CM19 5TR
Registered in England No. 3671653
_______________________________________________________________________ 



More information about the MIMEDefang mailing list