Mailing List Archive

On Thu, Aug 05, 2004 at 05:14:15PM +0100,
Damien Dye <damien@masterboss.demon.co.uk> wrote
a message of 35 lines which said:

> but since postfix get the mail from my fetchmail daemon the
> connections are not gona match published SPF records. is there
> another way of performing SPF checks. such as the headers that the
> ISP adds for connections to it for example.

I have the same problem with a site which receives mail by UUCP. I'm
afraid there is no clean solution: SPF is not an end-to-end technique
by a hop-by-hop one. Checking has to be done on the upstream relay
(the ISP, for you).

Otherwise, you may write a procmail rule which parses Received headers
(whose format is not standard) and who calls spfquery or spfqtool but
this would be quite brittle.

Remember to parse, not the most recent Received header (which was
added by fetchmail) but the one before (after, in reading order).

-------
Archives at http://archives.listbox.com/spf-help/current/
Donate! http://spf.pobox.com/donations.html
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-help@v2.listbox.com

>>Remember to parse, not the most recent Received header (which was
>>added by fetchmail) but the one before (after, in reading order).

By doing this you risk parsing a forged header - spammers very often
forge the received headers.

Parsing the received headers is a problem for two reasons:

1. There is hardly a standard for the format received headers, so
parsing is cumbersome and might not always work.
2. Received headers can be forged so are unreliable

For some, parsing these headers is probably the only option however SPF
will probably be much less effective and more vulnerable in this
scenario.

Jeremy.

-----Original Message-----
From: owner-spf-help@v2.listbox.com
[mailto:owner-spf-help@v2.listbox.com] On Behalf Of Stephane Bortzmeyer
Sent: Friday, August 06, 2004 8:53 AM
To: Damien Dye
Cc: spf-help@v2.listbox.com
Subject: [spf-help] Re: integration of SPF into incomming mail server

On Thu, Aug 05, 2004 at 05:14:15PM +0100,
Damien Dye <damien@masterboss.demon.co.uk> wrote
a message of 35 lines which said:

> but since postfix get the mail from my fetchmail daemon the
> connections are not gona match published SPF records. is there
> another way of performing SPF checks. such as the headers that the
> ISP adds for connections to it for example.

I have the same problem with a site which receives mail by UUCP. I'm
afraid there is no clean solution: SPF is not an end-to-end technique
by a hop-by-hop one. Checking has to be done on the upstream relay
(the ISP, for you).

Otherwise, you may write a procmail rule which parses Received headers
(whose format is not standard) and who calls spfquery or spfqtool but
this would be quite brittle.

Remember to parse, not the most recent Received header (which was
added by fetchmail) but the one before (after, in reading order).

-------
Archives at http://archives.listbox.com/spf-help/current/
Donate! http://spf.pobox.com/donations.html
To unsubscribe, change your address, or temporarily deactivate your
subscription,
please go to
http://v2.listbox.com/member/?listname=spf-help@v2.listbox.com



This mail was checked for malicious code and viruses
by GFI MailSecurity. GFI MailSecurity provides email content
checking, exploit detection, threats analysis and anti-virus for
Exchange & SMTP servers. Viruses, Trojans, dangerous
attachments and offensive content are removed automatically.
Key features include: multiple virus engines; email content and
attachment checking; an exploit shield; an HTML threats engine;
a Trojan & Executable Scanner; and more.

In addition to GFI MailSecurity, GFI also produces the
GFI MailEssentials anti-spam software, the GFI FAXmaker
fax server & GFI LANguard network security product ranges.
For more information on our products, please visit
http://www.gfi.com. This disclaimer was sent by
GFI MailEssentials for Exchange/SMTP.

-------
Archives at http://archives.listbox.com/spf-help/current/
Donate! http://spf.pobox.com/donations.html
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-help@v2.listbox.com

On Fri, Aug 06, 2004 at 10:28:52AM +0200,
Jeremy Pullicino <jeremyp@gfi.com> wrote
a message of 78 lines which said:

> >>Remember to parse, not the most recent Received header (which was
> >>added by fetchmail) but the one before (after, in reading order).
>
> By doing this you risk parsing a forged header - spammers very often
> forge the received headers.

I do not see how the second most-recent Received header could be
forged (the most recent is added by fetchmail, the second most recent
by your ISP, potentially forged Received headers are older).


-------
Archives at http://archives.listbox.com/spf-help/current/
Donate! http://spf.pobox.com/donations.html
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-help@v2.listbox.com

On Fri, Aug 06, 2004 at 10:28:52AM +0200, Jeremy Pullicino wrote:
> By doing this you risk parsing a forged header - spammers very often
> forge the received headers.
>
> Parsing the received headers is a problem for two reasons:
>
> 1. There is hardly a standard for the format received headers, so
> parsing is cumbersome and might not always work.
> 2. Received headers can be forged so are unreliable
>
> For some, parsing these headers is probably the only option however SPF
> will probably be much less effective and more vulnerable in this
> scenario.

I have noticed that for the upstream university account i mentioned earlier, parsing the headers was quite simple. I know what to expect in what portion of the message, because I can rely on the behaviour of the universities receiving smtp server. I am not looking for and/or parsing arbitrary received headers, just the ones I know are put there by the universities incoming email server.

For me, 1 & 2 have not proven to be problems.

Koen

--
K.F.J. Martens, Sonologic, http://www.sonologic.nl/
Networking, embedded systems, unix expertise, artificial intelligence.
Public PGP key: http://www.metro.cx/pubkey-gmc.asc
Wondering about the funny attachment your mail program
can't read? Visit http://www.openpgp.org/

-------
Archives at http://archives.listbox.com/spf-help/current/
Donate! http://spf.pobox.com/donations.html
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-help@v2.listbox.com

On Fri, Aug 06, 2004 at 09:53:23AM +0200,
Stephane Bortzmeyer <bortzmeyer@nic.fr> wrote
a message of 26 lines which said:

> Otherwise, you may write a procmail rule which parses Received
> headers (whose format is not standard) and who calls spfquery or
> spfqtool but this would be quite brittle.

I'm testing it nevertheless. The attached script, spf-for-procmail
calls spfquery (in Mail::SPF::Query) to find a SPF status. You can
then use it from procmail.

Here, I add the header:

:0fwh
# If the message is from the outside
* ^Received:.*from mx([0-9]+)\.nic.fr
| formail -A "Received-SPF: on `hostname`: SPF-`/usr/local/bin/spf-for-procmail`"

After that, I can act on it:

:0
* ^Received-SPF:.*SPF-fail
spam

or just let the bayesian filter learn if SPF is really useful.


-------
Archives at http://archives.listbox.com/spf-help/current/
Donate! http://spf.pobox.com/donations.html
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-help@v2.listbox.com

>> I am not looking for and/or parsing arbitrary received headers, just
the >> ones I know are put there by the universities incoming email
server.

What happens when the server configuration changes?

If this happens your SPF will break unless you upgrade your parsing to
reflect the new configuration.

Jeremy.

-----Original Message-----
From: owner-spf-help@v2.listbox.com
[mailto:owner-spf-help@v2.listbox.com] On Behalf Of Koen Martens
Sent: Friday, August 06, 2004 10:18 AM
To: spf-help@v2.listbox.com
Subject: Re: [spf-help] Re: integration of SPF into incomming mail
server

On Fri, Aug 06, 2004 at 10:28:52AM +0200, Jeremy Pullicino wrote:
> By doing this you risk parsing a forged header - spammers very often
> forge the received headers.
>
> Parsing the received headers is a problem for two reasons:
>
> 1. There is hardly a standard for the format received headers, so
> parsing is cumbersome and might not always work.
> 2. Received headers can be forged so are unreliable
>
> For some, parsing these headers is probably the only option however
SPF
> will probably be much less effective and more vulnerable in this
> scenario.

I have noticed that for the upstream university account i mentioned
earlier, parsing the headers was quite simple. I know what to expect in
what portion of the message, because I can rely on the behaviour of the
universities receiving smtp server. I am not looking for and/or parsing
arbitrary received headers, just the ones I know are put there by the
universities incoming email server.

For me, 1 & 2 have not proven to be problems.

Koen

--
K.F.J. Martens, Sonologic, http://www.sonologic.nl/
Networking, embedded systems, unix expertise, artificial intelligence.
Public PGP key: http://www.metro.cx/pubkey-gmc.asc
Wondering about the funny attachment your mail program
can't read? Visit http://www.openpgp.org/

-------
Archives at http://archives.listbox.com/spf-help/current/
Donate! http://spf.pobox.com/donations.html
To unsubscribe, change your address, or temporarily deactivate your
subscription,
please go to
http://v2.listbox.com/member/?listname=spf-help@v2.listbox.com



This mail was checked for malicious code and viruses
by GFI MailSecurity. GFI MailSecurity provides email content
checking, exploit detection, threats analysis and anti-virus for
Exchange & SMTP servers. Viruses, Trojans, dangerous
attachments and offensive content are removed automatically.
Key features include: multiple virus engines; email content and
attachment checking; an exploit shield; an HTML threats engine;
a Trojan & Executable Scanner; and more.

In addition to GFI MailSecurity, GFI also produces the
GFI MailEssentials anti-spam software, the GFI FAXmaker
fax server & GFI LANguard network security product ranges.
For more information on our products, please visit
http://www.gfi.com. This disclaimer was sent by
GFI MailEssentials for Exchange/SMTP.

-------
Archives at http://archives.listbox.com/spf-help/current/
Donate! http://spf.pobox.com/donations.html
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-help@v2.listbox.com

On Fri, Aug 06, 2004 at 03:09:24PM +0200,
Jeremy Pullicino <jeremyp@gfi.com> wrote
a message of 83 lines which said:

> What happens when the server configuration changes?
>
> If this happens your SPF will break unless you upgrade your parsing to
> reflect the new configuration.

That's why I said this solution is brittle but it is better than
nothing.

-------
Archives at http://archives.listbox.com/spf-help/current/
Donate! http://spf.pobox.com/donations.html
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-help@v2.listbox.com

OK al give this a go and see if my ISP is predicatible with the headers
it adds to the message

Stephane Bortzmeyer wrote:
> On Fri, Aug 06, 2004 at 09:53:23AM +0200,
> Stephane Bortzmeyer <bortzmeyer@nic.fr> wrote
> a message of 26 lines which said:
>
>
>>Otherwise, you may write a procmail rule which parses Received
>>headers (whose format is not standard) and who calls spfquery or
>>spfqtool but this would be quite brittle.
>
>
> I'm testing it nevertheless. The attached script, spf-for-procmail
> calls spfquery (in Mail::SPF::Query) to find a SPF status. You can
> then use it from procmail.
>
> Here, I add the header:
>
> :0fwh
> # If the message is from the outside
> * ^Received:.*from mx([0-9]+)\.nic.fr
> | formail -A "Received-SPF: on `hostname`: SPF-`/usr/local/bin/spf-for-procmail`"
>
> After that, I can act on it:
>
> :0
> * ^Received-SPF:.*SPF-fail
> spam
>
> or just let the bayesian filter learn if SPF is really useful.
>
>
> -------
> Archives at http://archives.listbox.com/spf-help/current/
> Donate! http://spf.pobox.com/donations.html
> To unsubscribe, change your address, or temporarily deactivate your subscription,
> please go to http://v2.listbox.com/member/?listname=spf-help@v2.listbox.com
>
>
> ------------------------------------------------------------------------
>
> #!/bin/sh
>
> # Requires: Perl module Mail::SPF::Query
>
> # Most recent trusted Received header. The value is 2 by default (1
> # for fetchmail and one for the ISP) but it can be more if there is a
> # complicated mail setup with several servers.
> TRUSTED_RECEIVED=2
>
> HEADERS_FILE=`mktemp`
> cat - > $HEADERS_FILE
>
> LAST_RECEIVED=`cat $HEADERS_FILE | formail -c -x Received |
> head -$TRUSTED_RECEIVED | tail -1`
>
> # The regexp depends on the SMTP
> # server
> # vvvvvvvvvvvvvvvvvvv
> LAST_SMTP_SENDER=`echo $LAST_RECEIVED | perl -ne 'm/\[([0-9\.]+)\]\) +by/;
> print $1'`
> HELO_HOST=`echo $LAST_RECEIVED | perl -ne 'm/^ *from +([a-zA-Z0-9\.\-]+)/;
> print $1'`
>
> SENDER=`cat $HEADERS_FILE | formail -c -x Return-Path`
>
> rm $HEADERS_FILE
>
> RESULT=`echo "$LAST_SMTP_SENDER $SENDER $HELO_HOST" | spfquery -f -`
> echo $RESULT
>
> -------
> Archives at http://archives.listbox.com/spf-help/current/
> Donate! http://spf.pobox.com/donations.html
> To unsubscribe, change your address, or temporarily deactivate your subscription,
> please go to http://v2.listbox.com/member/?listname=spf-help@v2.listbox.com

-------
Archives at http://archives.listbox.com/spf-help/current/
Donate! http://spf.pobox.com/donations.html
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-help@v2.listbox.com

Note that i'm not doing spf parsing, but sbl-xbl lookups.

Anyway, it's a quick hack I admit. I would rather have the university do the sbl-xbl lookups at the front gate, just as the original poster probably rather has his isp do the spf checks.

Frankly, a setup where you do fetchmail is a pain in the ass if you want to have more control over what gets in to your mailbox. Sometimes you can't get around it. As said, i only use it to receive the official university announcements, if the setup changes, i get more spam again through that channel and I can fix the script.

Koen

On Fri, Aug 06, 2004 at 03:09:24PM +0200, Jeremy Pullicino wrote:
> >> I am not looking for and/or parsing arbitrary received headers, just
> the >> ones I know are put there by the universities incoming email
> server.
>
> What happens when the server configuration changes?
>
> If this happens your SPF will break unless you upgrade your parsing to
> reflect the new configuration.
>
> Jeremy.
>
> -----Original Message-----
> From: owner-spf-help@v2.listbox.com
> [mailto:owner-spf-help@v2.listbox.com] On Behalf Of Koen Martens
> Sent: Friday, August 06, 2004 10:18 AM
> To: spf-help@v2.listbox.com
> Subject: Re: [spf-help] Re: integration of SPF into incomming mail
> server
>
> On Fri, Aug 06, 2004 at 10:28:52AM +0200, Jeremy Pullicino wrote:
> > By doing this you risk parsing a forged header - spammers very often
> > forge the received headers.
> >
> > Parsing the received headers is a problem for two reasons:
> >
> > 1. There is hardly a standard for the format received headers, so
> > parsing is cumbersome and might not always work.
> > 2. Received headers can be forged so are unreliable
> >
> > For some, parsing these headers is probably the only option however
> SPF
> > will probably be much less effective and more vulnerable in this
> > scenario.
>
> I have noticed that for the upstream university account i mentioned
> earlier, parsing the headers was quite simple. I know what to expect in
> what portion of the message, because I can rely on the behaviour of the
> universities receiving smtp server. I am not looking for and/or parsing
> arbitrary received headers, just the ones I know are put there by the
> universities incoming email server.
>
> For me, 1 & 2 have not proven to be problems.
>
> Koen
>
> --
> K.F.J. Martens, Sonologic, http://www.sonologic.nl/
> Networking, embedded systems, unix expertise, artificial intelligence.
> Public PGP key: http://www.metro.cx/pubkey-gmc.asc
> Wondering about the funny attachment your mail program
> can't read? Visit http://www.openpgp.org/
>
> -------
> Archives at http://archives.listbox.com/spf-help/current/
> Donate! http://spf.pobox.com/donations.html
> To unsubscribe, change your address, or temporarily deactivate your
> subscription,
> please go to
> http://v2.listbox.com/member/?listname=spf-help@v2.listbox.com
>
>
>
> This mail was checked for malicious code and viruses
> by GFI MailSecurity. GFI MailSecurity provides email content
> checking, exploit detection, threats analysis and anti-virus for
> Exchange & SMTP servers. Viruses, Trojans, dangerous
> attachments and offensive content are removed automatically.
> Key features include: multiple virus engines; email content and
> attachment checking; an exploit shield; an HTML threats engine;
> a Trojan & Executable Scanner; and more.
>
> In addition to GFI MailSecurity, GFI also produces the
> GFI MailEssentials anti-spam software, the GFI FAXmaker
> fax server & GFI LANguard network security product ranges.
> For more information on our products, please visit
> http://www.gfi.com. This disclaimer was sent by
> GFI MailEssentials for Exchange/SMTP.
>
> -------
> Archives at http://archives.listbox.com/spf-help/current/
> Donate! http://spf.pobox.com/donations.html
> To unsubscribe, change your address, or temporarily deactivate your subscription,
> please go to http://v2.listbox.com/member/?listname=spf-help@v2.listbox.com

--
K.F.J. Martens, Sonologic, http://www.sonologic.nl/
Networking, embedded systems, unix expertise, artificial intelligence.
Public PGP key: http://www.metro.cx/pubkey-gmc.asc
Wondering about the funny attachment your mail program
can't read? Visit http://www.openpgp.org/

-------
Archives at http://archives.listbox.com/spf-help/current/
Donate! http://spf.pobox.com/donations.html
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-help@v2.listbox.com