Mailing List Archive

On 14/08/2022 23:15, David Bürgin wrote:

> To clarify: Backscatter is caused by 'rejecting' mail with a bounce
> message, after first accepting it.

This is what was being suggested by some, I think everyone here knows
what backscatter means, and what it is.

--
Regards,
Noel Butler

This Email, including attachments, may contain legally privileged
information, therefore at all times remains confidential and subject to
copyright protected under international law. You may not disseminate
this message without the authors express written authority to do so.
If you are not the intended recipient, please notify the sender then
delete all copies of this message including attachments immediately.
Confidentiality, copyright, and legal privilege are not waived or lost
by reason of the mistaken delivery of this message.

>>On Sun, 2022-08-14 at 11:39 +1000, Noel Butler wrote: On 14/08/2022
>>3) It would be rather trivial to return spam to sender with a
>>suitable

>On 14/08/2022 22:37, Martin Gregorie wrote:
>>WTF, that has been a terrible idea since the 90s, given most spam is
>>spoofed, the end result of this will be your mail server getting the
>>poor reputation as source of backscatter and going into blacklists :)

On 15.08.22 12:00, Noel Butler wrote:
> greed - I don't do that, but almost as long as I've been on this list
>there have been advocates of it. As I said, I thought about it, but the
>effort of writing a filter to determine what, if anything should be
>bounced or rejected, has never seemed worth the effort for such a low
>volume mail used as myself.

IMHO if spam passes SPF and/or DKIM, bouncing it should at least result it
in being delivered to the correct sender (or they'll have to fix their SPF
records).

of course, only if you are willing to risk anger of misconfigured senders...

>When people advocate for it, it goes to show the only thing they have
>ever been responsible for is their own home mail server with accounts
>for them and maybe a friend or two on it, never for anything
>commercial, you've been around a great many years Martin, so I'm glad
>you resist the temptation of the fools.


--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
My mind is like a steel trap - rusty and illegal in 37 states.

Bill Cole wrote:
> Not exactly. There are 2 distinct domain lists internal to SA that exist
> to reduce false positives.
>
> 1. The URIDNSBL 'skip' list of domains which are ignored in body URIs.
> These are known to not *per se* have any correlation to the ham/spam
> classification decision.

IIRC the basic policy for this list is "these domains are never going to
be listed in the major DNSBLs, so don't bother looking them up because
the result will ALWAYS be negative".

If you're brushing the edges of the free lookup limits, this may also
help you stay under those limits a bit longer.

-kgd

Vincent Lefevre wrote:
> On 2022-08-13 14:05:43 -0400, joe a wrote:
>> On 8/13/2022 12:38 PM, Martin Gregorie wrote:
>> . . .
>>> 2) There's no mandatory need to REJECT spam. It has always been up to
>>> the recipient to decide whether to return it to the sender or not.
>>
>> Agreed in part. I see returning SPAM to sender as an exercise in futility
>> or perhaps further enabling. But I do prefer labeling as SPAM to outright
>> rejection in many cases.
>
> Rejecting mail (instead of accepting it and dropping it) is useful
> in case of false positives.

I'm a bit torn on this.

On the one hand, yes, the sender now knows for sure their message didn't
get through*.

On the other hand, the sender now calls *their* outgoing mail provider
to complain "You wouldn't let my message through!", and trying to
explain to someone that no, really, we can't do anything about this
because it's the recipient's system that doesn't like you is....
sometimes painfully tedious.

I have had a couple of cases where I tried to rephrase myself half a
dozen ways to explain this to a couple of senders; "you'll have to
contact them by some other method to get them to whitelist you, because
we literally CAN NOT force a message to be accepted". I'm pretty sure
that in one case I literally replied back "I don't know how else to put
this, we can't do anything because it's the recipient's mail system
that's rejecting the message".

On that basis I'm inclined to say that quarantining on the recipient end
is better, because the support burden then falls on the operator of the
misbehaving filter instead of the innocent sending platform.

I feel much the same about challenge-response filters; they offload the
problem from the person operating the filter onto everyone else.

-kgd

[*] Assuming they can make sense of the postmaster message in the first
place, which personal experience says "no, they really can't"... and
not just general end users, some "IT contractors" dealing with email
are, um, not entirely clued in to how SMTP works.

Vincent Lefevre <vincent@vinc17.net> writes:

> On 2022-08-13 14:05:43 -0400, joe a wrote:
>> On 8/13/2022 12:38 PM, Martin Gregorie wrote:
>> . . .
>> > 2) There's no mandatory need to REJECT spam. It has always been up to
>> > the recipient to decide whether to return it to the sender or not.
>>
>> Agreed in part. I see returning SPAM to sender as an exercise in futility
>> or perhaps further enabling. But I do prefer labeling as SPAM to outright
>> rejection in many cases.

Be careful in "returning". There is replying with 550 and not accepting
it, which ensures that *you* are not generating backscatter, and there
is sending a bounce later. I think that if you're going to reject it,
you should 550 it.

> Rejecting mail (instead of accepting it and dropping it) is useful
> in case of false positives.

This is a key point. A lot of mail ends up in spam folders that are so
full they don't get looked at, at a number of ISPs that have a poor
email recipient experience. I know people at AOL/Yahoo/Verizon and
Comcast that have mail end up in spam and in practice do not cope with
looking at it. (Further, this mail is wrongly classified, and people
can't in practice fix that.)

By rejecting spam with 550, it doesn't end up in the spam folder, and
that folder becomes easier to scan. And if legit mail is rejected, at
least the sender knows it didn't get there, even if the ISP is
intractable.

If you accept mail and then send it to /dev/null, then the recipient is
unaware that it was sent, and the sender is unaware that it wasn't
received, other than by implementing a human-human ack protocol.

So I'm a firm believer that at SMTP time, you need to pick one of

550 and you're done

accept and then sort into ham mailboxes and spam mailboxes, with the
idea that the user should be checking all of them

By choosing 550 you can turn up the aggressiveness of checking a bit
compared to if you don't.

On 16/08/2022 01:33, Greg Troxel wrote:

> If you accept mail and then send it to /dev/null, then the recipient is
> unaware that it was sent, and the sender is unaware that it wasn't
> received,

Exactly what happens to high scored spam, if its high is very obvious
trash and the recipient wont want to know, and well who cares what those
senders want to know :)

> So I'm a firm believer that at SMTP time, you need to pick one of
>
> 550 and you're done
>
> accept and then sort into ham mailboxes and spam mailboxes, with the
> idea that the user should be checking all of them

or use both,

1 block the very obvious and non compliant; 95%

2 spam folder the "just triggering spam rules" - a problem with pop3
users (yes, speaking from an ISP world in Oceana they heavily outweigh
number of imap users) so the labelled as spam stuff is mixed in their
normal inbox ;0.1%

3 /dev/null the other obvious ; 0.0001% (ultra low becasue step 1
catches most)

4 inbox the rest

As for spam folder checking.... not even I bother with mine except for
once or twice a year

--
Regards,
Noel Butler

This Email, including attachments, may contain legally privileged
information, therefore at all times remains confidential and subject to
copyright protected under international law. You may not disseminate
this message without the authors express written authority to do so.
If you are not the intended recipient, please notify the sender then
delete all copies of this message including attachments immediately.
Confidentiality, copyright, and legal privilege are not waived or lost
by reason of the mistaken delivery of this message.

On 2022-08-15 10:39:05 -0400, Kris Deugau wrote:
> Vincent Lefevre wrote:
> > Rejecting mail (instead of accepting it and dropping it) is useful
> > in case of false positives.
>
> I'm a bit torn on this.
>
> On the one hand, yes, the sender now knows for sure their message didn't get
> through*.
>
> On the other hand, the sender now calls *their* outgoing mail provider to
> complain "You wouldn't let my message through!", and trying to explain to
> someone that no, really, we can't do anything about this because it's the
> recipient's system that doesn't like you is.... sometimes painfully tedious.

Well, the outgoing mail provider (or IP address supplier) is sometimes
the culprit, e.g. by letting spam out.

--
Vincent Lef?vre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

On 2022-08-15 11:33:53 -0400, Greg Troxel wrote:
> Vincent Lefevre <vincent@vinc17.net> writes:
> > On 2022-08-13 14:05:43 -0400, joe a wrote:
> >> On 8/13/2022 12:38 PM, Martin Gregorie wrote:
> >> . . .
> >> > 2) There's no mandatory need to REJECT spam. It has always been up to
> >> > the recipient to decide whether to return it to the sender or not.
> >>
> >> Agreed in part. I see returning SPAM to sender as an exercise in futility
> >> or perhaps further enabling. But I do prefer labeling as SPAM to outright
> >> rejection in many cases.
>
> Be careful in "returning". There is replying with 550 and not accepting
> it, which ensures that *you* are not generating backscatter, and there
> is sending a bounce later. I think that if you're going to reject it,
> you should 550 it.

Yes, this is a 550 SMTP reply.

--
Vincent Lef?vre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Vincent Lefevre wrote:
> On 2022-08-15 10:39:05 -0400, Kris Deugau wrote:
>> Vincent Lefevre wrote:
>>> Rejecting mail (instead of accepting it and dropping it) is useful
>>> in case of false positives.
>>
>> I'm a bit torn on this.
>>
>> On the one hand, yes, the sender now knows for sure their message didn't get
>> through*.
>>
>> On the other hand, the sender now calls *their* outgoing mail provider to
>> complain "You wouldn't let my message through!", and trying to explain to
>> someone that no, really, we can't do anything about this because it's the
>> recipient's system that doesn't like you is.... sometimes painfully tedious.
>
> Well, the outgoing mail provider (or IP address supplier) is sometimes
> the culprit, e.g. by letting spam out.

True. But if spam filters everywhere were perfect, spam wouldn't be
such a big problem.

And, quite reasonably, most rejections for spam include very little or
no detail, so aside from DNSBL-based rejections the sending platform has
essentially zero information beyond "the receiving system doesn't like
us". Which can't be troubleshot from the sending side without at least
some kind of feedback from the receiving side.

-kgd

On 2022-08-16 12:05:43 -0400, Kris Deugau wrote:
> Vincent Lefevre wrote:
> > On 2022-08-15 10:39:05 -0400, Kris Deugau wrote:
> > > Vincent Lefevre wrote:
> > > > Rejecting mail (instead of accepting it and dropping it) is useful
> > > > in case of false positives.
> > >
> > > I'm a bit torn on this.
> > >
> > > On the one hand, yes, the sender now knows for sure their message didn't get
> > > through*.
> > >
> > > On the other hand, the sender now calls *their* outgoing mail provider to
> > > complain "You wouldn't let my message through!", and trying to explain to
> > > someone that no, really, we can't do anything about this because it's the
> > > recipient's system that doesn't like you is.... sometimes painfully tedious.
> >
> > Well, the outgoing mail provider (or IP address supplier) is sometimes
> > the culprit, e.g. by letting spam out.
>
> True. But if spam filters everywhere were perfect, spam wouldn't be such a
> big problem.
>
> And, quite reasonably, most rejections for spam include very little or no
> detail, so aside from DNSBL-based rejections the sending platform has
> essentially zero information beyond "the receiving system doesn't like us".
> Which can't be troubleshot from the sending side without at least some kind
> of feedback from the receiving side.

Minimal information should at least be included with the 550 status.
This is also useful for the receiving side in order to know whether
the antispam rules are effective (by checking the logs).

--
Vincent Lef?vre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Vincent Lefevre wrote:
> On 2022-08-16 12:05:43 -0400, Kris Deugau wrote:
>> And, quite reasonably, most rejections for spam include very little or no
>> detail, so aside from DNSBL-based rejections the sending platform has
>> essentially zero information beyond "the receiving system doesn't like us".
>> Which can't be troubleshot from the sending side without at least some kind
>> of feedback from the receiving side.
>
> Minimal information should at least be included with the 550 status.
> This is also useful for the receiving side in order to know whether
> the antispam rules are effective (by checking the logs).

Mmm. So how would you, as sender or sender's mail provider,
troubleshoot a message rejected with "550 Too spammy"? I have seen
several rejections that were equally clear and to the point, without
divulging any particular detail about what, exactly, was objectionable.

What details should the receiving system include in that 550, such that
legitimate senders can adjust or fix something in their message, that
spammers can't also abuse to slip their glop through that filter as
well? (TBH the spammers have the advantage here. For instance, if a
filter is rejecting on "miscellaneous terrifying MIME/HTML crap", the
spammer is going to have a FAR easier time tweaking their message than a
legitimate sender.... because the legitimate sender is just using
something like "Outlook stationery", which means they have basically
zero control over any MIME or HTML formatting that a filter might be
choking on.)

-kgd

On Thu, 2022-08-18 at 12:11 -0400, Kris Deugau wrote:
> Mmm.  So how would you, as sender or sender's mail provider,
> troubleshoot a message rejected with "550 Too spammy"?  I have seen
> several rejections that were equally clear and to the point, without
> divulging any particular detail about what, exactly, was
> objectionable.
>
> What details should the receiving system include in that 550, such
> that legitimate senders can adjust or fix something in their message,
> that spammers can't also abuse to slip their glop through that filter
> as well?

The only reasonably foolproof way I can think of gently telling friendly
senders why their message is being treated as spam while not helping
spammers to send more believable and/or less obvious spam requires
something line the following:

You should maintain some form of mail archive. It needn't be all that
big or complex: for this purpose all it needs to contain is a list of
valid addresses that you have previously sent mail to. If you keep this
information set then, as an initial guess the spam response logic can be
as simple as:

- extract the domain name from the incoming mail's From header and use 
it to find the domain IP. Use that IP to do a reverse domain lookup.

- if the reverse lookup fails, or the domain it retrieved does not match
the one in the From address, send a bare 550 REJECT because the failed
reverse lookup implies the sending domain is a forgery. 

This is a manual check I often use if I suspect a message of being
spam and get curious about it for some reason or other. FWIW my next
step is to use Lynx to see what the associated website (if any) is
associated with the domain - an amazing amount of spam sources have an
associated website - and its almost always an off-the-peg generic
page. I use Lynx for this because it is a text-only browser that can
also disable all cookie handling, so is a relatively safe way of
looking at possibly dodgy websites.

- if the mail archive shows that we've previously sent mail to the
sender of this message, either send a bounce or a 550 rejection
together with a polite explanation of why you think their message
might be spam.

- if mail has NOT previously been sent to the sender of this message,
send a bare 550 REJECT because (a) they may well be a spammer and (b)
you don't know them and so don't (yet) have any need to be nice to
them.

This is pretty much off the top of my pointy head, after a warmish day
spent driving round part of SE UK, so probably obvious flaws, but this
would be my starting point if I was planning to reject spam and similar
dross rather than simply tossing it in the wastebasket and it does at
least suggest a way of not telling a spammer why you dejected his junk.

Martin

On 2022-08-18 12:11:04 -0400, Kris Deugau wrote:
> Mmm. So how would you, as sender or sender's mail provider, troubleshoot a
> message rejected with "550 Too spammy"? I have seen several rejections that
> were equally clear and to the point, without divulging any particular detail
> about what, exactly, was objectionable.

I doubt that spammers take 550 messages into account, or even read them.

--
Vincent Lef?vre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

On 2022-08-18 19:40:33 +0100, Martin Gregorie wrote:
> - extract the domain name from the incoming mail's From header?and use?
> it to find the domain IP. Use that IP to do a reverse domain lookup.
>
> - if the reverse lookup fails, or the domain it retrieved does not match
> the one in the From address, send a bare 550 REJECT because the failed
> reverse lookup implies the sending domain is a forgery.?

It doesn't. There are IPs that host several domains, e.g. in case
of shared web hosting. For instance, I have 2 domains vinc17.net
and vinc17.org, and both are handled by the same machine, thus
with a single IP address. So, necessarily, the reverse lookup will
not match for one of these domains.

BTW, for spamassassin.apache.org, it resolves to 151.101.2.132, but
the reverse lookup fails.

And anyway, this is about mail, so the only thing that could really
be considered is the MX. But the MX domains may be different from
the "From:" domain, even if the domain has its own range of IP
addresses. For instance:

$ dig -t mx ens-lyon.fr
[...]
;; ANSWER SECTION:
ens-lyon.fr. 6754 IN MX 20 mxc.relay.renater.fr.
ens-lyon.fr. 6754 IN MX 20 mxd.relay.renater.fr.
ens-lyon.fr. 6754 IN MX 20 mxa.relay.renater.fr.
ens-lyon.fr. 6754 IN MX 20 mxb.relay.renater.fr.
[...]

The only thing that you may want to do is a reverse lookup of the
client IP, then check that the answer resolves back to the IP (among
the answers, as there may be several IP addresses).

--
Vincent Lef?vre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

On Tue, 2022-08-23 at 12:11 +0200, Vincent Lefevre wrote:
> On 2022-08-18 19:40:33 +0100, Martin Gregorie wrote:
> > - if the reverse lookup fails, or the domain it retrieved does not
> > match the one in the From address, send a bare 550 REJECT because
> > the failed
> > reverse lookup implies the sending domain is a forgery. 
>
> It doesn't. There are IPs that host several domains, e.g. in case
> of shared web hosting. For instance, I have 2 domains vinc17.net
> and vinc17.org, and both are handled by the same machine, thus
> with a single IP address. So, necessarily, the reverse lookup will
> not match for one of these domains.
>
Fair enough: I did say that some of this was off the top pf my head at
the end of a longish day.

Would doing the lookup trick on the URL in the Message-ID header be any
more reliable?

Martin

On 2022-08-23 14:31:55 +0100, Martin Gregorie wrote:
> Fair enough: I did say that some of this was off the top pf my head at
> the end of a longish day.
>
> Would doing the lookup trick on the URL in the Message-ID header be any
> more reliable?

DNS Lookup checking is valid only for IP -> FQDN -> IP, not
FQDN -> IP -> FQDN, whatever the FQDN.

Note also that the Message-ID is not a URL, just some kind of
identifier with some syntax and that must be unique. In general,
you shouldn't assume anything on the right-hand side of a
Message-ID. RFC 5322 just says:

As with addr-spec, a liberal syntax is given for the right-hand side
of the "@" in a msg-id. However, later in this section, the use of a
domain for the right-hand side of the "@" is RECOMMENDED.

In particular, it doesn't need to be resolvable, and there are
good reasons for which this may not be the case. For instance,
the Message-ID may be generated on the machine from which the
message is sent, with some internal hostname on the right-hand
side (this is better to ensure unicity), thus is not resolvable.

--
Vincent Lef?vre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

On Tue, 23 Aug 2022, Vincent Lefevre wrote:

> On 2022-08-18 12:11:04 -0400, Kris Deugau wrote:
>> Mmm. So how would you, as sender or sender's mail provider, troubleshoot a
>> message rejected with "550 Too spammy"? I have seen several rejections that
>> were equally clear and to the point, without divulging any particular detail
>> about what, exactly, was objectionable.
>
> I doubt that spammers take 550 messages into account, or even read them.

Agreed.

Perhaps dumping the list of SA rules that hit, absent scores. That's not a
bad violation of opsec as there are public evaluation tools available that
would return much the same information, and that would give something
helpful to discuss with the site admin when trying to resolve the
situation.


--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Law is too dangerous a tool to leave in the hands of
opposing tribes who just want to use it to
bludgeon one another. -- J.D. Tuccile
-----------------------------------------------------------------------
Tomorrow: the 1943rd anniversary of the destruction of Pompeii