Mailing List Archive

replay RBL queries one hour later
The last time I was hit by a not-recognized phishing campaign, no Ips nor
domains were present in RBL. When I took action one hour later I found that
several of them were listed.

So my idea is; is it possible to replay the queries one/two hours later?

I envision two methods:
- logging the queries, with Message-ids
- storing a copy of the message

If the second run hits new RBL, report to me, to take action.

Hope I was clear...
Re: replay RBL queries one hour later [ In reply to ]
On 25.02.23 15:34, hg user wrote:
>The last time I was hit by a not-recognized phishing campaign, no Ips nor
>domains were present in RBL. When I took action one hour later I found that
>several of them were listed.
>
>So my idea is; is it possible to replay the queries one/two hours later?

you can scan mail every time you want, the question is how do you want to do
that.

>I envision two methods:
>- logging the queries, with Message-ids
>- storing a copy of the message
>
>If the second run hits new RBL, report to me, to take action.

this could work, this way you could feed all mail multiple times to SA,
which would apparently increase usagfe of DNSBLs, they could block you then.

--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Microsoft dick is soft to do no harm
Re: replay RBL queries one hour later [ In reply to ]
On Sat, 25 Feb 2023, hg user wrote:

> The last time I was hit by a not-recognized phishing campaign, no Ips nor
> domains were present in RBL. When I took action one hour later I found that
> several of them were listed.
>
> So my idea is; is it possible to replay the queries one/two hours later?

Another more common approach to this situation is "greylisting", where the
first attempt to submit a message from an unrecognized source is
tempfailed for some period of time. The mailer will retry and the
submission will be accepted after the greylisting period has expired,
which may give RBLs time to list the IPs/domains/hashes/etc.

This also theoretically blocks fire-and-forget mass spammers who only try
submission once, but I don't know how common that model is these days.

https://duckduckgo.com/?q=milter-greylist

There are scenarios where this delay is unwelcome, for example commercial
accounts where you don't want a delay in receiving communications from
customers or potential customers. There are ways to tune it that may
mitigate these concerns somewhat.


--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
The Constitution is not a suicide pact, it is a restraining order
against government. And government, like any abusive person,
does not respect or obey restraining orders. -- Anonymous
-----------------------------------------------------------------------
1,001 days since the first private commercial manned orbital mission (SpaceX)
Re: replay RBL queries one hour later [ In reply to ]
On 2023-02-25 at 09:34:52 UTC-0500 (Sat, 25 Feb 2023 15:34:52 +0100)
hg user <mercurialuser@gmail.com>
is rumored to have said:

> The last time I was hit by a not-recognized phishing campaign, no Ips
> nor
> domains were present in RBL. When I took action one hour later I found
> that
> several of them were listed.
>
> So my idea is; is it possible to replay the queries one/two hours
> later?

If you write the code to do it, based on however you manage your mail,
you can do this. There's no way to put that sort of site-specific
tooling into SA itself. SA does not know anything about mail other than
the messages it is given. SA has no way to know what has happened to a
message after it has made its judgment.

> I envision two methods:
> - logging the queries, with Message-ids
> - storing a copy of the message
>
> If the second run hits new RBL, report to me, to take action.

It's certainly something that one could do.

It is not something that SpamAssassin itself does or ever will do.

A useful tool for doing this sort of thing involving SA is the
MIMEDdefang milter, which can use SA for filtering and also can do
anything else you can tell Perl to do with mail. I believe MailMunge (a
descendant of MIMEDefang) also has that capacity.

--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire
Re: replay RBL queries one hour later [ In reply to ]
Something to keep in mind about this idea of rescanning messages later -
once more anti-spam data is available - for use in training/reporting
spams - this probably should NOT be done days later because SOME senders
aggressively expire/recycle DKIM dns records. I guess that is to
minimize the ability for criminals to spoof DKIM? The result is that if
you implement this idea on days-old messages, you can end up with some
spam scoring that was ONLY due to the DKIM not being valid anymore,
where it was valid at the time the message was sent. This can lead to
many egregious false positives. But doing this "one hour later"
shouldn't have this problem.

Rob McEwen, invaluement
Re: replay RBL queries one hour later [ In reply to ]
Rob McEwen skrev den 2023-02-26 19:03:

...
> sent. This can lead to many egregious false positives. But doing this
> "one hour later" shouldn't have this problem.

message-id is timebased, so why invalidate it ? :)

i did that mistake on not dkim sign that header

in that regard i now have 2048 kbit size, where 4096 is a bit overkill
Re: replay RBL queries one hour later [ In reply to ]
Benny,

All I know for sure is this - for MANY legit emails - DKIM fails some
days later - when it had originally worked/validated at the time the
message was sent. I see this often in the real world when I rescan a
message to try to verify the impact on a message that a spam filtering
change caused - then notice that a very legit email that original passed
DKIM at the time the message was received - now suddenly fails DKIM
during this days-later rescan - and without ANY changes to the message
itself. I think that this is most likely caused by DNS records for that
DKIM being changed/updated. But whatever the cause, this is STILL a
reality that's worth noting, for anyone who is rescanning messages
later.

Rob McEwen, invaluement


------ Original Message ------
From "Benny Pedersen" <me@junc.eu>
To users@spamassassin.apache.org
Date 2/26/2023 1:37:53 PM
Subject Re: replay RBL queries one hour later

>Rob McEwen skrev den 2023-02-26 19:03:
>
>..
>>sent. This can lead to many egregious false positives. But doing this
>>"one hour later" shouldn't have this problem.
>
>message-id is timebased, so why invalidate it ? :)
>
>i did that mistake on not dkim sign that header
>
>in that regard i now have 2048 kbit size, where 4096 is a bit overkill
Re: replay RBL queries one hour later [ In reply to ]
Rob McEwen skrev den 2023-02-26 19:45:
> Benny,
>
> All I know for sure is this - for MANY legit emails - DKIM fails some
> days later - when it had originally worked/validated at the time the
> message was sent.

when i begined dkim signing i did that tought why would it be valid
after delivery, could it be good to only be valid until recipient
forwarder have recieved it ?, into days senario could harden arc more to
be used in forwards that on it self breaks dkim

postfix have queue life time 5d, so make dkim valid for 6 days ? :)

since then i do not expire this anymore

> I see this often in the real world when I rescan a
> message to try to verify the impact on a message that a spam filtering
> change caused - then notice that a very legit email that original
> passed DKIM at the time the message was received - now suddenly fails
> DKIM during this days-later rescan - and without ANY changes to the
> message itself.

why rescan ?

add reuse foo into local.cf for spamassassin so it not retesting dkim

> I think that this is most likely caused by DNS records
> for that DKIM being changed/updated. But whatever the cause, this is
> STILL a reality that's worth noting, for anyone who is rescanning
> messages later.

correct, how to solve that world on steriods ? :)
Re: replay RBL queries one hour later [ In reply to ]
Thank you to everybody that replied to my request. I knew I was not clear
in my message... :-)) sorry about it.

I have 2 paid RBL (so I don't care about number of queries) at the frontier
MTA. These RBLs reject a ton of connections and so the number of messages
reaching SA is already reduced.

Unfortunately, I can't greylist at the moment... well, actually I answer
with a 4xx code with one of the paid RBL... it's not *me* that greylists
but an external, official, specialized source. I know, borderline.

Back to my request, I see two possibilities.

A.
In the logs of the frontier MTA I have the connection IPs of the messages
that went through. A simple script can extract the IPs, |sort|uniq and then
dig/nslookup and note if they are now listed.
Unfortunately I don't know if the message was reported spam or quarantined
later but it may be detected spam not for RBL

B.
On the backend, zimbra logs all the messages stored in the mailboxes. A bit
more complex script can dump the not spam and not quarantined messages
received in a time range in a specific dir and submit each one to SA, the
production one or one dedicated to this job. In this way I will also check
the URL RBLs.
Using a different SA server allows us to use SA 4.0, or a different set of
plugins and rules, or for example enabling only RBLs checks, adding the
paid ones.

Still don't know if all this is worth the effort.




>
>
Re: replay RBL queries one hour later [ In reply to ]
Rob McEwen wrote:
> Benny,
>
> All I know for sure is this - for MANY legit emails - DKIM fails some
> days later

Hours.

I've recently learned about this, in the context of trying to
welcomelist legitimate senders. A 2-hour validity window for the DKIM
signature is pretty common. :(

- when it had originally worked/validated at the time the
> message was sent. I see this often in the real world when I rescan a
> message to try to verify the impact on a message that a spam filtering
> change caused - then notice that a very legit email that original passed
> DKIM at the time the message was received - now suddenly fails DKIM
> during this days-later rescan - and without ANY changes to the message
> itself. I think that this is most likely caused by DNS records for that
> DKIM being changed/updated.

On most of those messages I expect it's an attribute set on the
signature, not a rotated DKIM record.

Look for "t=..." and "x=..." in the DKIM-Signature header. t= is the
timestamp when it was signed, x= is when it expires.

-kgd