Mailing List Archive

Optimising DNS-based checks
Hi list,

I've extracted below the top lines of timing for my SA checks on
emails, and am wondering if these are along the lines of general
expectations and performance with some of the DNS-based checks? Other
than running a caching nameserver (done) are there any other pointers,
or are my results fairly typical? I've not seen much reporting or
discussion online on this. Not a major issue, as my server is not
highly loaded, but interested from the perspective of optimising and
understanding. The server is running a local caching nameserver which
is specified in local.cf for spamassassin use.

- Razor2 is typically about 2.9s, with 90% falling between the 1.3s
range between 1.9s and 3.2s
- DCC is interesting - 50% in less than 0.3s, but then the rest taking
all the way up to timeout at 10s
- SPF typically 0.25s, but 25% taking more than 0.8s
- Pyzor quick and consistent average < 0.4s

Simon




========================================================================================================================
SA Timing Percentiles % Time Total (ms) 0%
5% 25% 50% 75% 95% 100%

------------------------------------------------------------------------------------------------------------------------
tests_pri_20 (277) 51.44% 774766.00 948.00
1874.60 2827.00 2872.00 2910.00 3162.80 4180.00
check_razor2 (277) 51.39% 774057.00 947.00
1872.80 2825.00 2871.00 2907.00 3162.40 4177.00
tests_pri_10 (277) 26.14% 393733.00 286.00
288.00 290.00 294.00 4443.00 4456.20 10003.00
check_dcc (277) 26.06% 392435.00 285.00
286.00 287.00 289.00 4439.00 4447.00 10001.00
tests_pri_0 (277) 12.87% 193775.00 16.00
61.00 282.00 549.00 966.00 1919.00 2871.00
poll_dns_idle (267) 9.41% 141772.57 0.01
0.01 95.50 285.00 810.50 1639.00 2735.00
check_spf (277) 8.43% 126969.05 0.09
1.23 58.00 250.00 776.00 1600.20 2759.00
tests_pri_30 (277) 6.70% 100850.00 350.00
352.00 356.00 360.00 366.00 389.60 474.00
check_pyzor (277) 6.67% 100510.00 349.00
351.00 355.00 359.00 364.00 388.60 472.00
check_dkim_adsp (108) 1.70% 25648.85 0.87
1.12 1.54 52.00 323.25 908.90 2200.00
tests_pri_500 (277) 1.20% 18065.52 1.59
1.96 2.10 2.20 2.40 453.80 2388.00
total_txrep (277) 0.69% 10433.00 5.00
11.00 17.00 21.00 24.00 35.80 786.00
tests_pri_1000 (277) 0.62% 9397.00 6.00
10.00 15.00 19.00 21.00 30.20 621.00

--
Simon Wilson
M: 0400 12 11 16
Re: Optimising DNS-based checks [ In reply to ]
On Tue, 30 Mar 2021 11:45:57 +1000
Simon Wilson wrote:

> Hi list,
>
> I've extracted below the top lines of timing for my SA checks on
> emails, and am wondering if these are along the lines of general
> expectations and performance with some of the DNS-based checks?

You can work around the delays by having multiple spamd processes per
cpu or by reducing the timeouts. The latter may result in less
accuracy, so I wouldn't that unless you have a high enough load to
justify it.

Most DNS look-ups run in parallel with the regex rules. In 4.0/trunk
that also applies to Pyzor, Razor, and DCC.

If you have an Authentication-Results or Received-SPF header above your
first/lowest trusted header SA can use that instead of doing lookups.
Re: Optimising DNS-based checks [ In reply to ]
On Tue, 30 Mar 2021 14:16:16 +0100
RW wrote:


> by having multiple spamd processes per cpu

That should have been "per cpu core".

> Most DNS look-ups run in parallel with the regex rules. In 4.0/trunk
> that also applies to Pyzor, Razor, and DCC.
>
> If you have an Authentication-Results or Received-SPF header above
> your first/lowest trusted header SA can use that instead of doing

That should have been "trusted Received header".
Re: Optimising DNS-based checks [ In reply to ]
----- Message from RW <rwmaillists@googlemail.com> ---------
Date: Tue, 30 Mar 2021 14:30:01 +0100
From: RW <rwmaillists@googlemail.com>
Subject: Re: Optimising DNS-based checks
To: users@spamassassin.apache.org


> On Tue, 30 Mar 2021 14:16:16 +0100
> RW wrote:
>
>
>> by having multiple spamd processes per cpu
>
> That should have been "per cpu core".
>
>> Most DNS look-ups run in parallel with the regex rules. In 4.0/trunk
>> that also applies to Pyzor, Razor, and DCC.
>>
>> If you have an Authentication-Results or Received-SPF header above
>> your first/lowest trusted header SA can use that instead of doing
>
> That should have been "trusted Received header".


----- End message from RW <rwmaillists@googlemail.com> -----


Thanks RW.

I'm using v3.4.2 called from amavis when needed, with 3 x pre-forked
amavis processes - but I'm not familiar with how that would apply in
running multiple spam processes per core?

I *do* have Postfix adding a Received-SPF filter using
python-policyd-spf (called as last check in
smtpd_recipient_restrictions), so yes it would make sense for
spamassassin to trust the check already made - I'll see if I can work
out how to do that.

Simon.

--
Simon Wilson
M: 0400 12 11 16