Mailing List Archive

Random results with AskDns
Hi all,
When there are several hundreds of lookups, Askdns / Async abort many of them randomly even when 100% of queries got an answer.I use local dns cache but every run of SA produces different number of aborted remaining lookups. 
If you dig manually from command line any aborted query, answer is immediate.
I have not found any related bug in SA Bugzilla.. (pretty similar to this one  https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7875  ) After some debug, it seems that answers are not harvested properly... and around 30% of them are lost in every run.
It is not a timeout problem: both tcpdump and dns-cache log  show immediate answers to 100% of queries in less than 1 second.
May this be solved in the new AskDns John Hardin mentioned some days ago?

Thanks,

---------Pedreter.
Re: Random results with AskDns [ In reply to ]
SOLVED!
Just in case someone has this issue...  
Short version:
In heavy load environments, SA produces more UDP traffic (specially if answers are big, typically happens with TXT queries) than Linux kernel can handlewith default buffers (tested in Debian Buster), so many SA queries never get an answer and die on timeout.This not only affects final SA result, but performance.
Correct Kernel UD tunning solves the problem!
-----------Pedreter.

On Monday, March 1, 2021, 06:06:24 PM GMT+1, Pedro David Marco <pedrod_marco@yahoo.com> wrote:
>Hi all,>When there are several hundreds of lookups, Askdns / Async abort many of them randomly even when 100% of queries got an answer.
>I use local dns cache but every run of SA produces different number of aborted remaining lookups. >if you dig manually from command line any aborted query, answer is immediate.
>I have not found any related bug in SA Bugzilla.. (pretty similar to this one  7875 – AskDNS plugin does not correctly handle CNAMEs leading to TXTs  ) 
>After some debug, it seems that answers are not harvested properly... and around 30% of them are lost in every run.
>It is not a timeout problem: both tcpdump and dns-cache log  show immediate answers to 100% of queries in less than 1 second.
>May this be solved in the new AskDns John Hardin mentioned some days ago?
Re: Random results with AskDns [ In reply to ]
On 2021-03-02 16:26, Pedro David Marco wrote:

> Correct Kernel UD tunning solves the problem!

in verbose this is ?
Re: Random results with AskDns [ In reply to ]
On 02.03.21 15:26, Pedro David Marco wrote:
>Just in case someone has this issue...  
>Short version:
>In heavy load environments, SA produces more UDP traffic (specially if answers are big, typically happens with TXT queries) than Linux kernel can handlewith default buffers (tested in Debian Buster), so many SA queries never get an answer and die on timeout.This not only affects final SA result, but performance.
>Correct Kernel UD tunning solves the problem!

do you run local resolving (non-forwarding) DNS server?

> On Monday, March 1, 2021, 06:06:24 PM GMT+1, Pedro David Marco <pedrod_marco@yahoo.com> wrote:
> >Hi all,>When there are several hundreds of lookups, Askdns / Async abort many of them randomly even when 100% of queries got an answer.
>>I use local dns cache but every run of SA produces different number of aborted remaining lookups. >if you dig manually from command line any aborted query, answer is immediate.
>>I have not found any related bug in SA Bugzilla.. (pretty similar to this one  7875 – AskDNS plugin does not correctly handle CNAMEs leading to TXTs  ) 
>>After some debug, it seems that answers are not harvested properly... and around 30% of them are lost in every run.
>>It is not a timeout problem: both tcpdump and dns-cache log  show immediate answers to 100% of queries in less than 1 second.
>>May this be solved in the new AskDns John Hardin mentioned some days ago?
>
>

--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Due to unexpected conditions Windows 2000 will be released
in first quarter of year 1901
Re: Random results with AskDns [ In reply to ]
Tried both and with/without cache...

--------Pedreter...


On Tuesday, March 2, 2021, 04:46:08 PM GMT+1, Matus UHLAR - fantomas <uhlar@fantomas.sk> wrote:

On 02.03.21 15:26, Pedro David Marco wrote:
>Just in case someone has this issue...  
>Short version:
>In heavy load environments, SA produces more UDP traffic (specially if answers are big, typically happens with TXT queries) than Linux kernel can handlewith default buffers (tested in Debian Buster), so many SA queries never get an answer and die on timeout.This not only affects final SA result, but performance.
>Correct Kernel UD tunning solves the problem!

do you run local resolving (non-forwarding) DNS server?

>  On Monday, March 1, 2021, 06:06:24 PM GMT+1, Pedro David Marco <pedrod_marco@yahoo.com> wrote:
> >Hi all,>When there are several hundreds of lookups, Askdns / Async abort many of them randomly even when 100% of queries got an answer.
>>I use local dns cache but every run of SA produces different number of aborted remaining lookups. >if you dig manually from command line any aborted query, answer is immediate.
>>I have not found any related bug in SA Bugzilla.. (pretty similar to this one  7875 – AskDNS plugin does not correctly handle CNAMEs leading to TXTs  ) 
>>After some debug, it seems that answers are not harvested properly... and around 30% of them are lost in every run.
>>It is not a timeout problem: both tcpdump and dns-cache log  show immediate answers to 100% of queries in less than 1 second.
>>May this be solved in the new AskDns John Hardin mentioned some days ago?
>
>

--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Due to unexpected conditions Windows 2000 will be released
in first quarter of year 1901
Re: Random results with AskDns [ In reply to ]
I have set buffers to 20MB per core and results are great:  
# sysctl -w net.core.rmem_default=20971520

0% packet lost... with default value of 200KB packet-loss went easily above 30%
You can chek if you have this problem with:
# netstat -suna
look for errors in UDP area

----------Pedteter.


On Tuesday, March 2, 2021, 04:44:35 PM GMT+1, Benny Pedersen <me@junc.eu> wrote:
>On 2021-03-02 16:26, Pedro David Marco wrote:> Correct Kernel UD tunning solves the problem!>in verbose this is ?
Re: Random results with AskDns [ In reply to ]
On 2021-03-02 16:50, Pedro David Marco wrote:
> Tried both and with/without cache...

i think its a glibc problem, and if it is it could be solved with edns0
in local dns

force tcp on packet size over 512 byte

https://bobcares.com/blog/bind-edns/ default edns0 is now 4096, but
sometimes its can only be 512, check logs and read this link

i am not a dns expert, sorry
Re: Random results with AskDns [ In reply to ]
On 02.03.21 15:50, Pedro David Marco wrote:
> Tried both and with/without cache...

disabling cache will make problem worse.
However, the question was a bit different - if you run your DNS server
locally.
But it should not be forwarding for spam detection.

>
> On 02.03.21 15:26, Pedro David Marco wrote:
>>Just in case someone has this issue...  
>>Short version:
>>In heavy load environments, SA produces more UDP traffic (specially if answers are big, typically happens with TXT queries) than Linux kernel can handlewith default buffers (tested in Debian Buster), so many SA queries never get an answer and die on timeout.This not only affects final SA result, but performance.
>>Correct Kernel UD tunning solves the problem!

> On Tuesday, March 2, 2021, 04:46:08 PM GMT+1, Matus UHLAR - fantomas <uhlar@fantomas.sk> wrote:
>do you run local resolving (non-forwarding) DNS server?

>>  On Monday, March 1, 2021, 06:06:24 PM GMT+1, Pedro David Marco <pedrod_marco@yahoo.com> wrote:
>> >Hi all,>When there are several hundreds of lookups, Askdns / Async abort many of them randomly even when 100% of queries got an answer.
>>>I use local dns cache but every run of SA produces different number of aborted remaining lookups. >if you dig manually from command line any aborted query, answer is immediate.
>>>I have not found any related bug in SA Bugzilla.. (pretty similar to this one  7875 – AskDNS plugin does not correctly handle CNAMEs leading to TXTs  ) 
>>>After some debug, it seems that answers are not harvested properly... and around 30% of them are lost in every run.
>>>It is not a timeout problem: both tcpdump and dns-cache log  show immediate answers to 100% of queries in less than 1 second.
>>>May this be solved in the new AskDns John Hardin mentioned some days ago?

--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Your mouse has moved. Windows NT will now restart for changes to take
to take effect. [OK]
Re: Random results with AskDns [ In reply to ]
>On 2021-03-02 16:50, Pedro David Marco wrote:
>>Tried both and with/without cache...

On 02.03.21 18:26, Benny Pedersen wrote:
>i think its a glibc problem, and if it is it could be solved with
>edns0 in local dns
>
>force tcp on packet size over 512 byte

better not. excessive use of TCP can be a problem.

>https://bobcares.com/blog/bind-edns/ default edns0 is now 4096, but
>sometimes its can only be 512, check logs and read this link

logs of DNS server, like BIND. It can force maximum UDP size to e.g. 1500

>i am not a dns expert, sorry

--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Micro random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...