Mailing List Archive: handle_user and connect to spamd failed

handle_user and connect to spamd failed

Oct 18, 2021, 2:53 AM

Post #1 of 11 (1483 views)

I am setting up a new postfix/dovecot/spamassassin server to replace an
outdated one. It works correctly but I have two outstanding warnings in
the logs that I would like to eliminate if possible. I have trawled
through dozens of postings online but found no resolution for either
warning. The server is debian 10 buster.

1) "spamd: handle_user (userdir) unable to find user: 'dave'"
Postfix is using dovecot to resolve users but 'dave' is not a user. The
processing completes without it but I don't know what the process is
trying to do with the user that causes this message, nor how to resolve
it. I also have this on the old server and was advised to ignore it but
this seems ill-advised: if there is a message it's there for a purpose
or is in error.

2) "spamc: connect to spamd on ::1 failed, retrying (#1 of 3):
Connection refused"
This warning appears on receipt of a message. I do not have this message
on the old server. I believe spamc is trying to connect to spamd using
ipv6, which is turned off in postfix and, as far as I can determine, in
spamassassin as well.

The options in /etc/default/spamassassin are:
OPTIONS="--create-prefs -4 --max-children 5 --helper-home-dir
/var/lib/spamassassin -u debian-spamd"

I have tried adding options to make:
OPTIONS="--create-prefs -4 --ipv4-only --max-children 5
--helper-home-dir /var/lib/spamassassin -u debian-spamd -d 127.0.0.1"

This made no difference. I also have /etc/default/spamass-milter with
the options:
OPTIONS="-u spamass-milter -i 127.0.0.1 -4"

In postfix master.cf I have:
spamfilter unix - n n - - pipe
flags=Rq user=spamd argv=/usr/bin/spamfilter.sh -oi -f ${sender}
${recipient}

with /usr/bin/spamfilter.sh containing:
SENDMAIL=/usr/sbin/sendmail
SPAMASSASSIN=/usr/bin/spamc
MAX_MESSAGE_SIZE="-s 8000000"
logger <<<"Spam filter piping to SpamAssassin, then to: $SENDMAIL $@"
${SPAMASSASSIN} ${MAX_MESSAGE_SIZE} | ${SENDMAIL} "$@"
exit $?

This has been copied from the old server.

I would appreciate advice on how to remove these two errors. What other
information do you require, please?

--
Dave Stiles

Re: handle_user and connect to spamd failed [ In reply to ]

dbuergin at gluet

Oct 18, 2021, 3:15 AM

Post #2 of 11 (1483 views)

Permalink

One thing that needs clarification is how you want to integrate spam
filtering with SpamAssassin with Postfix:

Apparently, you are trying to do this with two different methods at
once? Once with spamass-milter, which is a milter that uses spamc to
integrate SpamAssassin with Postfix. And once with a self-made
spamfilter.sh script that also uses spamc but pipes into
/usr/sbin/sendmail?

It is not obvious what the intention is there so I recommend to clean
this up first.

Re: handle_user and connect to spamd failed [ In reply to ]

uhlar at fantomas

Oct 18, 2021, 3:20 AM

Post #3 of 11 (1483 views)

Permalink

On 18.10.21 10:53, Linkcheck wrote:
>I am setting up a new postfix/dovecot/spamassassin server to replace
>an outdated one. It works correctly but I have two outstanding
>warnings in the logs that I would like to eliminate if possible. I
>have trawled through dozens of postings online but found no resolution
>for either warning. The server is debian 10 buster.
>
>1) "spamd: handle_user (userdir) unable to find user: 'dave'"
>Postfix is using dovecot to resolve users but 'dave' is not a user.

spamd by default tries to find recipients' home directories and user
preferences in them.
try passing following option to spamd:

-x, --nouser-config, --user-config

>The processing completes without it but I don't know what the process
>is trying to do with the user that causes this message, nor how to
>resolve it. I also have this on the old server and was advised to
>ignore it but this seems ill-advised: if there is a message it's there
>for a purpose or is in error.
>
>2) "spamc: connect to spamd on ::1 failed, retrying (#1 of 3):
>Connection refused"
>This warning appears on receipt of a message. I do not have this
>message on the old server. I believe spamc is trying to connect to
>spamd using ipv6, which is turned off in postfix and, as far as I can
>determine, in spamassassin as well.
>
>The options in /etc/default/spamassassin are:
>OPTIONS="--create-prefs -4 --max-children 5 --helper-home-dir
>/var/lib/spamassassin -u debian-spamd"

invalid --helper-home-dir option:

-H directory, --helper-home-dir=directory

>I have tried adding options to make:
>OPTIONS="--create-prefs -4 --ipv4-only --max-children 5
>--helper-home-dir /var/lib/spamassassin -u debian-spamd -d 127.0.0.1"

spamd (server) options may not help with spamc. Especially not when you
instruct spamd to listen on ipve address and spamc tries to connect ipv6
address

>This made no difference. I also have /etc/default/spamass-milter with
>the options:
>OPTIONS="-u spamass-milter -i 127.0.0.1 -4"

>In postfix master.cf I have:
>spamfilter unix - n n - - pipe
> flags=Rq user=spamd argv=/usr/bin/spamfilter.sh -oi -f ${sender}
>${recipient}

you don't seem yo use spamass-milter, you should probably.

>with /usr/bin/spamfilter.sh containing:
>SENDMAIL=/usr/sbin/sendmail
>SPAMASSASSIN=/usr/bin/spamc
>MAX_MESSAGE_SIZE="-s 8000000"
>logger <<<"Spam filter piping to SpamAssassin, then to: $SENDMAIL $@"
>${SPAMASSASSIN} ${MAX_MESSAGE_SIZE} | ${SENDMAIL} "$@"
>exit $?

from spamc man page:

-d host[,host2], --dest=host[,host2]
In TCP/IP mode, connect to spamd server on given host (default:
localhost). Several hosts can be specified if separated by commas.

obviously "localhost" resolves to ::1 where spamd does not listen.
make spamd listen on ::1 or instruct spamd to connect to 127.0.0.1

--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Atheism is a non-prophet organization.

Re: handle_user and connect to spamd failed [ In reply to ]

spamassassin at linkcheck

Oct 18, 2021, 6:23 AM

Post #4 of 11 (1483 views)

Permalink

The problem with browsing online for an answer is that everyone seems to
have a different solution. Obviously, I've managed to combine two
solutions. Thanks for the pointer!

I've tried both types independantly now and opted for the milter, since
the other one does not use any of my own regex filters.

--
Dave Stiles

Re: handle_user and connect to spamd failed [ In reply to ]

spamassassin at linkcheck

Oct 18, 2021, 6:47 AM

Post #5 of 11 (1483 views)

Permalink

On 18/10/2021 11:20 am, Matus UHLAR - fantomas wrote:
> spamd by default tries to find recipients' home directories and user
> preferences in them. try passing following option to spamd:
>
> -x, --nouser-config, --user-config

Thanks. Where would I actually add that? Which file / command?

> -H directory, --helper-home-dir=directory

Is that the literal 'directory'? I took that to mean an actual directory.

> you don't seem yo use spamass-milter

See my reply above to David.

> instruct spamd to connect to 127.0.0.1

Sorry, I'm not sure where to do that. I've tried as noted in the OP; I
can't find anywhere else (remembering I've dropped spamfilter.sh).

--
Dave Stiles

Re: handle_user and connect to spamd failed [ In reply to ]

gdt at lexort

Oct 18, 2021, 7:27 AM

Post #6 of 11 (1483 views)

Permalink

Linkcheck <spamassassin@linkcheck.co.uk> writes:

>> instruct spamd to connect to 127.0.0.1
>
> Sorry, I'm not sure where to do that. I've tried as noted in the OP; I
> can't find anywhere else (remembering I've dropped spamfilter.sh).

I'm fuzzy on the details but hope this helps.

What's going on is basically

spamd might listen on ::1 an 127.0.0.1, or just 127.0.0.1. Sometimes
things listen on a socket which can get both. Use lsof, fstat,
etc. to see what spamd is listening on.

spamc probably does normal name resolution for localhost and gets ::1
and 127.0.0.1 in that order and tries them. If the connect to ::1
fails, it moves on. No harm done except for a log line and a few
wasted cycles

so options are:

make spamd listen on ::1 as well as 127.0.0.1. This is arguably the
right fix, and it shoudl happen out of the box.

tell spamc to use 127.0.0.1 when connecting to spamd, and you might
have to do this inside spamass-milter config or code

On my NetBSD 9 amd64 system with SA 3.4.6, spamd has two sockets open,
one on ::1 and one on 127.0.0.1. The logs show that ::1 is being used.

Re: handle_user and connect to spamd failed [ In reply to ]

dbfunk at engineering

Oct 18, 2021, 7:27 AM

Post #7 of 11 (1483 views)

Permalink

On Mon, 18 Oct 2021, Linkcheck wrote:

> On 18/10/2021 11:20 am, Matus UHLAR - fantomas wrote:
>> spamd by default tries to find recipients' home directories and user
>> preferences in them. try passing following option to spamd:
>>
>> -x, --nouser-config, --user-config
>
> Thanks. Where would I actually add that? Which file / command?

Those options need to get used in your spamd startup arguements.
They go in the same place you've got things like --max-children.
But if you're going that nouserconfig route, omit the --create-prefs option.

>
>> -H directory, --helper-home-dir=directory
>
> Is that the literal 'directory'? I took that to mean an actual directory.

Matus is saying that your '--helper-home-dir' option syntax in your spamd
settings is wrong. You say that you have those set to:

> OPTIONS="--create-prefs -4 --max-children 5 --helper-home-dir /var/lib/spamassassin -u debian-spamd"

Mattus is saying that it should be:

> OPTIONS="--create-prefs -4 --max-children 5 --helper-home-dir=/var/lib/spamassassin -u debian-spamd"

Or:

> OPTIONS="--create-prefs -4 --max-children 5 -H /var/lib/spamassassin -u debian-spamd"

IE the '--helper-home-dir' option needs an '=' with no spaces, or use the -H

--
Dave Funk University of Iowa
<dbfunk (at) engineering.uiowa.edu> College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center, 103 S Capitol St.
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{

Re: handle_user and connect to spamd failed [ In reply to ]

dbfunk at engineering

Oct 18, 2021, 7:40 AM

Post #8 of 11 (1483 views)

Permalink

On Mon, 18 Oct 2021, Linkcheck wrote:

> On 18/10/2021 11:20 am, Matus UHLAR - fantomas wrote:
>> spamd by default tries to find recipients' home directories and user
>> preferences in them. try passing following option to spamd:
>>
>> instruct spamd to connect to 127.0.0.1
>
> Sorry, I'm not sure where to do that. I've tried as noted in the OP; I can't
> find anywhere else (remembering I've dropped spamfilter.sh).

Actually that timeout error is coming from "spamc". spamass-milter uses spamc
under the hood to connect to spamd. It's spamc that is trying to connect to
"localhost" which contains that IPv6 reference.
Add an option to spamass-milter telling it to pass on to spamc the connect-to
host is 127.0.0.1 not localhost.

IE:
>This made no difference. I also have /etc/default/spamass-milter with the options:
> OPTIONS="-u spamass-milter -i 127.0.0.1 -4"

Add the option "-D 127.0.0.1" in that spamass-milter OPTIONS.

--
Dave Funk University of Iowa
<dbfunk (at) engineering.uiowa.edu> College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center, 103 S Capitol St.
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{

Re: handle_user and connect to spamd failed [ In reply to ]

uhlar at fantomas

Oct 18, 2021, 8:03 AM

Post #9 of 11 (1483 views)

Permalink

>On 18/10/2021 11:20 am, Matus UHLAR - fantomas wrote:
>>spamd by default tries to find recipients' home directories and user
>>preferences in them. try passing following option to spamd:
>>
>> ????? -x, --nouser-config, --user-config

On 18.10.21 14:47, Linkcheck wrote:
>Thanks. Where would I actually add that? Which file / command?

it's spamd option, put it where you configured the rest of spamd options.

>> -H directory, --helper-home-dir=directory
>
>Is that the literal 'directory'? I took that to mean an actual directory.

this is from spamd man page of spamd. I think Dave explained it already.
Note that in some cases you still can have per-user config files, or
per-user config settings (ldap and sql)

-x is useful if your users don't have their own SA prerefences

>> you don't seem yo use spamass-milter
>
>See my reply above to David.

>> instruct spamd to connect to 127.0.0.1

sorry, this had to be spamc.

>Sorry, I'm not sure where to do that. I've tried as noted in the OP; I
>can't find anywhere else (remembering I've dropped spamfilter.sh).

you called spamc from spamfilter.sh co you can forget this too.

--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Enter any 12-digit prime number to continue.

Re: handle_user and connect to spamd failed [ In reply to ]

dbfunk at engineering

Oct 19, 2021, 6:47 PM

Post #10 of 11 (1483 views)

Permalink

On Tue, 19 Oct 2021, Linkcheck wrote:

> Ok, thanks, Dave.
>
>> '--helper-home-dir' option needs an '='
>
> Also, --max-children?
>
> I have been playing with options based on suggestions here. I now have the
> spamassassin options as:
>
> OPTIONS="--nouser-config -4 -i 127.0.0.1 --max-children=5
> --helper-home-dir=/var/lib/spamassassin -u debian-spamd"
>
> and the spamass-milter options:
>
> OPTIONS="-u spamass-milter -- -d 127.0.0.1"
>
> Once I remembered that spamass-milter also needed to be restarted, along with
> spamassassin and postfix, I made more progress. :(
>
> That has fixed both warnings but the warning message "Could not retrieve
> sendmail macro 'i'" has returned; thought I'd got rid of that one for good. I
> tried adding 'i' to the postfix milter_connect_macros but no difference. I've
> never discovered what that macro is supposed to be nor whence/how it derives.
>
> Thanks to everyone who has contributed to this thread. If someone could round
> it off with the i macro solution that should be it.

spamass-milter wants the 'i' macro in both the milter_mail_macros and
milter_rcpt_macros postfix config parameters.
Putting it in the milter_connect_macros doesn't do any good, that's not where
spamass-milter looks for it.
(at least in the version 0.3.2 code that I looked at, YMMV version wise, grep
the Source Luke).

The 'i' macro is supposed to be the message queue-id value.

--
Dave Funk University of Iowa
<dbfunk (at) engineering.uiowa.edu> College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center, 103 S Capitol St.
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{

Re: handle_user and connect to spamd failed [ In reply to ]

spamassassin at linkcheck

Oct 20, 2021, 3:27 AM

Post #11 of 11 (1483 views)

Permalink

Brilliant! Thank you very much, David. No warnings, no errors now.

--
Dave Stiles

Mailing List Archive

Mailing List Archive

Attached Files: