http://bugzilla.spamassassin.org/show_bug.cgi?id=3513
------- Additional Comments From sidney@sidney.com 2004-06-16 17:01 -------
I'm not sure how it ever seemed to work :-)
Here are various issues related to the problem:
There are two types of tests that use spamd. The tests with names beginning with
"spamd" are testing spamd itself. They should only be run when spamd is being
installed on the system being tested. It does not make sense to run these on a
Windows system that is using SPAMD_SCRIPT to make use of spamd running elsewhere.
Some of the tests with names beginning with spamc require an instance of spamd
running for spamc to talk to. Those can run when spamd is not installed on the
machine being tested as long as the tests make few or no assumptions about any
custom configuration of the system running spamd.
Currently the only systems on which we do not install spamd are Windows.
SATest.pm sets a variable $SKIP_SPAMD_TESTS to true when the test is running on
Windows and there is no SPAMD_SCRIPT environment variable to override that.
One problem is that would cause the spamd* tests to run if SPAMD_SCRIPT is set,
which is not correct. They should always be skipped under Windows.
Another problem, the direct cause of this bug report, is that the spamc tests
are written assuming that the test machine can start and stop spamd. Under
Windows, SPAMD_SCRIPT lets the user of the machine specify a way of starting
spamd, but there is no reason to believe that the stop_spamd() subroutine will
work with it, or even that it canbe made to work.
This is fundamentally broken. If, for example, spamd is runnning on another
computer on the network, what we want to have happen is that the spamc tests
contact that spamd and no attempt is made to start and stop it.
A flaw in the current setup is that I tried to make testing seamlessly include
tests of spamc but I did it by trying to make it work with a fully installed
SpamAssassin existing on the same physical computer under Cygwin. That is not
seamless, and doesn't even really make sense. Why do a full install under Cygwin
just to be able to run a test on a Windows install?
I have two proposals. I'm in favor of the first, but I'll put out the second as
a possiblity for consideration.
Both proposals start with separating the two cases. $SKIP_SPAMD_TESTS would
always be true under Windows and would only affect the spamd* tests.
In the first proposal, $SKIP_SPAMD_TESTS would suppress the spamc tests that
require spamd unless an environment variable indicated the host name and port of
a spamd to use. It is up to the person running the tests to make sure that spamd
is running on that host and port, properly configured for the tests. If
$SKIP_SPAMD_TESTS is true, then start_spamd and stop_spamd return doing nothing.
I think that's the easiest to implement and easiest to understand for the person
installing SpamAssassinon a Windows machine.
The second proposal is to write a test framework daemon for spamc that produces
the proper responses that spamd would for the specific test cases we are using
to test spamc. The advantage is that it would allow running tests of spamc under
Windows without requiring access to spamd running under Cygwin or on another
machine. (On the other hand, the installation will do no good without access to
some spamd when you want to use it). The disadvantage is that someone would have
to write it. I only mention this as an alternative in case somebody thinks it is
the way to go.
I think the first proposal can go into 3.0. It changes only the tests, and it
only affects the test under Windows which currently does not work right anyway
if you try to include tests of spamc.
I'll work up a patch for the first proposal after leaving some time for
discussion here.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
------- Additional Comments From sidney@sidney.com 2004-06-16 17:01 -------
I'm not sure how it ever seemed to work :-)
Here are various issues related to the problem:
There are two types of tests that use spamd. The tests with names beginning with
"spamd" are testing spamd itself. They should only be run when spamd is being
installed on the system being tested. It does not make sense to run these on a
Windows system that is using SPAMD_SCRIPT to make use of spamd running elsewhere.
Some of the tests with names beginning with spamc require an instance of spamd
running for spamc to talk to. Those can run when spamd is not installed on the
machine being tested as long as the tests make few or no assumptions about any
custom configuration of the system running spamd.
Currently the only systems on which we do not install spamd are Windows.
SATest.pm sets a variable $SKIP_SPAMD_TESTS to true when the test is running on
Windows and there is no SPAMD_SCRIPT environment variable to override that.
One problem is that would cause the spamd* tests to run if SPAMD_SCRIPT is set,
which is not correct. They should always be skipped under Windows.
Another problem, the direct cause of this bug report, is that the spamc tests
are written assuming that the test machine can start and stop spamd. Under
Windows, SPAMD_SCRIPT lets the user of the machine specify a way of starting
spamd, but there is no reason to believe that the stop_spamd() subroutine will
work with it, or even that it canbe made to work.
This is fundamentally broken. If, for example, spamd is runnning on another
computer on the network, what we want to have happen is that the spamc tests
contact that spamd and no attempt is made to start and stop it.
A flaw in the current setup is that I tried to make testing seamlessly include
tests of spamc but I did it by trying to make it work with a fully installed
SpamAssassin existing on the same physical computer under Cygwin. That is not
seamless, and doesn't even really make sense. Why do a full install under Cygwin
just to be able to run a test on a Windows install?
I have two proposals. I'm in favor of the first, but I'll put out the second as
a possiblity for consideration.
Both proposals start with separating the two cases. $SKIP_SPAMD_TESTS would
always be true under Windows and would only affect the spamd* tests.
In the first proposal, $SKIP_SPAMD_TESTS would suppress the spamc tests that
require spamd unless an environment variable indicated the host name and port of
a spamd to use. It is up to the person running the tests to make sure that spamd
is running on that host and port, properly configured for the tests. If
$SKIP_SPAMD_TESTS is true, then start_spamd and stop_spamd return doing nothing.
I think that's the easiest to implement and easiest to understand for the person
installing SpamAssassinon a Windows machine.
The second proposal is to write a test framework daemon for spamc that produces
the proper responses that spamd would for the specific test cases we are using
to test spamc. The advantage is that it would allow running tests of spamc under
Windows without requiring access to spamd running under Cygwin or on another
machine. (On the other hand, the installation will do no good without access to
some spamd when you want to use it). The disadvantage is that someone would have
to write it. I only mention this as an alternative in case somebody thinks it is
the way to go.
I think the first proposal can go into 3.0. It changes only the tests, and it
only affects the test under Windows which currently does not work right anyway
if you try to include tests of spamc.
I'll work up a patch for the first proposal after leaving some time for
discussion here.
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.