Mailing List Archive: zLOG warnings and testrunner

zLOG warnings and testrunner

Sep 26, 2001, 9:35 PM

Post #1 of 18 (11377 views)

There of the test failures in the Zope suite are caused by warnings
logged by FileStorage. Two of the three are tests that I added
recently; they verify that a storage can be loaded even if the index
file is corrupted.

I don't want these tests to fail, because the code works. I'm not
sure whether I should change the log level or if there is something
wrong with the test framework.

The current FileStorage code logs a warning/problem `if the index file
is corrupted. I chose warning because the description of it in
zLOG/__init__.py seemed to match: "This isn't causing any immediate
problems, but deserves attention." If the index file is corrupted,
the FileStorage can continue, but the operator ought to know that a
file was corrupted and check to make sure that there isn't some more
serious problem.

Some specific questions:

Why does log messages cause tests to fail?

When should I use PROBLEM and higher severity levels with zLOG?

Should I use a different severity for the specific corrupted index
file messages?

Jeremy

Re: zLOG warnings and testrunner [ In reply to ]

Sep 26, 2001, 9:43 PM

Post #2 of 18 (11292 views)

It looks like two of the failures in the SearchIndex tests are also
caused by logging errors that turn into test failures. It looks like
the tests are expecting to continue despite the logged error.

Jeremy

Re: zLOG warnings and testrunner [ In reply to ]

andreas at zope

Sep 27, 2001, 5:50 AM

Post #3 of 18 (11295 views)

----- Original Message -----
From: "Jeremy Hylton" <jeremy@zope.com>
To: <evan@zope.com>; <zope-coders@zope.org>
Sent: Thursday, September 27, 2001 00:43
Subject: Re: [Zope-Coders] zLOG warnings and testrunner

> It looks like two of the failures in the SearchIndex tests are also
> caused by logging errors that turn into test failures. It looks like
> the tests are expecting to continue despite the logged error.

All stuff in lib/python/SearchIndex should be regarded as deprecated.
All modules are refactored inside Products/PluginIndexes. We keep
SearchIndex
only for backward compat. for third-party products. Maybe we should use
the warnings module to inform the users that the old SearchIndex stuff will
go
away someday ?

Andreas

Re: zLOG warnings and testrunner [ In reply to ]

chrism at digicool

Sep 27, 2001, 6:29 AM

Post #4 of 18 (11288 views)

Jeremy, you might try setting a STUPID_LOG_FILE envvar (no Im not
kidding) to a file, rerun the tests, and see what happens. I have no
idea why pyunit is choking on Zope log calls, especially because by
default nothing actually makes its way out of Zope without
STUPID_LOG_FILE set.

Jeremy Hylton wrote:
> There of the test failures in the Zope suite are caused by warnings
> logged by FileStorage. Two of the three are tests that I added
> recently; they verify that a storage can be loaded even if the index
> file is corrupted.
>
> I don't want these tests to fail, because the code works. I'm not
> sure whether I should change the log level or if there is something
> wrong with the test framework.
>
> The current FileStorage code logs a warning/problem `if the index file
> is corrupted. I chose warning because the description of it in
> zLOG/__init__.py seemed to match: "This isn't causing any immediate
> problems, but deserves attention." If the index file is corrupted,
> the FileStorage can continue, but the operator ought to know that a
> file was corrupted and check to make sure that there isn't some more
> serious problem.
>
> Some specific questions:
>
> Why does log messages cause tests to fail?
>
> When should I use PROBLEM and higher severity levels with zLOG?
>
> Should I use a different severity for the specific corrupted index
> file messages?
>
> Jeremy
>
>
>
>
>
>
> _______________________________________________
> Zope-Coders mailing list
> Zope-Coders@zope.org
> http://lists.zope.org/mailman/listinfo/zope-coders
>

--
Chris McDonough Zope Corporation
http://www.zope.org http://www.zope.com
"Killing hundreds of birds with thousands of stones"

Re: zLOG warnings and testrunner [ In reply to ]

chrism at digicool

Sep 27, 2001, 6:33 AM

Post #5 of 18 (11292 views)

> All stuff in lib/python/SearchIndex should be regarded as deprecated.
> All modules are refactored inside Products/PluginIndexes. We keep
> SearchIndex
> only for backward compat. for third-party products. Maybe we should use
> the warnings module to inform the users that the old SearchIndex stuff will
> go
> away someday ?

Is it really backwards compat with 3rd-party products or is it just to
allow non-plugin indexes to continue to work after a Zope upgrade from a
version without them?

Re: zLOG warnings and testrunner [ In reply to ]

andreas at zope

Sep 27, 2001, 6:33 AM

Post #6 of 18 (11295 views)

backwards compatible in the sense that old stuff still works
after an upgrade.

Andreas
----- Original Message -----
From: "Chris McDonough" <chrism@digicool.com>
To: "Andreas Jung" <andreas@zope.com>
Cc: <jeremy@zope.com>; <evan@zope.com>; <zope-coders@zope.org>
Sent: Thursday, September 27, 2001 09:33
Subject: Re: [Zope-Coders] zLOG warnings and testrunner

> > All stuff in lib/python/SearchIndex should be regarded as deprecated.
> > All modules are refactored inside Products/PluginIndexes. We keep
> > SearchIndex
> > only for backward compat. for third-party products. Maybe we should use
> > the warnings module to inform the users that the old SearchIndex stuff
will
> > go
> > away someday ?
>
> Is it really backwards compat with 3rd-party products or is it just to
> allow non-plugin indexes to continue to work after a Zope upgrade from a
> version without them?
>
>

Re: zLOG warnings and testrunner [ In reply to ]

Sep 27, 2001, 6:58 AM

Post #7 of 18 (11288 views)

>>>>> "AJ" == Andreas Jung <andreas@zope.com> writes:

>> It looks like two of the failures in the SearchIndex tests are
>> also caused by logging errors that turn into test failures. It
>> looks like the tests are expecting to continue despite the logged
>> error.

AJ> All stuff in lib/python/SearchIndex should be regarded as
AJ> deprecated. All modules are refactored inside
AJ> Products/PluginIndexes. We keep SearchIndex only for backward
AJ> compat. for third-party products. Maybe we should use the
AJ> warnings module to inform the users that the old SearchIndex
AJ> stuff will go away someday ?

Then we may also want to prevent testrunner from running the
SearchIndex tests. If we've got failing tests and no plans to fix
them, there's not much point in ever running them.

Jeremy

Re: zLOG warnings and testrunner [ In reply to ]

Sep 27, 2001, 7:02 AM

Post #8 of 18 (11280 views)

>>>>> "CMcD" == Chris McDonough <chrism@digicool.com> writes:

CMcD> Jeremy, you might try setting a STUPID_LOG_FILE envvar (no Im
CMcD> not kidding) to a file, rerun the tests, and see what happens.
CMcD> I have no idea why pyunit is choking on Zope log calls,
CMcD> especially because by default nothing actually makes its way
CMcD> out of Zope without STUPID_LOG_FILE set.

The Zope Testing.common module installs a log handler that raises an
AssertionError when a warning or higher is logged. So it's the policy
of the Zope testing framework, but I've no idea why :-).

Jeremy

Re: zLOG warnings and testrunner [ In reply to ]

andreas at zope

Sep 27, 2001, 7:09 AM

Post #9 of 18 (11298 views)

We can remove the tests or just rename the tests directory to
something like "tests_old".

Andreas
----- Original Message -----
From: "Jeremy Hylton" <jeremy@zope.com>
To: "Andreas Jung" <andreas@zope.com>
Cc: <jeremy@zope.com>; <evan@zope.com>; <zope-coders@zope.org>
Sent: Thursday, September 27, 2001 09:58
Subject: Re: [Zope-Coders] zLOG warnings and testrunner

> >>>>> "AJ" == Andreas Jung <andreas@zope.com> writes:
>
> >> It looks like two of the failures in the SearchIndex tests are
> >> also caused by logging errors that turn into test failures. It
> >> looks like the tests are expecting to continue despite the logged
> >> error.
>
> AJ> All stuff in lib/python/SearchIndex should be regarded as
> AJ> deprecated. All modules are refactored inside
> AJ> Products/PluginIndexes. We keep SearchIndex only for backward
> AJ> compat. for third-party products. Maybe we should use the
> AJ> warnings module to inform the users that the old SearchIndex
> AJ> stuff will go away someday ?
>
> Then we may also want to prevent testrunner from running the
> SearchIndex tests. If we've got failing tests and no plans to fix
> them, there's not much point in ever running them.
>
> Jeremy
>
>
>
> _______________________________________________
> Zope-Coders mailing list
> Zope-Coders@zope.org
> http://lists.zope.org/mailman/listinfo/zope-coders
>

Re: zLOG warnings and testrunner [ In reply to ]

Sep 27, 2001, 7:21 AM

Post #10 of 18 (11290 views)

Jeremy Hylton wrote:

> The Zope Testing.common module installs a log handler that raises an
> AssertionError when a warning or higher is logged. So it's the policy
> of the Zope testing framework, but I've no idea why :-).

As with so much in the framework, that's there because several tests
were doing it. There is also a function in there to turn off the log
handler, for tests that expect log messages and don't want to pay
attention to them.

Cheers,

Evan

Re: zLOG warnings and testrunner [ In reply to ]

Sep 27, 2001, 7:23 AM

Post #11 of 18 (11293 views)

I don't think it's a well thought-through policy. Maybe this behavior
should be settable via a function in Testing.common that should not be
invoked by default. Additionally, maybe instead of failing with a silly
uninformative message, it could say it failed because a log message above a
certain threshhold was seen.

----- Original Message -----
From: "Jeremy Hylton" <jeremy@zope.com>
To: <chrism@zope.com>
Cc: <jeremy@zope.com>; <evan@zope.com>; <zope-coders@zope.org>
Sent: Thursday, September 27, 2001 10:02 AM
Subject: Re: [Zope-Coders] zLOG warnings and testrunner

> >>>>> "CMcD" == Chris McDonough <chrism@digicool.com> writes:
>
> CMcD> Jeremy, you might try setting a STUPID_LOG_FILE envvar (no Im
> CMcD> not kidding) to a file, rerun the tests, and see what happens.
> CMcD> I have no idea why pyunit is choking on Zope log calls,
> CMcD> especially because by default nothing actually makes its way
> CMcD> out of Zope without STUPID_LOG_FILE set.
>
> The Zope Testing.common module installs a log handler that raises an
> AssertionError when a warning or higher is logged. So it's the policy
> of the Zope testing framework, but I've no idea why :-).
>
> Jeremy
>
>

Re: zLOG warnings and testrunner [ In reply to ]

Sep 27, 2001, 7:35 AM

Post #12 of 18 (11283 views)

Andreas Jung wrote:
>
> backwards compatible in the sense that old stuff still works
> after an upgrade.

There are *also* third-party products that depend on them.

Perhaps, in some future version, we should:

- Exclude them from regular distributions

- Provide a distrobution of them so people
can install them if they need to.

Jim

--
Jim Fulton mailto:jim@zope.com Python Powered!
CTO (888) 344-4332 http://www.python.org
Zope Corporation http://www.zope.com http://www.zope.org

Re: zLOG warnings and testrunner [ In reply to ]

Sep 27, 2001, 7:38 AM

Post #13 of 18 (11288 views)

Jeremy Hylton wrote:
>
> >>>>> "CMcD" == Chris McDonough <chrism@digicool.com> writes:
>
> CMcD> Jeremy, you might try setting a STUPID_LOG_FILE envvar (no Im
> CMcD> not kidding) to a file, rerun the tests, and see what happens.
> CMcD> I have no idea why pyunit is choking on Zope log calls,
> CMcD> especially because by default nothing actually makes its way
> CMcD> out of Zope without STUPID_LOG_FILE set.
>
> The Zope Testing.common module installs a log handler that raises an
> AssertionError when a warning or higher is logged. So it's the policy
> of the Zope testing framework, but I've no idea why :-).

It's done beause many errors are logged rather than raised
and indicate failure. This is an important and valuable feature.

I suggest that we make the default log/est threashold "ERROR"
and provide an testing API for lowering it.

Jim

--
Jim Fulton mailto:jim@zope.com Python Powered!
CTO (888) 344-4332 http://www.python.org
Zope Corporation http://www.zope.com http://www.zope.org

Re: zLOG warnings and testrunner [ In reply to ]

c.duncan at nlada

Sep 27, 2001, 8:26 AM

Post #14 of 18 (11294 views)

On Thursday 27 September 2001 08:50 am, Andreas Jung allegedly wrote:
> ----- Original Message -----
> From: "Jeremy Hylton" <jeremy@zope.com>
> To: <evan@zope.com>; <zope-coders@zope.org>
> Sent: Thursday, September 27, 2001 00:43
> Subject: Re: [Zope-Coders] zLOG warnings and testrunner
>
> > It looks like two of the failures in the SearchIndex tests are also
> > caused by logging errors that turn into test failures. It looks like
> > the tests are expecting to continue despite the logged error.
>
> All stuff in lib/python/SearchIndex should be regarded as deprecated.
> All modules are refactored inside Products/PluginIndexes. We keep
> SearchIndex
> only for backward compat. for third-party products. Maybe we should use
> the warnings module to inform the users that the old SearchIndex stuff will
> go
> away someday ?
>
> Andreas

I would say yes and do so as a rule. What is the policy for how long
deprecated stuff hangs around? Until the next maj version?

/---------------------------------------------------\
Casey Duncan, Sr. Web Developer
National Legal Aid and Defender Association
c.duncan@nlada.org
\---------------------------------------------------/

Re: zLOG warnings and testrunner [ In reply to ]

Sep 27, 2001, 9:24 AM

Post #15 of 18 (11287 views)

We should have a method available for unit tests that is analogous to
assertRaises(). The method assertLogs() would be used for an
operation where you expect the operation to execute without raising an
exception *and* to log an error message. It would better to be
explicit in the test method about what should log a message and what
shouldn't.

I think we should implement this by producing a zunit module that
extends unittest with whatever Zope specific features we need. The
write our unit tests to import zunit instead of unittest -- and use
zunit.main() instead of unittest.main().

We should be careful not to diverge too much from the pyunit
approach. Eventually, we may be able to push something like zLOG into
the Python standard library and push our zunit changes back into
pyunit.

Jeremy

Re: zLOG warnings and testrunner [ In reply to ]

Sep 27, 2001, 9:48 AM

Post #16 of 18 (11278 views)

> Maybe we should use
> the warnings module to inform the users that the old SearchIndex stuff will
> go
> away someday ?

Yes, please, we need some kind of warnings or something. Folks need some
way to know that code is deprecated. Otherwise they'll never know to
migrate to the new code.

-Amos

--
Amos Latteier amos@zope.com
Zope Corporation http://www.zope.com/

Re: zLOG warnings and testrunner [ In reply to ]

Sep 27, 2001, 9:49 AM

Post #17 of 18 (11287 views)

>>>>> "AL" == Amos Latteier <amos@zope.com> writes:

>> Maybe we should use the warnings module to inform the users that
>> the old SearchIndex stuff will go away someday ?

AL> Yes, please, we need some kind of warnings or something. Folks
AL> need some way to know that code is deprecated. Otherwise they'll
AL> never know to migrate to the new code.

Python 2.1 introduced the warnings module. It should do the trick.

Jeremy

Re: zLOG warnings and testrunner [ In reply to ]

chrisw at nipltd

Oct 3, 2001, 2:13 PM

Post #18 of 18 (11278 views)

> invoked by default. Additionally, maybe instead of failing with a silly
> uninformative message, it could say it failed because a log message above
a
> certain threshhold was seen.

How about even printing out what the log message was?

Chris