Using logging in the stdlib and its unit tests

classic Classic list List threaded Threaded
127 messages Options
1234 ... 7
Reply | Threaded
Open this post in threaded view
|

Using logging in the stdlib and its unit tests

Vinay Sajip
This issue was brought to my notice today:

http://bugs.python.org/issue10626

and reference was made in the comments to possible obstacles facing stdlib
maintainers who might wish to use logging in the stdlib and in its unit tests.

>From my perspective and as mentioned in the logging documentation, library code
which uses logging should add a NullHandler instance to any top-level logger,
which will avoid any "No handlers could be found for logger XXX" message if no
logging handlers have been set up. This applies to stdlib code, too, though it
would be good if a logger namespace could be agreed for stdlib usage. (The
logging package itself uses the logger "py.warnings" to redirect warnings to
logging when configured to do so. Perhaps we could standardize on "py.XXX" for
stdlib usage?)

I would suggest that when unit testing, rather than adding StreamHandlers to log
to stderr, that something like TestHandler and Matcher from this post:

http://plumberjack.blogspot.com/2010/09/unit-testing-and-logging.html

This will allow assertion checking of logged messages without resorting to
StringIO, getvalue() etc. If people think it's a good idea, I can add the
TestHandler/Matcher classes to the unit test infrastructure (they wouldn't
become part of the public API, at least until 3.3, but I presume they could be
used in the stdlib unit tests).

On the question of using logging in the stdlib and its unit tests, I would like
to throw this open to python-dev: is there anyone here who wants to use logging
but feels that they can't because of some shortcoming in logging? AFAIK there
should be no obstacles. The preferred approach in stdlib code is as I have
outlined above: add a NullHandler to top-level loggers to avoid
misconfiguration, document the logger names you use, and avoid adding any other
handlers by default. For unit testing, add a TestHandler (or equivalent) to your
top-level logger at the start of the test, make any assertions you need to
regarding what has been logged, remove the handler and close it at the end of
the test. I don't believe there is any inherent conflict or incompatibility
between logger usage in the stdlib, use of logging assertions in unit tests and
use of handlers by application code, but I am happy to have any mistake on my
part pointed out.

>From what I've seen, concurrent.futures adds a StreamHandler which is removed in
the unit test and replaced by StreamHandler pointing to a different stream.
This, I believe, should be changed in line with what I've said above.

Comments?

Regards,

Vinay Sajip

_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Using logging in the stdlib and its unit tests

Antoine Pitrou
On Tue, 7 Dec 2010 20:26:06 +0000 (UTC)
Vinay Sajip <[hidden email]> wrote:
>
> >From my perspective and as mentioned in the logging documentation, library code
> which uses logging should add a NullHandler instance to any top-level logger,
> which will avoid any "No handlers could be found for logger XXX" message if no
> logging handlers have been set up. This applies to stdlib code, too, though it
> would be good if a logger namespace could be agreed for stdlib usage. (The
> logging package itself uses the logger "py.warnings" to redirect warnings to
> logging when configured to do so. Perhaps we could standardize on "py.XXX" for
> stdlib usage?)

I thought "error" and "critical" messages were logged to stderr by
default? Isn't it the case?

If any library defining a logger must also add a NullHandler just in
case, ISTM that complicates a lot the use of logging (and could explain
its impopularity). Both for library writers and application writers,
apparently (since the former will have to add the NullHandler, and the
latter will have to override it so that error messages get printed out
rather than lost).

Regards

Antoine.


_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Using logging in the stdlib and its unit tests

Łukasz Langa-2
Wiadomość napisana przez Antoine Pitrou w dniu 2010-12-07, o godz. 21:50:

If any library defining a logger must also add a NullHandler just in
case, ISTM that complicates a lot the use of logging (and could explain
its impopularity). Both for library writers and application writers,
apparently (since the former will have to add the NullHandler, and the
latter will have to override it so that error messages get printed out
rather than lost).

ISTM that the stdlib is a special case here. If you're writing an application then the "No handlers could be found" message is actually useful because there's hardly any reason no to include one.

If you're writing a library, well, that depends. Part of libraries might want to have their specific default handlers, part of them might want to leave that to the library user (e.g. "No handlers can be found" by default) and part of them might want to silence logs by default.

One way or the other, we should really default to the convenience of application developers. This is currently the case.

-- 
Best regards,
Łukasz Langa
tel. +48 791 080 144


_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Using logging in the stdlib and its unit tests

Antoine Pitrou
On Tue, 7 Dec 2010 22:04:36 +0100
Łukasz Langa <[hidden email]> wrote:

> Wiadomość napisana przez Antoine Pitrou w dniu 2010-12-07, o godz. 21:50:
>
> > If any library defining a logger must also add a NullHandler just in
> > case, ISTM that complicates a lot the use of logging (and could explain
> > its impopularity). Both for library writers and application writers,
> > apparently (since the former will have to add the NullHandler, and the
> > latter will have to override it so that error messages get printed out
> > rather than lost).
>
> ISTM that the stdlib is a special case here. If you're writing an application then the "No handlers could be found" message is actually useful because there's hardly any reason no to include one.

Why do you say that? Not having to add a handler is certainly useful
when you are doing some quick prototyping or simply writing a script
(situations in which you still want to get error messages displayed
properly by the libraries).

> One way or the other, we should really default to the convenience of application developers. This is currently the case.

Why wouldn't there be a default convenience of printing out errors?
It's already the case for the root handler, so why would other handler
be treated differently?

>>> import logging
>>> logging.debug("foo")
>>> logging.error("bar")
ERROR:root:bar

Thanks

Antoine.
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Using logging in the stdlib and its unit tests

Eric V. Smith
In reply to this post by Vinay Sajip
On 12/07/2010 03:26 PM, Vinay Sajip wrote:

> I would suggest that when unit testing, rather than adding StreamHandlers to log
> to stderr, that something like TestHandler and Matcher from this post:
>
> http://plumberjack.blogspot.com/2010/09/unit-testing-and-logging.html
>
> This will allow assertion checking of logged messages without resorting to
> StringIO, getvalue() etc. If people think it's a good idea, I can add the
> TestHandler/Matcher classes to the unit test infrastructure (they wouldn't
> become part of the public API, at least until 3.3, but I presume they could be
> used in the stdlib unit tests).

Wouldn't it make more sense to add these to test.support? I don't think
we make any guarantees about its API being stable, although I have a
faint recollection of that being debated in the past.

Eric.
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Using logging in the stdlib and its unit tests

Łukasz Langa-2
In reply to this post by Antoine Pitrou
Wiadomość napisana przez Antoine Pitrou w dniu 2010-12-07, o godz. 22:19:

If you're writing an application then the "No handlers could be found" message is actually useful because there's hardly any reason no to include one.

Why do you say that? Not having to add a handler is certainly useful
when you are doing some quick prototyping or simply writing a script
(situations in which you still want to get error messages displayed
properly by the libraries).

One way or the other, we should really default to the convenience of application developers. This is currently the case.

Why wouldn't there be a default convenience of printing out errors?
It's already the case for the root handler, so why would other handler
be treated differently?

import logging
logging.debug("foo")
logging.error("bar")
ERROR:root:bar

If you're arguing that instead of writing "No handler", our logging library could just as easily default to a simplistic stderr handler for errors, then I agree. Both the convenience and consistency arguments you provided are convincing. See, that was 3 times a con* in one sentence!

Then again, warning an application developer that some handler is not configured that probably should be, is still valuable IMHO. Unless explicitly silenced.

-- 
Best regards,
Łukasz Langa
tel. +48 791 080 144


_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Using logging in the stdlib and its unit tests

Robert Kern-2
In reply to this post by Vinay Sajip
On 12/7/10 2:26 PM, Vinay Sajip wrote:

> This issue was brought to my notice today:
>
> http://bugs.python.org/issue10626
>
> and reference was made in the comments to possible obstacles facing stdlib
> maintainers who might wish to use logging in the stdlib and in its unit tests.
>
>> From my perspective and as mentioned in the logging documentation, library code
> which uses logging should add a NullHandler instance to any top-level logger,
> which will avoid any "No handlers could be found for logger XXX" message if no
> logging handlers have been set up.

I've done that before in my own library code, then quickly realized that it was
a bad idea. Adding a NullHandler silently prevents logging.basicConfig() from
working. Embedding that kind of restriction into library code (usually by a mere
import!) made users unhappy and added an additional burden on the library
developers who had to make sure to do that everywhere they used logging.

If I had my druthers, I would simply remove the "No handlers could be found for
logger XXX" message. If we always wrote entire applications from the ground up,
it makes a lot of sense. The person that writes the code that issues logs is the
same person that writes the code to configure logging for reading. If you add
logging in one place, you almost certainly want to use it somewhere else and not
setting up logging is probably an error. But when you want to write reusable
libraries, those roles become separated.

As a library author, I would dearly love to just add logging liberally without
placing any additional burden to the users of my library. If my users wants to
read those logs, he will configure logging. If he doesn't, he won't. With the
current behavior, I can't do that. If I add logging, he has to add code just to
silence a message that is meaningless to him (after I get the support emails
asking if things are broken and explain how to silence it). If I add a
NullHandler, I remove the ability for him to use logging.basicConfig(), the
easiest and most straightforward way for him to add logging to his application.

This is only my personal anecdotal experience, but the current behavior has
always wasted my time and never saved any of my time.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Using logging in the stdlib and its unit tests

Brett Cannon-2
In reply to this post by Eric V. Smith
On Tue, Dec 7, 2010 at 13:57, Eric Smith <[hidden email]> wrote:

> On 12/07/2010 03:26 PM, Vinay Sajip wrote:
>>
>> I would suggest that when unit testing, rather than adding StreamHandlers
>> to log
>> to stderr, that something like TestHandler and Matcher from this post:
>>
>> http://plumberjack.blogspot.com/2010/09/unit-testing-and-logging.html
>>
>> This will allow assertion checking of logged messages without resorting to
>> StringIO, getvalue() etc. If people think it's a good idea, I can add the
>> TestHandler/Matcher classes to the unit test infrastructure (they wouldn't
>> become part of the public API, at least until 3.3, but I presume they
>> could be
>> used in the stdlib unit tests).
>
> Wouldn't it make more sense to add these to test.support? I don't think we
> make any guarantees about its API being stable, although I have a faint
> recollection of that being debated in the past.

We make no guarantees. The debate was whether we accidentally painted
ourselves into a corner by documenting the module a tad for core
developer's benefit. The warning has been made much more explicit in
the docs that test.support is not meant for outside usage.
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Using logging in the stdlib and its unit tests

Robert Kern-2
In reply to this post by Robert Kern-2
On 12/7/10 4:59 PM, Robert Kern wrote:

> On 12/7/10 2:26 PM, Vinay Sajip wrote:
>> This issue was brought to my notice today:
>>
>> http://bugs.python.org/issue10626
>>
>> and reference was made in the comments to possible obstacles facing stdlib
>> maintainers who might wish to use logging in the stdlib and in its unit tests.
>>
>>> From my perspective and as mentioned in the logging documentation, library code
>> which uses logging should add a NullHandler instance to any top-level logger,
>> which will avoid any "No handlers could be found for logger XXX" message if no
>> logging handlers have been set up.
>
> I've done that before in my own library code, then quickly realized that it was
> a bad idea. Adding a NullHandler silently prevents logging.basicConfig() from
> working.

Only on the root handler. Forget this bit of my argument (and the statements
that directly follow from it). The rest of my argument should stand on its own,
though.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Using logging in the stdlib and its unit tests

Barry Warsaw
In reply to this post by Robert Kern-2
On Dec 07, 2010, at 04:59 PM, Robert Kern wrote:

>As a library author, I would dearly love to just add logging liberally
>without placing any additional burden to the users of my library. If my users
>wants to read those logs, he will configure logging. If he doesn't, he
>won't. With the current behavior, I can't do that. If I add logging, he has
>to add code just to silence a message that is meaningless to him (after I get
>the support emails asking if things are broken and explain how to silence
>it). If I add a NullHandler, I remove the ability for him to use
>logging.basicConfig(), the easiest and most straightforward way for him to
>add logging to his application.

+1

-Barry

_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%40nabble.com

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Using logging in the stdlib and its unit tests

Vinay Sajip
In reply to this post by Antoine Pitrou
Antoine Pitrou <solipsis <at> pitrou.net> writes:

>
> I thought "error" and "critical" messages were logged to stderr by
> default? Isn't it the case?
>

Only if you call basicConfig() or use the logging.debug(), logging.info(), etc.
module-level convenience functions (which call basicConfig under the hood).

> If any library defining a logger must also add a NullHandler just in
> case, ISTM that complicates a lot the use of logging (and could explain
> its impopularity). Both for library writers and application writers,
> apparently (since the former will have to add the NullHandler, and the
> latter will have to override it so that error messages get printed out
> rather than lost).
>

No - the application developer doesn't have to do anything special. Adding the
NullHandler is only needed for the top-level logger of a library package, so
it's hardly more onerous than doing an import. It doesn't complicate anything,
as far as I can see. It's to be done once per library, and trivial in comparison
with all the work needed to develop, test, and package a library. For the
stdlib, for example, if we were to standardize and say that all stdlib loggers
were to be located below the "py" namespace, then I could put the relevant
NullHandler-adding code in the logging package and no other stdlib maintainer
would have to do anything other than name their stdlib loggers beginning with
"py." (e.g. "py.concurrent.futures") for things to work as expected. If you
don't like "py", then pick "std", "stdlib" or whatever.

When is the NullHandler needed? Only for cases where an application developer
uses a library which does logging under the covers (for those users who might be
interested in logging its operations), but where that application developer
doesn't use logging themselves for that application. In such a case, a library
call which generated a logging event would cause logging to look for handlers,
and if one is not found, print that one-off "no handlers could be found for
logger XXX" message. This message is there because errors should not pass
silently; but in the case of an application developer oblivious of logging who
uses a library which uses logging under the hood, you can't argue that that
developer has made any error. So the library developer, by adding a NullHandler,
can improve things for their users by not forcing them to configure logging
handlers (in order to avoid the message).

#mylib.py
import logging

logger = logging.getLogger(__name__)
logger.addHandler(logging.NullHandler())

def func():
    logger.warning('Something happened!')

#myapp.py
import mylib

mylib.func()

If you run myapp.py, no message is printed. If you comment out the "addHandler"
line and re-run, then you see

No handlers could be found for logger "mylib"

which is the misconfiguration message. With the line commented back in, you as
an application developer get expected behaviour (no spurious messages about
missing handlers). When you decide to use logging in myapp.py, simply insert

import logging

logging.basicConfig()

before calling mylib.func(), and you get

WARNING:mylib:Something happened!

which is what you would expect.

Regards,

Vinay Sajip


_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Using logging in the stdlib and its unit tests

Vinay Sajip
In reply to this post by Robert Kern-2
Robert Kern <robert.kern <at> gmail.com> writes:


> If I had my druthers, I would simply remove the "No handlers could be
> found for logger XXX" message. If we always wrote entire applications
> from the ground up, it makes a lot of sense. The person that writes the
> code that issues logs is the same person that writes the code to configure
> logging for reading. If you add logging in one place, you almost certainly
> want to use it somewhere else and not setting up logging is probably an
> error. But when you want to write reusable  libraries, those roles become
> separated.

Exactly - we do use third-party libraries. When logging was added there was some
debate about whether this one-off message should be printed, and in balance it
was thought better to print this, not least because people would be unfamiliar
with logging (as it was then new) and so be not unlikely to misconfigure it. No
indication of this at all would be double-plus-ungood.

> As a library author, I would dearly love to just add logging liberally
> without placing any additional burden to the users of my library.
> If my users wants to read those logs, he will configure logging. If he
> doesn't, he won't. With the current behavior, I can't do that. If I add
> logging, he has to add code just to silence a message that is meaningless
> to him (after I get the support emails asking if things are broken and
> explain how to silence it). If I add a NullHandler, I remove the ability
> for him to use logging.basicConfig(), the easiest and most straightforward
> way for him to add logging to his application.

You don't remove the ability for him to use basicConfig() - that's another
mistake in your post (you already corrected the other mistake in your
self-response). See my example with mylib.py/myapp.py in another post on this
thread, in response to Antoine.

> This is only my personal anecdotal experience, but the current behavior has
> always wasted my time and never saved any of my time.
>

That's because I think you're doing it wrong, or else I've misunderstood your
use case. NullHandler is the way to go, and doesn't preclude the use of
basicConfig() UNLESS you choose to add the NullHandler to the root logger (not
what you're supposed to do - you're supposed to add it to the top-level logger
*of your library*, not the top-level logger of the logging hierarchy.)

See the mylib/myapp example and tell me which it is - your mistake or mine?

Regards,

Vinay Sajip


_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Using logging in the stdlib and its unit tests

Vinay Sajip
In reply to this post by Barry Warsaw
Barry Warsaw <barry <at> python.org> writes:

>
> On Dec 07, 2010, at 04:59 PM, Robert Kern wrote:
>
> >As a library author, I would dearly love to just add logging liberally
> >without placing any additional burden to the users of my library. If my users
> >wants to read those logs, he will configure logging. If he doesn't, he
> >won't. With the current behavior, I can't do that. If I add logging, he has
> >to add code just to silence a message that is meaningless to him (after I get
> >the support emails asking if things are broken and explain how to silence
> >it). If I add a NullHandler, I remove the ability for him to use
> >logging.basicConfig(), the easiest and most straightforward way for him to
> >add logging to his application.
>
> +1
>

Barry, if you mean +1 as in "I agree this is how it should work", then we're all
agreed. But I think Robert is wrong that NullHandler precludes use of
basicConfig - when NullHandler is added to a library logger (rather than the
root logger), basicConfig() works as expected. See the mylib/myapp example in my
other post on this thread.

Regards,

Vinay Sajip



_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Using logging in the stdlib and its unit tests

Antoine Pitrou
In reply to this post by Vinay Sajip
On Tue, 7 Dec 2010 23:45:39 +0000 (UTC)
Vinay Sajip <[hidden email]> wrote:
> Antoine Pitrou <solipsis <at> pitrou.net> writes:
>
> >
> > I thought "error" and "critical" messages were logged to stderr by
> > default? Isn't it the case?
> >
>
> Only if you call basicConfig() or use the logging.debug(), logging.info(), etc.
> module-level convenience functions (which call basicConfig under the hood).

Why wouldn't it be the default for all logging calls ? Such special
cases don't really make things easy to remember.

> When is the NullHandler needed? Only for cases where an application developer
> uses a library which does logging under the covers (for those users who might be
> interested in logging its operations), but where that application developer
> doesn't use logging themselves for that application.

You seem pretty tied up to the "application developer" situation. There
are cases (scripts, prototyping, etc.) where you certainly want to see
error messages (errors should not pass silently) but don't want to
configure logging for each of the libraries you use.

Having convenient and understandable defaults would go a long way
towards making logging more usable, IMO.

Regards

Antoine.


_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Using logging in the stdlib and its unit tests

Vinay Sajip
In reply to this post by Eric V. Smith
Eric Smith <eric <at> trueblade.com> writes:

 
> Wouldn't it make more sense to add these to test.support? I don't think
> we make any guarantees about its API being stable, although I have a
> faint recollection of that being debated in the past.
>

That's what I mean when I said "unit test infrastructure" - adding to
test.support. Sorry, I should have been more explicit.

Regards,

Vinay Sajip


_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Using logging in the stdlib and its unit tests

Eric V. Smith
On 12/07/2010 07:09 PM, Vinay Sajip wrote:

> Eric Smith<eric<at>  trueblade.com>  writes:
>
>
>> Wouldn't it make more sense to add these to test.support? I don't think
>> we make any guarantees about its API being stable, although I have a
>> faint recollection of that being debated in the past.
>>
>
> That's what I mean when I said "unit test infrastructure" - adding to
> test.support. Sorry, I should have been more explicit.

My bad. I read it as "unittest infrastructure".

Eric.

_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Using logging in the stdlib and its unit tests

Nick Coghlan
In reply to this post by Antoine Pitrou
On Wed, Dec 8, 2010 at 10:09 AM, Antoine Pitrou <[hidden email]> wrote:

> On Tue, 7 Dec 2010 23:45:39 +0000 (UTC)
> Vinay Sajip <[hidden email]> wrote:
>> Antoine Pitrou <solipsis <at> pitrou.net> writes:
>>
>> >
>> > I thought "error" and "critical" messages were logged to stderr by
>> > default? Isn't it the case?
>> >
>>
>> Only if you call basicConfig() or use the logging.debug(), logging.info(), etc.
>> module-level convenience functions (which call basicConfig under the hood).
>
> Why wouldn't it be the default for all logging calls ? Such special
> cases don't really make things easy to remember.

Indeed - I was very surprised to find just now that calling
"logging.warn('Whatever')" is not the same as doing "log =
logging.getLogger(); log.warn('Whatever')".

Adding a NullHandler isn't the right thing to do - the behaviour I
would want for any standard library logging that hasn't been
explicitly configured otherwise is to do what the root logger does
under basicConfig(): debug() and info() get suppressed, warn(),
error(), critical() and exception() go to stderr. It seems to me like
that would be more useful default behaviour in general than emitting a
warning about not finding a handler.

So my suggestion would be:
1. In the absence of a "config" call, make the "no handler" path in
loggers emit messages *as if* basicConfig() has been called (without
actually calling it)
2. Remove the implicit calls to basicConfig() from the module level
convenience function

How *feasible* that idea is to implement, I don't know.

Cheers,
Nick.
--
Nick Coghlan   |   [hidden email]   |   Brisbane, Australia
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Using logging in the stdlib and its unit tests

Vinay Sajip
In reply to this post by Antoine Pitrou
Antoine Pitrou <solipsis <at> pitrou.net> writes:

>
> Why wouldn't it be the default for all logging calls ? Such special
> cases don't really make things easy to remember.
>

One size doesn't fit all. Everything's documented reasonably well. If you use it
often, you remember it. If you use it seldom, you might need to look things up.
 

> You seem pretty tied up to the "application developer" situation. There
> are cases (scripts, prototyping, etc.) where you certainly want to see
> error messages (errors should not pass silently) but don't want to
> configure logging for each of the libraries you use.

You don't have to configure each of the libraries you use. The "application
developer" = "The developer who is not writing a library" = "The person writing
a script, prototyping, etc." I don't mean Application with a capital A. I'm just
using it as a convenient label to mean a library user who is not writing a
library themselves.

 
> Having convenient and understandable defaults would go a long way
> towards making logging more usable, IMO.
>

The defaults are I believe reasonably convenient, or at least not inconvenient:

1. If writing a library, add a NullHandler to the top-level logger of your
library, don't add any other handlers by default (i.e. on import), document the
names of the loggers you use. Otherwise, see the following.

2. If writing a simple script, prototyping etc. i.e. for casual use, use
logging.debug(), logging.info() etc. That's easy enough and convenient enough
for casual users to remember, I hope. All WARNING, ERROR and CRITICAL events in
your code and in that of any libraries you use will be printed to sys.stderr.

3. If slightly less than very casual use (e.g. you want to log to file, or want
a specific message format), call basicConfig(). Just the one extra line to
remember to do, hopefully not too inconvenient. All WARNING, ERROR and CRITICAL
events in your code and in that of any libraries you use will be printed to
sys.stderr, unless you specify a file to log to and/or a different logging
level, in which case what you specified will be used as destination/threshold
for logging messages.

4. For more advanced use, use programmatic configuration, fileConfig or
dictConfig to specify a configuration. Convenience is inversely proportional to
how involved the configuration is.

If you can specify what you think is inconvenient in the above, that would be
helpful.

Regards,

Vinay Sajip

_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Using logging in the stdlib and its unit tests

Barry Warsaw
In reply to this post by Vinay Sajip
On Dec 08, 2010, at 12:01 AM, Vinay Sajip wrote:

>Barry, if you mean +1 as in "I agree this is how it should work", then
>we're all agreed.

Yep, that's what I meant!

-Barry

_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%40nabble.com

signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Using logging in the stdlib and its unit tests

Vinay Sajip
In reply to this post by Nick Coghlan
Nick Coghlan <ncoghlan <at> gmail.com> writes:

> Indeed - I was very surprised to find just now that calling
> "logging.warn('Whatever')" is not the same as doing "log =
> logging.getLogger(); log.warn('Whatever')".

Don't know why you'd be surprised: it's been that way since logging was added to
Python, and the logging.debug() etc. are documented as convenience methods for
casual use in throwaway scripts, interactive sessions etc. The convenience is in
that you don't need to specify a logger (the root logger is used) and that
basicConfig() is called for you.
 

> Adding a NullHandler isn't the right thing to do - the behaviour I
> would want for any standard library logging that hasn't been
> explicitly configured otherwise is to do what the root logger does
> under basicConfig(): debug() and info() get suppressed, warn(),
> error(), critical() and exception() go to stderr. It seems to me like
> that would be more useful default behaviour in general than emitting a
> warning about not finding a handler.
>
> So my suggestion would be:
> 1. In the absence of a "config" call, make the "no handler" path in
> loggers emit messages *as if* basicConfig() has been called (without
> actually calling it)
> 2. Remove the implicit calls to basicConfig() from the module level
> convenience function
>
> How *feasible* that idea is to implement, I don't know.
>

What about "Special cases aren't special enough to break the rules."? Why should
logging from stdlib modules be treated differently from the case of logging from
library modules in general? It could be argued that the behaviour you're
proposing is confusing/inconsistent to some people.

Regards,

Vinay Sajip

_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
1234 ... 7