Suggested addition to PEP 8 for context managers

classic Classic list List threaded Threaded
30 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Suggested addition to PEP 8 for context managers

Raymond Hettinger-4
We should publish some advice on creating content managers.

Context managers are a general purpose tool but have a primary
use case of creating and releasing resources.  This creates an
expectation that that is what the context managers are doing unless
they explicitly say otherwise.

For example in the following calls, the content managers are responsible
for acquiring and releasing a resource.  This is a good and clean design:

    with open(filename) as f: ...      # will release the file resource when done

    with lock:                                     # will acquire and release the lock.

However, in the case that someone wants to create a context manager that does
something other than acquiring and releasing resources, then they should
create a separate content manager so that the behavior will be named.

In other words, if the behavior is going to be the common and expected case,
it is okay to add __enter__ and __exit__ to existing classes (as was done
for locks and files).  However, if the behavior is going to do something else,
then the __enter__ and __exit__ methods need to be in a new class or
factory function.

For example, given the typical uses of context managers, I would expect the
following code to automatically close the database connection:

     with sqlite3.connect(filename) as db:
          ...

Instead, the context manager implements a different behavior.  It would
have been better if that behavior had been given a name:

    db = sqlite3.connect(filename)
    with auto_commit_or_rollback(db):
          # do a transaction

Explicit beats implicit whenever the implicit behavior would deviate from expected norms.


Raymond
   
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Suggested addition to PEP 8 for context managers

Chris Angelico
On Mon, Apr 16, 2012 at 3:13 AM, Raymond Hettinger
<[hidden email]> wrote:
> Instead, the context manager implements a different behavior.  It would
> have been better if that behavior had been given a name:
>
>    db = sqlite3.connect(filename)
>    with auto_commit_or_rollback(db):
>          # do a transaction

I agree that it wants a name. If explicitness is the goal, would this
be more suitable?

db = sqlite3.connect(filename)
with db.begin_transaction() as trans:
  # do a transaction

This way, if a database engine supports multiple simultaneous
transactions, the same syntax can be used.

Chris Angelico
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Suggested addition to PEP 8 for context managers

Paul Moore
In reply to this post by Raymond Hettinger-4
On 15 April 2012 18:13, Raymond Hettinger <[hidden email]> wrote:
> We should publish some advice on creating content managers.
>
> Context managers are a general purpose tool but have a primary
> use case of creating and releasing resources.  This creates an
> expectation that that is what the context managers are doing unless
> they explicitly say otherwise.

I'd have said this was unnecessary, but the sqlite example shows it
isn't, so +1 from me.

As a database specialist, the sqlite behaviour you show is completely
non-intuitive :-(

Paul.
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Suggested addition to PEP 8 for context managers

Barry Warsaw
In reply to this post by Raymond Hettinger-4
On Apr 15, 2012, at 01:13 PM, Raymond Hettinger wrote:

>We should publish some advice on creating content managers.

I agree, I'm just not sure PEP 8 is the right place for it.

PEP 8 seems like it is structured more as mechanical guidelines for the look
and feel of code, not so much for the semantic content of the code.  As such,
including best practices for naming context managers would both appear
out-of-place, and possibly get lost in all the whitespace noise :).

Perhaps the contextlib docs are a better place for this?

-Barry
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Suggested addition to PEP 8 for context managers

Nam Nguyen
On Mon, Apr 16, 2012 at 8:30 AM, Barry Warsaw <[hidden email]> wrote:
> On Apr 15, 2012, at 01:13 PM, Raymond Hettinger wrote:
>
>>We should publish some advice on creating content managers.
>
> I agree, I'm just not sure PEP 8 is the right place for it.
>
> PEP 8 seems like it is structured more as mechanical guidelines for the look
> and feel of code, not so much for the semantic content of the code.  As such,

I'd like to piggyback this thread for a situtation to consider in PEP 8.

PEP 8 suggests no extra spaces after and before square brackets, and
colons. So code like this is appropriate:

a_list[1:3]

But I find it less readable in the case of:

a_list[pos + 1:-1]

The colon is seemingly lost in the right.

Would it be better to read like below?

a_list[pos + 1 : -1]

Any opinion?

Nam
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Suggested addition to PEP 8 for context managers

Matěj Cepl
On 16.4.2012 18:10, Nam Nguyen wrote:
> a_list[pos + 1 : -1]

or other way around

a_list[pos+1:-1]

?

Matěj
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Suggested addition to PEP 8 for context managers

Paul Moore
In reply to this post by Nam Nguyen
On 16 April 2012 17:10, Nam Nguyen <[hidden email]> wrote:

> PEP 8 suggests no extra spaces after and before square brackets, and
> colons. So code like this is appropriate:
>
> a_list[1:3]
>
> But I find it less readable in the case of:
>
> a_list[pos + 1:-1]
>
> The colon is seemingly lost in the right.
>
> Would it be better to read like below?
>
> a_list[pos + 1 : -1]
>
> Any opinion?

It says no space *before* a colon, not after. So the following should
be OK (and is what I'd use):

a_list[pos + 1: -1]

Paul.
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Suggested addition to PEP 8 for context managers

R. David Murray
In reply to this post by Matěj Cepl
On Tue, 17 Apr 2012 08:53:43 +0200, Matej Cepl <[hidden email]> wrote:
> On 16.4.2012 18:10, Nam Nguyen wrote:
> > a_list[pos + 1 : -1]
>
> or other way around
>
> a_list[pos+1:-1]


That's what I always use.  No spaces inside the brackets for me :)

If the expression gets unreadable that way, factor it out.

--David
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Suggested addition to PEP 8 for context managers

Jan Kaliszewski
In reply to this post by Paul Moore
Paul Moore dixit (2012-04-17, 08:14):

> On 16 April 2012 17:10, Nam Nguyen <[hidden email]> wrote:
> > PEP 8 suggests no extra spaces after and before square brackets, and
> > colons. So code like this is appropriate:
> >
> > a_list[1:3]
> >
> > But I find it less readable in the case of:
> >
> > a_list[pos + 1:-1]
> >
> > The colon is seemingly lost in the right.
> >
> > Would it be better to read like below?
> >
> > a_list[pos + 1 : -1]
> >
> > Any opinion?
>
> It says no space *before* a colon, not after. So the following should
> be OK (and is what I'd use):
>
> a_list[pos + 1: -1]

I'd prefer either:

    a_list[pos+1:-1]

or

    a_list[(pos + 1):-1]

Regards.
*j

_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Suggested addition to PEP 8 for context managers

Guido van Rossum
In reply to this post by Paul Moore
On Tue, Apr 17, 2012 at 12:14 AM, Paul Moore <[hidden email]> wrote:

> On 16 April 2012 17:10, Nam Nguyen <[hidden email]> wrote:
>> PEP 8 suggests no extra spaces after and before square brackets, and
>> colons. So code like this is appropriate:
>>
>> a_list[1:3]
>>
>> But I find it less readable in the case of:
>>
>> a_list[pos + 1:-1]
>>
>> The colon is seemingly lost in the right.
>>
>> Would it be better to read like below?
>>
>> a_list[pos + 1 : -1]
>>
>> Any opinion?
>
> It says no space *before* a colon, not after. So the following should
> be OK (and is what I'd use):
>
> a_list[pos + 1: -1]

I hope that's now what it says about slices -- that was meant for dict
displays. For slices it should be symmetrical. In this case I would
remove the spaces around the +, but it's okay to add spaces around the
: too. It does look odd to have an operator that binds tighter (the +)
surrounded by spaces while the operator that binds less tight (:) is
not. (And in this context, : should be considered an operator.)

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Suggested addition to PEP 8 for context managers

Barry Warsaw
In reply to this post by R. David Murray
On Apr 17, 2012, at 08:25 AM, R. David Murray wrote:

>On Tue, 17 Apr 2012 08:53:43 +0200, Matej Cepl <[hidden email]> wrote:
>> On 16.4.2012 18:10, Nam Nguyen wrote:
>> > a_list[pos + 1 : -1]
>>
>> or other way around
>>
>> a_list[pos+1:-1]
>
>
>That's what I always use.  No spaces inside the brackets for me :)
>
>If the expression gets unreadable that way, factor it out.

+1

-Barry
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Suggested addition to PEP 8 for context managers

Hrvoje Niksic-2
In reply to this post by Guido van Rossum
On 04/17/2012 04:21 PM, Guido van Rossum wrote:
> I hope that's now what it says about slices -- that was meant for dict
> displays. For slices it should be symmetrical. In this case I would
> remove the spaces around the +, but it's okay to add spaces around the
> : too. It does look odd to have an operator that binds tighter (the +)
> surrounded by spaces while the operator that binds less tight (:) is
> not.

The same oddity occurs with expressions in kwargs calls:

func(pos1, pos2, keyword=foo + bar)

I find myself wanting to add parentheses arround the + to make the code
clearer.
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Suggested addition to PEP 8 for context managers

Floris Bruynooghe-3
In reply to this post by Barry Warsaw
On 17 April 2012 16:36, Barry Warsaw <[hidden email]> wrote:

> On Apr 17, 2012, at 08:25 AM, R. David Murray wrote:
>
>>On Tue, 17 Apr 2012 08:53:43 +0200, Matej Cepl <[hidden email]> wrote:
>>> On 16.4.2012 18:10, Nam Nguyen wrote:
>>> > a_list[pos + 1 : -1]
>>>
>>> or other way around
>>>
>>> a_list[pos+1:-1]
>>
>>
>>That's what I always use.  No spaces inside the brackets for me :)
>>
>>If the expression gets unreadable that way, factor it out.

Ditto here.

And since this is OT by now, one of the other pep8 annoyances I
have[0] is the blanket whitespace around arithmetic operators,
including **.  To me the first just looks ugly:

>>> 1024 ** 2
>>> 1024**2

Certainly when the expressions are larger.

Regards,
Floris

--
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Suggested addition to PEP 8 for context managers

Guido van Rossum
On Wed, Apr 18, 2012 at 1:29 AM, Floris Bruynooghe <[hidden email]> wrote:
> And since this is OT by now, one of the other pep8 annoyances I
> have[0] is the blanket whitespace around arithmetic operators,
> including **.  To me the first just looks ugly:
>
>>>> 1024 ** 2
>>>> 1024**2
>
> Certainly when the expressions are larger.

I don't believe PEP 8 requires whitespace around all binary operators.
Where do you read that?

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Suggested addition to PEP 8 for context managers

Chris Angelico
On Thu, Apr 19, 2012 at 12:47 AM, Guido van Rossum <[hidden email]> wrote:
> I don't believe PEP 8 requires whitespace around all binary operators.
> Where do you read that?

Quoting from http://www.python.org/dev/peps/pep-0008/#other-recommendations
(with elision):

Use spaces around arithmetic operators:
   No:
      i=i+1
      submitted +=1
      x = x*2 - 1
      hypot2 = x*x + y*y
      c = (a+b) * (a-b)

End quote.

In my code, whether Python or any other language, I tend to follow the
principle that whitespace is completely optional in these expressions,
but if spaces surround any operator, they should (generally) also
surround all operators of lower precedence in the same expression. So
I would quite happily accept all of the expressions above (except
'submitted', which is inconsistent), but would prefer not to see
something like:

c=(a + b)*(a - b)

which is also forbidden by PEP 8.

ChrisA
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Suggested addition to PEP 8 for context managers

Georg Brandl-2
On 18.04.2012 17:47, Chris Angelico wrote:

> On Thu, Apr 19, 2012 at 12:47 AM, Guido van Rossum<[hidden email]>  wrote:
>>  I don't believe PEP 8 requires whitespace around all binary operators.
>>  Where do you read that?
>
> Quoting from http://www.python.org/dev/peps/pep-0008/#other-recommendations
> (with elision):
>
> Use spaces around arithmetic operators:
>     No:
>        i=i+1
>        submitted +=1
>        x = x*2 - 1
>        hypot2 = x*x + y*y
>        c = (a+b) * (a-b)
>
> End quote.

I agree that this could be reworded.  Especially when the operands are
as short as in the examples, the last three "No"s read better to me than
the "Yes" entries.  In this case, spacing serves for visually grouping
expressions by precedence, which otherwise could also be indicated by
(semantically unnecessary) parens.

But in all cases discussed here, PEP8 should not be seen as a law.
Its second section ("A Foolish Consistency is the Hobgoblin of Little
Minds") is quite prominent for a reason.

Georg

_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Suggested addition to PEP 8 for context managers

Guido van Rossum
On Wed, Apr 18, 2012 at 9:25 AM, Georg Brandl <[hidden email]> wrote:

> On 18.04.2012 17:47, Chris Angelico wrote:
>>
>> On Thu, Apr 19, 2012 at 12:47 AM, Guido van Rossum<[hidden email]>
>>  wrote:
>>>
>>>  I don't believe PEP 8 requires whitespace around all binary operators.
>>>  Where do you read that?
>>
>>
>> Quoting from
>> http://www.python.org/dev/peps/pep-0008/#other-recommendations
>> (with elision):
>>
>> Use spaces around arithmetic operators:
>>    No:
>>       i=i+1
>>       submitted +=1
>>       x = x*2 - 1
>>       hypot2 = x*x + y*y
>>       c = (a+b) * (a-b)
>>
>> End quote.
>
>
> I agree that this could be reworded.  Especially when the operands are
> as short as in the examples, the last three "No"s read better to me than
> the "Yes" entries.  In this case, spacing serves for visually grouping
> expressions by precedence, which otherwise could also be indicated by
> (semantically unnecessary) parens.

Indeed. I don't know who put that in, it wasn't me.

> But in all cases discussed here, PEP8 should not be seen as a law.
> Its second section ("A Foolish Consistency is the Hobgoblin of Little
> Minds") is quite prominent for a reason.

I think whoever put that blanket rule in the PEP fell prey to this.

Let's change this to something more reasonable, e.g.

"""
If operators with different priorities are used, consider adding
whitespace around the operators with the lowest priority(ies). This is
very much to taste, however, never use more than one space, and always
have the same amount of whitespace on both sides of a binary operator.
"""

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Suggested addition to PEP 8 for context managers

Steven D'Aprano-8
In reply to this post by Hrvoje Niksic-2
Hrvoje Niksic wrote:

> The same oddity occurs with expressions in kwargs calls:
>
> func(pos1, pos2, keyword=foo + bar)
>
> I find myself wanting to add parentheses arround the + to make the code
> clearer.

Then why don't you?

In the above example, spaces around the + are not only optional but
discouraged, this would be preferred:

func(pos1, pos2, keyword=foo+bar)

but if you insist on using spaces (perhaps because it is part of a larger
expression) just use parentheses.

func(pos1, pos2, keyword=(foo*spam*ham*eggs + bar/spam**cheese))


Strictly speaking they're not needed, but if they make it easier to read (and
I think they do) then why would you not use them?



--
Steven
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Suggested addition to PEP 8 for context managers

Eli Bendersky
In reply to this post by Guido van Rossum
>>> Quoting from
>>> http://www.python.org/dev/peps/pep-0008/#other-recommendations
>>> (with elision):
>>>
>>> Use spaces around arithmetic operators:
>>>    No:
>>>       i=i+1
>>>       submitted +=1
>>>       x = x*2 - 1
>>>       hypot2 = x*x + y*y
>>>       c = (a+b) * (a-b)
>>>
>>> End quote.
>>
>>
>> I agree that this could be reworded.  Especially when the operands are
>> as short as in the examples, the last three "No"s read better to me than
>> the "Yes" entries.  In this case, spacing serves for visually grouping
>> expressions by precedence, which otherwise could also be indicated by
>> (semantically unnecessary) parens.
>
> Indeed. I don't know who put that in, it wasn't me.
>
>> But in all cases discussed here, PEP8 should not be seen as a law.
>> Its second section ("A Foolish Consistency is the Hobgoblin of Little
>> Minds") is quite prominent for a reason.
>
> I think whoever put that blanket rule in the PEP fell prey to this.
>
> Let's change this to something more reasonable, e.g.
>
> """
> If operators with different priorities are used, consider adding
> whitespace around the operators with the lowest priority(ies). This is
> very much to taste, however, never use more than one space, and always
> have the same amount of whitespace on both sides of a binary operator.
> """

+1, a very welcome change to a piece of PEP8 I've always felt
uncomfortable with. Tiny nitpick: I'd just replace the comma following
"however" with a period or semicolon.

Eli
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Suggested addition to PEP 8 for context managers

Chris Angelico
On Thu, Apr 19, 2012 at 1:06 PM, Eli Bendersky <[hidden email]> wrote:

> (quoting GvR)
>> Let's change this to something more reasonable, e.g.
>>
>> """
>> If operators with different priorities are used, consider adding
>> whitespace around the operators with the lowest priority(ies). This is
>> very much to taste, however, never use more than one space, and always
>> have the same amount of whitespace on both sides of a binary operator.
>> """
>
> +1, a very welcome change to a piece of PEP8 I've always felt
> uncomfortable with. Tiny nitpick: I'd just replace the comma following
> "however" with a period or semicolon.

Following or preceding? Either works, but there's a slight shift of
meaning depending on which punctuation gets the upgrade. What was the
original intent of the paragraph?

Chris Angelico
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
12