C API: Making a context manager

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

C API: Making a context manager

Chris Kaynor
I am currently rewritting a class using the Python C API to improve
performance of it, however I have not been able to find any
documentation about how to make a context manager using the C API.

The code I am working to produce is the following (its a method of a class):

@contextlib.contextmanager
def connected(self, *args, **kwargs):
        connection = self.connect(*args, **kwargs)
        try:
                yield
        finally:
                connection.disconnect()

For this, my first question is: is there any built-in method to make
this type of method in the C API? If not, is there a slot on the type
object I am missing for __enter__ and __exit__, or should just be
defined using the PyMethodDef struct on the class (presumably named
the same as the Python functions)?

Chris
--
http://mail.python.org/mailman/listinfo/python-list
Reply | Threaded
Open this post in threaded view
|

Re: C API: Making a context manager

briancurtin
On Mon, Oct 31, 2011 at 13:34, Chris Kaynor <[hidden email]> wrote:

> I am currently rewritting a class using the Python C API to improve
> performance of it, however I have not been able to find any
> documentation about how to make a context manager using the C API.
>
> The code I am working to produce is the following (its a method of a class):
>
> @contextlib.contextmanager
> def connected(self, *args, **kwargs):
>        connection = self.connect(*args, **kwargs)
>        try:
>                yield
>        finally:
>                connection.disconnect()
>
> For this, my first question is: is there any built-in method to make
> this type of method in the C API? If not, is there a slot on the type
> object I am missing for __enter__ and __exit__, or should just be
> defined using the PyMethodDef struct on the class (presumably named
> the same as the Python functions)?

You'd just add "__enter__" and "__exit__" in the PyMethodDef. If you
have the CPython source, we do it in there in a few places. Off the
top of my head, PC\winreg.c contains at least one class that works as
a context manager (PyHKEY), although there are a few others scattered
around the source.
--
http://mail.python.org/mailman/listinfo/python-list
Reply | Threaded
Open this post in threaded view
|

Re: C API: Making a context manager

Chris Kaynor
On Mon, Oct 31, 2011 at 12:15 PM, Brian Curtin <[hidden email]> wrote:
>
> You'd just add "__enter__" and "__exit__" in the PyMethodDef. If you
> have the CPython source, we do it in there in a few places. Off the
> top of my head, PC\winreg.c contains at least one class that works as
> a context manager (PyHKEY), although there are a few others scattered
> around the source.
>

That is what I figured. I was just hoping there was some helper class
similar to the contextmanager decorator that would make it easier to
use, however at the same time it makes sense that there is not.

Thanks,
Chris
--
http://mail.python.org/mailman/listinfo/python-list
Reply | Threaded
Open this post in threaded view
|

Re: C API: Making a context manager

Stefan Behnel-3
In reply to this post by Chris Kaynor
Chris Kaynor, 31.10.2011 19:34:
> I am currently rewritting a class using the Python C API to improve
> performance of it, however I have not been able to find any
> documentation about how to make a context manager using the C API.

You should take a look at Cython. It makes these things *so* much easier.


> The code I am working to produce is the following (its a method of a class):
>
> @contextlib.contextmanager
> def connected(self, *args, **kwargs):
> connection = self.connect(*args, **kwargs)
> try:
> yield
> finally:
> connection.disconnect()

You can use the above in Cython code unmodified, and it's likely going to
be good enough (depending on what kind of connection we are talking about
here).

In case that's also performance critical, however, it's likely faster to
spell it out as a class, i.e.

     ...
         def connected(self, *args, **kwargs):
             return ConnectionManager(self, args, kwargs)


     cdef class ConnectionManager:
         cdef object args, kwargs, connection, connect

         def __init__(self, connector, args, kwargs):
             self.args, self.kwargs = args, kwargs
             self.connect = connector.connect
         def __enter__(self):
             self.connection = self.connect(*self.args, **self.kwargs)
         def __exit__(self, *exc):
             self.connection.disconnect()
             return True # or False? I always forget which means what

Not that much longer either.

Stefan

--
http://mail.python.org/mailman/listinfo/python-list
Reply | Threaded
Open this post in threaded view
|

Re: C API: Making a context manager

Chris Kaynor
On Tue, Nov 1, 2011 at 8:57 AM, Stefan Behnel <[hidden email]> wrote:
> Chris Kaynor, 31.10.2011 19:34:
>>
>> I am currently rewritting a class using the Python C API to improve
>> performance of it, however I have not been able to find any
>> documentation about how to make a context manager using the C API.
>
> You should take a look at Cython. It makes these things *so* much easier.

Unfortunately, all of the code has to be fully compatible with CPython
2.6 - it needs to function inside of Maya, which has CPython 2.6
embedded, and to which we do not have the source code.

While not all parts of our code base are used inside of Maya, most of
the performance-critical items are to some degree or another.

In this particular case, the connected context manager is not heavily
used (outside of unittests) and itself is not performance critical,
but the much of the rest of the package (and thus the class) it is
part of is.

Chris
--
http://mail.python.org/mailman/listinfo/python-list
Reply | Threaded
Open this post in threaded view
|

Re: C API: Making a context manager

Stefan Behnel-3
Chris Kaynor, 01.11.2011 17:19:

> On Tue, Nov 1, 2011 at 8:57 AM, Stefan Behnel wrote:
>> Chris Kaynor, 31.10.2011 19:34:
>>> I am currently rewritting a class using the Python C API to improve
>>> performance of it, however I have not been able to find any
>>> documentation about how to make a context manager using the C API.
>>
>> You should take a look at Cython. It makes these things *so* much easier.
>
> Unfortunately, all of the code has to be fully compatible with CPython
> 2.6 - it needs to function inside of Maya, which has CPython 2.6
> embedded, and to which we do not have the source code.

This sounds like you're misunderstanding what Cython is. Cython compiles
(and optimises) your Python code into fast C code that uses the C-API (and
that can happily call into external C code etc.). So you get basically the
same (and sometimes better) speed, but without all the C-level hassle and
maintenance issues. The C code that Cython generates is fully compatible
with CPython 2.4 up to the latest 3.3, and that includes 2.6.


> While not all parts of our code base are used inside of Maya, most of
> the performance-critical items are to some degree or another.
>
> In this particular case, the connected context manager is not heavily
> used (outside of unittests) and itself is not performance critical,
> but the much of the rest of the package (and thus the class) it is
> part of is.

In that case, I advise you to leave the context manager code as is, and
just compile the module that you are trying to speed up (and which, IIUC is
currently written in Python) with Cython, then optimise the parts of it
that need more speed by injecting static typing. Here's a quick howto:

http://docs.cython.org/src/quickstart/cythonize.html

Stefan

--
http://mail.python.org/mailman/listinfo/python-list