Quantcast

socket, _socket, and socket.py

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

socket, _socket, and socket.py

Zachary Gramana
Hi:

I've been working on adapting Mercurial to run on IronPython 2.7, and
ran into http://ironpython.codeplex.com/workitem/26852 which has stopped
me from getting `hg clone` working over SSL.

I noticed that for the ssl module, the IPY team mirrored the CPython
pattern of placing the platform-specific code in the _ssl compiled
module, and then wrapped the platform-independent around it in ssl.py
(almost entirely shared with CPython 2.7).

The socket module, however, does not.  I admit I have a limited
understanding of the code, but at first blush, it appears that adopting
the _socket.cs/socket.py isn't out-of-the-question. Is there a story
behind this, or am I missing something obvious to everyone else?

The immediate benefit would be getting a free implementation of
_socketobject, _dummy, and dup(); it also improves DRY conformance, and
would help to limit behavioral differences with respect to other
implementations.

Thanks!

Regards,
Zack

P.S. Any advice on tackling issue #26852 is very warmly appreciated.
_______________________________________________
Users mailing list
[hidden email]
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: socket, _socket, and socket.py

Curt Hagenlocher
Originally, we weren't allowed to redistribute the Python standard library with IronPython. So it made sense to implement the socket module directly. When IronPython started shipping the standard lib, it could have been changed, but never was. I think it makes sense to follow the CPython pattern.

On Tue, May 17, 2011 at 2:33 PM, Zachary Gramana <[hidden email]> wrote:
Hi:

I've been working on adapting Mercurial to run on IronPython 2.7, and ran into http://ironpython.codeplex.com/workitem/26852 which has stopped me from getting `hg clone` working over SSL.

I noticed that for the ssl module, the IPY team mirrored the CPython pattern of placing the platform-specific code in the _ssl compiled module, and then wrapped the platform-independent around it in ssl.py (almost entirely shared with CPython 2.7).

The socket module, however, does not.  I admit I have a limited understanding of the code, but at first blush, it appears that adopting the _socket.cs/socket.py isn't out-of-the-question. Is there a story behind this, or am I missing something obvious to everyone else?

The immediate benefit would be getting a free implementation of _socketobject, _dummy, and dup(); it also improves DRY conformance, and would help to limit behavioral differences with respect to other implementations.

Thanks!

Regards,
Zack

P.S. Any advice on tackling issue #26852 is very warmly appreciated.
_______________________________________________
Users mailing list
[hidden email]
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com


_______________________________________________
Users mailing list
[hidden email]
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: socket, _socket, and socket.py

Dino Viehland

There is actually more of a story to this though – Cpython’s socket.py relies upon reference counting for handling dup.  See this comment from socket.py:

 

# Wrapper around platform socket objects. This implements

# a platform-independent dup() functionality. The

# implementation currently relies on reference counting

# to close the underlying socket object.

class _socketobject(object):

 

Because IronPython doesn’t have a reference counting GC we cannot simply use socket.py.  So someone would need to patch both IronPython and socket.py so this would work.  For the record it looks like Jython also has a custom socket.py.

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Curt Hagenlocher
Sent: Tuesday, May 17, 2011 2:59 PM
To: Discussion of IronPython
Subject: Re: [IronPython] socket, _socket, and socket.py

 

Originally, we weren't allowed to redistribute the Python standard library with IronPython. So it made sense to implement the socket module directly. When IronPython started shipping the standard lib, it could have been changed, but never was. I think it makes sense to follow the CPython pattern.

On Tue, May 17, 2011 at 2:33 PM, Zachary Gramana <[hidden email]> wrote:

Hi:

I've been working on adapting Mercurial to run on IronPython 2.7, and ran into http://ironpython.codeplex.com/workitem/26852 which has stopped me from getting `hg clone` working over SSL.

I noticed that for the ssl module, the IPY team mirrored the CPython pattern of placing the platform-specific code in the _ssl compiled module, and then wrapped the platform-independent around it in ssl.py (almost entirely shared with CPython 2.7).

The socket module, however, does not.  I admit I have a limited understanding of the code, but at first blush, it appears that adopting the _socket.cs/socket.py isn't out-of-the-question. Is there a story behind this, or am I missing something obvious to everyone else?

The immediate benefit would be getting a free implementation of _socketobject, _dummy, and dup(); it also improves DRY conformance, and would help to limit behavioral differences with respect to other implementations.

Thanks!

Regards,
Zack

P.S. Any advice on tackling issue #26852 is very warmly appreciated.
_______________________________________________
Users mailing list
[hidden email]
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

 


_______________________________________________
Users mailing list
[hidden email]
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: socket, _socket, and socket.py

Zachary Gramana
In reply to this post by Curt Hagenlocher
Thanks Curt.  That's encouraging news to hear.

I've noticed that some Python projects have a dependency on being able
to standard library methods and types.  For example, Mercurial replaces
buffer() with their own fakebuffer(), but ipy socket reads still return
the original buffer.  I ended up patching around their monkey patch,
which, understandably, they weren't too keen on importing back into
their repo. An eventual move to _socket + socket.py wrapper could help
improve compatibility with those projects.

These type of hidden dependencies on vm-specific behaviors seem to the
biggest source of issues I've come across so far (in 2.7, anyway).  I'm
continually amazed at how big of a step forward the team made from 2.6
-> 2.7 with respect to compatibility.


On 5/17/2011 5:58 PM, Curt Hagenlocher wrote:

> Originally, we weren't allowed to redistribute the Python standard
> library with IronPython. So it made sense to implement the socket module
> directly. When IronPython started shipping the standard lib, it could
> have been changed, but never was. I think it makes sense to follow the
> CPython pattern.
>
> On Tue, May 17, 2011 at 2:33 PM, Zachary Gramana
> <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi:
>
>     I've been working on adapting Mercurial to run on IronPython 2.7,
>     and ran into http://ironpython.codeplex.com/workitem/26852 which has
>     stopped me from getting `hg clone` working over SSL.
>
>     I noticed that for the ssl module, the IPY team mirrored the CPython
>     pattern of placing the platform-specific code in the _ssl compiled
>     module, and then wrapped the platform-independent around it in
>     ssl.py (almost entirely shared with CPython 2.7).
>
>     The socket module, however, does not.  I admit I have a limited
>     understanding of the code, but at first blush, it appears that
>     adopting the _socket.cs/socket.py isn't out-of-the-question. Is
>     there a story behind this, or am I missing something obvious to
>     everyone else?
>
>     The immediate benefit would be getting a free implementation of
>     _socketobject, _dummy, and dup(); it also improves DRY conformance,
>     and would help to limit behavioral differences with respect to other
>     implementations.
>
>     Thanks!
>
>     Regards,
>     Zack
>
>     P.S. Any advice on tackling issue #26852 is very warmly appreciated.
>     _______________________________________________
>     Users mailing list
>     [hidden email] <mailto:[hidden email]>
>     http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>
>
>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
_______________________________________________
Users mailing list
[hidden email]
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: socket, _socket, and socket.py

Zachary Gramana
In reply to this post by Dino Viehland
I noticed this, with a little disappointment. I also noticed that they
implemented a sort of ersatz/platform agnostic reference tracking system
in ssl.py.  Seeing the _makefile_refs system got me thinking about the
whole ssl.py/_ssl.pyd paradigm and sockets.cs, since I remembered you
tackled this issue once before (or one very similar to it:
http://lists.ironpython.com/pipermail/users-ironpython.com/2007-May/004946.html).

Perhaps something along the lines of _makefile_refs and
GC.SuppressFinalize might get us closer?

On 5/17/2011 6:06 PM, Dino Viehland wrote:

> There is actually more of a story to this though – Cpython’s socket.py
> relies upon reference counting for handling dup.  See this comment from
> socket.py:
>
> # Wrapper around platform socket objects. This implements
>
> # a platform-independent dup() functionality. The
>
> # implementation currently relies on reference counting
>
> # to close the underlying socket object.
>
> class _socketobject(object):
>
> Because IronPython doesn’t have a reference counting GC we cannot simply
> use socket.py.  So someone would need to patch both IronPython and
> socket.py so this would work.  For the record it looks like Jython also
> has a custom socket.py.
>
_______________________________________________
Users mailing list
[hidden email]
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: socket, _socket, and socket.py

Dino Viehland
I think manual ref count tracking (like _make_filerefs) + defining __del__ on the socket objects would fix it.  But there may be issues w/ that - when we first encountered this I pinged python-dev and I recall Guido saying that there were problems getting anything else working on Windows.  So there may be some oddity that prevents this from working but I don't know what that is off the top of my head.

> -----Original Message-----
> From: [hidden email] [mailto:users-
> [hidden email]] On Behalf Of Zachary Gramana
> Sent: Tuesday, May 17, 2011 3:38 PM
> To: Discussion of IronPython
> Subject: Re: [IronPython] socket, _socket, and socket.py
>
> I noticed this, with a little disappointment. I also noticed that they
> implemented a sort of ersatz/platform agnostic reference tracking system in
> ssl.py.  Seeing the _makefile_refs system got me thinking about the whole
> ssl.py/_ssl.pyd paradigm and sockets.cs, since I remembered you tackled this
> issue once before (or one very similar to it:
> http://lists.ironpython.com/pipermail/users-ironpython.com/2007-
> May/004946.html).
>
> Perhaps something along the lines of _makefile_refs and
> GC.SuppressFinalize might get us closer?
>
> On 5/17/2011 6:06 PM, Dino Viehland wrote:
> > There is actually more of a story to this though - Cpython's socket.py
> > relies upon reference counting for handling dup.  See this comment
> > from
> > socket.py:
> >
> > # Wrapper around platform socket objects. This implements
> >
> > # a platform-independent dup() functionality. The
> >
> > # implementation currently relies on reference counting
> >
> > # to close the underlying socket object.
> >
> > class _socketobject(object):
> >
> > Because IronPython doesn't have a reference counting GC we cannot
> > simply use socket.py.  So someone would need to patch both IronPython
> > and socket.py so this would work.  For the record it looks like Jython
> > also has a custom socket.py.
> >
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
_______________________________________________
Users mailing list
[hidden email]
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: socket, _socket, and socket.py

Curt Hagenlocher
In reply to this post by Dino Viehland
Ah, right, I'd forgotten that -- we would have had to have distributed a modified socket.py, and that was infeasible even after we were delivering the standard library. This feels like a place where IronPython could collaborate on a joint change with Jython and have the result distributed inside the standard library using a test for sys.platform.

On Tue, May 17, 2011 at 3:06 PM, Dino Viehland <[hidden email]> wrote:

There is actually more of a story to this though – Cpython’s socket.py relies upon reference counting for handling dup.  See this comment from socket.py:

 

# Wrapper around platform socket objects. This implements

# a platform-independent dup() functionality. The

# implementation currently relies on reference counting

# to close the underlying socket object.

class _socketobject(object):

 

Because IronPython doesn’t have a reference counting GC we cannot simply use socket.py.  So someone would need to patch both IronPython and socket.py so this would work.  For the record it looks like Jython also has a custom socket.py.

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Curt Hagenlocher
Sent: Tuesday, May 17, 2011 2:59 PM
To: Discussion of IronPython
Subject: Re: [IronPython] socket, _socket, and socket.py

 

Originally, we weren't allowed to redistribute the Python standard library with IronPython. So it made sense to implement the socket module directly. When IronPython started shipping the standard lib, it could have been changed, but never was. I think it makes sense to follow the CPython pattern.

On Tue, May 17, 2011 at 2:33 PM, Zachary Gramana <[hidden email]> wrote:

Hi:

I've been working on adapting Mercurial to run on IronPython 2.7, and ran into http://ironpython.codeplex.com/workitem/26852 which has stopped me from getting `hg clone` working over SSL.

I noticed that for the ssl module, the IPY team mirrored the CPython pattern of placing the platform-specific code in the _ssl compiled module, and then wrapped the platform-independent around it in ssl.py (almost entirely shared with CPython 2.7).

The socket module, however, does not.  I admit I have a limited understanding of the code, but at first blush, it appears that adopting the _socket.cs/socket.py isn't out-of-the-question. Is there a story behind this, or am I missing something obvious to everyone else?

The immediate benefit would be getting a free implementation of _socketobject, _dummy, and dup(); it also improves DRY conformance, and would help to limit behavioral differences with respect to other implementations.

Thanks!

Regards,
Zack

P.S. Any advice on tackling issue #26852 is very warmly appreciated.
_______________________________________________
Users mailing list
[hidden email]
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

 


_______________________________________________
Users mailing list
[hidden email]
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com



_______________________________________________
Users mailing list
[hidden email]
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: socket, _socket, and socket.py

Zachary Gramana
In reply to this post by Dino Viehland
That's good enough for me.  I'm going to give it a go.

On 5/17/2011 6:48 PM, Dino Viehland wrote:

> I think manual ref count tracking (like _make_filerefs) + defining __del__ on the socket objects would fix it.  But there may be issues w/ that - when we first encountered this I pinged python-dev and I recall Guido saying that there were problems getting anything else working on Windows.  So there may be some oddity that prevents this from working but I don't know what that is off the top of my head.
>
>> -----Original Message-----
>> From: [hidden email] [mailto:users-
>> [hidden email]] On Behalf Of Zachary Gramana
>> Sent: Tuesday, May 17, 2011 3:38 PM
>> To: Discussion of IronPython
>> Subject: Re: [IronPython] socket, _socket, and socket.py
>>
>> I noticed this, with a little disappointment. I also noticed that they
>> implemented a sort of ersatz/platform agnostic reference tracking system in
>> ssl.py.  Seeing the _makefile_refs system got me thinking about the whole
>> ssl.py/_ssl.pyd paradigm and sockets.cs, since I remembered you tackled this
>> issue once before (or one very similar to it:
>> http://lists.ironpython.com/pipermail/users-ironpython.com/2007-
>> May/004946.html).
>>
>> Perhaps something along the lines of _makefile_refs and
>> GC.SuppressFinalize might get us closer?
>>
>> On 5/17/2011 6:06 PM, Dino Viehland wrote:
>>> There is actually more of a story to this though - Cpython's socket.py
>>> relies upon reference counting for handling dup.  See this comment
>>> from
>>> socket.py:
>>>
>>> # Wrapper around platform socket objects. This implements
>>>
>>> # a platform-independent dup() functionality. The
>>>
>>> # implementation currently relies on reference counting
>>>
>>> # to close the underlying socket object.
>>>
>>> class _socketobject(object):
>>>
>>> Because IronPython doesn't have a reference counting GC we cannot
>>> simply use socket.py.  So someone would need to patch both IronPython
>>> and socket.py so this would work.  For the record it looks like Jython
>>> also has a custom socket.py.
>>>
>> _______________________________________________
>> Users mailing list
>> [hidden email]
>> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
_______________________________________________
Users mailing list
[hidden email]
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Loading...