IronPython and CPython httplib and ssl modules produce IOError

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

IronPython and CPython httplib and ssl modules produce IOError

Lukáš Duběda
Hi there,

I hope someone out there could help me out with this problem.

I have a API written in Python (CPython that is) that relies on
some of the CPython modules, such as ssl and httplib.

The problem is, when I call the API and its methods via IronPython
I get IOError: "System.IO.IOException: Unable to write data to the
transport connection: Cannot access a disposed object."

The weirdest thing is, if I make the call for the second time,
all goes through beautifully. Then again, thrid time errors out, but
fourth goes through just fine. And so on so forth.

Here's the exact script excerpt (with the thorough error output) at
pastebin: http://pastebin.com/TKbKDtPb

I call the sg.find_one() method the first time, which produces the
IOError. Then at the end of the error output, I call it again and
as you can see, it returns the correct object.

Please, could anyone tell me how to deal with this without rewriting
the given API to use .NET objects?

Thanks a lot in advance, cheers,

--
Lukáš Duběda
Director
[T] +420 602 444 164

duber studio(tm)
[M] [hidden email]
[W] http://www.duber.cz

[A] R.A.Dvorského 601, Praha 10
[A] 10900, Czech Republic, Europe
_______________________________________________
Users mailing list
[hidden email]
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Reply | Threaded
Open this post in threaded view
|

Re: IronPython and CPython httplib and ssl modules produce IOError

Zachary Gramana
Lukas,

I'm working on this issue, or at least an issue that generates the same
exception.  I do have a working patch, which I hope to contribute back
after a little more work on it.

Check your API to see if you are making any calls to socket.makefile().
  If you do, and subsequently close the fileobject (but not the socket
object), the underlying .NET Socket will also get closed (and garbage
collected), even though other objects are still referencing it.
However, the Python socket will not be aware of this corrupt state, and
subsequent socket operations will raise the exception.

Not sure why subsequent attempts work for you, other than that perhaps
your API has error handling code that re-create the connection.

Hope this might be helpful.

Best Regards, Zack



On 6/10/2011 7:55 AM, Lukáš Duběda wrote:

> Hi there,
>
> I hope someone out there could help me out with this problem.
>
> I have a API written in Python (CPython that is) that relies on
> some of the CPython modules, such as ssl and httplib.
>
> The problem is, when I call the API and its methods via IronPython
> I get IOError: "System.IO.IOException: Unable to write data to the
> transport connection: Cannot access a disposed object."
>
> The weirdest thing is, if I make the call for the second time,
> all goes through beautifully. Then again, thrid time errors out, but
> fourth goes through just fine. And so on so forth.
>
> Here's the exact script excerpt (with the thorough error output) at
> pastebin: http://pastebin.com/TKbKDtPb
>
> I call the sg.find_one() method the first time, which produces the
> IOError. Then at the end of the error output, I call it again and
> as you can see, it returns the correct object.
>
> Please, could anyone tell me how to deal with this without rewriting
> the given API to use .NET objects?
>
> Thanks a lot in advance, cheers,
>
_______________________________________________
Users mailing list
[hidden email]
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Reply | Threaded
Open this post in threaded view
|

Re: IronPython and CPython httplib and ssl modules produce IOError

Lukáš Duběda
Thank you, Zack,

unfortunately, editing the API I got isn't an option, so the
only workaround I found was to catch the exception and completely
ignore it, which is terrible, but at least it works.

So you're saying that this is actually a "bug" in IronPython
that's being worked at?

Thanks again, much appretiated!

Lukáš Duběda
Director
[T] +420 602 444 164

duber studio(tm)
[M] [hidden email]
[W] http://www.duber.cz

[A] R.A.Dvorského 601, Praha 10
[A] 10900, Czech Republic, Europe

On 10.6.2011 16:32, Zachary Gramana wrote:

> Lukas,
>
> I'm working on this issue, or at least an issue that generates the same
> exception. I do have a working patch, which I hope to contribute back
> after a little more work on it.
>
> Check your API to see if you are making any calls to socket.makefile().
> If you do, and subsequently close the fileobject (but not the socket
> object), the underlying .NET Socket will also get closed (and garbage
> collected), even though other objects are still referencing it. However,
> the Python socket will not be aware of this corrupt state, and
> subsequent socket operations will raise the exception.
>
> Not sure why subsequent attempts work for you, other than that perhaps
> your API has error handling code that re-create the connection.
>
> Hope this might be helpful.
>
> Best Regards, Zack
>
>
>
> On 6/10/2011 7:55 AM, Lukáš Duběda wrote:
>> Hi there,
>>
>> I hope someone out there could help me out with this problem.
>>
>> I have a API written in Python (CPython that is) that relies on
>> some of the CPython modules, such as ssl and httplib.
>>
>> The problem is, when I call the API and its methods via IronPython
>> I get IOError: "System.IO.IOException: Unable to write data to the
>> transport connection: Cannot access a disposed object."
>>
>> The weirdest thing is, if I make the call for the second time,
>> all goes through beautifully. Then again, thrid time errors out, but
>> fourth goes through just fine. And so on so forth.
>>
>> Here's the exact script excerpt (with the thorough error output) at
>> pastebin: http://pastebin.com/TKbKDtPb
>>
>> I call the sg.find_one() method the first time, which produces the
>> IOError. Then at the end of the error output, I call it again and
>> as you can see, it returns the correct object.
>>
>> Please, could anyone tell me how to deal with this without rewriting
>> the given API to use .NET objects?
>>
>> Thanks a lot in advance, cheers,
>>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
_______________________________________________
Users mailing list
[hidden email]
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Reply | Threaded
Open this post in threaded view
|

Re: IronPython and CPython httplib and ssl modules produce IOError

Zachary Gramana
I can't say for certain that issue you are experiencing is caused by the
bug I'm working on, but it looks very likely.

I'll let you know when I submit the patch back to the project.

On 6/10/2011 1:49 PM, Lukáš Duběda wrote:

> Thank you, Zack,
>
> unfortunately, editing the API I got isn't an option, so the
> only workaround I found was to catch the exception and completely
> ignore it, which is terrible, but at least it works.
>
> So you're saying that this is actually a "bug" in IronPython
> that's being worked at?
>
> Thanks again, much appretiated!
>
> Lukáš Duběda
> Director
> [T] +420 602 444 164
>
> duber studio(tm)
> [M] [hidden email]
> [W] http://www.duber.cz
>
> [A] R.A.Dvorského 601, Praha 10
> [A] 10900, Czech Republic, Europe
>
> On 10.6.2011 16:32, Zachary Gramana wrote:
>> Lukas,
>>
>> I'm working on this issue, or at least an issue that generates the same
>> exception. I do have a working patch, which I hope to contribute back
>> after a little more work on it.
>>
>> Check your API to see if you are making any calls to socket.makefile().
>> If you do, and subsequently close the fileobject (but not the socket
>> object), the underlying .NET Socket will also get closed (and garbage
>> collected), even though other objects are still referencing it. However,
>> the Python socket will not be aware of this corrupt state, and
>> subsequent socket operations will raise the exception.
>>
>> Not sure why subsequent attempts work for you, other than that perhaps
>> your API has error handling code that re-create the connection.
>>
>> Hope this might be helpful.
>>
>> Best Regards, Zack
>>
>>
>>
>> On 6/10/2011 7:55 AM, Lukáš Duběda wrote:
>>> Hi there,
>>>
>>> I hope someone out there could help me out with this problem.
>>>
>>> I have a API written in Python (CPython that is) that relies on
>>> some of the CPython modules, such as ssl and httplib.
>>>
>>> The problem is, when I call the API and its methods via IronPython
>>> I get IOError: "System.IO.IOException: Unable to write data to the
>>> transport connection: Cannot access a disposed object."
>>>
>>> The weirdest thing is, if I make the call for the second time,
>>> all goes through beautifully. Then again, thrid time errors out, but
>>> fourth goes through just fine. And so on so forth.
>>>
>>> Here's the exact script excerpt (with the thorough error output) at
>>> pastebin: http://pastebin.com/TKbKDtPb
>>>
>>> I call the sg.find_one() method the first time, which produces the
>>> IOError. Then at the end of the error output, I call it again and
>>> as you can see, it returns the correct object.
>>>
>>> Please, could anyone tell me how to deal with this without rewriting
>>> the given API to use .NET objects?
>>>
>>> Thanks a lot in advance, cheers,
>>>
>> _______________________________________________
>> Users mailing list
>> [hidden email]
>> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
_______________________________________________
Users mailing list
[hidden email]
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Reply | Threaded
Open this post in threaded view
|

Re: IronPython and CPython httplib and ssl modules produce IOError

Lukáš Duběda
Great! That'll save me and my clients a lot of
headaches, actually. :)

Thank you,

Lukáš Duběda
Director
[T] +420 602 444 164

duber studio(tm)
[M] [hidden email]
[W] http://www.duber.cz

[A] R.A.Dvorského 601, Praha 10
[A] 10900, Czech Republic, Europe

On 10.6.2011 19:58, Zachary Gramana wrote:

> I can't say for certain that issue you are experiencing is caused by the
> bug I'm working on, but it looks very likely.
>
> I'll let you know when I submit the patch back to the project.
>
> On 6/10/2011 1:49 PM, Lukáš Duběda wrote:
>> Thank you, Zack,
>>
>> unfortunately, editing the API I got isn't an option, so the
>> only workaround I found was to catch the exception and completely
>> ignore it, which is terrible, but at least it works.
>>
>> So you're saying that this is actually a "bug" in IronPython
>> that's being worked at?
>>
>> Thanks again, much appretiated!
>>
>> Lukáš Duběda
>> Director
>> [T] +420 602 444 164
>>
>> duber studio(tm)
>> [M] [hidden email]
>> [W] http://www.duber.cz
>>
>> [A] R.A.Dvorského 601, Praha 10
>> [A] 10900, Czech Republic, Europe
>>
>> On 10.6.2011 16:32, Zachary Gramana wrote:
>>> Lukas,
>>>
>>> I'm working on this issue, or at least an issue that generates the same
>>> exception. I do have a working patch, which I hope to contribute back
>>> after a little more work on it.
>>>
>>> Check your API to see if you are making any calls to socket.makefile().
>>> If you do, and subsequently close the fileobject (but not the socket
>>> object), the underlying .NET Socket will also get closed (and garbage
>>> collected), even though other objects are still referencing it. However,
>>> the Python socket will not be aware of this corrupt state, and
>>> subsequent socket operations will raise the exception.
>>>
>>> Not sure why subsequent attempts work for you, other than that perhaps
>>> your API has error handling code that re-create the connection.
>>>
>>> Hope this might be helpful.
>>>
>>> Best Regards, Zack
>>>
>>>
>>>
>>> On 6/10/2011 7:55 AM, Lukáš Duběda wrote:
>>>> Hi there,
>>>>
>>>> I hope someone out there could help me out with this problem.
>>>>
>>>> I have a API written in Python (CPython that is) that relies on
>>>> some of the CPython modules, such as ssl and httplib.
>>>>
>>>> The problem is, when I call the API and its methods via IronPython
>>>> I get IOError: "System.IO.IOException: Unable to write data to the
>>>> transport connection: Cannot access a disposed object."
>>>>
>>>> The weirdest thing is, if I make the call for the second time,
>>>> all goes through beautifully. Then again, thrid time errors out, but
>>>> fourth goes through just fine. And so on so forth.
>>>>
>>>> Here's the exact script excerpt (with the thorough error output) at
>>>> pastebin: http://pastebin.com/TKbKDtPb
>>>>
>>>> I call the sg.find_one() method the first time, which produces the
>>>> IOError. Then at the end of the error output, I call it again and
>>>> as you can see, it returns the correct object.
>>>>
>>>> Please, could anyone tell me how to deal with this without rewriting
>>>> the given API to use .NET objects?
>>>>
>>>> Thanks a lot in advance, cheers,
>>>>
>>> _______________________________________________
>>> Users mailing list
>>> [hidden email]
>>> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>> _______________________________________________
>> Users mailing list
>> [hidden email]
>> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
_______________________________________________
Users mailing list
[hidden email]
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Reply | Threaded
Open this post in threaded view
|

Re: IronPython and CPython httplib and ssl modules produce IOError

jalopyuser
In reply to this post by Lukáš Duběda
Lukáš Duběda <[hidden email]> wrote:

> Hi there,
>
> I hope someone out there could help me out with this problem.
>
> I have a API written in Python (CPython that is) that relies on
> some of the CPython modules, such as ssl and httplib.
>
> The problem is, when I call the API and its methods via IronPython
> I get IOError: "System.IO.IOException: Unable to write data to the
> transport connection: Cannot access a disposed object."

I re-wrote the SSL module for CPython back in 2007, and as part of that
delved into the socket and httplib modules, further than I wanted to go.
The httplib module relies on the socket module's not actually closing
the socket when it gets GC'd.  It passes the file descriptor off to a
callback function, and drops the socket object, assuming that the file
descriptor will stay open so the callback function can read from it,
until someone (presumably the callback function) closes it.  (This
apparently was originally done to work around the absence of dup() on
Windows.)  I believe it was re-engineered in py3, but am not sure what
the status is in py2.  I could easily see this leading to something
similar to what you're seeing.

Bill
_______________________________________________
Users mailing list
[hidden email]
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Reply | Threaded
Open this post in threaded view
|

Re: IronPython and CPython httplib and ssl modules produce IOError

Lukáš Duběda
Hi there, Bill,

thanks a lot for the insight of the issue, I can see
how my programs could error out like that now.

I'll see what I can do for now, but, unfortunately
the fastest and easiest workaround I used was:

try:
     main()
except IOError:
     main()

which is painful even when looked at. :(

Thanks again, cheers,

Lukáš Duběda
Director
[T] +420 602 444 164

duber studio(tm)
[M] [hidden email]
[W] http://www.duber.cz

[A] R.A.Dvorského 601, Praha 10
[A] 10900, Czech Republic, Europe

On 12.6.2011 19:34, Bill Janssen wrote:

> Lukáš Duběda<[hidden email]>  wrote:
>
>> Hi there,
>>
>> I hope someone out there could help me out with this problem.
>>
>> I have a API written in Python (CPython that is) that relies on
>> some of the CPython modules, such as ssl and httplib.
>>
>> The problem is, when I call the API and its methods via IronPython
>> I get IOError: "System.IO.IOException: Unable to write data to the
>> transport connection: Cannot access a disposed object."
>
> I re-wrote the SSL module for CPython back in 2007, and as part of that
> delved into the socket and httplib modules, further than I wanted to go.
> The httplib module relies on the socket module's not actually closing
> the socket when it gets GC'd.  It passes the file descriptor off to a
> callback function, and drops the socket object, assuming that the file
> descriptor will stay open so the callback function can read from it,
> until someone (presumably the callback function) closes it.  (This
> apparently was originally done to work around the absence of dup() on
> Windows.)  I believe it was re-engineered in py3, but am not sure what
> the status is in py2.  I could easily see this leading to something
> similar to what you're seeing.
>
> Bill
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
_______________________________________________
Users mailing list
[hidden email]
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com