Quantcast

Silent crashes in calls to a COM object in Python

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Silent crashes in calls to a COM object in Python

Deniz Pelvan

Hello guys,I am having trouble with the Python interpreter crashing silently (no exceptions or errors thrown) when I make a call to a very CPU-intensive method in my COM/ATL component using Pywin32. The method is an FFT analysis for audio files and hits about 25% CPU on a single core and runs for about 1 minutes. I have tested the COM method intensively via normal win32 applications and there are no problems with it (I can run the same method over and over again 1000s of times in a win32 app).- Also the same Python script (with a call to this COM method) will run successfully on different machines and will fail at the same machine only randomly (i.e. on some machines, it never fails, while on others, it has ~50% failure rate). - This specific method is the most CPU-intensive one in the COM component and other methods do not fail on any of the machines. Because of this, I think the problem might have something to do with either the CPU usage or a timeout feature on Python or pywin32. Are there any tools that would help me narrow down the cause or maybe increase the CPU/time limits for Python/PyWin32? I haven't seen anything in the documentation that'd help.Thanks in advanceDeniz Pelvan
     
_______________________________________________
python-win32 mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-win32
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Silent crashes in calls to a COM object in Python

Mark Hammond-4
Python and pywin32 don't impose any time limits for timeouts and I can't
think of what could cause this other than (say) an exception handler in
your script that ignores exceptions and terminates the process.  Even if
Python actually crashed I'd expect Windows to show the "app crashed"
dialog.  If you have MSVC you could try running the python process under
it to see if it reports anything strange.

Mark

On 24/04/2012 8:09 AM, Deniz Pelvan wrote:

>
> Hello guys,I am having trouble with the Python interpreter crashing
> silently (no exceptions or errors thrown) when I make a call to a
> very CPU-intensive method in my COM/ATL component using Pywin32. The
> method is an FFT analysis for audio files and hits about 25% CPU on a
> single core and runs for about 1 minutes. I have tested the COM
> method intensively via normal win32 applications and there are no
> problems with it (I can run the same method over and over again 1000s
> of times in a win32 app).- Also the same Python script (with a call
> to this COM method) will run successfully on different machines and
> will fail at the same machine only randomly (i.e. on some machines,
> it never fails, while on others, it has ~50% failure rate). - This
> specific method is the most CPU-intensive one in the COM component
> and other methods do not fail on any of the machines. Because of
> this, I think the problem might have something to do with either the
> CPU usage or a timeout feature on Python or pywin32. Are there any
> tools that would help me narrow down the cause or maybe increase the
> CPU/time limits for Python/PyWin32? I haven't seen anything in the
> documentation that'd help.Thanks in advanceDeniz Pelvan
> _______________________________________________ python-win32 mailing
> list [hidden email]
> http://mail.python.org/mailman/listinfo/python-win32
>


_______________________________________________
python-win32 mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-win32
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Silent crashes in calls to a COM object in Python

Deniz Pelvan
Hello Mark,
As I mentioned, I ran my COM interface with another Win32 test app under a MSVC debugger for hours without any problem. I attached a debugger to the Python process but with Python, the script goes all the way to that call, waits about a minute in that call and then without throwing any exceptions or errors, the script stops running (i.e. the next line in the script never gets called) and Python drops back to the prompt. Nothing gets caught by the debugger. 
I was wondering if there are any options either in the Python interpreter or in pywin32 (or maybe a debug version) that can show what is happening internally, especially over the COM dispatch...
Thanks in advance
Deniz


> Date: Tue, 24 Apr 2012 11:08:12 +1000

> From: [hidden email]
> To: [hidden email]
> CC: [hidden email]
> Subject: Re: [python-win32] Silent crashes in calls to a COM object in Python
>
> Python and pywin32 don't impose any time limits for timeouts and I can't
> think of what could cause this other than (say) an exception handler in
> your script that ignores exceptions and terminates the process. Even if
> Python actually crashed I'd expect Windows to show the "app crashed"
> dialog. If you have MSVC you could try running the python process under
> it to see if it reports anything strange.
>
> Mark
>
> On 24/04/2012 8:09 AM, Deniz Pelvan wrote:
> >
> > Hello guys,I am having trouble with the Python interpreter crashing
> > silently (no exceptions or errors thrown) when I make a call to a
> > very CPU-intensive method in my COM/ATL component using Pywin32. The
> > method is an FFT analysis for audio files and hits about 25% CPU on a
> > single core and runs for about 1 minutes. I have tested the COM
> > method intensively via normal win32 applications and there are no
> > problems with it (I can run the same method over and over again 1000s
> > of times in a win32 app).- Also the same Python script (with a call
> > to this COM method) will run successfully on different machines and
> > will fail at the same machine only randomly (i.e. on some machines,
> > it never fails, while on others, it has ~50% failure rate). - This
> > specific method is the most CPU-intensive one in the COM component
> > and other methods do not fail on any of the machines. Because of
> > this, I think the problem might have something to do with either the
> > CPU usage or a timeout feature on Python or pywin32. Are there any
> > tools that would help me narrow down the cause or maybe increase the
> > CPU/time limits for Python/PyWin32? I haven't seen anything in the
> > documentation that'd help.Thanks in advanceDeniz Pelvan
> > _______________________________________________ python-win32 mailing
> > list [hidden email]
> > http://mail.python.org/mailman/listinfo/python-win32
> >
>
>

_______________________________________________
python-win32 mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-win32
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Silent crashes in calls to a COM object in Python

Mark Hammond-3
On 24/04/2012 2:55 PM, Deniz Pelvan wrote:
> Hello Mark,
> As I mentioned, I ran my COM interface with another Win32 test app under
> a MSVC debugger for hours without any problem. I attached a debugger to
> the Python process but with Python, the script goes all the way to that
> call, waits about a minute in that call and then without throwing any
> exceptions or errors, the script stops running (i.e. the next line in
> the script never gets called) and Python drops back to the prompt.
> Nothing gets caught by the debugger.

I'm not quite with you here - I assume Python is using an inproc version
of your object, so the COM call made by Python is actually a direct call
into your object.  In that scenario, I can't see how execution will
return to Python without your object actually returning.

If it is not an inproc object, then it may be that the default COM
marshaller is indeed timing out, but I'm not familiar with that or under
what circumstances this will happen but it might be worth looking at
what "apartment" things are running in, and/or any other differences
between the environment your test app runs under and that of Python.

I'm also not sure what prompt you are referring to - the Python prompt
or a cmdline prompt?  Either way, with the debugger you should be able
to see the call to IDispatch::Invoke made by Python (hrm - but sadly you
will probably need to build pywin32 yourself so you get debugging
symbols).  If you can see that call to Invoke(), you should be able to
determine if that call never returns (in which case it is almost
certainly the marshaller) or if it returns with an error code or 0.

There isn't much else in Python or pywin32 that can help - depending on
how you construct the COM object, there are one or 2 places where the
Python code calls "dispatchobject.Invoke()" (or
dispatchobject.InvokeTypes()) which is basically a direct call from the
interpreter into win32com/src/PyIDispatch.cpp - the debugger is the best
tool for that...

Mark


> I was wondering if there are any options either in the Python
> interpreter or in pywin32 (or maybe a debug version) that can show what
> is happening internally, especially over the COM dispatch...
> Thanks in advance
> Deniz
>
>
>  > Date: Tue, 24 Apr 2012 11:08:12 +1000
>  > From: [hidden email]
>  > To: [hidden email]
>  > CC: [hidden email]
>  > Subject: Re: [python-win32] Silent crashes in calls to a COM object
> in Python
>  >
>  > Python and pywin32 don't impose any time limits for timeouts and I can't
>  > think of what could cause this other than (say) an exception handler in
>  > your script that ignores exceptions and terminates the process. Even if
>  > Python actually crashed I'd expect Windows to show the "app crashed"
>  > dialog. If you have MSVC you could try running the python process under
>  > it to see if it reports anything strange.
>  >
>  > Mark
>  >
>  > On 24/04/2012 8:09 AM, Deniz Pelvan wrote:
>  > >
>  > > Hello guys,I am having trouble with the Python interpreter crashing
>  > > silently (no exceptions or errors thrown) when I make a call to a
>  > > very CPU-intensive method in my COM/ATL component using Pywin32. The
>  > > method is an FFT analysis for audio files and hits about 25% CPU on a
>  > > single core and runs for about 1 minutes. I have tested the COM
>  > > method intensively via normal win32 applications and there are no
>  > > problems with it (I can run the same method over and over again 1000s
>  > > of times in a win32 app).- Also the same Python script (with a call
>  > > to this COM method) will run successfully on different machines and
>  > > will fail at the same machine only randomly (i.e. on some machines,
>  > > it never fails, while on others, it has ~50% failure rate). - This
>  > > specific method is the most CPU-intensive one in the COM component
>  > > and other methods do not fail on any of the machines. Because of
>  > > this, I think the problem might have something to do with either the
>  > > CPU usage or a timeout feature on Python or pywin32. Are there any
>  > > tools that would help me narrow down the cause or maybe increase the
>  > > CPU/time limits for Python/PyWin32? I haven't seen anything in the
>  > > documentation that'd help.Thanks in advanceDeniz Pelvan
>  > > _______________________________________________ python-win32 mailing
>  > > list [hidden email]
>  > > http://mail.python.org/mailman/listinfo/python-win32
>  > >
>  >
>  >


_______________________________________________
python-win32 mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-win32
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Silent crashes in calls to a COM object in Python

Deniz Pelvan

Hello again Mark,I am not sure what you mean by direct call? The COM object is loaded inproc (i.e. the COM object is not in a service proc and I don't see any temp processes created by python or pywin32) but I thought you were using the IDispatch interface to find and load the dll & object rather than a direct interface call (which would require type library etc. knowledge). As for the test app, it is not using the IDispatch interface so the IDispatch might be causing trouble.The prompt I mentioned is the Python interpreter prompt, btw. I will try to dig deeper into the python interpreter with the debugger and see if there is anything else I can catch. Thanks for the helpDeniz
----------------------------------------

> Date: Tue, 24 Apr 2012 15:07:39 +1000
> From: [hidden email]
> To: [hidden email]
> CC: [hidden email]
> Subject: Re: [python-win32] Silent crashes in calls to a COM object in Python
>
> On 24/04/2012 2:55 PM, Deniz Pelvan wrote:
> > Hello Mark,
> > As I mentioned, I ran my COM interface with another Win32 test app under
> > a MSVC debugger for hours without any problem. I attached a debugger to
> > the Python process but with Python, the script goes all the way to that
> > call, waits about a minute in that call and then without throwing any
> > exceptions or errors, the script stops running (i.e. the next line in
> > the script never gets called) and Python drops back to the prompt.
> > Nothing gets caught by the debugger.
>
> I'm not quite with you here - I assume Python is using an inproc version
> of your object, so the COM call made by Python is actually a direct call
> into your object. In that scenario, I can't see how execution will
> return to Python without your object actually returning.
>
> If it is not an inproc object, then it may be that the default COM
> marshaller is indeed timing out, but I'm not familiar with that or under
> what circumstances this will happen but it might be worth looking at
> what "apartment" things are running in, and/or any other differences
> between the environment your test app runs under and that of Python.
>
> I'm also not sure what prompt you are referring to - the Python prompt
> or a cmdline prompt? Either way, with the debugger you should be able
> to see the call to IDispatch::Invoke made by Python (hrm - but sadly you
> will probably need to build pywin32 yourself so you get debugging
> symbols). If you can see that call to Invoke(), you should be able to
> determine if that call never returns (in which case it is almost
> certainly the marshaller) or if it returns with an error code or 0.
>
> There isn't much else in Python or pywin32 that can help - depending on
> how you construct the COM object, there are one or 2 places where the
> Python code calls "dispatchobject.Invoke()" (or
> dispatchobject.InvokeTypes()) which is basically a direct call from the
> interpreter into win32com/src/PyIDispatch.cpp - the debugger is the best
> tool for that...
>
> Mark
>
>
> > I was wondering if there are any options either in the Python
> > interpreter or in pywin32 (or maybe a debug version) that can show what
> > is happening internally, especially over the COM dispatch...
> > Thanks in advance
> > Deniz
> >
> >
> > > Date: Tue, 24 Apr 2012 11:08:12 +1000
> > > From: [hidden email]
> > > To: [hidden email]
> > > CC: [hidden email]
> > > Subject: Re: [python-win32] Silent crashes in calls to a COM object
> > in Python
> > >
> > > Python and pywin32 don't impose any time limits for timeouts and I can't
> > > think of what could cause this other than (say) an exception handler in
> > > your script that ignores exceptions and terminates the process. Even if
> > > Python actually crashed I'd expect Windows to show the "app crashed"
> > > dialog. If you have MSVC you could try running the python process under
> > > it to see if it reports anything strange.
> > >
> > > Mark
> > >
> > > On 24/04/2012 8:09 AM, Deniz Pelvan wrote:
> > > >
> > > > Hello guys,I am having trouble with the Python interpreter crashing
> > > > silently (no exceptions or errors thrown) when I make a call to a
> > > > very CPU-intensive method in my COM/ATL component using Pywin32. The
> > > > method is an FFT analysis for audio files and hits about 25% CPU on a
> > > > single core and runs for about 1 minutes. I have tested the COM
> > > > method intensively via normal win32 applications and there are no
> > > > problems with it (I can run the same method over and over again 1000s
> > > > of times in a win32 app).- Also the same Python script (with a call
> > > > to this COM method) will run successfully on different machines and
> > > > will fail at the same machine only randomly (i.e. on some machines,
> > > > it never fails, while on others, it has ~50% failure rate). - This
> > > > specific method is the most CPU-intensive one in the COM component
> > > > and other methods do not fail on any of the machines. Because of
> > > > this, I think the problem might have something to do with either the
> > > > CPU usage or a timeout feature on Python or pywin32. Are there any
> > > > tools that would help me narrow down the cause or maybe increase the
> > > > CPU/time limits for Python/PyWin32? I haven't seen anything in the
> > > > documentation that'd help.Thanks in advanceDeniz Pelvan
> > > > _______________________________________________ python-win32 mailing
> > > > list [hidden email]
> > > > http://mail.python.org/mailman/listinfo/python-win32
> > > >
> > >
> > >
>
>
     
_______________________________________________
python-win32 mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-win32
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Silent crashes in calls to a COM object in Python

Mark Hammond-4
On 25/04/2012 7:14 AM, Deniz Pelvan wrote:
>
> Hello again Mark,I am not sure what you mean by direct call? The COM
> object is loaded inproc (i.e. the COM object is not in a service proc
> and I don't see any temp processes created by python or pywin32) but
> I thought you were using the IDispatch interface to find and load the
> dll & object rather than a direct interface call (which would require
> type library etc. knowledge).

IDispatch is just an interface like any other.  If you call an inproc
interface, you are making a normal direct call across DLLs, almost as if
you had linked to the DLL directly - just the mechanics of that
"linking" change.

> As for the test app, it is not using
> the IDispatch interface so the IDispatch might be causing trouble.

This sounds very likely.

> The
> prompt I mentioned is the Python interpreter prompt, btw.

So it sounds like one of your IDispatch interfaces is prematurely
returning S_OK - ie, nothing is crashing but your method is just
returning before you expect it.

Mark

I will try

> to dig deeper into the python interpreter with the debugger and see
> if there is anything else I can catch. Thanks for the helpDeniz
> ----------------------------------------
>> Date: Tue, 24 Apr 2012 15:07:39 +1000 From:
>> [hidden email] To: [hidden email] CC:
>> [hidden email] Subject: Re: [python-win32] Silent crashes
>> in calls to a COM object in Python
>>
>> On 24/04/2012 2:55 PM, Deniz Pelvan wrote:
>>> Hello Mark, As I mentioned, I ran my COM interface with another
>>> Win32 test app under a MSVC debugger for hours without any
>>> problem. I attached a debugger to the Python process but with
>>> Python, the script goes all the way to that call, waits about a
>>> minute in that call and then without throwing any exceptions or
>>> errors, the script stops running (i.e. the next line in the
>>> script never gets called) and Python drops back to the prompt.
>>> Nothing gets caught by the debugger.
>>
>> I'm not quite with you here - I assume Python is using an inproc
>> version of your object, so the COM call made by Python is actually
>> a direct call into your object. In that scenario, I can't see how
>> execution will return to Python without your object actually
>> returning.
>>
>> If it is not an inproc object, then it may be that the default COM
>> marshaller is indeed timing out, but I'm not familiar with that or
>> under what circumstances this will happen but it might be worth
>> looking at what "apartment" things are running in, and/or any other
>> differences between the environment your test app runs under and
>> that of Python.
>>
>> I'm also not sure what prompt you are referring to - the Python
>> prompt or a cmdline prompt? Either way, with the debugger you
>> should be able to see the call to IDispatch::Invoke made by Python
>> (hrm - but sadly you will probably need to build pywin32 yourself
>> so you get debugging symbols). If you can see that call to
>> Invoke(), you should be able to determine if that call never
>> returns (in which case it is almost certainly the marshaller) or if
>> it returns with an error code or 0.
>>
>> There isn't much else in Python or pywin32 that can help -
>> depending on how you construct the COM object, there are one or 2
>> places where the Python code calls "dispatchobject.Invoke()" (or
>> dispatchobject.InvokeTypes()) which is basically a direct call from
>> the interpreter into win32com/src/PyIDispatch.cpp - the debugger is
>> the best tool for that...
>>
>> Mark
>>
>>
>>> I was wondering if there are any options either in the Python
>>> interpreter or in pywin32 (or maybe a debug version) that can
>>> show what is happening internally, especially over the COM
>>> dispatch... Thanks in advance Deniz
>>>
>>>
>>>> Date: Tue, 24 Apr 2012 11:08:12 +1000 From:
>>>> [hidden email] To: [hidden email] CC:
>>>> [hidden email] Subject: Re: [python-win32] Silent
>>>> crashes in calls to a COM object
>>> in Python
>>>>
>>>> Python and pywin32 don't impose any time limits for timeouts
>>>> and I can't think of what could cause this other than (say) an
>>>> exception handler in your script that ignores exceptions and
>>>> terminates the process. Even if Python actually crashed I'd
>>>> expect Windows to show the "app crashed" dialog. If you have
>>>> MSVC you could try running the python process under it to see
>>>> if it reports anything strange.
>>>>
>>>> Mark
>>>>
>>>> On 24/04/2012 8:09 AM, Deniz Pelvan wrote:
>>>>>
>>>>> Hello guys,I am having trouble with the Python interpreter
>>>>> crashing silently (no exceptions or errors thrown) when I
>>>>> make a call to a very CPU-intensive method in my COM/ATL
>>>>> component using Pywin32. The method is an FFT analysis for
>>>>> audio files and hits about 25% CPU on a single core and runs
>>>>> for about 1 minutes. I have tested the COM method intensively
>>>>> via normal win32 applications and there are no problems with
>>>>> it (I can run the same method over and over again 1000s of
>>>>> times in a win32 app).- Also the same Python script (with a
>>>>> call to this COM method) will run successfully on different
>>>>> machines and will fail at the same machine only randomly
>>>>> (i.e. on some machines, it never fails, while on others, it
>>>>> has ~50% failure rate). - This specific method is the most
>>>>> CPU-intensive one in the COM component and other methods do
>>>>> not fail on any of the machines. Because of this, I think the
>>>>> problem might have something to do with either the CPU usage
>>>>> or a timeout feature on Python or pywin32. Are there any
>>>>> tools that would help me narrow down the cause or maybe
>>>>> increase the CPU/time limits for Python/PyWin32? I haven't
>>>>> seen anything in the documentation that'd help.Thanks in
>>>>> advanceDeniz Pelvan
>>>>> _______________________________________________ python-win32
>>>>> mailing list [hidden email]
>>>>> http://mail.python.org/mailman/listinfo/python-win32
>>>>>
>>>>
>>>>
>>
>>
>  _______________________________________________ python-win32 mailing
> list [hidden email]
> http://mail.python.org/mailman/listinfo/python-win32
>


_______________________________________________
python-win32 mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-win32
Loading...