Re: Having issues using subprocess module

[Oscar Steele]

Hi all,

Im just getting started with Jython.  I have a small codebase (Python 2.7 based) that I am converting over.  I seem to be down to one last issue.  I was using the multiprocessing module to spawn subprocesses.  I have switched over to subprocess.

The issue I am having is how to kill the subprocesses.  The Popen(...).pid is alway None (and no I am not specifying shell=True), so I can't use os.kill().  Also, the Popen object has no terminate() or send_signal() method.

Any chance this is an issue with jython 2.5.3b1 and I should roll back to the last release?

Note, I printed out dir(Popen()) and I do see methods like communicate(), wait(), and poll() but not terminate() or send_signal().

All help appreciated.

- Ozzy
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users

[Chris Clark]

On Tuesday 2012-05-15 14:32 (-0700), Oscar Steele
<[hidden email]> wrote:

> Im just getting started with Jython.  I have a small codebase (Python 2.7 based) that I am converting over.  I seem to be down to one last issue.  I was using the multiprocessing module to spawn subprocesses.  I have switched over to subprocess.
>
> The issue I am having is how to kill the subprocesses.  The Popen(...).pid is alway None (and no I am not specifying shell=True), so I can't use os.kill().  Also, the Popen object has no terminate() or send_signal() method.
>
> Any chance this is an issue with jython 2.5.3b1 and I should roll back to the last release?
>
> Note, I printed out dir(Popen()) and I do see methods like communicate(), wait(), and poll() but not terminate() or send_signal().
>
> All help appreciated.
>

Have you considered switching to threading instead? Under Java threads
the GIL isn't present so you should be able to get full use of the CPU
and avoid subprocesses (unless you really need to spawn a different
exe.....).

Chris


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users

[Oscar Steele]

Yes I considered that, but it really should be a separate process.  That aside, unlike subprocess module, threads has no api for the parent to kill the child.  The subprocesses in my case are long lived servers that run independently of the parent process, so the parent can not wait on the children and the children can not wait on the parent.  But there has to be a mechanism for the parent to interrupt the children and tell them to shut down (e.g. process and signal).  Perhaps there is a way to do this with threads that I missed????

Sent from my iPad

On May 15, 2012, at 2:35 PM, Chris Clark <[hidden email]> wrote:

> On Tuesday 2012-05-15 14:32 (-0700), Oscar Steele <[hidden email]> wrote:
>> Im just getting started with Jython.  I have a small codebase (Python 2.7 based) that I am converting over.  I seem to be down to one last issue.  I was using the multiprocessing module to spawn subprocesses.  I have switched over to subprocess.
>>
>> The issue I am having is how to kill the subprocesses.  The Popen(...).pid is alway None (and no I am not specifying shell=True), so I can't use os.kill().  Also, the Popen object has no terminate() or send_signal() method.
>>
>> Any chance this is an issue with jython 2.5.3b1 and I should roll back to the last release?
>>
>> Note, I printed out dir(Popen()) and I do see methods like communicate(), wait(), and poll() but not terminate() or send_signal().
>>
>> All help appreciated.
>>
>
> Have you considered switching to threading instead? Under Java threads the GIL isn't present so you should be able to get full use of the CPU and avoid subprocesses (unless you really need to spawn a different exe.....).
>
> Chris
>

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users

[Pekka Klärck]

2012/5/16 Oscar Steele <[hidden email]>:
>
> Note, I printed out dir(Popen()) and I do see methods like communicate(), wait(), and poll() but not terminate() or send_signal().

According to the documentation [1], both terminate() and send_signal()
were added in Python 2.6 and thus they missing from Jython 2.5.x isn't
surprising. I just noticed there's Jython 2.7 alpha 1 available [2] so
you could try that. If Frank and others haven't yet updated
subprocess, I'm sure they are very interested in any help regarding to
that.

[1] http://docs.python.org/library/subprocess.html
[2] http://fwierzbicki.blogspot.com/2012/05/jython-27-alpha1-released.html

Cheers,
    .peke
--
Agile Tester/Developer/Consultant :: http://eliga.fi
Lead Developer of Robot Framework :: http://robotframework.org

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users

[Philip Jenvey-3]

On May 21, 2012, at 4:59 AM, Pekka Klärck wrote:

> 2012/5/16 Oscar Steele <[hidden email]>:
>>
>> Note, I printed out dir(Popen()) and I do see methods like communicate(), wait(), and poll() but not terminate() or send_signal().
>
> According to the documentation [1], both terminate() and send_signal()
> were added in Python 2.6 and thus they missing from Jython 2.5.x isn't
> surprising. I just noticed there's Jython 2.7 alpha 1 available [2] so
> you could try that. If Frank and others haven't yet updated
> subprocess, I'm sure they are very interested in any help regarding to
> that.
>
> [1] http://docs.python.org/library/subprocess.html
> [2] http://fwierzbicki.blogspot.com/2012/05/jython-27-alpha1-released.html

This came up on the IRC channel recently (maybe that was also Oscar?)

2.6 indeed added those and kill(). Jython 2.7 should be able to support kill but it probably won't be able to fully support the other API calls (without fully hacking around java.lang.Process/Builder). Note that the 2.7 alpha shipped with a broken kill(), however.

There's actually an undocumented way to kill the process in 2.5 by accessing Popen's _process attribute (the underlying java.lang.Process), e.g.:

$ dist/bin/jython
Jython 2.5.3+ (2.5:c62b1a8fa99b, May 21 2012, 12:56:50)
[Java HotSpot(TM) 64-Bit Server VM (Apple Inc.)] on java1.6.0_31
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> p = subprocess.Popen(['/bin/sleep', '60'])      
>>> p._process
java.lang.UNIXProcess@40395aaf
>>> p._process.destroy()
>>>

--
Philip Jenvey


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users