KVM incompatibility with jython?

[Quanah Gibson-Mount-3]

We have a customer who is having problems with our jython based code.
Their server is a RHEL5 Update 6 64-bit system, and we have no issues with
jython on that OS in our environment.  The difference between the two
environments that I can find is that while we use VMware ESX, they are
using KVM.

When I am on their system and import os, there are relatively few functions
available.  Is this a known bug with jython?  Are there additional pieces
that need importing to get jython to work in a KVM VM?

--Quanah

--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration

------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users

[Quanah Gibson-Mount-3]

--On Wednesday, July 06, 2011 10:05 AM -0700 Quanah Gibson-Mount
<[hidden email]> wrote:

> When I am on their system and import os, there are relatively few
> functions  available.  Is this a known bug with jython?  Are there
> additional pieces  that need importing to get jython to work in a KVM VM?

On their system, dir(os) *looks* ok:

>>> import os
>>> dir(os)
['EX_OK', 'F_OK', 'O_APPEND', 'O_CREAT', 'O_EXCL', 'O_RDONLY', 'O_RDWR',
'O_SYNC', 'O_TRUNC', 'O_WRONLY', 'R_OK', 'SEEK_CUR', 'SEEK_END',
'SEEK_SET', 'W_OK', 'X_OK', '__all__', '__doc__', '__file__', '__loader__',
'__name__', '_exists', '_exit', '_get_exports_list', '_get_shell_commands',
'_name', '_native_posix', '_posix_impl', '_wrap_close', 'access', 'altsep',
'chdir', 'chmod', 'chown', 'close', 'curdir', 'defpath', 'devnull',
'environ', 'errno', 'error', 'extsep', 'fdatasync', 'fdopen', 'fsync',
'ftruncate', 'getcwd', 'getcwdu', 'getegid', 'getenv', 'geteuid', 'getgid',
'getlogin', 'getpgrp', 'getppid', 'getuid', 'isatty', 'kill', 'lchmod',
'lchown', 'linesep', 'link', 'listdir', 'lseek', 'lstat', 'makedirs',
'mkdir', 'name', 'open', 'pardir', 'path', 'pathsep', 'popen', 'popen2',
'popen3', 'popen4', 'putenv', 'read', 'readlink', 'remove', 'removedirs',
'rename', 'renames', 'rmdir', 'sep', 'setpgrp', 'setsid', 'stat',
'stat_result', 'strerror', 'symlink', 'sys', 'system', 'unlink',
'unsetenv', 'urandom', 'utime', 'wait', 'waitpid', 'walk', 'write']
>>> os.getpid()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'getpid'

--Quanah

--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration

------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users

[Alex Grönholm-3]

06.07.2011 20:37, Quanah Gibson-Mount kirjoitti:
> --On Wednesday, July 06, 2011 10:05 AM -0700 Quanah Gibson-Mount
> <[hidden email]>  wrote:
>
>> When I am on their system and import os, there are relatively few
>> functions  available.  Is this a known bug with jython?  Are there
>> additional pieces  that need importing to get jython to work in a KVM VM?
This has nothing whatsoever to do with the KVM environment. This is
caused by limitations of the Java Virtual Machine, as result of which
many low level functions are not available.

> On their system, dir(os) *looks* ok:
>
>>>> import os
>>>> dir(os)
> ['EX_OK', 'F_OK', 'O_APPEND', 'O_CREAT', 'O_EXCL', 'O_RDONLY', 'O_RDWR',
> 'O_SYNC', 'O_TRUNC', 'O_WRONLY', 'R_OK', 'SEEK_CUR', 'SEEK_END',
> 'SEEK_SET', 'W_OK', 'X_OK', '__all__', '__doc__', '__file__', '__loader__',
> '__name__', '_exists', '_exit', '_get_exports_list', '_get_shell_commands',
> '_name', '_native_posix', '_posix_impl', '_wrap_close', 'access', 'altsep',
> 'chdir', 'chmod', 'chown', 'close', 'curdir', 'defpath', 'devnull',
> 'environ', 'errno', 'error', 'extsep', 'fdatasync', 'fdopen', 'fsync',
> 'ftruncate', 'getcwd', 'getcwdu', 'getegid', 'getenv', 'geteuid', 'getgid',
> 'getlogin', 'getpgrp', 'getppid', 'getuid', 'isatty', 'kill', 'lchmod',
> 'lchown', 'linesep', 'link', 'listdir', 'lseek', 'lstat', 'makedirs',
> 'mkdir', 'name', 'open', 'pardir', 'path', 'pathsep', 'popen', 'popen2',
> 'popen3', 'popen4', 'putenv', 'read', 'readlink', 'remove', 'removedirs',
> 'rename', 'renames', 'rmdir', 'sep', 'setpgrp', 'setsid', 'stat',
> 'stat_result', 'strerror', 'symlink', 'sys', 'system', 'unlink',
> 'unsetenv', 'urandom', 'utime', 'wait', 'waitpid', 'walk', 'write']
>>>> os.getpid()
> Traceback (most recent call last):
>    File "<stdin>", line 1, in<module>
> AttributeError: 'module' object has no attribute 'getpid'
>
> --Quanah
>
> --
>
> Quanah Gibson-Mount
> Sr. Member of Technical Staff
> Zimbra, Inc
> A Division of VMware, Inc.
> --------------------
> Zimbra ::  the leader in open source messaging and collaboration
>
> ------------------------------------------------------------------------------
> All of the data generated in your IT infrastructure is seriously valuable.
> Why? It contains a definitive record of application performance, security
> threats, fraudulent activity, and more. Splunk takes this data and makes
> sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-d2d-c2
> _______________________________________________
> Jython-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/jython-users

------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users

[Quanah Gibson-Mount-3]

--On Wednesday, July 06, 2011 9:33 PM +0300 Alex Grönholm
<[hidden email]> wrote:

> 06.07.2011 20:37, Quanah Gibson-Mount kirjoitti:
>> --On Wednesday, July 06, 2011 10:05 AM -0700 Quanah Gibson-Mount
>> <[hidden email]>  wrote:
>>
>>> When I am on their system and import os, there are relatively few
>>> functions  available.  Is this a known bug with jython?  Are there
>>> additional pieces  that need importing to get jython to work in a KVM
>>> VM?
> This has nothing whatsoever to do with the KVM environment. This is
> caused by limitations of the Java Virtual Machine, as result of which
> many low level functions are not available.

Sorry, but you are wrong here, and if you'd paid attention to what I
started with, then you'd already know that your answer is incorrect.  As I
stated:

This works fine on our ESX RHEL5_64 box.
This does not work on the KVM RHEL5_64 box.

In fact, this works fine on:
RHEL4 32-bit
RHEL4 64-bit
RHEL5 32-bit
RHEL5 64-bit
RHEL6 64-bit
Fedora 11 32-bit
Fedora 11 64-bit
Fedora 13 64-bit
SLES10 23-bit
SLES10 64-bit
SLES11 64-bit
Debian5 32-bit
Debian5 64-bit
Debian6 64-bit
Ubuntu8 32-bit
Ubuntu8 64-bit
Ubuntu10 64-bit
Mac OSX 10.6 64-bit

You see, I build my software on numerous platforms.  On all of these
platforms, os.getpid() works just fine for me when called from inside
jython.  The *only* case I've found where it does not work fine in jython
is on RHEL5 64-bit when the VM is inside of KVM instead of VMware ESX or
Xen.

--Quanah


--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration

------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users

[Chris Clark-2]

Quanah Gibson-Mount wrote:

> --On Wednesday, July 06, 2011 9:33 PM +0300 Alex Grönholm
> <[hidden email]> wrote:
>
>  
>> 06.07.2011 20:37, Quanah Gibson-Mount kirjoitti:
>>    
>>> --On Wednesday, July 06, 2011 10:05 AM -0700 Quanah Gibson-Mount
>>> <[hidden email]>  wrote:
>>>
>>>      
>>>> When I am on their system and import os, there are relatively few
>>>> functions  available.  Is this a known bug with jython?  Are there
>>>> additional pieces  that need importing to get jython to work in a KVM
>>>> VM?
>>>>        
>> This has nothing whatsoever to do with the KVM environment. This is
>> caused by limitations of the Java Virtual Machine, as result of which
>> many low level functions are not available.
>>    
>
> Sorry, but you are wrong here, and if you'd paid attention to what I
> started with, then you'd already know that your answer is incorrect.  As I
> stated:
>
> This works fine on our ESX RHEL5_64 box.
> This does not work on the KVM RHEL5_64 box.
>  

Out of curiosity, what JVM is used (in both environments).

I've seen similar missing functionality when the JVM was missing it too.
If you are sure the only difference is the (CPU) VM, then I'd recommend
debugging the os module (it imports a lot of its "stuff" from other
modules, posix for instance). But checking the JVM is the low hang fruit
here.

Chris


------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users

[Quanah Gibson-Mount-3]

--On Wednesday, July 06, 2011 12:27 PM -0700 Chris Clark
<[hidden email]> wrote:

> I've seen similar missing functionality when the JVM was missing it too.
> If you are sure the only difference is the (CPU) VM, then I'd recommend
> debugging the os module (it imports a lot of its "stuff" from other
> modules, posix for instance). But checking the JVM is the low hang fruit

We ship the JVM as a part of our product, and our command to start jython
(zmpython) instantiates our copy of the JVM.  So the JVM is identical in
both environments (Which I also confirmed by checking what it started up
with).

I tried importing various bits when reading os.py, but I can't see where
exactly it tries to pull in getpid() from, so got somewhat stuck there.
Some of the posix commands are definitely there, but others are not. :/

--Quanah


--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration

------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users

[Chris Clark-2]

Quanah Gibson-Mount wrote:

> --On Wednesday, July 06, 2011 12:27 PM -0700 Chris Clark
> <[hidden email]> wrote:
>
>> I've seen similar missing functionality when the JVM was missing it too.
>> If you are sure the only difference is the (CPU) VM, then I'd recommend
>> debugging the os module (it imports a lot of its "stuff" from other
>> modules, posix for instance). But checking the JVM is the low hang fruit
>
> We ship the JVM as a part of our product, and our command to start
> jython (zmpython) instantiates our copy of the JVM.  So the JVM is
> identical in both environments (Which I also confirmed by checking
> what it started up with).
>
> I tried importing various bits when reading os.py, but I can't see
> where exactly it tries to pull in getpid() from, so got somewhat stuck
> there. Some of the posix commands are definitely there, but others are
> not. :/

It sounds like you have your own JVM, is that correct?

I've noticed that Open JDK 1.6 (with Jython) is missing os.getpid()
(under esx). I would try the Oracle/Sun one.

     > java -jar jython252.jar
    Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06)
    [Java HotSpot(TM) Client VM (Sun Microsystems Inc.)] on java1.5.0_06
    Type "help", "copyright", "credits" or "license" for more information.
     >>> import posix
     >>> posix.getpid()
    20154

NOTE this demo is from a Solaris box (as my linux box with Jython 2.5.2
is virtual and only has openjdk).

Chris



------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users

[Quanah Gibson-Mount-3]

--On Wednesday, July 06, 2011 12:44 PM -0700 Chris Clark
<[hidden email]> wrote:

> It sounds like you have your own JVM, is that correct?
>
> I've noticed that Open JDK 1.6 (with Jython) is missing os.getpid()
> (under esx). I would try the Oracle/Sun one.

We ship Oracle's JDK (1.6.0u25) in this case.  We don't use OpenJDK. ;)

--Quanah


--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration

------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users

[Chris Clark-2]

Quanah Gibson-Mount wrote:
> --On Wednesday, July 06, 2011 12:44 PM -0700 Chris Clark
> <[hidden email]> wrote:
>
>> It sounds like you have your own JVM, is that correct?
>>
>> I've noticed that Open JDK 1.6 (with Jython) is missing os.getpid()
>> (under esx). I would try the Oracle/Sun one.
>
> We ship Oracle's JDK (1.6.0u25) in this case.  We don't use OpenJDK. ;)


Weird.

I think at this stage your best bet is to pull down the source and
instrument/debug the posix module. The Java docs are here
http://www.jython.org/javadoc/org/python/modules/posix/PythonPOSIXHandler.html

Or you could implement your own getpid() and monkey patch it in.

    ingres@clach04perl:~/python$ java -jar jython252.jar
    Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06)
    [OpenJDK Client VM (Sun Microsystems Inc.)] on java1.6.0_0
    Type "help", "copyright", "credits" or "license" for more information.
     >>> import posix
     >>> posix.getpid()
    31005
     >>> from java.lang.management import ManagementFactory
     >>> int(ManagementFactory.getRuntimeMXBean().getName().split('@',
    1)[0])
    31005


Chris


------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users

[Quanah Gibson-Mount-3]

--On Wednesday, July 06, 2011 1:32 PM -0700 Chris Clark
<[hidden email]> wrote:

> Quanah Gibson-Mount wrote:
>> --On Wednesday, July 06, 2011 12:44 PM -0700 Chris Clark
>> <[hidden email]> wrote:
>>
>>> It sounds like you have your own JVM, is that correct?
>>>
>>> I've noticed that Open JDK 1.6 (with Jython) is missing os.getpid()
>>> (under esx). I would try the Oracle/Sun one.
>>
>> We ship Oracle's JDK (1.6.0u25) in this case.  We don't use OpenJDK. ;)
>
>
> Weird.
>
> I think at this stage your best bet is to pull down the source and
> instrument/debug the posix module. The Java docs are here
> http://www.jython.org/javadoc/org/python/modules/posix/PythonPOSIXHandler
> .html
>
> Or you could implement your own getpid() and monkey patch it in.
>
>     ingres@clach04perl:~/python$ java -jar jython252.jar
>     Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06)
>     [OpenJDK Client VM (Sun Microsystems Inc.)] on java1.6.0_0
>     Type "help", "copyright", "credits" or "license" for more information.
>      >>> import posix
>      >>> posix.getpid()
>     31005
>      >>> from java.lang.management import ManagementFactory
>      >>> int(ManagementFactory.getRuntimeMXBean().getName().split('@',
>     1)[0])
>     31005

Hi Chris,

Thanks, that's interesting.  The java function definitely works, while the
posix function does not:

Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06)
[Java HotSpot(TM) 64-Bit Server VM (Sun Microsystems Inc.)] on java1.6.0_25
Type "help", "copyright", "credits" or "license" for more information.
>>> import posix
>>> posix.getpid()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: type object 'posix' has no attribute 'getpid'
>>> from java.lang.management import ManagementFactory
>>> int(ManagementFactory.getRuntimeMXBean().getName().split('@',1)[0])
12602

--Quanah


--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration

------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users

[Dan Stromberg-2]

On Wed, Jul 6, 2011 at 12:48 PM, Quanah Gibson-Mount <[hidden email]> wrote:

--On Wednesday, July 06, 2011 12:44 PM -0700 Chris Clark

<[hidden email]> wrote:

> It sounds like you have your own JVM, is that correct?
>
> I've noticed that Open JDK 1.6 (with Jython) is missing os.getpid()
> (under esx). I would try the Oracle/Sun one.

We ship Oracle's JDK (1.6.0u25) in this case.  We don't use OpenJDK. ;)

--Quanah

Is it possible the system has more than one JDK?  If your frontend wrapper script is doing a $PATH search for the java executable, it could perhaps use the wrong one.

------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users

[Quanah Gibson-Mount-3]

--On Wednesday, July 06, 2011 3:19 PM -0700 Dan Stromberg
<[hidden email]> wrote:

> Is it possible the system has more than one JDK?  If your frontend
> wrapper script is doing a $PATH search for the java executable, it could
> perhaps use the wrong one.

The wrapper specifically pulls in the java we tell it to, so I'm positive
it is using our JDK.  I also verified on the problem system it is the same
JDK 1.6.0u25 as the working system.   Which it also reports when it starts
up:

Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06)
[Java HotSpot(TM) 64-Bit Server VM (Sun Microsystems Inc.)] on java1.6.0_25

Has anyone used jython on a KVM VM?

--Quanah

--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration

------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users

[Dan Stromberg-2]

On Wed, Jul 6, 2011 at 3:27 PM, Quanah Gibson-Mount <[hidden email]> wrote:

--On Wednesday, July 06, 2011 3:19 PM -0700 Dan Stromberg <[hidden email]> wrote:

Is it possible the system has more than one JDK?  If your frontend
wrapper script is doing a $PATH search for the java executable, it could
perhaps use the wrong one.

The wrapper specifically pulls in the java we tell it to, so I'm positive it is using our JDK.  I also verified on the problem system it is the same JDK 1.6.0u25 as the working system.   Which it also reports when it starts up:

Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06)

[Java HotSpot(TM) 64-Bit Server VM (Sun Microsystems Inc.)] on java1.6.0_25

Has anyone used jython on a KVM VM?

It's actually pretty unlikely that it's a KVM thing, though I cannot say with certainty that it is impossible.

Maybe you could generate a list of md5sums for all the files on each system, and then diff the results to find differences:

good-system> find / -xdev -type f -print | sort | xargs md5sum > /tmp/good.md5s
bad-system> find / -xdev -type f -print | sort | xargs md5sum > /tmp/bad.md5s
bad-system> scp /tmp/bad.md5s good-system:/tmp/.
good-system> diff -u /tmp/good.md5s /tmp/bad.md5s

?

Note that the above assumes that you have no files with newlines in their filenames - there's a more general, though more complicated, way if you have such filenames.

Furthermore, this assumes that only your root filesystems are of interest - if there are more filesystems than that on the systems, they could matter.

(BTW, even on systems installed via the same automatic procedure, there will commonly be a few hundred differences or more)

------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users

[yyamano]

Quanah,

Could you run the following code on the KVM?

import os
print os._native_posix
print os._posix_impl

from java.lang import System
print System.getProperty("os.name");

--
Yuji Yamano
OGIS International, Inc.


------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users

[Quanah Gibson-Mount-3]

--On Thursday, July 07, 2011 12:28 PM -0700 Yuji YAMANO
<[hidden email]> wrote:

> Quanah,
>
> Could you run the following code on the KVM?
>
> import os
> print os._native_posix
> print os._posix_impl
>
> from java.lang import System
> print System.getProperty("os.name");

Non working system (KVM):

Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06)
[Java HotSpot(TM) 64-Bit Server VM (Sun Microsystems Inc.)] on java1.6.0_25
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>>
>>> print os._native_posix
False
>>> print os._posix_impl
org.python.posix.LazyPOSIX@4a005364
>>> from java.lang import System
>>> print System.getProperty("os.name");
Linux


Working system (ESX):
Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06)
[Java HotSpot(TM) 64-Bit Server VM (Sun Microsystems Inc.)] on java1.6.0_26
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> print os._native_posix
True
>>> print os._posix_impl
org.python.posix.LazyPOSIX@4c9fd062
>>> from java.lang import System
>>> print System.getProperty("os.name");
Linux

So for the working one, I see it says it is using native posix.  For the
non-working one, I see it is not using native posix.  No clue why there is
a difference. :/

--Quanah

--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration

------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users

[yyamano]

On Thu, 07 Jul 2011 12:41:42 -0700, Quanah Gibson-Mount <[hidden email]> wrote:

> So for the working one, I see it says it is using native posix.  For the
> non-working one, I see it is not using native posix.  No clue why there is
> a difference. :/

Could you invoke jython with -vv option?
You can get debug information for POSIX module loading.

https://github.com/jruby/jnr-posix/blob/master/src/org/jruby/ext/posix/POSIXFactory.java

static POSIX loadPOSIX(POSIXHandler handler, boolean useNativePOSIX) {

[snip]

       try {
           if (Platform.IS_MAC) {
               posix = loadMacOSPOSIX(handler);
           } else if (Platform.IS_LINUX) {
               posix = loadLinuxPOSIX(handler);
           } else if (Platform.IS_FREEBSD) {

[snip]

           // ENEBO: Should printing be done through a handler+log method?
           if (handler.isVerbose()) {
               if (posix != null) {
                   System.err.println("Successfully loaded native POSIX impl.");
               } else {
                   System.err.println("Failed to load native POSIX impl; falling back on Java impl. Unsupported OS.");
               }
            }
        } catch (Throwable t) {
            if (handler.isVerbose()) {
                System.err.println("Failed to load native POSIX impl; falling back on Java impl. Stacktrace follows.");
                t.printStackTrace();
            }
        }

--
Yuji Yamano
OGIS International, Inc.


------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users

[Quanah Gibson-Mount-3]

--On Thursday, July 07, 2011 4:19 PM -0700 Yuji YAMANO
<[hidden email]> wrote:

> On Thu, 07 Jul 2011 12:41:42 -0700, Quanah Gibson-Mount
> <[hidden email]> wrote:
>
>> So for the working one, I see it says it is using native posix.  For the
>> non-working one, I see it is not using native posix.  No clue why there
>> is  a difference. :/
>
> Could you invoke jython with -vv option?
> You can get debug information for POSIX module loading.

Bingo:

[zimbrasupport@kami ~]$ /opt/zimbra/bin/zmpython
initializer: 'META-INF/services/org.python.core.JythonInitializer' not
found on sun.misc.Launcher$AppClassLoader@77cde100
initializer: 'META-INF/services/org.python.core.JythonInitializer' not
found on sun.misc.Launcher$ExtClassLoader@6d6f0472
Failed to load native POSIX impl; falling back on Java impl. Stacktrace
follows.
java.lang.UnsatisfiedLinkError: /tmp/jffi9200968577849985291.tmp:
/tmp/jffi9200968577849985291.tmp: failed to map segment from shared object:
Operation not permitted
        at java.lang.ClassLoader$NativeLibrary.load(Native Method)
        at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1807)
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1703)
        at java.lang.Runtime.load0(Runtime.java:770)
        at java.lang.System.load(System.java:1003)
        at com.kenai.jffi.Init.loadFromJar(Init.java:127)
        at com.kenai.jffi.Init.load(Init.java:64)
        at
com.kenai.jffi.Foreign$InstanceHolder.getInstanceHolder(Foreign.java:35)
        at com.kenai.jffi.Foreign$InstanceHolder.<clinit>(Foreign.java:31)
        at com.kenai.jffi.Foreign.getInstance(Foreign.java:81)
        at com.kenai.jffi.Internals.getErrnoSaveFunction(Internals.java:30)
        at
com.kenai.jaffl.provider.jffi.StubCompiler.getErrnoSaveFunction(StubCompiler.java:85)
        at
com.kenai.jaffl.provider.jffi.StubCompiler.<clinit>(StubCompiler.java:35)
        at
com.kenai.jaffl.provider.jffi.AsmLibraryLoader.generateInterfaceImpl(AsmLibraryLoader.java:161)
        at
com.kenai.jaffl.provider.jffi.AsmLibraryLoader.loadLibrary(AsmLibraryLoader.java:125)
        at
com.kenai.jaffl.provider.jffi.Provider.loadLibrary(Provider.java:31)
        at
com.kenai.jaffl.provider.jffi.Provider.loadLibrary(Provider.java:25)
        at com.kenai.jaffl.Library.loadLibrary(Library.java:76)
        at
org.python.posix.POSIXFactory$LinuxLibCProvider$SingletonHolder.<clinit>(POSIXFactory.java:105)
        at
org.python.posix.POSIXFactory$LinuxLibCProvider.getLibC(POSIXFactory.java:109)
        at org.python.posix.BaseNativePOSIX.<init>(BaseNativePOSIX.java:29)
        at org.python.posix.LinuxPOSIX.<init>(LinuxPOSIX.java:17)
        at
org.python.posix.POSIXFactory.loadLinuxPOSIX(POSIXFactory.java:67)
        at org.python.posix.POSIXFactory.loadPOSIX(POSIXFactory.java:28)
        at org.python.posix.LazyPOSIX.loadPOSIX(LazyPOSIX.java:28)
        at org.python.posix.LazyPOSIX.posix(LazyPOSIX.java:24)
        at org.python.posix.LazyPOSIX.isatty(LazyPOSIX.java:135)
        at org.python.core.io.StreamIO.isatty(StreamIO.java:234)
        at
org.python.core.io.BufferedIOMixin.isatty(BufferedIOMixin.java:90)
        at org.python.core.io.TextIOBase.isatty(TextIOBase.java:175)
        at org.python.core.PyFile.file_isatty(PyFile.java:485)
        at org.python.core.PyFile.isatty(PyFile.java:480)
        at
org.python.core.PySystemState.initEncoding(PySystemState.java:245)
        at org.python.core.PySystemState.<init>(PySystemState.java:189)
        at
org.python.core.PySystemState.doInitialize(PySystemState.java:890)
        at org.python.core.PySystemState.initialize(PySystemState.java:800)
        at org.python.core.PySystemState.initialize(PySystemState.java:750)
        at org.python.core.PySystemState.initialize(PySystemState.java:743)
        at org.python.util.jython.run(jython.java:150)
        at org.python.util.jython.main(jython.java:129)
import: trying org.python.core.exceptions as builtin modules in
SysPathJavaLoader
import: 'exceptions' as org.python.core.exceptions in builtin modules
Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06)
[Java HotSpot(TM) 64-Bit Server VM (Sun Microsystems Inc.)] on java1.6.0_25



--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration

------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users

[Quanah Gibson-Mount-3]

--On Thursday, July 07, 2011 4:28 PM -0700 Quanah Gibson-Mount
<[hidden email]> wrote:

> --On Thursday, July 07, 2011 4:19 PM -0700 Yuji YAMANO
> <[hidden email]> wrote:
>
>> On Thu, 07 Jul 2011 12:41:42 -0700, Quanah Gibson-Mount
>> <[hidden email]> wrote:
>>
>>> So for the working one, I see it says it is using native posix.  For the
>>> non-working one, I see it is not using native posix.  No clue why there
>>> is  a difference. :/
>>
>> Could you invoke jython with -vv option?
>> You can get debug information for POSIX module loading.
>
> Bingo:
>
> [zimbrasupport@kami ~]$ /opt/zimbra/bin/zmpython
> initializer: 'META-INF/services/org.python.core.JythonInitializer' not
> found on sun.misc.Launcher$AppClassLoader@77cde100
> initializer: 'META-INF/services/org.python.core.JythonInitializer' not
> found on sun.misc.Launcher$ExtClassLoader@6d6f0472
> Failed to load native POSIX impl; falling back on Java impl. Stacktrace
> follows.
> java.lang.UnsatisfiedLinkError: /tmp/jffi9200968577849985291.tmp:
> /tmp/jffi9200968577849985291.tmp: failed to map segment from shared
> object:  Operation not permitted
>         at java.lang.ClassLoader$NativeLibrary.load(Native Method)

It appears this is because they mounted /tmp noexec, so it is not a KVM
issue at all.

Thanks very much!

--Quanah


--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration

------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users