Quantcast

Compilation unit size exploding in JVM

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Compilation unit size exploding in JVM

Jeff Mitchell
Was just reminded about an issue I had back in December; I don't have logfile evidence handy but may be able to generate some if needed.

There seems to be an emitted unit sizing issue in 2.7 -- ie: python code that ran just fine in 2.5 sometimes blows up in 2.7 when the JVM tries to invoke it; the on the fly compilation may be dumping in too many tokens or something.

The workaround I had at the time was to try a few earlier builds of 2.7 or just go back to 2.5 .. ie: the standard 2.7 from the website, vs the most recent from the source repo. IIRC, the more recent the release, the larger the space used, and the higher chance the JVM would reject it. 2.5 had no problems whatsoever.

JVM being openjdk in this case on a linux VM.

Without the logfiles on hand its hard to pull out the correct keywords here,but does this ring a bell with anyone?

I seem to recall there was no easy JVM tweak to work around it short of recompiling the JVM (ugh), but I expected the issue may just take care of itself later, and it wasn't critical for me at the time.

Anyone know if this is a known issue? resolved? or an ongoing concern?

jeff


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Compilation unit size exploding in JVM

Stefan Richthofer
Jeff, is it issue #527524 (http://bugs.jython.org/issue527524) that you observed?
In http://bugs.jython.org/issue2473 Jim mentions that "In general, 2.7.0 does a better job with large methods than 2.5.x, due to collection literal splitting", so it surprises me that you observed things to have gone worse in 2.7.0.
 
- If you observed some different issue, it would be good if you could specify what behavior "blows up" actually describes. Ideal would be to file an issue with steps to reproduce, if none exists yet.
 
- If it is #527524, you can try the newest Jython 2.7.1 trunk-version. Provide a CPython 2.7 bytecode-file (.pyc) of the problematic module (e.g. by running 'python -m py_compile filename') in the same folder. (It is only needed in compilation phase; Jython will embed that bytecode into the resulting class-file for problematic methods, so if you precompile to class-files you won't have to distribute the pyc-file.)
 
Best
 
Stefan
 
 
Gesendet: Donnerstag, 12. Januar 2017 um 21:01 Uhr
Von: "Jeff Mitchell" <[hidden email]>
An: [hidden email]
Betreff: [Jython-users] Compilation unit size exploding in JVM
Was just reminded about an issue I had back in December; I don't have logfile evidence handy but may be able to generate some if needed.
 
There seems to be an emitted unit sizing issue in 2.7 -- ie: python code that ran just fine in 2.5 sometimes blows up in 2.7 when the JVM tries to invoke it; the on the fly compilation may be dumping in too many tokens or something.
 
The workaround I had at the time was to try a few earlier builds of 2.7 or just go back to 2.5 .. ie: the standard 2.7 from the website, vs the most recent from the source repo. IIRC, the more recent the release, the larger the space used, and the higher chance the JVM would reject it. 2.5 had no problems whatsoever.
 
JVM being openjdk in this case on a linux VM.
 
Without the logfiles on hand its hard to pull out the correct keywords here,but does this ring a bell with anyone?
 
I seem to recall there was no easy JVM tweak to work around it short of recompiling the JVM (ugh), but I expected the issue may just take care of itself later, and it wasn't critical for me at the time.
 
Anyone know if this is a known issue? resolved? or an ongoing concern?
 
jeff
 
------------------------------------------------------------------------------ Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi_______________________________________________ Jython-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/jython-users

------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Compilation unit size exploding in JVM

Jeff Mitchell
Awesome thanks; when I had the issue, it was with fairly small .py modules, but I was importing some of the DB modules as well.

I'll see if i can reproduce on the current trunk and get back to you; thanks!

jeff

On Fri, Jan 13, 2017 at 2:57 AM, Stefan Richthofer <[hidden email]> wrote:
Jeff, is it issue #527524 (http://bugs.jython.org/issue527524) that you observed?
In http://bugs.jython.org/issue2473 Jim mentions that "In general, 2.7.0 does a better job with large methods than 2.5.x, due to collection literal splitting", so it surprises me that you observed things to have gone worse in 2.7.0.
 
- If you observed some different issue, it would be good if you could specify what behavior "blows up" actually describes. Ideal would be to file an issue with steps to reproduce, if none exists yet.
 
- If it is #527524, you can try the newest Jython 2.7.1 trunk-version. Provide a CPython 2.7 bytecode-file (.pyc) of the problematic module (e.g. by running 'python -m py_compile filename') in the same folder. (It is only needed in compilation phase; Jython will embed that bytecode into the resulting class-file for problematic methods, so if you precompile to class-files you won't have to distribute the pyc-file.)
 
Best
 
Stefan
 
 
Gesendet: Donnerstag, 12. Januar 2017 um 21:01 Uhr
Von: "Jeff Mitchell" <[hidden email]>
An: [hidden email]
Betreff: [Jython-users] Compilation unit size exploding in JVM
Was just reminded about an issue I had back in December; I don't have logfile evidence handy but may be able to generate some if needed.
 
There seems to be an emitted unit sizing issue in 2.7 -- ie: python code that ran just fine in 2.5 sometimes blows up in 2.7 when the JVM tries to invoke it; the on the fly compilation may be dumping in too many tokens or something.
 
The workaround I had at the time was to try a few earlier builds of 2.7 or just go back to 2.5 .. ie: the standard 2.7 from the website, vs the most recent from the source repo. IIRC, the more recent the release, the larger the space used, and the higher chance the JVM would reject it. 2.5 had no problems whatsoever.
 
JVM being openjdk in this case on a linux VM.
 
Without the logfiles on hand its hard to pull out the correct keywords here,but does this ring a bell with anyone?
 
I seem to recall there was no easy JVM tweak to work around it short of recompiling the JVM (ugh), but I expected the issue may just take care of itself later, and it wasn't critical for me at the time.
 
Anyone know if this is a known issue? resolved? or an ongoing concern?
 
jeff
 
------------------------------------------------------------------------------ Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi_______________________________________________ Jython-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/jython-users


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
Jython-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-users
Loading...