Memory leak in Jython 2.7.1 RC2?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Memory leak in Jython 2.7.1 RC2?

Tobias.Brieden

Hi,

 

we are using Jython embedded in an Open Source Java application to allow our users to script certain behavior. We are really looking forward to the release of Jython 2.7.1, since we are currently using Jython 2.5.1 and weren't able to upgrade in the meantime due to various Jython bugs. Therefore, I am lurking on the developer mailing list to be up to date with the current development and I wanted to give you some early feedback to the RC2 of 2.7.1.

 

One problem we encountered in an earlier Jython version (2.5.2) was the memory leak described in this bug report: http://bugs.jython.org/issue2026 . The bug is marked as fixed and I wanted to double check this. Therefore, I used the code snipped contained in the bug report to reconstruct the issue. The original issue seems to be resolved, but I encountered another problem. If I run the aforementioned code for a short period of time, the consumed heap increases quite fast: ~120MB after ~60 seconds. I inspected the heap dump and most the memory is consumed by a ConcurrentHashMap in PySystemState. I took a quick peek at the source and found this:

 

PySystemState has a static final ConcurrentHashMap named sysClosers [0]. If a new PySystemStateCloser is created, a Key-Value-Pair of a WeakReference to a PySystemState and a reference to the PySystemStateCloser is added to the map [1], but never removed. Even if the WeakReference is finalized, if looks like the value will still remain in the map, since there is no removal code for broken references.

 

 

I haven't created a bug report for this issue, since the RC is not officially announced, but I can do this if requested.

 

 

Thanks for your efforts!

 

Tobias

 

 

[0] https://hg.python.org/jython/file/d4cd06b8c8c7/src/org/python/core/PySystemState.java#l199

[1] https://hg.python.org/jython/file/d4cd06b8c8c7/src/org/python/core/PySystemState.java#l1575

 

 


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Jython-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-dev
Reply | Threaded
Open this post in threaded view
|

Re: Memory leak in Jython 2.7.1 RC2?

Stefan Richthofer
Tobias,
thanks for testing 2.7.1 RC2!
I'd recommend to file this as a new issue on bugs.jython.org.
It is better tracked there and not so likely to get lost. We can have additional discussion there.
 
Best
 
-Stefan
 
Gesendet: Donnerstag, 01. Juni 2017 um 13:00 Uhr
Von: [hidden email]
An: [hidden email]
Betreff: [Jython-dev] Memory leak in Jython 2.7.1 RC2?

Hi,

 

we are using Jython embedded in an Open Source Java application to allow our users to script certain behavior. We are really looking forward to the release of Jython 2.7.1, since we are currently using Jython 2.5.1 and weren't able to upgrade in the meantime due to various Jython bugs. Therefore, I am lurking on the developer mailing list to be up to date with the current development and I wanted to give you some early feedback to the RC2 of 2.7.1.

 

One problem we encountered in an earlier Jython version (2.5.2) was the memory leak described in this bug report: http://bugs.jython.org/issue2026 . The bug is marked as fixed and I wanted to double check this. Therefore, I used the code snipped contained in the bug report to reconstruct the issue. The original issue seems to be resolved, but I encountered another problem. If I run the aforementioned code for a short period of time, the consumed heap increases quite fast: ~120MB after ~60 seconds. I inspected the heap dump and most the memory is consumed by a ConcurrentHashMap in PySystemState. I took a quick peek at the source and found this:

 

PySystemState has a static final ConcurrentHashMap named sysClosers [0]. If a new PySystemStateCloser is created, a Key-Value-Pair of a WeakReference to a PySystemState and a reference to the PySystemStateCloser is added to the map [1], but never removed. Even if the WeakReference is finalized, if looks like the value will still remain in the map, since there is no removal code for broken references.

 

 

I haven't created a bug report for this issue, since the RC is not officially announced, but I can do this if requested.

 

 

Thanks for your efforts!

 

Tobias

 

 

[0] https://hg.python.org/jython/file/d4cd06b8c8c7/src/org/python/core/PySystemState.java#l199

[1] https://hg.python.org/jython/file/d4cd06b8c8c7/src/org/python/core/PySystemState.java#l1575

 

 

------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________ Jython-dev mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/jython-dev

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Jython-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-dev
Reply | Threaded
Open this post in threaded view
|

Re: Memory leak in Jython 2.7.1 RC2?

Tobias.Brieden
In reply to this post by Tobias.Brieden

I created a bug report: http://bugs.jython.org/issue2597

 

By the way, is there any chance that the bug mentioned the following bug report is tackled in 2.7.1?:  http://bugs.jython.org/issue2513

 

 

Tobias

 

Von: Stefan Richthofer [mailto:[hidden email]]
Gesendet: Freitag, 2. Juni 2017 14:17
An: Brieden, Tobias
Cc: Jython Developers
Betreff: Re: [Jython-dev] Memory leak in Jython 2.7.1 RC2?

 

Tobias,

thanks for testing 2.7.1 RC2!

I'd recommend to file this as a new issue on bugs.jython.org.

It is better tracked there and not so likely to get lost. We can have additional discussion there.

 

Best

 

-Stefan

 

Gesendet: Donnerstag, 01. Juni 2017 um 13:00 Uhr
Von: [hidden email]
An: [hidden email]
Betreff: [Jython-dev] Memory leak in Jython 2.7.1 RC2?

Hi,

 

we are using Jython embedded in an Open Source Java application to allow our users to script certain behavior. We are really looking forward to the release of Jython 2.7.1, since we are currently using Jython 2.5.1 and weren't able to upgrade in the meantime due to various Jython bugs. Therefore, I am lurking on the developer mailing list to be up to date with the current development and I wanted to give you some early feedback to the RC2 of 2.7.1.

 

One problem we encountered in an earlier Jython version (2.5.2) was the memory leak described in this bug report: http://bugs.jython.org/issue2026 . The bug is marked as fixed and I wanted to double check this. Therefore, I used the code snipped contained in the bug report to reconstruct the issue. The original issue seems to be resolved, but I encountered another problem. If I run the aforementioned code for a short period of time, the consumed heap increases quite fast: ~120MB after ~60 seconds. I inspected the heap dump and most the memory is consumed by a ConcurrentHashMap in PySystemState. I took a quick peek at the source and found this:

 

PySystemState has a static final ConcurrentHashMap named sysClosers [0]. If a new PySystemStateCloser is created, a Key-Value-Pair of a WeakReference to a PySystemState and a reference to the PySystemStateCloser is added to the map [1], but never removed. Even if the WeakReference is finalized, if looks like the value will still remain in the map, since there is no removal code for broken references.

 

 

I haven't created a bug report for this issue, since the RC is not officially announced, but I can do this if requested.

 

 

Thanks for your efforts!

 

Tobias

 

 

[0] https://hg.python.org/jython/file/d4cd06b8c8c7/src/org/python/core/PySystemState.java#l199

[1] https://hg.python.org/jython/file/d4cd06b8c8c7/src/org/python/core/PySystemState.java#l1575

 

 

------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________ Jython-dev mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/jython-dev


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Jython-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-dev
Reply | Threaded
Open this post in threaded view
|

Re: Memory leak in Jython 2.7.1 RC2?

Stefan Richthofer
Tobias,
 
in http://bugs.jython.org/issue2597 I suggest a simple solution for this bug. If that turns out to be sufficient, a fix is straight forward. I suppose 2.7.1 RC2 cannot turn into final already because of http://bugs.jython.org/issue2596.
If nobody objects that we will need an RC3 phase, #2597 will be fixed then for 2.7.1.
However, we should wait a while to see if more stuff for RC3 pops up.

-Stefan
 

Gesendet: Mittwoch, 07. Juni 2017 um 09:16 Uhr
Von: [hidden email]
An: [hidden email]
Betreff: Re: [Jython-dev] Memory leak in Jython 2.7.1 RC2?

I created a bug report: http://bugs.jython.org/issue2597[http://bugs.jython.org/issue2597]
 
By the way, is there any chance that the bug mentioned the following bug report is tackled in 2.7.1?:  http://bugs.jython.org/issue2513[http://bugs.jython.org/issue2513]
 

 
Tobias
 

Von: Stefan Richthofer [mailto:[hidden email]]
Gesendet: Freitag, 2. Juni 2017 14:17
An: Brieden, Tobias
Cc: Jython Developers
Betreff: Re: [Jython-dev] Memory leak in Jython 2.7.1 RC2?
 

Tobias,

thanks for testing 2.7.1 RC2!

I'd recommend to file this as a new issue on bugs.jython.org.

It is better tracked there and not so likely to get lost. We can have additional discussion there.

 

Best

 

-Stefan

 

Gesendet: Donnerstag, 01. Juni 2017 um 13:00 Uhr
Von: [hidden email][mailto:[hidden email]]
An: [hidden email][mailto:[hidden email]]
Betreff: [Jython-dev] Memory leak in Jython 2.7.1 RC2?

Hi,
 
we are using Jython embedded in an Open Source Java application to allow our users to script certain behavior. We are really looking forward to the release of Jython 2.7.1, since we are currently using Jython 2.5.1 and weren't able to upgrade in the meantime due to various Jython bugs. Therefore, I am lurking on the developer mailing list to be up to date with the current development and I wanted to give you some early feedback to the RC2 of 2.7.1.
 
One problem we encountered in an earlier Jython version (2.5.2) was the memory leak described in this bug report: http://bugs.jython.org/issue2026[http://bugs.jython.org/issue2026] . The bug is marked as fixed and I wanted to double check this. Therefore, I used the code snipped contained in the bug report to reconstruct the issue. The original issue seems to be resolved, but I encountered another problem. If I run the aforementioned code for a short period of time, the consumed heap increases quite fast: ~120MB after ~60 seconds. I inspected the heap dump and most the memory is consumed by a ConcurrentHashMap in PySystemState. I took a quick peek at the source and found this:
 
PySystemState has a static final ConcurrentHashMap named sysClosers [0]. If a new PySystemStateCloser is created, a Key-Value-Pair of a WeakReference to a PySystemState and a reference to the PySystemStateCloser is added to the map [1], but never removed. Even if the WeakReference is finalized, if looks like the value will still remain in the map, since there is no removal code for broken references.
 
 
I haven't created a bug report for this issue, since the RC is not officially announced, but I can do this if requested.
 
 
Thanks for your efforts!
 
Tobias
 
 
[0] https://hg.python.org/jython/file/d4cd06b8c8c7/src/org/python/core/PySystemState.java#l199[https://hg.python.org/jython/file/d4cd06b8c8c7/src/org/python/core/PySystemState.java#l199]
[1] https://hg.python.org/jython/file/d4cd06b8c8c7/src/org/python/core/PySystemState.java#l1575[https://hg.python.org/jython/file/d4cd06b8c8c7/src/org/python/core/PySystemState.java#l1575]
 
 
------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________[http://sdm.link/slashdot_______________________________________________] Jython-dev mailing list [hidden email][mailto:[hidden email]] https://lists.sourceforge.net/lists/listinfo/jython-dev
------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________[http://sdm.link/slashdot_______________________________________________] Jython-dev mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/jython-dev[https://lists.sourceforge.net/lists/listinfo/jython-dev]

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Jython-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-dev