Debug potential memory leaks

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Debug potential memory leaks

Andreas Pakulat
Hi,

I'm having a small 'systray' application that monitors a jenkins service and it seems there's some kind of memory leak, especially on MacOSX. After having it run for a while (with the machine going to sleep now and again) it starts eating quite some memory - if I understand top/ActivityMonitor correctly - and amd wondering now how I'd best debug a PyQt application's memory consumption.

So what tools do you guys usually use to debug such memory leaks? I've tried valgrind on Linux but there's no good hint there, possibly also due to the way Python works.

Andreas

_______________________________________________
PyQt mailing list    [hidden email]
http://www.riverbankcomputing.com/mailman/listinfo/pyqt
Reply | Threaded
Open this post in threaded view
|

Re: Debug potential memory leaks

Phil Thompson-5
On 07/04/2015 8:47 pm, Andreas Pakulat wrote:

> Hi,
>
> I'm having a small 'systray' application that monitors a jenkins
> service
> and it seems there's some kind of memory leak, especially on MacOSX.
> After
> having it run for a while (with the machine going to sleep now and
> again)
> it starts eating quite some memory - if I understand
> top/ActivityMonitor
> correctly - and amd wondering now how I'd best debug a PyQt
> application's
> memory consumption.
>
> So what tools do you guys usually use to debug such memory leaks? I've
> tried valgrind on Linux but there's no good hint there, possibly also
> due
> to the way Python works.

You could try a debug version of Python that will track objects. If that
shows objects are not being collected then it is probably a PyQt
problem. Otherwise it might be more likely to be a Qt problem.

Phil
_______________________________________________
PyQt mailing list    [hidden email]
http://www.riverbankcomputing.com/mailman/listinfo/pyqt
Reply | Threaded
Open this post in threaded view
|

Re: Debug potential memory leaks

Kovid Goyal
In reply to this post by Andreas Pakulat
See the various functions in:
https://github.com/kovidgoyal/calibre/blob/master/src/calibre/utils/mem.py

On Tue, Apr 07, 2015 at 09:47:36PM +0200, Andreas Pakulat wrote:

> Hi,
>
> I'm having a small 'systray' application that monitors a jenkins service
> and it seems there's some kind of memory leak, especially on MacOSX. After
> having it run for a while (with the machine going to sleep now and again)
> it starts eating quite some memory - if I understand top/ActivityMonitor
> correctly - and amd wondering now how I'd best debug a PyQt application's
> memory consumption.
>
> So what tools do you guys usually use to debug such memory leaks? I've
> tried valgrind on Linux but there's no good hint there, possibly also due
> to the way Python works.
>
> Andreas

> _______________________________________________
> PyQt mailing list    [hidden email]
> http://www.riverbankcomputing.com/mailman/listinfo/pyqt


--
_____________________________________

Dr. Kovid Goyal
http://www.kovidgoyal.net
http://calibre-ebook.com
_____________________________________
_______________________________________________
PyQt mailing list    [hidden email]
http://www.riverbankcomputing.com/mailman/listinfo/pyqt
Reply | Threaded
Open this post in threaded view
|

Re: Debug potential memory leaks

Andreas Pakulat
Hi,

On Wed, Apr 8, 2015 at 5:24 AM, Kovid Goyal <[hidden email]> wrote:
See the various functions in:
https://github.com/kovidgoyal/calibre/blob/master/src/calibre/utils/mem.py

Ah, the gc-histogram may already point to the right place. A suspiciously matching count of qactions, tuple's and even functions that are ever-growing... Thanks for that.

Andreas

_______________________________________________
PyQt mailing list    [hidden email]
http://www.riverbankcomputing.com/mailman/listinfo/pyqt
Reply | Threaded
Open this post in threaded view
|

Re: Debug potential memory leaks

Andreas Pakulat
In reply to this post by Phil Thompson-5
Hi Phil,

On Tue, Apr 7, 2015 at 9:55 PM, Phil Thompson <[hidden email]> wrote:
On 07/04/2015 8:47 pm, Andreas Pakulat wrote:
Hi,

I'm having a small 'systray' application that monitors a jenkins service
and it seems there's some kind of memory leak, especially on MacOSX. After
having it run for a while (with the machine going to sleep now and again)
it starts eating quite some memory - if I understand top/ActivityMonitor
correctly - and amd wondering now how I'd best debug a PyQt application's
memory consumption.

So what tools do you guys usually use to debug such memory leaks? I've
tried valgrind on Linux but there's no good hint there, possibly also due
to the way Python works.

You could try a debug version of Python that will track objects. If that shows objects are not being collected then it is probably a PyQt problem. Otherwise it might be more likely to be a Qt problem.

I'll check that option if I can determine its not my own code hanging on to objects it shouldn't hang onto anymore...

Andreas

_______________________________________________
PyQt mailing list    [hidden email]
http://www.riverbankcomputing.com/mailman/listinfo/pyqt
Reply | Threaded
Open this post in threaded view
|

Re: Debug potential memory leaks

Andreas Pakulat
Hi,

On Wed, Apr 8, 2015 at 10:42 AM, Andreas Pakulat <[hidden email]> wrote:
Hi Phil,

On Tue, Apr 7, 2015 at 9:55 PM, Phil Thompson <[hidden email]> wrote:
On 07/04/2015 8:47 pm, Andreas Pakulat wrote:
Hi,

I'm having a small 'systray' application that monitors a jenkins service
and it seems there's some kind of memory leak, especially on MacOSX. After
having it run for a while (with the machine going to sleep now and again)
it starts eating quite some memory - if I understand top/ActivityMonitor
correctly - and amd wondering now how I'd best debug a PyQt application's
memory consumption.

So what tools do you guys usually use to debug such memory leaks? I've
tried valgrind on Linux but there's no good hint there, possibly also due
to the way Python works.

You could try a debug version of Python that will track objects. If that shows objects are not being collected then it is probably a PyQt problem. Otherwise it might be more likely to be a Qt problem.

I'll check that option if I can determine its not my own code hanging on to objects it shouldn't hang onto anymore...

So, it seems like I'm leaking QAction's and they're not garbage-collected for quite a while even when they're not connected to anymore anymore. I can 'fix' the problem by setting their parent QObject to None and explicitly calling gc.collect, without the latter the objgraph module (http://mg.pov.lt/objgraph.py) shows them as  objects without any references to them even after another minute of the application running.

Maybe a little background on what the app does helps to understand this a little better. It polls a jenkins instance for states of the various jobs and for each job a QAction is created and added to the menu of a QSysTray object. Each time the data is retrieved all existing QActions are removed from teh menu via removeAction and the dictionary that kept the instances is cleared. Then a new set of actions is created and added to the menu for the possibly-changed set of jobs and their changed icons. The parent object of the actions is the systray's menu, hence having to set their parent to None when I want to get rid of them makes sense.

What I find surprising though is that Python's gc does not clean them up even after a while unless I explicitly call gc.collect. Does someone have an explanation for this?

Andreas

_______________________________________________
PyQt mailing list    [hidden email]
http://www.riverbankcomputing.com/mailman/listinfo/pyqt