Possible memory leak with CBV's ?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Possible memory leak with CBV's ?

Andrei Antoukh-2
I've been experimenting, and I found a strange behavior. In the View class if I define the __ del__ method, and is never executed. In instances of other classes, which are members of View, feel the same behavior (eg HttpRequest is not removed by a garbage collector).

The fault lies primarily in this line (django/views/generic/base.py):

def view(request, *args, **kwargs):
    # [...]
    if hasattr(self, 'get') and not hasattr(self, 'head'):
        self.head = self.get

    # [...]


When you create alias methods once the class has been instantiated seems to make some references (cyclical?) which prevent the garbage collector remove them.

I have reproduced the same behavior with simple classes to verify the problem: https://gist.github.com/4475138
If a custom view defines a head method, the problem disappears.

So, is how it should work? I'm wrong about something?

Thank you very much.

--
Andrey Antukh - Андрей Антух - <[hidden email]>
http://www.niwi.be/about.html
http://www.kaleidos.net/A5694F/

"Linux is for people who hate Windows, BSD is for people who love UNIX"
"Social Engineer -> Because there is no patch for human stupidity"

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Reply | Threaded
Open this post in threaded view
|

Re: Possible memory leak with CBV's ?

Anssi Kääriäinen
On 7 tammi, 16:06, Andrey Antukh <[hidden email]> wrote:

> I've been experimenting, and I found a strange behavior. In the View class
> if I define the __ del__ method, and is never executed. In instances of
> other classes, which are members of View, feel the same behavior (eg
> HttpRequest is not removed by a garbage collector).
>
> The fault lies primarily in this line (django/views/generic/base.py):
>
> def view(request, *args, **kwargs):
>     # [...]
> *    if hasattr(self, 'get') and not hasattr(self, 'head'):
>         self.head = self.get*
>     # [...]
>
> When you create alias methods once the class has been instantiated seems to
> make some references (cyclical?) which prevent the garbage collector remove
> them.
>
> I have reproduced the same behavior with simple classes to verify the
> problem:https://gist.github.com/4475138
> If a custom view defines a head method, the problem disappears.
>
> So, is how it should work? I'm wrong about something?
>
> Thank you very much.

The reason is that if you define a __del__ method for a class, and an
instance of that class is part of a reference cycle, then Python will
not release any object which is reachable from the cycle. This is
ugly, and once you know this you know to avoid __del__ methods if
possible. More here: http://docs.python.org/2/library/gc.html#gc.garbage

You can try to have a special cleaner object - the idea is that on
__init__ of an in-cycle object you assign a cleaner object to self.
Pass the cleaner object the data you want to clean up. The data passed
must not be part of the cycle or the trick doesn't work. When the in-
cycle object is garbage collected, then the cleaner object will be
garbage collected too (only reference to it is from the in-cycle
object). Now, the __del__ of the cleaner object is called and you can
do cleanup there (again, assuming that the cleaner is not part of a
cycle). This is ugly but if you have a cyclic object and you need to
have __del__ I don't know of anything better (ideas welcome!).

This idea is implemented here: https://github.com/akaariai/django_pooled/blob/master/base.py#L134
- the idea is that when a ConnectionWrapper is created we also create
PoolReleaser. The PoolReleaser has the needed data to release a
connection to the pool. When the ConnectionWrapper is gc'd, the
PoolReleaser's __del__ is called and we can release the connection
back to pool.

Avoid __del__ if possible,
 - Anssi

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

Reply | Threaded
Open this post in threaded view
|

Re: Possible memory leak with CBV's ?

Andrei Antoukh-2
Thank you very much for the great explanation.
I'll try to avoid using __del__ methods whenever possible.

Andrey

2013/1/7 Anssi Kääriäinen <[hidden email]>
Avoid __del__ if possible,



--
Andrey Antukh - Андрей Антух - <[hidden email]>
http://www.niwi.be/about.html
http://www.kaleidos.net/A5694F/

"Linux is for people who hate Windows, BSD is for people who love UNIX"
"Social Engineer -> Because there is no patch for human stupidity"

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to [hidden email].
To unsubscribe from this group, send email to [hidden email].
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.