[Django] #30796: Chaining select_related mutates original QuerySet.

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[Django] #30796: Chaining select_related mutates original QuerySet.

Django
#30796: Chaining select_related mutates original QuerySet.
-------------------------------------+-------------------------------------
               Reporter:  Darren     |          Owner:  nobody
  Maki                               |
                   Type:  Bug        |         Status:  new
              Component:  Database   |        Version:  2.2
  layer (models, ORM)                |       Keywords:  select_related
               Severity:  Normal     |  prefetch_related mutate queryset
           Triage Stage:             |      Has patch:  0
  Unreviewed                         |
    Needs documentation:  0          |    Needs tests:  0
Patch needs improvement:  0          |  Easy pickings:  0
                  UI/UX:  0          |
-------------------------------------+-------------------------------------
 When creating a new QuerySet from an existing QuerySet that has had
 'select_related' applied, if you apply another 'select_related' to the new
 QuerySet it will mutate the original QuerySet to also have the extra
 'select_related'.

 '''models.py'''

 {{{
 from django.db import models

 class ModelA(models.Model):
     pass

 class ModelB(models.Model):
     pass

 class ModelC(models.Model):
     model_a = models.ForeignKey('foobar.ModelA', on_delete=models.CASCADE)
     model_b = models.ForeignKey('foobar.ModelB', on_delete=models.CASCADE)
 }}}

 '''test.py'''

 {{{
 query_1 = ModelC.objects.select_related('model_a')
 print('QUERY 1:', str(query_1.query))

 query_2 = query_1.select_related('model_b')
 print('QUERY 2:', str(query_2.query))

 print('QUERY 1:', str(query_1.query))

 if str(query_1.query) == str(query_2.query):
     print('\n!!! The two queries are the same !!!\n')
 }}}

 '''output'''

 {{{
 QUERY 1: SELECT "foobar_modelc"."id", "foobar_modelc"."model_a_id",
 "foobar_modelc"."model_b_id", "foobar_modela"."id" FROM "foobar_modelc"
 INNER JOIN "foobar_modela" ON ("foobar_modelc"."model_a_id" =
 "foobar_modela"."id") 139663365718536
 QUERY 2: SELECT "foobar_modelc"."id", "foobar_modelc"."model_a_id",
 "foobar_modelc"."model_b_id", "foobar_modela"."id", "foobar_modelb"."id"
 FROM "foobar_modelc" INNER JOIN "foobar_modela" ON
 ("foobar_modelc"."model_a_id" = "foobar_modela"."id") INNER JOIN
 "foobar_modelb" ON ("foobar_modelc"."model_b_id" = "foobar_modelb"."id")
 139663366175376
 QUERY 1: SELECT "foobar_modelc"."id", "foobar_modelc"."model_a_id",
 "foobar_modelc"."model_b_id", "foobar_modela"."id", "foobar_modelb"."id"
 FROM "foobar_modelc" INNER JOIN "foobar_modela" ON
 ("foobar_modelc"."model_a_id" = "foobar_modela"."id") INNER JOIN
 "foobar_modelb" ON ("foobar_modelc"."model_b_id" = "foobar_modelb"."id")
 139663365718536

 !!! The two queries are the same !!!

 }}}

 The expectation is that the original QuerySet is not mutated, and the two
 queries are different. This behavior also happens with 'prefetch_related'.

 Since the QuerySet methods call 'self._clone()', and state that they
 return 'a new QuerySet instance', this behavior does not seem correct.

--
Ticket URL: <https://code.djangoproject.com/ticket/30796>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/055.c00f5aad0b2877f8fa2beb8af9e8859e%40djangoproject.com.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #30796: Chaining select_related mutates original QuerySet.

Django
#30796: Chaining select_related mutates original QuerySet.
-------------------------------------+-------------------------------------
     Reporter:  Darren Maki          |                    Owner:  nobody
         Type:  Bug                  |                   Status:  new
    Component:  Database layer       |                  Version:  2.2
  (models, ORM)                      |
     Severity:  Normal               |               Resolution:
     Keywords:  select_related       |             Triage Stage:
  prefetch_related mutate queryset   |  Unreviewed
    Has patch:  0                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------
Description changed by Darren Maki:

Old description:

> When creating a new QuerySet from an existing QuerySet that has had
> 'select_related' applied, if you apply another 'select_related' to the
> new QuerySet it will mutate the original QuerySet to also have the extra
> 'select_related'.
>
> '''models.py'''
>
> {{{
> from django.db import models
>
> class ModelA(models.Model):
>     pass
>
> class ModelB(models.Model):
>     pass
>
> class ModelC(models.Model):
>     model_a = models.ForeignKey('foobar.ModelA',
> on_delete=models.CASCADE)
>     model_b = models.ForeignKey('foobar.ModelB',
> on_delete=models.CASCADE)
> }}}
>
> '''test.py'''
>
> {{{
> query_1 = ModelC.objects.select_related('model_a')
> print('QUERY 1:', str(query_1.query))
>
> query_2 = query_1.select_related('model_b')
> print('QUERY 2:', str(query_2.query))
>
> print('QUERY 1:', str(query_1.query))
>
> if str(query_1.query) == str(query_2.query):
>     print('\n!!! The two queries are the same !!!\n')
> }}}
>
> '''output'''
>
> {{{
> QUERY 1: SELECT "foobar_modelc"."id", "foobar_modelc"."model_a_id",
> "foobar_modelc"."model_b_id", "foobar_modela"."id" FROM "foobar_modelc"
> INNER JOIN "foobar_modela" ON ("foobar_modelc"."model_a_id" =
> "foobar_modela"."id") 139663365718536
> QUERY 2: SELECT "foobar_modelc"."id", "foobar_modelc"."model_a_id",
> "foobar_modelc"."model_b_id", "foobar_modela"."id", "foobar_modelb"."id"
> FROM "foobar_modelc" INNER JOIN "foobar_modela" ON
> ("foobar_modelc"."model_a_id" = "foobar_modela"."id") INNER JOIN
> "foobar_modelb" ON ("foobar_modelc"."model_b_id" = "foobar_modelb"."id")
> 139663366175376
> QUERY 1: SELECT "foobar_modelc"."id", "foobar_modelc"."model_a_id",
> "foobar_modelc"."model_b_id", "foobar_modela"."id", "foobar_modelb"."id"
> FROM "foobar_modelc" INNER JOIN "foobar_modela" ON
> ("foobar_modelc"."model_a_id" = "foobar_modela"."id") INNER JOIN
> "foobar_modelb" ON ("foobar_modelc"."model_b_id" = "foobar_modelb"."id")
> 139663365718536
>
> !!! The two queries are the same !!!
>
> }}}
>
> The expectation is that the original QuerySet is not mutated, and the two
> queries are different. This behavior also happens with
> 'prefetch_related'.
>
> Since the QuerySet methods call 'self._clone()', and state that they
> return 'a new QuerySet instance', this behavior does not seem correct.
New description:

 When creating a new QuerySet from an existing QuerySet that has had
 'select_related' applied, if you apply another 'select_related' to the new
 QuerySet it will mutate the original QuerySet to also have the extra
 'select_related'.

 '''models.py'''

 {{{
 from django.db import models

 class ModelA(models.Model):
     pass

 class ModelB(models.Model):
     pass

 class ModelC(models.Model):
     model_a = models.ForeignKey('foobar.ModelA', on_delete=models.CASCADE)
     model_b = models.ForeignKey('foobar.ModelB', on_delete=models.CASCADE)
 }}}

 '''test.py'''

 {{{
 query_1 = ModelC.objects.select_related('model_a')
 print('QUERY 1:', str(query_1.query))

 query_2 = query_1.select_related('model_b')
 print('QUERY 2:', str(query_2.query))

 print('QUERY 1:', str(query_1.query))

 if str(query_1.query) == str(query_2.query):
     print('\n!!! The two queries are the same !!!\n')
 }}}

 '''output'''

 {{{
 QUERY 1: SELECT "foobar_modelc"."id", "foobar_modelc"."model_a_id",
 "foobar_modelc"."model_b_id", "foobar_modela"."id" FROM "foobar_modelc"
 INNER JOIN "foobar_modela" ON ("foobar_modelc"."model_a_id" =
 "foobar_modela"."id")
 QUERY 2: SELECT "foobar_modelc"."id", "foobar_modelc"."model_a_id",
 "foobar_modelc"."model_b_id", "foobar_modela"."id", "foobar_modelb"."id"
 FROM "foobar_modelc" INNER JOIN "foobar_modela" ON
 ("foobar_modelc"."model_a_id" = "foobar_modela"."id") INNER JOIN
 "foobar_modelb" ON ("foobar_modelc"."model_b_id" = "foobar_modelb"."id")
 QUERY 1: SELECT "foobar_modelc"."id", "foobar_modelc"."model_a_id",
 "foobar_modelc"."model_b_id", "foobar_modela"."id", "foobar_modelb"."id"
 FROM "foobar_modelc" INNER JOIN "foobar_modela" ON
 ("foobar_modelc"."model_a_id" = "foobar_modela"."id") INNER JOIN
 "foobar_modelb" ON ("foobar_modelc"."model_b_id" = "foobar_modelb"."id")

 !!! The two queries are the same !!!

 }}}

 The expectation is that the original QuerySet is not mutated, and the two
 queries are different. This behavior also happens with 'prefetch_related'.

 Since the QuerySet methods call 'self._clone()', and state that they
 return 'a new QuerySet instance', this behavior does not seem correct.

--

--
Ticket URL: <https://code.djangoproject.com/ticket/30796#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/070.117cb5405af9819cb5114a025575693d%40djangoproject.com.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #30796: Chaining select_related mutates original QuerySet.

Django
In reply to this post by Django
#30796: Chaining select_related mutates original QuerySet.
-------------------------------------+-------------------------------------
     Reporter:  Darren Maki          |                    Owner:  nobody
         Type:  Bug                  |                   Status:  new
    Component:  Database layer       |                  Version:  2.2
  (models, ORM)                      |
     Severity:  Normal               |               Resolution:
     Keywords:  select_related       |             Triage Stage:  Accepted
  prefetch_related mutate queryset   |
    Has patch:  0                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------
Changes (by Simon Charette):

 * stage:  Unreviewed => Accepted


Comment:

 This seems to have been happening forever.

 `sql.Query.select_related` is made a `dict` on `.add_select_related` but
 never copied on `.clone`.

--
Ticket URL: <https://code.djangoproject.com/ticket/30796#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/070.8ff0a0ffe0fae42fc8ca5dbd6b549c64%40djangoproject.com.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #30796: Chaining select_related mutates original QuerySet.

Django
In reply to this post by Django
#30796: Chaining select_related mutates original QuerySet.
-------------------------------------+-------------------------------------
     Reporter:  Darren Maki          |                    Owner:  Simon
                                     |  Charette
         Type:  Bug                  |                   Status:  assigned
    Component:  Database layer       |                  Version:  2.2
  (models, ORM)                      |
     Severity:  Normal               |               Resolution:
     Keywords:  select_related       |             Triage Stage:  Accepted
  prefetch_related mutate queryset   |
    Has patch:  0                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------
Changes (by Simon Charette):

 * status:  new => assigned
 * owner:  nobody => Simon Charette


--
Ticket URL: <https://code.djangoproject.com/ticket/30796#comment:3>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/070.9a80435d33800b2123a36181a80f3213%40djangoproject.com.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #30796: Chaining select_related mutates original QuerySet.

Django
In reply to this post by Django
#30796: Chaining select_related mutates original QuerySet.
-------------------------------------+-------------------------------------
     Reporter:  Darren Maki          |                    Owner:  Simon
                                     |  Charette
         Type:  Bug                  |                   Status:  assigned
    Component:  Database layer       |                  Version:  2.2
  (models, ORM)                      |
     Severity:  Normal               |               Resolution:
     Keywords:  select_related       |             Triage Stage:  Accepted
  prefetch_related mutate queryset   |
    Has patch:  1                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------
Changes (by Simon Charette):

 * has_patch:  0 => 1


Comment:

 Fixed in https://github.com/django/django/pull/11810

--
Ticket URL: <https://code.djangoproject.com/ticket/30796#comment:4>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/070.ef1e40cbfd25b8406f4dda59058b4d72%40djangoproject.com.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #30796: Chaining select_related mutates original QuerySet.

Django
In reply to this post by Django
#30796: Chaining select_related mutates original QuerySet.
-------------------------------------+-------------------------------------
     Reporter:  Darren Maki          |                    Owner:  Simon
                                     |  Charette
         Type:  Bug                  |                   Status:  closed
    Component:  Database layer       |                  Version:  2.2
  (models, ORM)                      |
     Severity:  Normal               |               Resolution:  fixed
     Keywords:  select_related       |             Triage Stage:  Accepted
  prefetch_related mutate queryset   |
    Has patch:  1                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak <felisiak.mariusz@…>):

 * status:  assigned => closed
 * resolution:   => fixed


Comment:

 In [changeset:"37f8f293775d0b672da8ae369d9a4e17f1db7851" 37f8f293]:
 {{{
 #!CommitTicketReference repository=""
 revision="37f8f293775d0b672da8ae369d9a4e17f1db7851"
 Fixed #30796 -- Prevented select_related() from mutating a queryset on
 chaining.

 Thanks Darren Maki for the report.
 }}}

--
Ticket URL: <https://code.djangoproject.com/ticket/30796#comment:5>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/070.65860fe918ed54cbc0a11d15f825ccff%40djangoproject.com.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #30796: Chaining select_related mutates original QuerySet.

Django
In reply to this post by Django
#30796: Chaining select_related mutates original QuerySet.
-------------------------------------+-------------------------------------
     Reporter:  Darren Maki          |                    Owner:  Simon
                                     |  Charette
         Type:  Bug                  |                   Status:  closed
    Component:  Database layer       |                  Version:  2.2
  (models, ORM)                      |
     Severity:  Normal               |               Resolution:  fixed
     Keywords:  select_related       |             Triage Stage:  Accepted
  prefetch_related mutate queryset   |
    Has patch:  1                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------

Comment (by Mariusz Felisiak <felisiak.mariusz@…>):

 In [changeset:"6b7bd079a6fc4e84553262b65074dbe0af456b03" 6b7bd079]:
 {{{
 #!CommitTicketReference repository=""
 revision="6b7bd079a6fc4e84553262b65074dbe0af456b03"
 [3.0.x] Fixed #30796 -- Prevented select_related() from mutating a
 queryset on chaining.

 Thanks Darren Maki for the report.

 Backport of 37f8f293775d0b672da8ae369d9a4e17f1db7851 from master
 }}}

--
Ticket URL: <https://code.djangoproject.com/ticket/30796#comment:6>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/070.f1efc64ce51b5675d841aaff74953fae%40djangoproject.com.