[Django] #29016: Reuse of UpdateQuery breaks delete some updates

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[Django] #29016: Reuse of UpdateQuery breaks delete some updates

Django
#29016: Reuse of UpdateQuery breaks delete some updates
-------------------------------------+-------------------------------------
               Reporter:  Étienne    |          Owner:  nobody
  Loks                               |
                   Type:  Bug        |         Status:  new
              Component:  Database   |        Version:  1.11
  layer (models, ORM)                |
               Severity:  Normal     |       Keywords:
           Triage Stage:             |      Has patch:  0
  Unreviewed                         |
    Needs documentation:  0          |    Needs tests:  0
Patch needs improvement:  0          |  Easy pickings:  0
                  UI/UX:  0          |
-------------------------------------+-------------------------------------
 On a model A, when deleting a foreign key pointing to a model B, some
 other foreign key of the model A pointing to the same model B may be
 nullified.

 I have isolated this behaviour on a simple project:

 {{{
 #!div style="font-size: 80%"
 models.py:
   {{{#!python
 from django.db import models


 class ChildModel(models.Model):
     name = models.CharField(max_length=200)


 class ParentModel(models.Model):
     name = models.CharField(max_length=200)
     child_1 = models.ForeignKey(ChildModel, on_delete=models.SET_NULL,
                                 related_name='parents_1', null=True)
     child_2 = models.ForeignKey(ChildModel, on_delete=models.SET_NULL,
                                 related_name='parents_2', null=True)
   }}}
 Django shell session:
   {{{#!python
 from testapp.models import ParentModel, ChildModel

 child_1 = ChildModel.objects.create(name="child_1")
 child_2 = ChildModel.objects.create(name="child_2")
 parent_1 = ParentModel.objects.create(name="parent 1", child_1=child_1,
 child_2=child_2)
 parent_2 = ParentModel.objects.create(name="parent 2", child_1=child_2,
 child_2=child_1)

 child_1.delete()
 parent_1 = ParentModel.objects.get(pk=parent_1.pk)
 parent_2 = ParentModel.objects.get(pk=parent_2.pk)
 # parent_1.child_2 and parent_2.child_1 should be normaly equal to child_2
 but...
 parent_1.child_2 is not None and parent_2.child_1 is not None
 # False is returned
   }}}
 }}}

 This simple project has been tested on an SQLite database. The same
 behaviour has been first discovered on a PostgreSQL database.

 A mis-reuse of an UpdateQuery seems to be the cause of this bug.
 The attached patch fixes the issue.

 After search on the django bug tracker I have found another issue with the
 same patch attached #28099.
 I have opened this new ticket because the issue seems to be more severe (I
 have experienced large data loss) and more general.

 This issue has been found on version 1.11 and 2.0 of Django.

--
Ticket URL: <https://code.djangoproject.com/ticket/29016>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/055.7ba0f9f770082fb44d33ac37f001701f%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29016: Reuse of UpdateQuery breaks delete some updates

Django
#29016: Reuse of UpdateQuery breaks delete some updates
-------------------------------------+-------------------------------------
     Reporter:  Étienne Loks         |                    Owner:  nobody
         Type:  Bug                  |                   Status:  new
    Component:  Database layer       |                  Version:  1.11
  (models, ORM)                      |
     Severity:  Normal               |               Resolution:
     Keywords:                       |             Triage Stage:
                                     |  Unreviewed
    Has patch:  0                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------
Changes (by Étienne Loks):

 * Attachment "fix_delete_on_update.patch" added.

 Simple patch version (no regression test yet)

--
Ticket URL: <https://code.djangoproject.com/ticket/29016>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/070.59296534484c3e7170bf06dd10938034%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29016: Reuse of UpdateQuery breaks some delete updates (was: Reuse of UpdateQuery breaks delete some updates)

Django
In reply to this post by Django
#29016: Reuse of UpdateQuery breaks some delete updates
-------------------------------------+-------------------------------------
     Reporter:  Étienne Loks         |                    Owner:  nobody
         Type:  Bug                  |                   Status:  new
    Component:  Database layer       |                  Version:  1.11
  (models, ORM)                      |
     Severity:  Normal               |               Resolution:
     Keywords:                       |             Triage Stage:
                                     |  Unreviewed
    Has patch:  0                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------

--
Ticket URL: <https://code.djangoproject.com/ticket/29016#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/070.59db96e5e11890f1f01346e065611515%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29016: Reuse of UpdateQuery breaks some delete updates

Django
In reply to this post by Django
#29016: Reuse of UpdateQuery breaks some delete updates
-------------------------------------+-------------------------------------
     Reporter:  Étienne Loks         |                    Owner:  nobody
         Type:  Bug                  |                   Status:  new
    Component:  Database layer       |                  Version:  1.11
  (models, ORM)                      |
     Severity:  Normal               |               Resolution:
     Keywords:                       |             Triage Stage:
                                     |  Unreviewed
    Has patch:  1                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------
Changes (by Étienne Loks):

 * has_patch:  0 => 1


Old description:

> On a model A, when deleting a foreign key pointing to a model B, some
> other foreign key of the model A pointing to the same model B may be
> nullified.
>
> I have isolated this behaviour on a simple project:
>
> {{{
> #!div style="font-size: 80%"
> models.py:
>   {{{#!python
> from django.db import models
>

> class ChildModel(models.Model):
>     name = models.CharField(max_length=200)
>

> class ParentModel(models.Model):
>     name = models.CharField(max_length=200)
>     child_1 = models.ForeignKey(ChildModel, on_delete=models.SET_NULL,
>                                 related_name='parents_1', null=True)
>     child_2 = models.ForeignKey(ChildModel, on_delete=models.SET_NULL,
>                                 related_name='parents_2', null=True)
>   }}}
> Django shell session:
>   {{{#!python
> from testapp.models import ParentModel, ChildModel
>
> child_1 = ChildModel.objects.create(name="child_1")
> child_2 = ChildModel.objects.create(name="child_2")
> parent_1 = ParentModel.objects.create(name="parent 1", child_1=child_1,
> child_2=child_2)
> parent_2 = ParentModel.objects.create(name="parent 2", child_1=child_2,
> child_2=child_1)
>
> child_1.delete()
> parent_1 = ParentModel.objects.get(pk=parent_1.pk)
> parent_2 = ParentModel.objects.get(pk=parent_2.pk)
> # parent_1.child_2 and parent_2.child_1 should be normaly equal to
> child_2 but...
> parent_1.child_2 is not None and parent_2.child_1 is not None
> # False is returned
>   }}}
> }}}
>
> This simple project has been tested on an SQLite database. The same
> behaviour has been first discovered on a PostgreSQL database.
>
> A mis-reuse of an UpdateQuery seems to be the cause of this bug.
> The attached patch fixes the issue.
>
> After search on the django bug tracker I have found another issue with
> the same patch attached #28099.
> I have opened this new ticket because the issue seems to be more severe
> (I have experienced large data loss) and more general.
>
> This issue has been found on version 1.11 and 2.0 of Django.
New description:

 On a model A, when deleting a foreign key pointing to a model B, some
 other foreign key of the model A pointing to the same model B may be
 nullified.

 I have isolated this behaviour on a simple project:

 {{{
 #!div style="font-size: 80%"
 models.py:
   {{{#!python
 from django.db import models


 class ChildModel(models.Model):
     name = models.CharField(max_length=200)


 class ParentModel(models.Model):
     name = models.CharField(max_length=200)
     child_1 = models.ForeignKey(ChildModel, on_delete=models.SET_NULL,
                                 related_name='parents_1', null=True)
     child_2 = models.ForeignKey(ChildModel, on_delete=models.SET_NULL,
                                 related_name='parents_2', null=True)
   }}}
 Django shell session:
   {{{#!python
 from testapp.models import ParentModel, ChildModel

 child_1 = ChildModel.objects.create(name="child_1")
 child_2 = ChildModel.objects.create(name="child_2")
 parent_1 = ParentModel.objects.create(name="parent 1", child_1=child_1,
 child_2=child_2)
 parent_2 = ParentModel.objects.create(name="parent 2", child_1=child_2,
 child_2=child_1)

 child_1.delete()
 parent_1 = ParentModel.objects.get(pk=parent_1.pk)
 parent_2 = ParentModel.objects.get(pk=parent_2.pk)
 # parent_1.child_2 and parent_2.child_1 should be normaly equal to child_2
 but...
 parent_1.child_2 is not None and parent_2.child_1 is not None
 # False is returned
   }}}
 }}}

 This simple project has been tested on an SQLite database. The same
 behaviour has been first discovered on a PostgreSQL database.

 A mis-reuse of an UpdateQuery seems to be the cause of this bug.

 After search on the django bug tracker I have found another issue with the
 same patch attached #28099.
 I have opened this new ticket because the issue seems to be more severe (I
 have experienced large data loss) and more general.

 This issue has been found on version 1.11 and 2.0 of Django.

 I have created a new branch on my github account with patch and test
 associated: https://github.com/Nimn/django/tree/ticket_29016

--

--
Ticket URL: <https://code.djangoproject.com/ticket/29016#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/070.7ebc189f584340c9640f132d1171be10%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29016: Reuse of UpdateQuery breaks some delete updates

Django
In reply to this post by Django
#29016: Reuse of UpdateQuery breaks some delete updates
-------------------------------------+-------------------------------------
     Reporter:  Étienne Loks         |                    Owner:  nobody
         Type:  Bug                  |                   Status:  new
    Component:  Database layer       |                  Version:  1.11
  (models, ORM)                      |
     Severity:  Release blocker      |               Resolution:
     Keywords:                       |             Triage Stage:  Accepted
    Has patch:  1                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------
Changes (by Tim Graham):

 * severity:  Normal => Release blocker
 * stage:  Unreviewed => Accepted


Comment:

 I think that qualifies for a backport based on the "data loss" criteria.
 Rather than adding entirely new models, can you reuse an existing model
 (perhaps another `ForeignKey` will need to be added on an existing model)
 either in `delete` or `delete_regress`?

--
Ticket URL: <https://code.djangoproject.com/ticket/29016#comment:3>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/070.ffdc9e98a804b7ab77595b85bec2c169%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29016: Reuse of UpdateQuery breaks some delete updates

Django
In reply to this post by Django
#29016: Reuse of UpdateQuery breaks some delete updates
-------------------------------------+-------------------------------------
     Reporter:  Étienne Loks         |                    Owner:  Étienne
                                     |  Loks
         Type:  Bug                  |                   Status:  assigned
    Component:  Database layer       |                  Version:  1.11
  (models, ORM)                      |
     Severity:  Release blocker      |               Resolution:
     Keywords:                       |             Triage Stage:  Accepted
    Has patch:  1                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------
Changes (by Étienne Loks):

 * owner:  nobody => Étienne Loks
 * status:  new => assigned


Comment:

 I have changed a model in `delete_regress` to add two foreign keys (it
 seems clearer to me). I have sent a pull request for the master branch.

 If it really qualifies to a backport (I agree with this qualification),
 how can I proceed to propose a patch? I miss a proper documentation.

--
Ticket URL: <https://code.djangoproject.com/ticket/29016#comment:4>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/070.86ad7fb0152b86c0369ad33821f9ee0a%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29016: Reuse of UpdateQuery breaks some delete updates

Django
In reply to this post by Django
#29016: Reuse of UpdateQuery breaks some delete updates
-------------------------------------+-------------------------------------
     Reporter:  Étienne Loks         |                    Owner:  Étienne
                                     |  Loks
         Type:  Bug                  |                   Status:  assigned
    Component:  Database layer       |                  Version:  1.11
  (models, ORM)                      |
     Severity:  Release blocker      |               Resolution:
     Keywords:                       |             Triage Stage:  Ready for
                                     |  checkin
    Has patch:  1                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------
Changes (by Tim Graham):

 * stage:  Accepted => Ready for checkin


Comment:

 Looks good, I made some cosmetic edits and added release notes.

--
Ticket URL: <https://code.djangoproject.com/ticket/29016#comment:5>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/070.ff517d576d2e2221381bd93ad48a6052%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29016: Reuse of UpdateQuery breaks some delete updates

Django
In reply to this post by Django
#29016: Reuse of UpdateQuery breaks some delete updates
-------------------------------------+-------------------------------------
     Reporter:  Étienne Loks         |                    Owner:  Étienne
                                     |  Loks
         Type:  Bug                  |                   Status:  closed
    Component:  Database layer       |                  Version:  1.11
  (models, ORM)                      |
     Severity:  Release blocker      |               Resolution:  fixed
     Keywords:                       |             Triage Stage:  Ready for
                                     |  checkin
    Has patch:  1                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------
Changes (by Tim Graham <timograham@…>):

 * status:  assigned => closed
 * resolution:   => fixed


Comment:

 In [changeset:"9a621edf624a4eb1f1645fca628a9e432f0de776" 9a621edf]:
 {{{
 #!CommitTicketReference repository=""
 revision="9a621edf624a4eb1f1645fca628a9e432f0de776"
 Fixed #29016 -- Fixed incorrect foreign key nullification on related
 instance deletion.
 }}}

--
Ticket URL: <https://code.djangoproject.com/ticket/29016#comment:6>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/070.92a3399b210f74221b0f45a870ce250d%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29016: Reuse of UpdateQuery breaks some delete updates

Django
In reply to this post by Django
#29016: Reuse of UpdateQuery breaks some delete updates
-------------------------------------+-------------------------------------
     Reporter:  Étienne Loks         |                    Owner:  Étienne
                                     |  Loks
         Type:  Bug                  |                   Status:  closed
    Component:  Database layer       |                  Version:  1.11
  (models, ORM)                      |
     Severity:  Release blocker      |               Resolution:  fixed
     Keywords:                       |             Triage Stage:  Ready for
                                     |  checkin
    Has patch:  1                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------

Comment (by Tim Graham <timograham@…>):

 In [changeset:"8f2e3857ce5a50839ee362690a6a6d92fc8ff0ea" 8f2e3857]:
 {{{
 #!CommitTicketReference repository=""
 revision="8f2e3857ce5a50839ee362690a6a6d92fc8ff0ea"
 [2.0.x] Fixed #29016 -- Fixed incorrect foreign key nullification on
 related instance deletion.

 Backport of 9a621edf624a4eb1f1645fca628a9e432f0de776 from master
 }}}

--
Ticket URL: <https://code.djangoproject.com/ticket/29016#comment:7>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/070.b1ee5321ddc81a8d4429f9aef5899ee0%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29016: Reuse of UpdateQuery breaks some delete updates

Django
In reply to this post by Django
#29016: Reuse of UpdateQuery breaks some delete updates
-------------------------------------+-------------------------------------
     Reporter:  Étienne Loks         |                    Owner:  Étienne
                                     |  Loks
         Type:  Bug                  |                   Status:  closed
    Component:  Database layer       |                  Version:  1.11
  (models, ORM)                      |
     Severity:  Release blocker      |               Resolution:  fixed
     Keywords:                       |             Triage Stage:  Ready for
                                     |  checkin
    Has patch:  1                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------

Comment (by Tim Graham <timograham@…>):

 In [changeset:"419705bbe84e27c3d5be85f198a0352a6724927e" 419705bb]:
 {{{
 #!CommitTicketReference repository=""
 revision="419705bbe84e27c3d5be85f198a0352a6724927e"
 [1.11.x] Fixed #29016 -- Fixed incorrect foreign key nullification on
 related instance deletion.

 Backport of 9a621edf624a4eb1f1645fca628a9e432f0de776 from master
 }}}

--
Ticket URL: <https://code.djangoproject.com/ticket/29016#comment:8>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/070.ecf1a492186f6e7834264ed0765bdd29%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.