[Django] #29182: SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

classic Classic list List threaded Threaded
45 messages Options
123
Reply | Threaded
Open this post in threaded view
|

[Django] #29182: SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

Django
#29182: SQLite database migration breaks ForeignKey constraint, leaving
<table_name>__old in db schema
-------------------------------------+-------------------------------------
               Reporter:  ezaquarii  |          Owner:  nobody
                   Type:             |         Status:  new
  Uncategorized                      |
              Component:             |        Version:  2.0
  Migrations                         |
               Severity:  Normal     |       Keywords:  sqlite migration
           Triage Stage:             |      Has patch:  0
  Unreviewed                         |
    Needs documentation:  0          |    Needs tests:  0
Patch needs improvement:  0          |  Easy pickings:  0
                  UI/UX:  0          |
-------------------------------------+-------------------------------------
 SQLite table alteration uses rename and copy method. When running a
 database migration from Django command (via `call_command('migrate')`),
 database is left with references to <table_name>__old table that has been
 dropped after migration.

 This happens only when running SQLite DB migrations from management
 command.
 Running migrations via `./manage.py migrate` does not cause any issues.

 This is the demonstration:
 https://github.com/ezaquarii/openvpn-at-home/tree/sqlite-migration-bug

 Steps to reproduce the bug
 1. build the project with `make devel`

 2. enter Python virtualenv by `source backend/env/bin/activate` and enter
 `backend` subdirectory

 3. run `./manage.py bootstrap` to initialize the database

 4. bootstrap command runs db migration by calling
 `call_command('migrate')` in
 `openvpnathome/apps/management/management/commands/bootstrap.py:37`:
     - `openvpn_server` table is created
     - `openvpn_client` table is created; it uses foreign key to
 `openvpn_server`
     -  another `openvpn_server` migration is run, adding dummy field
     - `openvpn_server` table is renamed to `openvpn_server__old`
     - `openvpn_client` foreign key is re-linked to `openvpn_server__old`
     - new table `openvpn_server` is created and data from
 `openvpn_server__old` is copied with applied alterations
     - `openvpn_server__old` is dropped
     - `openvpn_client` still references `openvpn_sever__old` - ForeignKey
 constraint is **not** updated and DB is left in unusable state
 6. try creating `Client` object in django shell - SQL cannot be executed
 due to missing table


 Exact commands (available in provided `bug_repro.sh` script):
 {{{
 make devel
 cd backend
 source ./env/bin/activate
 ./manage.py bootstrap

 echo '
 Now type that:

 ./manage.py shell
 In [1]: from openvpnathome.apps.openvpn.models import Server, Client

 In [2]: Client.objects.create()

 ... snip ...

 OperationalError: no such table: main.openvpn_server__old
 '
 }}}

 Exported database schema clearly references deleted `openvpn_server__old`
 table:

 {{{
 CREATE TABLE "openvpn_client" ("id" integer NOT NULL PRIMARY KEY
 AUTOINCREMENT, "created" datetime NOT NULL, "name" varchar(64) NOT NULL,
 "cert_id" integer NOT NULL REFERENCES "x509_cert" ("id") DEFERRABLE
 INITIALLY DEFERRED, "owner_id" integer NOT NULL REFERENCES "accounts_user"
 ("id") DEFERRABLE INITIALLY DEFERRED, "server_id" integer NOT NULL
 REFERENCES "openvpn_server__old" ("id") DEFERRABLE INITIALLY DEFERRED);
 }}}

 Please refer to `db_schema.sql` file placed in the `backend` subdirectory.

--
Ticket URL: <https://code.djangoproject.com/ticket/29182>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/052.2c20ce961fadeb6c9cd3d18bba23281e%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29182: SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

Django
#29182: SQLite database migration breaks ForeignKey constraint, leaving
<table_name>__old in db schema
----------------------------------+--------------------------------------
     Reporter:  ezaquarii         |                    Owner:  nobody
         Type:  Uncategorized     |                   Status:  new
    Component:  Migrations        |                  Version:  2.0
     Severity:  Normal            |               Resolution:
     Keywords:  sqlite migration  |             Triage Stage:  Unreviewed
    Has patch:  0                 |      Needs documentation:  0
  Needs tests:  0                 |  Patch needs improvement:  0
Easy pickings:  0                 |                    UI/UX:  0
----------------------------------+--------------------------------------
Changes (by ezaquarii):

 * Attachment "bug_repro.sh" added.

 Bug reproduction steps as shell script

--
Ticket URL: <https://code.djangoproject.com/ticket/29182>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.aae98087b4f71e388f33f01eed4c54cf%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29182: SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

Django
In reply to this post by Django
#29182: SQLite database migration breaks ForeignKey constraint, leaving
<table_name>__old in db schema
----------------------------------+--------------------------------------
     Reporter:  ezaquarii         |                    Owner:  nobody
         Type:  Uncategorized     |                   Status:  new
    Component:  Migrations        |                  Version:  2.0
     Severity:  Normal            |               Resolution:
     Keywords:  sqlite migration  |             Triage Stage:  Unreviewed
    Has patch:  0                 |      Needs documentation:  0
  Needs tests:  0                 |  Patch needs improvement:  0
Easy pickings:  0                 |                    UI/UX:  0
----------------------------------+--------------------------------------
Changes (by ezaquarii):

 * Attachment "db_schema.sql" added.

 Database schema after running migrations - see references to temporary
 openvpn_server__old table

--
Ticket URL: <https://code.djangoproject.com/ticket/29182>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.bcb346cbf78772006da6a5454141c2c7%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29182: SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

Django
In reply to this post by Django
#29182: SQLite database migration breaks ForeignKey constraint, leaving
<table_name>__old in db schema
----------------------------------+--------------------------------------
     Reporter:  ezaquarii         |                    Owner:  nobody
         Type:  Uncategorized     |                   Status:  new
    Component:  Migrations        |                  Version:  2.0
     Severity:  Normal            |               Resolution:
     Keywords:  sqlite migration  |             Triage Stage:  Unreviewed
    Has patch:  0                 |      Needs documentation:  0
  Needs tests:  0                 |  Patch needs improvement:  0
Easy pickings:  0                 |                    UI/UX:  0
----------------------------------+--------------------------------------
Description changed by ezaquarii:

Old description:

> SQLite table alteration uses rename and copy method. When running a
> database migration from Django command (via `call_command('migrate')`),
> database is left with references to <table_name>__old table that has been
> dropped after migration.
>
> This happens only when running SQLite DB migrations from management
> command.
> Running migrations via `./manage.py migrate` does not cause any issues.
>
> This is the demonstration:
> https://github.com/ezaquarii/openvpn-at-home/tree/sqlite-migration-bug
>
> Steps to reproduce the bug
> 1. build the project with `make devel`
>
> 2. enter Python virtualenv by `source backend/env/bin/activate` and enter
> `backend` subdirectory
>
> 3. run `./manage.py bootstrap` to initialize the database
>
> 4. bootstrap command runs db migration by calling
> `call_command('migrate')` in
> `openvpnathome/apps/management/management/commands/bootstrap.py:37`:
>     - `openvpn_server` table is created
>     - `openvpn_client` table is created; it uses foreign key to
> `openvpn_server`
>     -  another `openvpn_server` migration is run, adding dummy field
>     - `openvpn_server` table is renamed to `openvpn_server__old`
>     - `openvpn_client` foreign key is re-linked to `openvpn_server__old`
>     - new table `openvpn_server` is created and data from
> `openvpn_server__old` is copied with applied alterations
>     - `openvpn_server__old` is dropped
>     - `openvpn_client` still references `openvpn_sever__old` - ForeignKey
> constraint is **not** updated and DB is left in unusable state
> 6. try creating `Client` object in django shell - SQL cannot be executed
> due to missing table
>

> Exact commands (available in provided `bug_repro.sh` script):
> {{{
> make devel
> cd backend
> source ./env/bin/activate
> ./manage.py bootstrap
>
> echo '
> Now type that:
>
> ./manage.py shell
> In [1]: from openvpnathome.apps.openvpn.models import Server, Client
>
> In [2]: Client.objects.create()
>
> ... snip ...
>
> OperationalError: no such table: main.openvpn_server__old
> '
> }}}
>
> Exported database schema clearly references deleted `openvpn_server__old`
> table:
>
> {{{
> CREATE TABLE "openvpn_client" ("id" integer NOT NULL PRIMARY KEY
> AUTOINCREMENT, "created" datetime NOT NULL, "name" varchar(64) NOT NULL,
> "cert_id" integer NOT NULL REFERENCES "x509_cert" ("id") DEFERRABLE
> INITIALLY DEFERRED, "owner_id" integer NOT NULL REFERENCES
> "accounts_user" ("id") DEFERRABLE INITIALLY DEFERRED, "server_id" integer
> NOT NULL REFERENCES "openvpn_server__old" ("id") DEFERRABLE INITIALLY
> DEFERRED);
> }}}
>
> Please refer to `db_schema.sql` file placed in the `backend`
> subdirectory.
New description:

 SQLite table alteration uses rename and copy method. When running a
 database migration from Django command (via `call_command('migrate')`),
 database is left with references to <table_name>__old table that has been
 dropped after migration.

 This happens only when running SQLite DB migrations from management
 command.
 Running migrations via `./manage.py migrate` does not cause any issues.

 This is the demonstration:
 https://github.com/ezaquarii/openvpn-at-home/tree/sqlite-migration-bug

 Steps to reproduce the bug
 1. build the project with `make devel`

 2. enter Python virtualenv by `source backend/env/bin/activate` and enter
 `backend` subdirectory

 3. run `./manage.py bootstrap` to initialize the database

 4. bootstrap command runs db migration by calling
 `call_command('migrate')` in
 `openvpnathome/apps/management/management/commands/bootstrap.py:37`:
     - `openvpn_server` table is created
     - `openvpn_client` table is created; it uses foreign key to
 `openvpn_server`
     -  another `openvpn_server` migration is run, adding dummy field
     - `openvpn_server` table is renamed to `openvpn_server__old`
     - `openvpn_client` foreign key is re-linked to `openvpn_server__old`
     - new table `openvpn_server` is created and data from
 `openvpn_server__old` is copied with applied alterations
     - `openvpn_server__old` is dropped
     - `openvpn_client` still references `openvpn_sever__old` - ForeignKey
 constraint is **not** updated and DB is left in unusable state
 6. try creating `Client` object in django shell - SQL cannot be executed
 due to missing table


 Exact commands (available in provided `bug_repro.sh` script):
 {{{
 make devel
 cd backend
 source ./env/bin/activate
 ./manage.py bootstrap

 echo '
 Now type that:

 ./manage.py shell
 In [1]: from openvpnathome.apps.openvpn.models import Server, Client

 In [2]: Client.objects.create()

 ... snip ...

 OperationalError: no such table: main.openvpn_server__old
 '
 }}}

 Exported database schema clearly references deleted `openvpn_server__old`
 table:

 {{{
 CREATE TABLE "openvpn_client" ("id" integer NOT NULL PRIMARY KEY
 AUTOINCREMENT, "created" datetime NOT NULL, "name" varchar(64) NOT NULL,
 "cert_id" integer NOT NULL REFERENCES "x509_cert" ("id") DEFERRABLE
 INITIALLY DEFERRED, "owner_id" integer NOT NULL REFERENCES "accounts_user"
 ("id") DEFERRABLE INITIALLY DEFERRED, "server_id" integer NOT NULL
 REFERENCES "openvpn_server__old" ("id") DEFERRABLE INITIALLY DEFERRED);
 }}}

 Please refer to `db_schema.sql` file placed in the `backend` subdirectory.

 Tested with Django versions: 2.0.1 and 2.0.2

--

--
Ticket URL: <https://code.djangoproject.com/ticket/29182#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.a19b3758f5302b67186c56b4cba3a8a6%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29182: SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

Django
In reply to this post by Django
#29182: SQLite database migration breaks ForeignKey constraint, leaving
<table_name>__old in db schema
----------------------------------+--------------------------------------
     Reporter:  ezaquarii         |                    Owner:  nobody
         Type:  Uncategorized     |                   Status:  new
    Component:  Migrations        |                  Version:  2.0
     Severity:  Normal            |               Resolution:
     Keywords:  sqlite migration  |             Triage Stage:  Unreviewed
    Has patch:  0                 |      Needs documentation:  0
  Needs tests:  0                 |  Patch needs improvement:  0
Easy pickings:  0                 |                    UI/UX:  0
----------------------------------+--------------------------------------

Comment (by Tim Graham):

 Could you provide a more minimal project that reproduces the issue? In
 particular, the steps to reproduce shouldn't require installing a bunch of
 NodeJS packages. Thanks!

--
Ticket URL: <https://code.djangoproject.com/ticket/29182#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.81648b27e7561d6d4dae007c31a5ca94%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29182: SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

Django
In reply to this post by Django
#29182: SQLite database migration breaks ForeignKey constraint, leaving
<table_name>__old in db schema
----------------------------------+--------------------------------------
     Reporter:  ezaquarii         |                    Owner:  nobody
         Type:  Uncategorized     |                   Status:  new
    Component:  Migrations        |                  Version:  2.0
     Severity:  Normal            |               Resolution:
     Keywords:  sqlite migration  |             Triage Stage:  Unreviewed
    Has patch:  0                 |      Needs documentation:  0
  Needs tests:  0                 |  Patch needs improvement:  0
Easy pickings:  0                 |                    UI/UX:  0
----------------------------------+--------------------------------------

Old description:

> SQLite table alteration uses rename and copy method. When running a
> database migration from Django command (via `call_command('migrate')`),
> database is left with references to <table_name>__old table that has been
> dropped after migration.
>
> This happens only when running SQLite DB migrations from management
> command.
> Running migrations via `./manage.py migrate` does not cause any issues.
>
> This is the demonstration:
> https://github.com/ezaquarii/openvpn-at-home/tree/sqlite-migration-bug
>
> Steps to reproduce the bug
> 1. build the project with `make devel`
>
> 2. enter Python virtualenv by `source backend/env/bin/activate` and enter
> `backend` subdirectory
>
> 3. run `./manage.py bootstrap` to initialize the database
>
> 4. bootstrap command runs db migration by calling
> `call_command('migrate')` in
> `openvpnathome/apps/management/management/commands/bootstrap.py:37`:
>     - `openvpn_server` table is created
>     - `openvpn_client` table is created; it uses foreign key to
> `openvpn_server`
>     -  another `openvpn_server` migration is run, adding dummy field
>     - `openvpn_server` table is renamed to `openvpn_server__old`
>     - `openvpn_client` foreign key is re-linked to `openvpn_server__old`
>     - new table `openvpn_server` is created and data from
> `openvpn_server__old` is copied with applied alterations
>     - `openvpn_server__old` is dropped
>     - `openvpn_client` still references `openvpn_sever__old` - ForeignKey
> constraint is **not** updated and DB is left in unusable state
> 6. try creating `Client` object in django shell - SQL cannot be executed
> due to missing table
>

> Exact commands (available in provided `bug_repro.sh` script):
> {{{
> make devel
> cd backend
> source ./env/bin/activate
> ./manage.py bootstrap
>
> echo '
> Now type that:
>
> ./manage.py shell
> In [1]: from openvpnathome.apps.openvpn.models import Server, Client
>
> In [2]: Client.objects.create()
>
> ... snip ...
>
> OperationalError: no such table: main.openvpn_server__old
> '
> }}}
>
> Exported database schema clearly references deleted `openvpn_server__old`
> table:
>
> {{{
> CREATE TABLE "openvpn_client" ("id" integer NOT NULL PRIMARY KEY
> AUTOINCREMENT, "created" datetime NOT NULL, "name" varchar(64) NOT NULL,
> "cert_id" integer NOT NULL REFERENCES "x509_cert" ("id") DEFERRABLE
> INITIALLY DEFERRED, "owner_id" integer NOT NULL REFERENCES
> "accounts_user" ("id") DEFERRABLE INITIALLY DEFERRED, "server_id" integer
> NOT NULL REFERENCES "openvpn_server__old" ("id") DEFERRABLE INITIALLY
> DEFERRED);
> }}}
>
> Please refer to `db_schema.sql` file placed in the `backend`
> subdirectory.
>
> Tested with Django versions: 2.0.1 and 2.0.2
New description:

 SQLite table alteration uses rename and copy method. When running a
 database migration from Django command (via `call_command('migrate')`),
 database is left with references to <table_name>__old table that has been
 dropped after migration.

 This happens only when running SQLite DB migrations from management
 command.
 Running migrations via `./manage.py migrate` does not cause any issues.

 This is the demonstration:
 https://github.com/ezaquarii/openvpn-at-home/tree/sqlite-migration-bug

 Steps to reproduce the bug
 1. build the project with `make devel`

 2. enter Python virtualenv by `source backend/env/bin/activate` and enter
 `backend` subdirectory

 3. run `./manage.py bootstrap` to initialize the database

 4. bootstrap command runs db migration by calling
 `call_command('migrate')` in
 `openvpnathome/apps/management/management/commands/bootstrap.py:37`:
     - `openvpn_server` table is created
     - `openvpn_client` table is created; it uses foreign key to
 `openvpn_server`
     -  another `openvpn_server` migration is run, adding dummy field
     - `openvpn_server` table is renamed to `openvpn_server__old`
     - `openvpn_client` foreign key is re-linked to `openvpn_server__old`
     - new table `openvpn_server` is created and data from
 `openvpn_server__old` is copied with applied alterations
     - `openvpn_server__old` is dropped
     - `openvpn_client` still references `openvpn_sever__old` - ForeignKey
 constraint is **not** updated and DB is left in unusable state
 6. try creating `Client` object in django shell - SQL cannot be executed
 due to missing table


 Exact commands (available in provided `bug_repro.sh` script):
 {{{
 cd backend
 make devel
 source ./env/bin/activate
 ./manage.py bootstrap

 echo '
 Now type that:

 ./manage.py shell
 In [1]: from openvpnathome.apps.openvpn.models import Server, Client

 In [2]: Client.objects.create()

 ... snip ...

 OperationalError: no such table: main.openvpn_server__old
 '
 }}}

 Exported database schema clearly references deleted `openvpn_server__old`
 table:

 {{{
 CREATE TABLE "openvpn_client" ("id" integer NOT NULL PRIMARY KEY
 AUTOINCREMENT, "created" datetime NOT NULL, "name" varchar(64) NOT NULL,
 "cert_id" integer NOT NULL REFERENCES "x509_cert" ("id") DEFERRABLE
 INITIALLY DEFERRED, "owner_id" integer NOT NULL REFERENCES "accounts_user"
 ("id") DEFERRABLE INITIALLY DEFERRED, "server_id" integer NOT NULL
 REFERENCES "openvpn_server__old" ("id") DEFERRABLE INITIALLY DEFERRED);
 }}}

 Please refer to `db_schema.sql` file placed in the `backend` subdirectory.

 Tested with Django versions: 2.0.1 and 2.0.2

--

Comment (by ezaquarii):

 Please check this repository:

 https://github.com/ezaquarii/django-sqlite-migration-bug


 {{{
         with transaction.atomic():
             call_command('migrate')
 }}}

 I think this is the issue. It works ok without transaction.

--
Ticket URL: <https://code.djangoproject.com/ticket/29182#comment:3>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.93a74f8ed3a596dc6b6ddbb58ab0cf5b%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29182: SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

Django
In reply to this post by Django
#29182: SQLite database migration breaks ForeignKey constraint, leaving
<table_name>__old in db schema
----------------------------------+--------------------------------------
     Reporter:  ezaquarii         |                    Owner:  nobody
         Type:  Uncategorized     |                   Status:  new
    Component:  Migrations        |                  Version:  2.0
     Severity:  Normal            |               Resolution:
     Keywords:  sqlite migration  |             Triage Stage:  Unreviewed
    Has patch:  0                 |      Needs documentation:  0
  Needs tests:  0                 |  Patch needs improvement:  0
Easy pickings:  0                 |                    UI/UX:  0
----------------------------------+--------------------------------------
Description changed by ezaquarii:

Old description:

> SQLite table alteration uses rename and copy method. When running a
> database migration from Django command (via `call_command('migrate')`),
> database is left with references to <table_name>__old table that has been
> dropped after migration.
>
> This happens only when running SQLite DB migrations from management
> command.
> Running migrations via `./manage.py migrate` does not cause any issues.
>
> This is the demonstration:
> https://github.com/ezaquarii/openvpn-at-home/tree/sqlite-migration-bug
>
> Steps to reproduce the bug
> 1. build the project with `make devel`
>
> 2. enter Python virtualenv by `source backend/env/bin/activate` and enter
> `backend` subdirectory
>
> 3. run `./manage.py bootstrap` to initialize the database
>
> 4. bootstrap command runs db migration by calling
> `call_command('migrate')` in
> `openvpnathome/apps/management/management/commands/bootstrap.py:37`:
>     - `openvpn_server` table is created
>     - `openvpn_client` table is created; it uses foreign key to
> `openvpn_server`
>     -  another `openvpn_server` migration is run, adding dummy field
>     - `openvpn_server` table is renamed to `openvpn_server__old`
>     - `openvpn_client` foreign key is re-linked to `openvpn_server__old`
>     - new table `openvpn_server` is created and data from
> `openvpn_server__old` is copied with applied alterations
>     - `openvpn_server__old` is dropped
>     - `openvpn_client` still references `openvpn_sever__old` - ForeignKey
> constraint is **not** updated and DB is left in unusable state
> 6. try creating `Client` object in django shell - SQL cannot be executed
> due to missing table
>

> Exact commands (available in provided `bug_repro.sh` script):
> {{{
> cd backend
> make devel
> source ./env/bin/activate
> ./manage.py bootstrap
>
> echo '
> Now type that:
>
> ./manage.py shell
> In [1]: from openvpnathome.apps.openvpn.models import Server, Client
>
> In [2]: Client.objects.create()
>
> ... snip ...
>
> OperationalError: no such table: main.openvpn_server__old
> '
> }}}
>
> Exported database schema clearly references deleted `openvpn_server__old`
> table:
>
> {{{
> CREATE TABLE "openvpn_client" ("id" integer NOT NULL PRIMARY KEY
> AUTOINCREMENT, "created" datetime NOT NULL, "name" varchar(64) NOT NULL,
> "cert_id" integer NOT NULL REFERENCES "x509_cert" ("id") DEFERRABLE
> INITIALLY DEFERRED, "owner_id" integer NOT NULL REFERENCES
> "accounts_user" ("id") DEFERRABLE INITIALLY DEFERRED, "server_id" integer
> NOT NULL REFERENCES "openvpn_server__old" ("id") DEFERRABLE INITIALLY
> DEFERRED);
> }}}
>
> Please refer to `db_schema.sql` file placed in the `backend`
> subdirectory.
>
> Tested with Django versions: 2.0.1 and 2.0.2
New description:

 SQLite table alteration uses rename and copy method. When running a
 database migration from Django command (via `call_command('migrate')`),
 database is left with references to <table_name>__old table that has been
 dropped after migration.

 This happens only when running SQLite DB migrations from management
 command under transaction.
 Running migrations via `./manage.py migrate` does not cause any issues.

 This is the demonstration:
 https://github.com/ezaquarii/django-sqlite-migration-bug

 Steps to reproduce the bug
 1. run ./bootstrap.sh
 2. from import app.models import *
 3. Bravo.objects.create()
 4. SQL cannot be executed due to missing table `main.app_alpha__old`

     - `app/management/commands/bootstrap.py:9`: run the migration
     - `alpha` table is created
     - `bravo` table is created; it uses foreign key to `alpha`
     -  another `alpha` migration is run, adding dummy field
     - `alpha` table is renamed to `alpha__old`
     - `bravo` foreign key is re-linked to `alpha__old`
     - new table `alpha` is created and data from `alpha__old` is copied
 with applied alterations
     - `alpha__old` is dropped
     - `bravo` still references `alpha__old` - ForeignKey constraint is
 **not** updated and DB is left in unusable state


 {{{
 CREATE TABLE "app_bravo" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT,
 "dummy" varchar(10) NOT NULL, "alpha_id" integer NOT NULL REFERENCES
 "app_alpha__old" ("id") DEFERRABLE INITIALLY DEFERRED);
 }}}

 Tested with Django versions: 2.0.1 and 2.0.2

--

--
Ticket URL: <https://code.djangoproject.com/ticket/29182#comment:4>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.9a985504ad0190cb91037ecbf1f0f4d5%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29182: SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

Django
In reply to this post by Django
#29182: SQLite database migration breaks ForeignKey constraint, leaving
<table_name>__old in db schema
----------------------------------+--------------------------------------
     Reporter:  ezaquarii         |                    Owner:  nobody
         Type:  Uncategorized     |                   Status:  new
    Component:  Migrations        |                  Version:  2.0
     Severity:  Normal            |               Resolution:
     Keywords:  sqlite migration  |             Triage Stage:  Unreviewed
    Has patch:  0                 |      Needs documentation:  0
  Needs tests:  0                 |  Patch needs improvement:  0
Easy pickings:  0                 |                    UI/UX:  0
----------------------------------+--------------------------------------
Description changed by ezaquarii:

Old description:

> SQLite table alteration uses rename and copy method. When running a
> database migration from Django command (via `call_command('migrate')`),
> database is left with references to <table_name>__old table that has been
> dropped after migration.
>
> This happens only when running SQLite DB migrations from management
> command under transaction.
> Running migrations via `./manage.py migrate` does not cause any issues.
>
> This is the demonstration:
> https://github.com/ezaquarii/django-sqlite-migration-bug
>
> Steps to reproduce the bug
> 1. run ./bootstrap.sh
> 2. from import app.models import *
> 3. Bravo.objects.create()
> 4. SQL cannot be executed due to missing table `main.app_alpha__old`
>
>     - `app/management/commands/bootstrap.py:9`: run the migration
>     - `alpha` table is created
>     - `bravo` table is created; it uses foreign key to `alpha`
>     -  another `alpha` migration is run, adding dummy field
>     - `alpha` table is renamed to `alpha__old`
>     - `bravo` foreign key is re-linked to `alpha__old`
>     - new table `alpha` is created and data from `alpha__old` is copied
> with applied alterations
>     - `alpha__old` is dropped
>     - `bravo` still references `alpha__old` - ForeignKey constraint is
> **not** updated and DB is left in unusable state
>

> {{{
> CREATE TABLE "app_bravo" ("id" integer NOT NULL PRIMARY KEY
> AUTOINCREMENT, "dummy" varchar(10) NOT NULL, "alpha_id" integer NOT NULL
> REFERENCES "app_alpha__old" ("id") DEFERRABLE INITIALLY DEFERRED);
> }}}
>
> Tested with Django versions: 2.0.1 and 2.0.2

New description:

 SQLite table alteration uses rename and copy method. When running a
 database migration from Django command (via `call_command('migrate')`),
 database is left with references to `<table_name>__old` table that has
 been dropped after migration.

 This happens only when running SQLite DB migrations from management
 command under transaction.
 Running migrations via `./manage.py migrate` does not cause any issues.

 This is the demonstration:
 https://github.com/ezaquarii/django-sqlite-migration-bug

 Steps to reproduce the bug
 1. run ./bootstrap.sh
 2. from import app.models import *
 3. Bravo.objects.create()
 4. SQL cannot be executed due to missing table `main.app_alpha__old`

     - `app/management/commands/bootstrap.py:9`: run the migration
     - `alpha` table is created
     - `bravo` table is created; it uses foreign key to `alpha`
     -  another `alpha` migration is run, adding dummy field
     - `alpha` table is renamed to `alpha__old`
     - `bravo` foreign key is re-linked to `alpha__old`
     - new table `alpha` is created and data from `alpha__old` is copied
 with applied alterations
     - `alpha__old` is dropped
     - `bravo` still references `alpha__old` - ForeignKey constraint is
 **not** updated and DB is left in unusable state


 {{{
 CREATE TABLE "app_bravo" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT,
 "dummy" varchar(10) NOT NULL, "alpha_id" integer NOT NULL REFERENCES
 "app_alpha__old" ("id") DEFERRABLE INITIALLY DEFERRED);
 }}}

 Tested with Django versions: 2.0.1 and 2.0.2

--

--
Ticket URL: <https://code.djangoproject.com/ticket/29182#comment:5>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.1fa7abf94f7741c2d699519c43940437%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29182: SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

Django
In reply to this post by Django
#29182: SQLite database migration breaks ForeignKey constraint, leaving
<table_name>__old in db schema
----------------------------------+------------------------------------
     Reporter:  ezaquarii         |                    Owner:  nobody
         Type:  Uncategorized     |                   Status:  new
    Component:  Migrations        |                  Version:  2.0
     Severity:  Normal            |               Resolution:
     Keywords:  sqlite migration  |             Triage Stage:  Accepted
    Has patch:  0                 |      Needs documentation:  0
  Needs tests:  0                 |  Patch needs improvement:  0
Easy pickings:  0                 |                    UI/UX:  0
----------------------------------+------------------------------------
Changes (by Simon Charette):

 * cc: Simon Charette (added)
 * stage:  Unreviewed => Accepted


Comment:

 Hey ezaquarii, thank you for your great report.

 You can get a bit of context about why all the SQLite constraint
 workarounds are required in #28849 but in this case I suspect odd things
 happen because, as you might know, savepoints are used in nested
 `atomic()` blocks.

 The migration executor was not designed to be used within a transaction as
 it does some pretty intense transaction management itself so I'm accepting
 on the basis that we should do a better job at raising an error in this
 case. The fact that this doesn't happen when `./manage.py migrate` is
 called directly is a good indicator that other weird things might be
 happening under the hood because this case is completely untested by the
 suite right now. For example, do the backend that we know transactional
 DDL support for all behave the same way with regards to savepoints?

 I guess you were trying to come up with some kind of deploying script that
 either fails or succeeds to apply multiple migrations?

--
Ticket URL: <https://code.djangoproject.com/ticket/29182#comment:6>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.9ad793cd54be316951e7a2cfc6292102%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29182: SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

Django
In reply to this post by Django
#29182: SQLite database migration breaks ForeignKey constraint, leaving
<table_name>__old in db schema
--------------------------------------+------------------------------------
     Reporter:  ezaquarii             |                    Owner:  nobody
         Type:  Cleanup/optimization  |                   Status:  new
    Component:  Migrations            |                  Version:  2.0
     Severity:  Normal                |               Resolution:
     Keywords:  sqlite migration      |             Triage Stage:  Accepted
    Has patch:  0                     |      Needs documentation:  0
  Needs tests:  0                     |  Patch needs improvement:  0
Easy pickings:  0                     |                    UI/UX:  0
--------------------------------------+------------------------------------
Changes (by Tim Graham):

 * type:  Uncategorized => Cleanup/optimization


--
Ticket URL: <https://code.djangoproject.com/ticket/29182#comment:7>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.605611b3331d376e57a7e8eabf4ae965%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29182: SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

Django
In reply to this post by Django
#29182: SQLite database migration breaks ForeignKey constraint, leaving
<table_name>__old in db schema
----------------------------------+------------------------------------
     Reporter:  ezaquarii         |                    Owner:  nobody
         Type:  Bug               |                   Status:  new
    Component:  Migrations        |                  Version:  2.0
     Severity:  Release blocker   |               Resolution:
     Keywords:  sqlite migration  |             Triage Stage:  Accepted
    Has patch:  0                 |      Needs documentation:  0
  Needs tests:  0                 |  Patch needs improvement:  0
Easy pickings:  0                 |                    UI/UX:  0
----------------------------------+------------------------------------
Changes (by Florian Apolloner):

 * type:  Cleanup/optimization => Bug
 * severity:  Normal => Release blocker


Comment:

 Changing this to a full blown bug and release blocker since I am able to
 reproduce this with `./manage.py migrate` and very simple app.

 Relevant versions:
 Sqlite: 3.26.0 2018-12-01 12:34:55
 bf8c1b2b7a5960c282e543b9c293686dccff272512d08865f4600fb58238alt1
 python sqlite3 version: 2.6.0

 Same test models:
 {{{
 #!python
 from django.db import models

 class File(models.Model):
     content = models.ForeignKey("FileContent", on_delete=models.PROTECT)

 class FileContent(models.Model):
     pass

 class PlayBook(models.Model):
     file = models.ForeignKey("File", on_delete=models.PROTECT)
 }}}

 This generates the following initial migration (on latest master):
 {{{
 #!python
 # Generated by Django 2.2.dev20181204152138 on 2018-12-04 16:49

 from django.db import migrations, models
 import django.db.models.deletion


 class Migration(migrations.Migration):

     initial = True

     dependencies = [
     ]

     operations = [
         migrations.CreateModel(
             name='File',
             fields=[
                 ('id', models.AutoField(auto_created=True,
 primary_key=True, serialize=False, verbose_name='ID')),
             ],
         ),
         migrations.CreateModel(
             name='FileContent',
             fields=[
                 ('id', models.AutoField(auto_created=True,
 primary_key=True, serialize=False, verbose_name='ID')),
             ],
         ),
         migrations.CreateModel(
             name='PlayBook',
             fields=[
                 ('id', models.AutoField(auto_created=True,
 primary_key=True, serialize=False, verbose_name='ID')),
                 ('file',
 models.ForeignKey(on_delete=django.db.models.deletion.PROTECT,
 to='testing.File')),
             ],
         ),
         migrations.AddField(
             model_name='file',
             name='content',
 field=models.ForeignKey(on_delete=django.db.models.deletion.PROTECT,
 to='testing.FileContent'),
         ),
     ]
 }}}

 Running this migration results in the following schema in SQLite:
 {{{
 #!sql
 CREATE TABLE IF NOT EXISTS "django_migrations"(
   "id" integer NOT NULL PRIMARY KEY AUTOINCREMENT,
   "app" varchar(255) NOT NULL,
   "name" varchar(255) NOT NULL,
   "applied" datetime NOT NULL
 );
 CREATE TABLE IF NOT EXISTS "testing_filecontent"(
   "id" integer NOT NULL PRIMARY KEY AUTOINCREMENT
 );
 CREATE TABLE IF NOT EXISTS "testing_playbook"(
   "id" integer NOT NULL PRIMARY KEY AUTOINCREMENT,
   "file_id" integer NOT NULL REFERENCES "testing_file__old"("id")
 DEFERRABLE INITIALLY DEFERRED
 );
 CREATE TABLE IF NOT EXISTS "testing_file"(
   "id" integer NOT NULL PRIMARY KEY AUTOINCREMENT,
   "content_id" integer NOT NULL REFERENCES "testing_filecontent"("id")
 DEFERRABLE INITIALLY DEFERRED
 );
 CREATE INDEX "testing_playbook_file_id_debe0476" ON "testing_playbook"(
   "file_id"
 );
 CREATE INDEX "testing_file_content_id_4682b86d" ON "testing_file"(
   "content_id"
 );
 }}}

 From the looks of it, we will need to do some similar to
 https://github.com/django/django/blob/196b420fcb0cbdd82970e2b9aea80251bde82056/django/db/backends/sqlite3/schema.py#L108-L121
 whenever we rename a table which has foreignkeys pointing to it.

--
Ticket URL: <https://code.djangoproject.com/ticket/29182#comment:8>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.82727c654fc1d1707f151f9060ccc39b%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29182: SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

Django
In reply to this post by Django
#29182: SQLite database migration breaks ForeignKey constraint, leaving
<table_name>__old in db schema
----------------------------------+------------------------------------
     Reporter:  ezaquarii         |                    Owner:  nobody
         Type:  Bug               |                   Status:  new
    Component:  Migrations        |                  Version:  2.0
     Severity:  Release blocker   |               Resolution:
     Keywords:  sqlite migration  |             Triage Stage:  Accepted
    Has patch:  0                 |      Needs documentation:  0
  Needs tests:  0                 |  Patch needs improvement:  0
Easy pickings:  0                 |                    UI/UX:  0
----------------------------------+------------------------------------

Comment (by Simon Charette):

 Thanks for the extra details Florian. I'll try to have a look at it this
 week.

--
Ticket URL: <https://code.djangoproject.com/ticket/29182#comment:9>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.750fac8b966c6659a5cae4919f740e9c%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29182: SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

Django
In reply to this post by Django
#29182: SQLite database migration breaks ForeignKey constraint, leaving
<table_name>__old in db schema
----------------------------------+------------------------------------
     Reporter:  ezaquarii         |                    Owner:  nobody
         Type:  Bug               |                   Status:  new
    Component:  Migrations        |                  Version:  2.0
     Severity:  Release blocker   |               Resolution:
     Keywords:  sqlite migration  |             Triage Stage:  Accepted
    Has patch:  0                 |      Needs documentation:  0
  Needs tests:  0                 |  Patch needs improvement:  0
Easy pickings:  0                 |                    UI/UX:  0
----------------------------------+------------------------------------
Changes (by Florian Apolloner):

 * cc: Florian Apolloner (added)


--
Ticket URL: <https://code.djangoproject.com/ticket/29182#comment:10>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.3aa3e16d2899aaa2039edc118cf1186b%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29182: SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

Django
In reply to this post by Django
#29182: SQLite database migration breaks ForeignKey constraint, leaving
<table_name>__old in db schema
----------------------------------+------------------------------------
     Reporter:  ezaquarii         |                    Owner:  nobody
         Type:  Bug               |                   Status:  new
    Component:  Migrations        |                  Version:  2.0
     Severity:  Release blocker   |               Resolution:
     Keywords:  sqlite migration  |             Triage Stage:  Accepted
    Has patch:  0                 |      Needs documentation:  0
  Needs tests:  0                 |  Patch needs improvement:  0
Easy pickings:  0                 |                    UI/UX:  0
----------------------------------+------------------------------------

Comment (by Florian Apolloner):

 Fwiw I think we will have to disable `can_rollback_ddl` on sqlite to get a
 sane behavior, otherwise pretty much every migration out there will break.

--
Ticket URL: <https://code.djangoproject.com/ticket/29182#comment:11>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.a93f4f7977b4a13ed99ab14a933c7082%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29182: SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

Django
In reply to this post by Django
#29182: SQLite database migration breaks ForeignKey constraint, leaving
<table_name>__old in db schema
----------------------------------+------------------------------------
     Reporter:  ezaquarii         |                    Owner:  nobody
         Type:  Bug               |                   Status:  new
    Component:  Migrations        |                  Version:  2.0
     Severity:  Release blocker   |               Resolution:
     Keywords:  sqlite migration  |             Triage Stage:  Accepted
    Has patch:  0                 |      Needs documentation:  0
  Needs tests:  0                 |  Patch needs improvement:  0
Easy pickings:  0                 |                    UI/UX:  0
----------------------------------+------------------------------------
Changes (by Florian Apolloner):

 * Attachment "initial_patch.diff" added.


--
Ticket URL: <https://code.djangoproject.com/ticket/29182>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.be859f1af997922a8757329ae71dbe82%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29182: SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

Django
In reply to this post by Django
#29182: SQLite database migration breaks ForeignKey constraint, leaving
<table_name>__old in db schema
----------------------------------+------------------------------------
     Reporter:  ezaquarii         |                    Owner:  nobody
         Type:  Bug               |                   Status:  new
    Component:  Migrations        |                  Version:  2.0
     Severity:  Release blocker   |               Resolution:
     Keywords:  sqlite migration  |             Triage Stage:  Accepted
    Has patch:  0                 |      Needs documentation:  0
  Needs tests:  0                 |  Patch needs improvement:  0
Easy pickings:  0                 |                    UI/UX:  0
----------------------------------+------------------------------------

Comment (by Florian Apolloner):

 I have added an initial patch which fixes my issue -- it should serve only
 as starting point for some ideas and also has a few questions left…

--
Ticket URL: <https://code.djangoproject.com/ticket/29182#comment:12>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.1a5198b57f9d975f32673fb9918549cb%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29182: SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

Django
In reply to this post by Django
#29182: SQLite database migration breaks ForeignKey constraint, leaving
<table_name>__old in db schema
----------------------------------+------------------------------------
     Reporter:  ezaquarii         |                    Owner:  nobody
         Type:  Bug               |                   Status:  new
    Component:  Migrations        |                  Version:  2.0
     Severity:  Release blocker   |               Resolution:
     Keywords:  sqlite migration  |             Triage Stage:  Accepted
    Has patch:  0                 |      Needs documentation:  0
  Needs tests:  0                 |  Patch needs improvement:  0
Easy pickings:  0                 |                    UI/UX:  0
----------------------------------+------------------------------------

Comment (by Simon Charette):

 Florian, if you still have a few minutes to dedicate to this issue I'd
 appreciate if you could add a
 [https://github.com/django/django/pull/9389/files#diff-
 888b45a0a8f38ee67e8d22403cf994dbR1151 schema test] based on your
 investigation reproducing the issue. It's surprising that this wasn't
 caught by an existing foreign key addition test.

--
Ticket URL: <https://code.djangoproject.com/ticket/29182#comment:13>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.5d3887e891622077140ff4d1f8bb6fcb%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29182: SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

Django
In reply to this post by Django
#29182: SQLite database migration breaks ForeignKey constraint, leaving
<table_name>__old in db schema
----------------------------------+------------------------------------
     Reporter:  ezaquarii         |                    Owner:  nobody
         Type:  Bug               |                   Status:  new
    Component:  Migrations        |                  Version:  2.0
     Severity:  Release blocker   |               Resolution:
     Keywords:  sqlite migration  |             Triage Stage:  Accepted
    Has patch:  0                 |      Needs documentation:  0
  Needs tests:  0                 |  Patch needs improvement:  0
Easy pickings:  0                 |                    UI/UX:  0
----------------------------------+------------------------------------

Comment (by Florian Apolloner):

 I'll try to do so tomorrow, I've never written a schema test :)

--
Ticket URL: <https://code.djangoproject.com/ticket/29182#comment:14>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.866c7ba2e2360550cb51158d276aba1c%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29182: SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

Django
In reply to this post by Django
#29182: SQLite database migration breaks ForeignKey constraint, leaving
<table_name>__old in db schema
----------------------------------+------------------------------------
     Reporter:  ezaquarii         |                    Owner:  nobody
         Type:  Bug               |                   Status:  new
    Component:  Migrations        |                  Version:  2.0
     Severity:  Release blocker   |               Resolution:
     Keywords:  sqlite migration  |             Triage Stage:  Accepted
    Has patch:  0                 |      Needs documentation:  0
  Needs tests:  0                 |  Patch needs improvement:  0
Easy pickings:  0                 |                    UI/UX:  0
----------------------------------+------------------------------------

Comment (by Simon Charette):

 I did a bit of investigation on my side and there must be something else
 at play here as I cannot reproduce against `stable/2.1.x` and `master`
 with SQLite version 3.22.0 2018-01-22 18:45:57 and `sqlite3.version` 2.6.0
 on Python 3.6.6.


 {{{
 $ sqlite3 project/db.sqlite3
 SQLite version 3.22.0 2018-01-22 18:45:57
 Enter ".help" for usage hints.
 sqlite> .schema
 CREATE TABLE IF NOT EXISTS "django_migrations" ("id" integer NOT NULL
 PRIMARY KEY AUTOINCREMENT, "app" varchar(255) NOT NULL, "name"
 varchar(255) NOT NULL, "applied" datetime NOT NULL);
 CREATE TABLE sqlite_sequence(name,seq);
 CREATE TABLE IF NOT EXISTS "ticket_29182_filecontent" ("id" integer NOT
 NULL PRIMARY KEY AUTOINCREMENT);
 CREATE TABLE IF NOT EXISTS "ticket_29182_playbook" ("id" integer NOT NULL
 PRIMARY KEY AUTOINCREMENT, "file_id" integer NOT NULL REFERENCES
 "ticket_29182_file" ("id") DEFERRABLE INITIALLY DEFERRED);
 CREATE TABLE IF NOT EXISTS "ticket_29182_file" ("id" integer NOT NULL
 PRIMARY KEY AUTOINCREMENT, "content_id" integer NOT NULL REFERENCES
 "ticket_29182_filecontent" ("id") DEFERRABLE INITIALLY DEFERRED);
 CREATE INDEX "ticket_29182_playbook_file_id_f3bc0e50" ON
 "ticket_29182_playbook" ("file_id");
 CREATE INDEX "ticket_29182_file_content_id_7815674c" ON
 "ticket_29182_file" ("content_id");
 }}}

 I also couldn't get a regression test to fail either. Florian could you
 confirm one of these added tests fail for you on your local setup:
 https://github.com/django/django/compare/master...charettes:ticket-29182

--
Ticket URL: <https://code.djangoproject.com/ticket/29182#comment:15>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.6c4fa52eb8f89773bbac011283ad378e%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #29182: SQLite database migration breaks ForeignKey constraint, leaving <table_name>__old in db schema

Django
In reply to this post by Django
#29182: SQLite database migration breaks ForeignKey constraint, leaving
<table_name>__old in db schema
----------------------------------+------------------------------------
     Reporter:  ezaquarii         |                    Owner:  nobody
         Type:  Bug               |                   Status:  new
    Component:  Migrations        |                  Version:  2.0
     Severity:  Release blocker   |               Resolution:
     Keywords:  sqlite migration  |             Triage Stage:  Accepted
    Has patch:  0                 |      Needs documentation:  0
  Needs tests:  0                 |  Patch needs improvement:  0
Easy pickings:  0                 |                    UI/UX:  0
----------------------------------+------------------------------------

Comment (by Simon Charette):

 Cannot reproduce with `SQLite 3.23.1` and `3.24.0 2018-06-04 14:10:15`
 either, I'll try to get my hands on 3.26+ tomorrow. Could you provide the
 output of

 {{{#!python
 import sqlite3
 sqlite3.connect('file::memory:').cursor().execute('PRAGMA
 compile_options').fetchall()
 }}}

--
Ticket URL: <https://code.djangoproject.com/ticket/29182#comment:16>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.21ed72bf7d66ce32d7ce8ee8596992b3%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
123