[Django] #28752: Django.setup() should be idempotent

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[Django] #28752: Django.setup() should be idempotent

Django
#28752: Django.setup() should be idempotent
------------------------------------------+------------------------
               Reporter:  pascal chambon  |          Owner:  nobody
                   Type:  Bug             |         Status:  new
              Component:  Core (Other)    |        Version:  1.11
               Severity:  Normal          |       Keywords:
           Triage Stage:  Unreviewed      |      Has patch:  0
    Needs documentation:  0               |    Needs tests:  0
Patch needs improvement:  0               |  Easy pickings:  0
                  UI/UX:  0               |
------------------------------------------+------------------------
 I've been bitten numerous times by the impredictable behaviour of django
 when django.setup() was called numerous times.

 In the old days I had exceptions, now it's mainly subtle breakages of
 logging configuration.

 I couldn't find, in the issue tracker or the dev mailing list statements
 about this subject, others than request from other users encountering the
 problem.

 For example this ticket concerned script+importable modules :
 https://code.djangoproject.com/ticket/26152

 The latest case in date for me is pytest-django having troubles with these
 multiple setup() calls : https://github.com/pytest-dev/pytest-
 django/issues/531 , due to multiple fixtures attempting this auto-setup.

 Would it be OK to make django.setup() idempotent, or even expose a
 "is_ready" flag for easier introspection ?

--
Ticket URL: <https://code.djangoproject.com/ticket/28752>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/048.0d98c845609066b6033ab7908a17e110%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #28752: django.setup() should be idempotent (was: Django.setup() should be idempotent)

Django
#28752: django.setup() should be idempotent
--------------------------------+--------------------------------------
     Reporter:  pascal chambon  |                    Owner:  nobody
         Type:  Bug             |                   Status:  closed
    Component:  Core (Other)    |                  Version:  1.11
     Severity:  Normal          |               Resolution:  duplicate
     Keywords:                  |             Triage Stage:  Unreviewed
    Has patch:  0               |      Needs documentation:  0
  Needs tests:  0               |  Patch needs improvement:  0
Easy pickings:  0               |                    UI/UX:  0
--------------------------------+--------------------------------------
Changes (by Tim Graham):

 * status:  new => closed
 * resolution:   => duplicate


Comment:

 I believe this is addressed in Django 2.0 by #27176. If not, please reopen
 with a minimal project that reproduces the problem you're encountering.

--
Ticket URL: <https://code.djangoproject.com/ticket/28752#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/063.8ea2c20f45330ee19c34def43a53a4ac%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #28752: django.setup() should be idempotent

Django
In reply to this post by Django
#28752: django.setup() should be idempotent
--------------------------------+--------------------------------------
     Reporter:  pascal chambon  |                    Owner:  nobody
         Type:  Bug             |                   Status:  new
    Component:  Core (Other)    |                  Version:  1.11
     Severity:  Normal          |               Resolution:
     Keywords:                  |             Triage Stage:  Unreviewed
    Has patch:  0               |      Needs documentation:  0
  Needs tests:  0               |  Patch needs improvement:  0
Easy pickings:  0               |                    UI/UX:  0
--------------------------------+--------------------------------------
Changes (by pascal chambon):

 * status:  closed => new
 * resolution:  duplicate =>


Old description:

> I've been bitten numerous times by the impredictable behaviour of django
> when django.setup() was called numerous times.
>
> In the old days I had exceptions, now it's mainly subtle breakages of
> logging configuration.
>
> I couldn't find, in the issue tracker or the dev mailing list statements
> about this subject, others than request from other users encountering the
> problem.
>
> For example this ticket concerned script+importable modules :
> https://code.djangoproject.com/ticket/26152
>
> The latest case in date for me is pytest-django having troubles with
> these multiple setup() calls : https://github.com/pytest-dev/pytest-
> django/issues/531 , due to multiple fixtures attempting this auto-setup.
>
> Would it be OK to make django.setup() idempotent, or even expose a
> "is_ready" flag for easier introspection ?
New description:

 Alas the ticket #27176 dealt only with the "apps.populate" part, but the
 whole setup() must be protected, else we'll always have weird side effects
 on duplicate calls.

 Here is a testcase showing the reset of the "django" logger level, for
 example, when calling setup() multiple times.

 Depending on the exact LOGGING dict (with disable_existing_loggers etc.),
 even the shape of the logging tree might be changed.

 {{{

     def test_duplicated_setup_calls(self):
         import django, logging
         #from django.conf import settings
         #print(settings.LOGGING_CONFIG, settings.LOGGING)

         django.setup()
         logging.getLogger('django').setLevel(logging.DEBUG)
         assert logging.getLogger('django').level == logging.DEBUG

         django.setup()
         assert logging.getLogger('django').level == logging.DEBUG  #
 raises

 }}}

--

--
Ticket URL: <https://code.djangoproject.com/ticket/28752#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/063.dc42aee0058368e02acc12fc377cf910%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #28752: django.setup() should be idempotent

Django
In reply to this post by Django
#28752: django.setup() should be idempotent
--------------------------------+--------------------------------------
     Reporter:  pascal chambon  |                    Owner:  nobody
         Type:  Bug             |                   Status:  new
    Component:  Core (Other)    |                  Version:  1.11
     Severity:  Normal          |               Resolution:
     Keywords:                  |             Triage Stage:  Unreviewed
    Has patch:  0               |      Needs documentation:  0
  Needs tests:  0               |  Patch needs improvement:  0
Easy pickings:  0               |                    UI/UX:  0
--------------------------------+--------------------------------------

Comment (by Tim Graham):

 Thanks. Can you explain the use case for calling `django.setup()` multiple
 times?

--
Ticket URL: <https://code.djangoproject.com/ticket/28752#comment:3>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/063.8bc93df3b756458eb538d2687dfeed8e%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #28752: django.setup() should be idempotent

Django
In reply to this post by Django
#28752: django.setup() should be idempotent
--------------------------------+--------------------------------------
     Reporter:  pascal chambon  |                    Owner:  nobody
         Type:  Bug             |                   Status:  new
    Component:  Core (Other)    |                  Version:  1.11
     Severity:  Normal          |               Resolution:
     Keywords:                  |             Triage Stage:  Unreviewed
    Has patch:  0               |      Needs documentation:  0
  Needs tests:  0               |  Patch needs improvement:  0
Easy pickings:  0               |                    UI/UX:  0
--------------------------------+--------------------------------------
Description changed by pascal chambon:

Old description:

> Alas the ticket #27176 dealt only with the "apps.populate" part, but the
> whole setup() must be protected, else we'll always have weird side
> effects on duplicate calls.
>
> Here is a testcase showing the reset of the "django" logger level, for
> example, when calling setup() multiple times.
>
> Depending on the exact LOGGING dict (with disable_existing_loggers etc.),
> even the shape of the logging tree might be changed.
>
> {{{
>
>     def test_duplicated_setup_calls(self):
>         import django, logging
>         #from django.conf import settings
>         #print(settings.LOGGING_CONFIG, settings.LOGGING)
>
>         django.setup()
>         logging.getLogger('django').setLevel(logging.DEBUG)
>         assert logging.getLogger('django').level == logging.DEBUG
>
>         django.setup()
>         assert logging.getLogger('django').level == logging.DEBUG  #
> raises
>
> }}}
New description:

 Calling django.setup() multiple times is useless, BUT it can happen in
 lots of cases, that's why imho this case should be handled by the
 framework to avoid nasty side effects.

 These "duplicate calls" often involve the collision between manage.py
 commands, tests, custom scripts, and external launchers like pytest-
 django. Plus maybe some corner cases when unittest-style TestCases and
 pytest-style test functions are mixed in the same project.

 Users have to do a real gym to call setup() "at some moment" in all these
 use cases, yet try to prevent multiple calls of this initialization step
 (like the 'if__name__ == "main"' protection). So far my only way out was
 often to check for (not really undocumented) states of the framework
 before calling setup().

--

--
Ticket URL: <https://code.djangoproject.com/ticket/28752#comment:4>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/063.6eb230d0d65a353f178a9eefeaedf05a%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #28752: django.setup() should be idempotent

Django
In reply to this post by Django
#28752: django.setup() should be idempotent
--------------------------------+--------------------------------------
     Reporter:  pascal chambon  |                    Owner:  nobody
         Type:  Bug             |                   Status:  new
    Component:  Core (Other)    |                  Version:  1.11
     Severity:  Normal          |               Resolution:
     Keywords:                  |             Triage Stage:  Unreviewed
    Has patch:  0               |      Needs documentation:  0
  Needs tests:  0               |  Patch needs improvement:  0
Easy pickings:  0               |                    UI/UX:  0
--------------------------------+--------------------------------------
Description changed by pascal chambon:

Old description:

> Calling django.setup() multiple times is useless, BUT it can happen in
> lots of cases, that's why imho this case should be handled by the
> framework to avoid nasty side effects.
>
> These "duplicate calls" often involve the collision between manage.py
> commands, tests, custom scripts, and external launchers like pytest-
> django. Plus maybe some corner cases when unittest-style TestCases and
> pytest-style test functions are mixed in the same project.
>
> Users have to do a real gym to call setup() "at some moment" in all these
> use cases, yet try to prevent multiple calls of this initialization step
> (like the 'if__name__ == "main"' protection). So far my only way out was
> often to check for (not really undocumented) states of the framework
> before calling setup().
New description:

 I've been bitten numerous times by the impredictable behaviour of django
 when django.setup() was called numerous times.

 In the old days I had exceptions, now it's mainly subtle breakages of
 logging configuration.

 I couldn't find, in the issue tracker or the dev mailing list statements
 about this subject, others than request from other users encountering the
 problem.

 For example this ticket concerned script+importable modules :
 https://code.djangoproject.com/ticket/26152

 The latest case in date for me is pytest-django having troubles with
 these multiple setup() calls : https://github.com/pytest-dev/pytest-
 django/issues/531 , due to multiple fixtures attempting this auto-setup.

 Would it be OK to make django.setup() idempotent, or even expose a
 "is_ready" flag for easier introspection ?

--

--
Ticket URL: <https://code.djangoproject.com/ticket/28752#comment:5>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/063.bb8716fa01d239c1cba2d61c449c0111%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #28752: django.setup() should be idempotent

Django
In reply to this post by Django
#28752: django.setup() should be idempotent
--------------------------------+--------------------------------------
     Reporter:  pascal chambon  |                    Owner:  nobody
         Type:  Bug             |                   Status:  new
    Component:  Core (Other)    |                  Version:  1.11
     Severity:  Normal          |               Resolution:
     Keywords:                  |             Triage Stage:  Unreviewed
    Has patch:  0               |      Needs documentation:  0
  Needs tests:  0               |  Patch needs improvement:  0
Easy pickings:  0               |                    UI/UX:  0
--------------------------------+--------------------------------------
Description changed by pascal chambon:

Old description:

> I've been bitten numerous times by the impredictable behaviour of django
> when django.setup() was called numerous times.
>
> In the old days I had exceptions, now it's mainly subtle breakages of
> logging configuration.
>
> I couldn't find, in the issue tracker or the dev mailing list statements
> about this subject, others than request from other users encountering the
> problem.
>
> For example this ticket concerned script+importable modules :
> https://code.djangoproject.com/ticket/26152
>
> The latest case in date for me is pytest-django having troubles with
> these multiple setup() calls : https://github.com/pytest-dev/pytest-
> django/issues/531 , due to multiple fixtures attempting this auto-setup.
>
> Would it be OK to make django.setup() idempotent, or even expose a
> "is_ready" flag for easier introspection ?
New description:

 I've been bitten numerous times by the impredictable behaviour of django
 when django.setup() was called numerous times.

 In the old days I had exceptions, now it's mainly subtle breakages of
 logging configuration.

 I couldn't find, in the issue tracker or the dev mailing list statements
 about this subject, others than request from other users encountering the
 problem.

 For example this ticket concerned script+importable modules :
 https://code.djangoproject.com/ticket/26152

 The latest case in date for me is pytest-django having troubles with
 these multiple setup() calls : https://github.com/pytest-dev/pytest-
 django/issues/531 , due to multiple fixtures attempting this auto-setup.

 Would it be OK to make django.setup() idempotent, or even expose a
 "is_ready" flag for easier introspection ?


 -- here are some updates, comments get rejected as spam --



 Calling django.setup() multiple times is useless, BUT it can happen in
 lots of cases, that's why imho this case should be handled by the
 framework to avoid nasty side effects.

 These "duplicate calls" often involve the collision between manage.py
 commands, tests, custom scripts, and external launchers like pytest-
 django. Plus maybe some corner cases when unittest-style TestCases and
 pytest-style test functions are mixed in the same project.

 Users have to do a real gym to call setup() "at some moment" in all these
 use cases, yet try to prevent multiple calls of this initialization step
 (like the 'if__name__ == "main"' protection). So far my only way out was
 often to check for (not really undocumented) states of the framework
 before calling setup().

--

--
Ticket URL: <https://code.djangoproject.com/ticket/28752#comment:6>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/063.e6c335fd0c7e14567411475589baa07b%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #28752: django.setup() should be idempotent

Django
In reply to this post by Django
#28752: django.setup() should be idempotent
--------------------------------+--------------------------------------
     Reporter:  pascal chambon  |                    Owner:  nobody
         Type:  Bug             |                   Status:  new
    Component:  Core (Other)    |                  Version:  1.11
     Severity:  Normal          |               Resolution:
     Keywords:                  |             Triage Stage:  Unreviewed
    Has patch:  0               |      Needs documentation:  0
  Needs tests:  0               |  Patch needs improvement:  0
Easy pickings:  0               |                    UI/UX:  0
--------------------------------+--------------------------------------

Comment (by pascal chambon):

 My comments get marked as spam, I updated the description above.

--
Ticket URL: <https://code.djangoproject.com/ticket/28752#comment:7>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/063.116ef74d417ccaef059468739910d807%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #28752: django.setup() should be idempotent

Django
In reply to this post by Django
#28752: django.setup() should be idempotent
--------------------------------+--------------------------------------
     Reporter:  pascal chambon  |                    Owner:  nobody
         Type:  Bug             |                   Status:  new
    Component:  Core (Other)    |                  Version:  1.11
     Severity:  Normal          |               Resolution:
     Keywords:                  |             Triage Stage:  Unreviewed
    Has patch:  0               |      Needs documentation:  0
  Needs tests:  0               |  Patch needs improvement:  0
Easy pickings:  0               |                    UI/UX:  0
--------------------------------+--------------------------------------
Changes (by Tim Graham):

 * cc: Aymeric Augustin (added)


Old description:

> I've been bitten numerous times by the impredictable behaviour of django
> when django.setup() was called numerous times.
>
> In the old days I had exceptions, now it's mainly subtle breakages of
> logging configuration.
>
> I couldn't find, in the issue tracker or the dev mailing list statements
> about this subject, others than request from other users encountering the
> problem.
>
> For example this ticket concerned script+importable modules :
> https://code.djangoproject.com/ticket/26152
>
> The latest case in date for me is pytest-django having troubles with
> these multiple setup() calls : https://github.com/pytest-dev/pytest-
> django/issues/531 , due to multiple fixtures attempting this auto-setup.
>
> Would it be OK to make django.setup() idempotent, or even expose a
> "is_ready" flag for easier introspection ?
>

> -- here are some updates, comments get rejected as spam --
>

>
> Calling django.setup() multiple times is useless, BUT it can happen in
> lots of cases, that's why imho this case should be handled by the
> framework to avoid nasty side effects.
>
> These "duplicate calls" often involve the collision between manage.py
> commands, tests, custom scripts, and external launchers like pytest-
> django. Plus maybe some corner cases when unittest-style TestCases and
> pytest-style test functions are mixed in the same project.
>
> Users have to do a real gym to call setup() "at some moment" in all these
> use cases, yet try to prevent multiple calls of this initialization step
> (like the 'if__name__ == "main"' protection). So far my only way out was
> often to check for (not really undocumented) states of the framework
> before calling setup().
New description:

 I've been bitten numerous times by the impredictable behaviour of django
 when django.setup() was called numerous times.

 In the old days I had exceptions, now it's mainly subtle breakages of
 logging configuration.

 I couldn't find, in the issue tracker or the dev mailing list statements
 about this subject, others than request from other users encountering the
 problem.

 For example #26152 concerned script+importable modules.

 The latest case in date for me is pytest-django having troubles with these
 multiple setup() calls : https://github.com/pytest-dev/pytest-
 django/issues/531 , due to multiple fixtures attempting this auto-setup.

 Would it be OK to make django.setup() idempotent, or even expose an
 "is_ready" flag for easier introspection ?

 -- here are some updates, comments get rejected as spam --

 Calling django.setup() multiple times is useless, BUT it can happen in
 lots of cases, that's why imho this case should be handled by the
 framework to avoid nasty side effects.

 These "duplicate calls" often involve the collision between manage.py
 commands, tests, custom scripts, and external launchers like pytest-
 django. Plus maybe some corner cases when unittest-style TestCases and
 pytest-style test functions are mixed in the same project.

 Users have to do a real gym to call setup() "at some moment" in all these
 use cases, yet try to prevent multiple calls of this initialization step
 (like the 'if__name__ == "main"' protection). So far my only way out was
 often to check for (not really undocumented) states of the framework
 before calling setup().

--

Comment:

 Aymeric, any input?

--
Ticket URL: <https://code.djangoproject.com/ticket/28752#comment:8>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/063.6ea71d2e9b8faa73d3eefe7b47657f27%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #28752: django.setup() should be idempotent

Django
In reply to this post by Django
#28752: django.setup() should be idempotent
--------------------------------+--------------------------------------
     Reporter:  pascal chambon  |                    Owner:  nobody
         Type:  Bug             |                   Status:  new
    Component:  Core (Other)    |                  Version:  1.11
     Severity:  Normal          |               Resolution:
     Keywords:                  |             Triage Stage:  Unreviewed
    Has patch:  0               |      Needs documentation:  0
  Needs tests:  0               |  Patch needs improvement:  0
Easy pickings:  0               |                    UI/UX:  0
--------------------------------+--------------------------------------

Comment (by Aymeric Augustin):

 `django.setup()` is already idempotent. However it isn't reentrant — I
 believe that's what you're actually asking for — and it cannot be made
 reentrant without breaking its invariants. See #27176 for details.

 I don't think we should make changes to Django for accomodating pytest
 code that does `django.setup = lambda: None`.

--
Ticket URL: <https://code.djangoproject.com/ticket/28752#comment:9>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/063.fdf9070a8299762ec3dc51547c08b7b7%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #28752: django.setup() should not be runnable multiple times (was: django.setup() should be idempotent)

Django
In reply to this post by Django
#28752: django.setup() should not be runnable multiple times
--------------------------------+--------------------------------------
     Reporter:  pascal chambon  |                    Owner:  nobody
         Type:  Bug             |                   Status:  new
    Component:  Core (Other)    |                  Version:  1.11
     Severity:  Normal          |               Resolution:
     Keywords:                  |             Triage Stage:  Unreviewed
    Has patch:  0               |      Needs documentation:  0
  Needs tests:  0               |  Patch needs improvement:  0
Easy pickings:  0               |                    UI/UX:  0
--------------------------------+--------------------------------------

--
Ticket URL: <https://code.djangoproject.com/ticket/28752#comment:10>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/063.badd52874b7be9f41786359f627cdf6c%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #28752: django.setup() should not be runnable multiple times

Django
In reply to this post by Django
#28752: django.setup() should not be runnable multiple times
--------------------------------+--------------------------------------
     Reporter:  pascal chambon  |                    Owner:  nobody
         Type:  Bug             |                   Status:  new
    Component:  Core (Other)    |                  Version:  1.11
     Severity:  Normal          |               Resolution:
     Keywords:                  |             Triage Stage:  Unreviewed
    Has patch:  0               |      Needs documentation:  0
  Needs tests:  0               |  Patch needs improvement:  0
Easy pickings:  0               |                    UI/UX:  0
--------------------------------+--------------------------------------

Comment (by pascal chambon):

 OK I think my vocabulary was wrong, it's not (really) an idempotence
 problem, since django.setup() does more or less the same things on both
 calls (just skipping apps population phase on the second).

 It's not a reentrancy problem, i.e not a problem with multiple threads (or
 signal interrupts) entering django.setup() concurrently.

 It's really just a problem of "multiple successive calls of
 django.setup()", which are doing silent errors and weird modifications,
 simply because only the first call of django.setup() makes sense.

 Raising an exception on subsequent calls would be a possibility, but it
 would be a useless hassle, since users just want is to ensure that django
 was initialized at some point.
 That's why I propose we just do a no-op on subsequent calls to
 django.setup().

 (By teh way I don't understand your statement about the "django.setup =
 lambda: None" snippet - it was just a quick and dirty hack to prevent the
 multiple runs of django.setup(), which broke the LOGGING config)

--
Ticket URL: <https://code.djangoproject.com/ticket/28752#comment:11>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/063.38ac0044071c1d2077b7d55b7afe94d3%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #28752: django.setup() should not be runnable multiple times

Django
In reply to this post by Django
#28752: django.setup() should not be runnable multiple times
--------------------------------+--------------------------------------
     Reporter:  pascal chambon  |                    Owner:  nobody
         Type:  Bug             |                   Status:  new
    Component:  Core (Other)    |                  Version:  1.11
     Severity:  Normal          |               Resolution:
     Keywords:                  |             Triage Stage:  Unreviewed
    Has patch:  0               |      Needs documentation:  0
  Needs tests:  0               |  Patch needs improvement:  0
Easy pickings:  0               |                    UI/UX:  0
--------------------------------+--------------------------------------

Old description:

> I've been bitten numerous times by the impredictable behaviour of django
> when django.setup() was called numerous times.
>
> In the old days I had exceptions, now it's mainly subtle breakages of
> logging configuration.
>
> I couldn't find, in the issue tracker or the dev mailing list statements
> about this subject, others than request from other users encountering the
> problem.
>
> For example #26152 concerned script+importable modules.
>
> The latest case in date for me is pytest-django having troubles with
> these multiple setup() calls : https://github.com/pytest-dev/pytest-
> django/issues/531 , due to multiple fixtures attempting this auto-setup.
>
> Would it be OK to make django.setup() idempotent, or even expose an
> "is_ready" flag for easier introspection ?
>
> -- here are some updates, comments get rejected as spam --
>
> Calling django.setup() multiple times is useless, BUT it can happen in
> lots of cases, that's why imho this case should be handled by the
> framework to avoid nasty side effects.
>
> These "duplicate calls" often involve the collision between manage.py
> commands, tests, custom scripts, and external launchers like pytest-
> django. Plus maybe some corner cases when unittest-style TestCases and
> pytest-style test functions are mixed in the same project.
>
> Users have to do a real gym to call setup() "at some moment" in all these
> use cases, yet try to prevent multiple calls of this initialization step
> (like the 'if__name__ == "main"' protection). So far my only way out was
> often to check for (not really undocumented) states of the framework
> before calling setup().
New description:

 I've been bitten numerous times by the impredictable behaviour of django
 when django.setup() was called numerous times.

 In the old days I had exceptions, now it's mainly subtle breakages of
 logging configuration.

 I couldn't find, in the issue tracker or the dev mailing list statements
 about this subject, others than request from other users encountering the
 problem.

 For example #26152 concerned script+importable modules.

 The latest case in date for me is pytest-django having troubles with these
 multiple setup() calls : https://github.com/pytest-dev/pytest-
 django/issues/531 , due to multiple fixtures attempting this auto-setup.

 Would it be OK to make django.setup() idempotent, or even expose an
 "is_ready" flag for easier introspection ?

 -- here are some updates, comments get rejected as spam --

 Calling django.setup() multiple times is useless, BUT it can happen in
 lots of cases, that's why imho this case should be handled by the
 framework to avoid nasty side effects.

 These "duplicate calls" often involve the collision between manage.py
 commands, tests, custom scripts, and external launchers like pytest-
 django. Plus maybe some corner cases when unittest-style TestCases and
 pytest-style test functions are mixed in the same project.

 Users have to do a real gym to call setup() "at some moment" in all these
 use cases, yet try to prevent multiple calls of this initialization step
 (like the `if__name__ == "main"'` protection). So far my only way out was
 often to check for (not really undocumented) states of the framework
 before calling setup().

--

Comment (by Tim Graham):

 I don't know. Does that change risk breaking working code where multiple
 calls to `django.setup()` has an intended effect?

--
Ticket URL: <https://code.djangoproject.com/ticket/28752#comment:12>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/063.69f40a5106663af7af7fd4c49c878d53%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #28752: django.setup() should not be runnable multiple times

Django
In reply to this post by Django
#28752: django.setup() should not be runnable multiple times
--------------------------------+--------------------------------------
     Reporter:  pascal chambon  |                    Owner:  nobody
         Type:  Bug             |                   Status:  new
    Component:  Core (Other)    |                  Version:  1.11
     Severity:  Normal          |               Resolution:
     Keywords:                  |             Triage Stage:  Unreviewed
    Has patch:  0               |      Needs documentation:  0
  Needs tests:  0               |  Patch needs improvement:  0
Easy pickings:  0               |                    UI/UX:  0
--------------------------------+--------------------------------------

Comment (by pascal chambon):

 Well, I'm usually quite dedicated to retrocompatibility (see django-
 compat-patcher package), but for once any breakage would be due to a
 strange misuse of django.setup().

 In the code below, we see that django.setup() performs 3 steps :

 - overridding logging configuration with django settings
 - setting the script prefix
 - populating the apps registry

 The last 2 steps are now idempotent it seems.

 Only overridding logging breaks some setups (eg. with pytest fixtures),
 and I can't find any use case where it would be a wanted behaviour.
 If users want to reset logging several times, they may as well call
 configure_logging() by themselves.

 Until django.setup() is protected against double executions, we'll have
 weird bugs surfacing each time we add new steps to it ''(are these
 idempotent, or reentrant, or runnable multiple times...)'', so I'd
 advocate fixing this once and for all.

 On "how" to do it, I think a threading lock + a boolean guard would be
 easy and sufficient, wouldn't they ? With a system to raise an error if
 django.setup() ends up being called multiple times by the same thread
 (which often smells like missing "if __name__ == '__main'__" conditions in
 imported scripts).


 {{{

 configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)

 if set_prefix:
         set_script_prefix(
             '/' if settings.FORCE_SCRIPT_NAME is None else
 settings.FORCE_SCRIPT_NAME
         )

 apps.populate(settings.INSTALLED_APPS)
 }}}

--
Ticket URL: <https://code.djangoproject.com/ticket/28752#comment:13>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/063.00205178636b3654314e006480dbea93%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #28752: Prevent django.setup() from running multiple times (was: django.setup() should not be runnable multiple times)

Django
In reply to this post by Django
#28752: Prevent django.setup() from running multiple times
--------------------------------------+------------------------------------
     Reporter:  pascal chambon        |                    Owner:  nobody
         Type:  Cleanup/optimization  |                   Status:  new
    Component:  Core (Other)          |                  Version:  1.11
     Severity:  Normal                |               Resolution:
     Keywords:                        |             Triage Stage:  Accepted
    Has patch:  0                     |      Needs documentation:  0
  Needs tests:  0                     |  Patch needs improvement:  0
Easy pickings:  0                     |                    UI/UX:  0
--------------------------------------+------------------------------------
Changes (by Tim Graham):

 * type:  Bug => Cleanup/optimization
 * stage:  Unreviewed => Accepted


Comment:

 I'm not completely sure if a change is feasible but I guess we could
 evaluate a patch.

--
Ticket URL: <https://code.djangoproject.com/ticket/28752#comment:14>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/063.86c8be4bc2817c3027b7fe7995f5883e%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #28752: Prevent django.setup() from running multiple times

Django
In reply to this post by Django
#28752: Prevent django.setup() from running multiple times
--------------------------------------+------------------------------------
     Reporter:  pascal chambon        |                    Owner:  nobody
         Type:  Cleanup/optimization  |                   Status:  new
    Component:  Core (Other)          |                  Version:  1.11
     Severity:  Normal                |               Resolution:
     Keywords:                        |             Triage Stage:  Accepted
    Has patch:  0                     |      Needs documentation:  0
  Needs tests:  0                     |  Patch needs improvement:  0
Easy pickings:  0                     |                    UI/UX:  0
--------------------------------------+------------------------------------

Comment (by Daniel Hahler):

 This would be nice to have indeed.

 Pascal, do you plan to provide a patch for this?

 btw: the following can be used as a workaround (also used by pytest-
 django):
 {{{
     import django.apps

     if not django.apps.apps.ready:
         django.setup()
 }}}

--
Ticket URL: <https://code.djangoproject.com/ticket/28752#comment:15>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/063.aa7d38b0cb9868b346a9fa8fee8bda18%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #28752: Prevent django.setup() from running multiple times

Django
In reply to this post by Django
#28752: Prevent django.setup() from running multiple times
--------------------------------------+------------------------------------
     Reporter:  pascal chambon        |                    Owner:  nobody
         Type:  Cleanup/optimization  |                   Status:  new
    Component:  Core (Other)          |                  Version:  1.11
     Severity:  Normal                |               Resolution:
     Keywords:                        |             Triage Stage:  Accepted
    Has patch:  0                     |      Needs documentation:  0
  Needs tests:  0                     |  Patch needs improvement:  0
Easy pickings:  0                     |                    UI/UX:  0
--------------------------------------+------------------------------------

Comment (by pascal chambon):

 Thanks for reminding me of this ticket, my latest struggles with pytest-
 django made me think a more global solution to the problem of django setup
 and complex environments, and this ticket is for me now superseded by
 https://code.djangoproject.com/ticket/30536 - hoping that the new setting
 I introduce will be OK too.

--
Ticket URL: <https://code.djangoproject.com/ticket/28752#comment:16>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/063.e3201b02745aa1ad23340f02845d5b17%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #28752: Prevent django.setup() from running multiple times

Django
In reply to this post by Django
#28752: Prevent django.setup() from running multiple times
--------------------------------------+------------------------------------
     Reporter:  pascal chambon        |                    Owner:  nobody
         Type:  Cleanup/optimization  |                   Status:  new
    Component:  Core (Other)          |                  Version:  1.11
     Severity:  Normal                |               Resolution:
     Keywords:                        |             Triage Stage:  Accepted
    Has patch:  0                     |      Needs documentation:  0
  Needs tests:  0                     |  Patch needs improvement:  0
Easy pickings:  0                     |                    UI/UX:  0
--------------------------------------+------------------------------------

Comment (by Carlton Gibson):

 Let's keep the discussion from #30536 here. (It's ≈the same ticket AFAICS,
 and the history here is informative.)

 [https://github.com/django/django/pull/11435 PR there is here].

 One additional element added there is to make the the `setup()` code
 ''pluggable''. (i.e. whilst we make `setup()` exit if run multiple times,
 allow user code to be run before/after as well.) A setting is proposed for
 this. ''MUST it be a setting…?''

--
Ticket URL: <https://code.djangoproject.com/ticket/28752#comment:17>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/063.307296b0b80c926d0c32c17e77b1cd12%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #28752: Prevent django.setup() from running multiple times

Django
In reply to this post by Django
#28752: Prevent django.setup() from running multiple times
--------------------------------------+------------------------------------
     Reporter:  pascal chambon        |                    Owner:  nobody
         Type:  Cleanup/optimization  |                   Status:  new
    Component:  Core (Other)          |                  Version:  1.11
     Severity:  Normal                |               Resolution:
     Keywords:                        |             Triage Stage:  Accepted
    Has patch:  0                     |      Needs documentation:  0
  Needs tests:  0                     |  Patch needs improvement:  1
Easy pickings:  0                     |                    UI/UX:  0
--------------------------------------+------------------------------------
Changes (by Carlton Gibson):

 * needs_better_patch:  0 => 1


Comment:

 Looking at the PR, there are two things going on there:

 1. Making `setup()` ''idempotent''. Let's handle that part here.
 2. Allowing specifying custom logic. Let's handle that on #30536.

 (The one looks less controversial than the other...)

 Pascal, if you could split the PR into two, targeting the one as `Fixed
 #28752 -- ...` that would be great.

--
Ticket URL: <https://code.djangoproject.com/ticket/28752#comment:18>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/063.a399c13c4b3804797fb4eb8397b7213a%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Django] #28752: Prevent django.setup() from running multiple times

Django
In reply to this post by Django
#28752: Prevent django.setup() from running multiple times
--------------------------------------+------------------------------------
     Reporter:  pascal chambon        |                    Owner:  nobody
         Type:  Cleanup/optimization  |                   Status:  new
    Component:  Core (Other)          |                  Version:  1.11
     Severity:  Normal                |               Resolution:
     Keywords:                        |             Triage Stage:  Accepted
    Has patch:  0                     |      Needs documentation:  0
  Needs tests:  0                     |  Patch needs improvement:  1
Easy pickings:  0                     |                    UI/UX:  0
--------------------------------------+------------------------------------

Comment (by pascal chambon):

 OK let's handle 1 feature at a time  :)

 Here is the PR only for idempotence (and django.is_ready flag) :
 https://github.com/django/django/pull/11440

 One of the CI checks failed for obscure reasons (package django not
 found), any easy way to restart it ?

--
Ticket URL: <https://code.djangoproject.com/ticket/28752#comment:19>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/063.ac784c35a676c8481f622409c6fe86df%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
12