Quantcast

[Django] #27854: Make `collectstatic` warn (rather than blow up) on missing directories

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[Django] #27854: Make `collectstatic` warn (rather than blow up) on missing directories

Django
#27854: Make `collectstatic` warn (rather than blow up) on missing directories
-----------------------------------------------+------------------------
               Reporter:  David Evans          |          Owner:  nobody
                   Type:  Uncategorized        |         Status:  new
              Component:  contrib.staticfiles  |        Version:  1.10
               Severity:  Normal               |       Keywords:
           Triage Stage:  Unreviewed           |      Has patch:  0
    Needs documentation:  0                    |    Needs tests:  0
Patch needs improvement:  0                    |  Easy pickings:  0
                  UI/UX:  0                    |
-----------------------------------------------+------------------------
 At present if the `STATICFILES_DIRS` setting contains references to
 directories which do not exist then the whole command will die with an
 `OSError`.

 A situation I've seen bite a few newcomers to Django is that they will
 have an empty static directory which (being empty) won't get tracked by
 git. This means that `collectstatic` works when they run it locally, but
 blows up with a (to them) cryptic error when they try to deploy.

 If we made `collectstatic` simply log a warning in the case of non-
 existent directories and continue processing then we could avoid this
 problem and remove one possible source of frustration for newcomers.

 If this approach seems acceptable to others I am happy to submit an
 appropriate patch.

--
Ticket URL: <https://code.djangoproject.com/ticket/27854>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/049.a073579d693a0f9435ac0981bca4bfec%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Django] #27854: Make `collectstatic` warn (rather than blow up) on missing directories

Django
#27854: Make `collectstatic` warn (rather than blow up) on missing directories
-------------------------------------+-------------------------------------
     Reporter:  David Evans          |                    Owner:  nobody
         Type:  Uncategorized        |                   Status:  new
    Component:  contrib.staticfiles  |                  Version:  1.10
     Severity:  Normal               |               Resolution:
     Keywords:                       |             Triage Stage:
                                     |  Unreviewed
    Has patch:  0                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------

Comment (by Tim Graham):

 Maybe the current error message could be improved. What does it say?

 Did you research the origin of the current exception to see if you can
 find any design decisions there?

--
Ticket URL: <https://code.djangoproject.com/ticket/27854#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/064.5ce2487ab32b9d81544cbea0a3a3a0c0%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Django] #27854: Make `collectstatic` warn (rather than blow up) on missing directories

Django
In reply to this post by Django
#27854: Make `collectstatic` warn (rather than blow up) on missing directories
-------------------------------------+-------------------------------------
     Reporter:  David Evans          |                    Owner:  nobody
         Type:  Uncategorized        |                   Status:  new
    Component:  contrib.staticfiles  |                  Version:  1.10
     Severity:  Normal               |               Resolution:
     Keywords:                       |             Triage Stage:
                                     |  Unreviewed
    Has patch:  0                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------

Comment (by David Evans):

 Replying to [comment:1 Tim Graham]:
 > Maybe the current error message could be improved. What does it say?
 >
 > Did you research the origin of the current exception to see if you can
 find any design decisions there?

 The exception is just `OSError: [Errno 2] No such file or directory`

 This bubbles up from directly from the call to `os.listdir` here:
 https://github.com/django/django/blob/1f7ca858664491589ba400419a491dd0a9af5dff/django/core/files/storage.py#L312

 So I don't think there was any deliberate design decision here.

 Also, I don't think improving the error message will solve this specific
 problem because:
 a. If the user is deploying to Heroku then they are just told that the
 collectstatic step was skipped due to errors, but they have to run the
 command themselves to see what those errors were.
 b. An error message about missing directories is confusing to them because
 they can see the directory right there on their filesystem. The fact that
 git doesn't track empty directories is then yet another bit of weird
 computer arcanery that we need to expose them to when they are just trying
 to deploy "My First Website".

--
Ticket URL: <https://code.djangoproject.com/ticket/27854#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/064.6f1752076a1b6680ca4c7fce4fcf16c6%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Django] #27854: Make `collectstatic` warn (rather than blow up) on missing directories

Django
In reply to this post by Django
#27854: Make `collectstatic` warn (rather than blow up) on missing directories
-------------------------------------+-------------------------------------
     Reporter:  David Evans          |                    Owner:  nobody
         Type:  Uncategorized        |                   Status:  new
    Component:  contrib.staticfiles  |                  Version:  1.10
     Severity:  Normal               |               Resolution:
     Keywords:                       |             Triage Stage:
                                     |  Unreviewed
    Has patch:  0                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------

Comment (by Aymeric Augustin):

 In the case of Heroku which swallows the output of collectstatic, if
 `STATICFILES_DIRS` doesn't have a correct value, the change suggested here
 will make it pretend that it succeeded, even though it didn't collect
 files, while it currently says it's failing. I think that's a problem:
 "errors shouldn't pass silently" :-)

--
Ticket URL: <https://code.djangoproject.com/ticket/27854#comment:3>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/064.9db3cbcf944f9fe9b9a579192d20b753%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Django] #27854: Make `collectstatic` warn (rather than blow up) on missing directories

Django
In reply to this post by Django
#27854: Make `collectstatic` warn (rather than blow up) on missing directories
-------------------------------------+-------------------------------------
     Reporter:  David Evans          |                    Owner:  nobody
         Type:  Uncategorized        |                   Status:  new
    Component:  contrib.staticfiles  |                  Version:  1.10
     Severity:  Normal               |               Resolution:
     Keywords:                       |             Triage Stage:
                                     |  Unreviewed
    Has patch:  0                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------

Comment (by David Evans):

 Replying to [comment:3 Aymeric Augustin]:
 > In the case of Heroku which swallows the output of collectstatic, if
 `STATICFILES_DIRS` doesn't have a correct value, the change suggested here
 will make it pretend that it succeeded, even though it didn't collect
 files, while it currently says it's failing. I think that's a problem:
 "errors shouldn't pass silently" :-)

 Yes, absolutely agree that errors shouldn't pass silently. In the case of
 the Heroku buildpack, it does suppress some of the output but any warning
 messages will get displayed:
 https://github.com/heroku/heroku-buildpack-
 python/blob/677dfeec119f28b4d1a8f679b38b2d4e407f4533/bin/steps/collectstatic#L33

 Would another option be for `collectstatic` to note that a directory was
 missing, issue a warning, ''proceed with collecting the rest of the static
 files'', and then exit with status > 0 at the end? That way, the user
 would still have a working set of static files.

--
Ticket URL: <https://code.djangoproject.com/ticket/27854#comment:4>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/064.5d45eedf7284de5e1e582a0131271993%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Django] #27854: Make `collectstatic` warn (rather than blow up) on missing directories

Django
In reply to this post by Django
#27854: Make `collectstatic` warn (rather than blow up) on missing directories
-------------------------------------+-------------------------------------
     Reporter:  David Evans          |                    Owner:  nobody
         Type:  Uncategorized        |                   Status:  new
    Component:  contrib.staticfiles  |                  Version:  1.10
     Severity:  Normal               |               Resolution:
     Keywords:                       |             Triage Stage:
                                     |  Unreviewed
    Has patch:  0                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------

Comment (by Aymeric Augustin):

 As long as that doesn't obscure errors, yes, it should be an improvement.

 Historically we've removed lots of "helpful" error handling which made
 other errors harder to diagnose. Often the raw stack trace is the best way
 to see what went wrong. I wanted to share that experience with the first
 case that came to my mind.

--
Ticket URL: <https://code.djangoproject.com/ticket/27854#comment:5>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/064.6ab81b2453d3d709a48f662b7f322b9b%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Django] #27854: Make `collectstatic` warn (rather than blow up) on missing directories

Django
In reply to this post by Django
#27854: Make `collectstatic` warn (rather than blow up) on missing directories
-------------------------------------+-------------------------------------
     Reporter:  David Evans          |                    Owner:  nobody
         Type:  Uncategorized        |                   Status:  new
    Component:  contrib.staticfiles  |                  Version:  1.10
     Severity:  Normal               |               Resolution:
     Keywords:                       |             Triage Stage:
                                     |  Unreviewed
    Has patch:  0                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------

Comment (by Tim Graham):

 Another trend is moving runtime warnings to the system check framework,
 e.g. 0ec4dc91e0e7befdd06aa0613b5d0fbe3c785ee7. Could that be appropriate
 here?

--
Ticket URL: <https://code.djangoproject.com/ticket/27854#comment:6>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/064.892e674b1acf35cd7c393ee94068e0af%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Django] #27854: Make `collectstatic` warn (rather than blow up) on missing directories

Django
In reply to this post by Django
#27854: Make `collectstatic` warn (rather than blow up) on missing directories
-------------------------------------+-------------------------------------
     Reporter:  David Evans          |                    Owner:  nobody
         Type:  Uncategorized        |                   Status:  new
    Component:  contrib.staticfiles  |                  Version:  1.10
     Severity:  Normal               |               Resolution:
     Keywords:                       |             Triage Stage:
                                     |  Unreviewed
    Has patch:  0                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------

Comment (by David Evans):

 That's an interesting idea. So referencing a non-existent directory in
 `STATICFILES_DIRS` would trigger a warning in the system check framework,
 but wouldn't prevent `collectstatic` from running. Aymeric, would that
 address your concerns?

--
Ticket URL: <https://code.djangoproject.com/ticket/27854#comment:7>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/064.aa501aec320e4ce64dd0c874e81556c3%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Django] #27854: Make `collectstatic` warn (rather than blow up) on missing directories

Django
In reply to this post by Django
#27854: Make `collectstatic` warn (rather than blow up) on missing directories
--------------------------------------+------------------------------------
     Reporter:  David Evans           |                    Owner:  nobody
         Type:  Cleanup/optimization  |                   Status:  new
    Component:  contrib.staticfiles   |                  Version:  1.10
     Severity:  Normal                |               Resolution:
     Keywords:                        |             Triage Stage:  Accepted
    Has patch:  0                     |      Needs documentation:  0
  Needs tests:  0                     |  Patch needs improvement:  0
Easy pickings:  0                     |                    UI/UX:  0
--------------------------------------+------------------------------------
Changes (by Tim Graham):

 * type:  Uncategorized => Cleanup/optimization
 * stage:  Unreviewed => Accepted


--
Ticket URL: <https://code.djangoproject.com/ticket/27854#comment:8>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/064.65e6cf0011adf8aac8f61edfe62138f4%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Django] #27854: Make `collectstatic` warn (rather than blow up) on missing directories

Django
In reply to this post by Django
#27854: Make `collectstatic` warn (rather than blow up) on missing directories
-------------------------------------+-------------------------------------
     Reporter:  David Evans          |                    Owner:
         Type:                       |  maheshjurel
  Cleanup/optimization               |                   Status:  assigned
    Component:  contrib.staticfiles  |                  Version:  1.10
     Severity:  Normal               |               Resolution:
     Keywords:                       |             Triage Stage:  Accepted
    Has patch:  0                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------
Changes (by maheshjurel):

 * owner:  nobody => maheshjurel
 * status:  new => assigned


--
Ticket URL: <https://code.djangoproject.com/ticket/27854#comment:9>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/064.3e171b718f78da81ff5ab39f16259262%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Django] #27854: Make `collectstatic` warn (rather than blow up) on missing directories

Django
In reply to this post by Django
#27854: Make `collectstatic` warn (rather than blow up) on missing directories
-------------------------------------+-------------------------------------
     Reporter:  David Evans          |                    Owner:
         Type:                       |  maheshjurel
  Cleanup/optimization               |                   Status:  assigned
    Component:  contrib.staticfiles  |                  Version:  1.10
     Severity:  Normal               |               Resolution:
     Keywords:                       |             Triage Stage:  Accepted
    Has patch:  0                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------

Comment (by maheshjurel):

 Changes have been made to show warning (without stopping execution) if any
 of the static files directory does not exists. Earlier the situation was
 that if we have more than one directory paths in  `STATICFILES_DIRS` and
 if the first one is missing then it wouldn't process the next directory
 and fails giving the exception logs.
 But now, with the current changes it will show a message in the format
 that `"Path [<missing_dir_path>] does not exists. Skipping..."` and also
 process all the directories (next ones) as expected.
 A pull request has been created
 [https://github.com/django/django/pull/8137]

--
Ticket URL: <https://code.djangoproject.com/ticket/27854#comment:10>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/064.bb2129f7806b3d1c384af5d3ba16ddca%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Django] #27854: Make `collectstatic` warn (rather than blow up) on missing directories

Django
In reply to this post by Django
#27854: Make `collectstatic` warn (rather than blow up) on missing directories
-------------------------------------+-------------------------------------
     Reporter:  David Evans          |                    Owner:
         Type:                       |  maheshjurel
  Cleanup/optimization               |                   Status:  closed
    Component:  contrib.staticfiles  |                  Version:  1.10
     Severity:  Normal               |               Resolution:  fixed
     Keywords:                       |             Triage Stage:  Accepted
    Has patch:  0                    |      Needs documentation:  0
  Needs tests:  0                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------
Changes (by maheshjurel):

 * status:  assigned => closed
 * resolution:   => fixed


--
Ticket URL: <https://code.djangoproject.com/ticket/27854#comment:11>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/064.e51b381faaf4e048766eef8d6d315b87%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Django] #27854: Make `collectstatic` warn (rather than blow up) on missing directories

Django
In reply to this post by Django
#27854: Make `collectstatic` warn (rather than blow up) on missing directories
-------------------------------------+-------------------------------------
     Reporter:  David Evans          |                    Owner:
         Type:                       |  maheshjurel
  Cleanup/optimization               |                   Status:  new
    Component:  contrib.staticfiles  |                  Version:  1.10
     Severity:  Normal               |               Resolution:
     Keywords:                       |             Triage Stage:  Accepted
    Has patch:  1                    |      Needs documentation:  0
  Needs tests:  1                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------
Changes (by Tim Graham):

 * status:  closed => new
 * has_patch:  0 => 1
 * resolution:  fixed =>
 * needs_tests:  0 => 1


Comment:

 The ticket is marked fixed when the patch is committed to the Django
 repository.  Please read
 [https://docs.djangoproject.com/en/dev/internals/contributing/triaging-
 tickets/ Triaging Tickets]. A test is also required. Please uncheck "Needs
 tests" on the ticket after updating the PR.

--
Ticket URL: <https://code.djangoproject.com/ticket/27854#comment:12>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/064.3ade743f6135ac4019382366571f5922%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Django] #27854: Make `collectstatic` warn (rather than blow up) on missing directories

Django
In reply to this post by Django
#27854: Make `collectstatic` warn (rather than blow up) on missing directories
-------------------------------------+-------------------------------------
     Reporter:  David Evans          |                    Owner:  Mahesh
         Type:                       |  Kumar Jurel
  Cleanup/optimization               |                   Status:  assigned
    Component:  contrib.staticfiles  |                  Version:  1.10
     Severity:  Normal               |               Resolution:
     Keywords:                       |             Triage Stage:  Accepted
    Has patch:  1                    |      Needs documentation:  0
  Needs tests:  1                    |  Patch needs improvement:  0
Easy pickings:  0                    |                    UI/UX:  0
-------------------------------------+-------------------------------------
Changes (by Mahesh Kumar Jurel):

 * status:  new => assigned


--
Ticket URL: <https://code.djangoproject.com/ticket/27854#comment:13>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups "Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/064.1a775ed2f1904288bc89dfa890a0db45%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
Loading...