UTF-8 name of a staticfiles test is breaking Django installation

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

UTF-8 name of a staticfiles test is breaking Django installation

Marcin Nowak
Hi all,

Here is a file which breaks installation on fresh Debian system:
 
https://github.com/django/django/blob/master/tests/staticfiles_tests/apps/test/static/test/%E2%8A%97.txt

I'm using Buildout and virtualenv with Python 2.7.9.

The full traceback is:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/project/env/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 2271, in main
    **kw
  File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/home/webcar/env/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 409, in run
    self.easy_install(spec, not self.no_deps)
  File "/home/webcar/env/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 645, in easy_install
    return self.install_item(None, spec, tmpdir, deps, True)
  File "/home/webcar/env/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 694, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/home/webcar/env/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 845, in install_eggs
    unpack_archive(dist_filename, tmpdir, self.unpack_progress)
  File "/home/webcar/env/lib/python2.7/site-packages/setuptools/archive_util.py", line 52, in unpack_archive
    driver(filename, extract_dir, progress_filter)
  File "/home/webcar/env/lib/python2.7/site-packages/setuptools/archive_util.py", line 148, in unpack_tarfile
    prelim_dst = os.path.join(extract_dir, *name.split('/'))
  File "/home/webcar/env/lib/python2.7/posixpath.py", line 80, in join
    path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1: ordinal not in range(128)
An error occurred when trying to install Django 1.8.14. Look above this message for any errors that were output by easy_install.

The path to a file which fails is:
/tmp/easy_install-A3XRR9 Django-1.8.14/tests/staticfiles_tests/apps/test/static/test/⊗.txt

Is this file really needed in Django dist?

Marcin
 

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/967785a0-3df7-4454-a2dd-7c5e6875991d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: UTF-8 name of a staticfiles test is breaking Django installation

Marcin Nowak


On Tuesday, August 2, 2016 at 12:40:27 PM UTC+2, Marcin Nowak wrote:

I'm using Buildout and virtualenv with Python 2.7.9.


and setuptools 25.1.2 


M.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/cb9383d4-2ced-4ea2-9a9d-bb9be6e43086%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: UTF-8 name of a staticfiles test is breaking Django installation

Marcin Nowak

I'm using Buildout and virtualenv with Python 2.7.9.


and setuptools 25.1.2 


It works with setuptools 17.0, but displays warning:

Getting distribution for 'Django==1.8.14'.
'tests/staticfiles_tests/apps/test/static/test/⊗.txt' not ANSI_X3.4-1968 encodable -- skipping
warning: no previously-included files matching '__pycache__' found under directory '*'
warning: no previously-included files matching '*.py[co]' found under directory '*'
'tests/staticfiles_tests/apps/test/static/test/⊗.txt' not ANSI_X3.4-1968 encodable -- skipping
Got Django 1.8.14.
 

M.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/9914693c-56cd-43f2-ac19-b763628ce740%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: UTF-8 name of a staticfiles test is breaking Django installation

Tim Graham-2
A similar report was https://code.djangoproject.com/ticket/24761. I'm not sure if this is a bug in those install tools or if it's something we should try to fix in Django. You can use `git blame tests/staticfiles_tests/apps/test/static/test/⊗.txt` to find the commit where the file was introduced to learn the reason for it.

On Tuesday, August 2, 2016 at 6:45:28 AM UTC-4, Marcin Nowak wrote:

I'm using Buildout and virtualenv with Python 2.7.9.


and setuptools 25.1.2 


It works with setuptools 17.0, but displays warning:

Getting distribution for 'Django==1.8.14'.
'tests/staticfiles_tests/apps/test/static/test/⊗.txt' not ANSI_X3.4-1968 encodable -- skipping
warning: no previously-included files matching '__pycache__' found under directory '*'
warning: no previously-included files matching '*.py[co]' found under directory '*'
'tests/staticfiles_tests/apps/test/static/test/⊗.txt' not ANSI_X3.4-1968 encodable -- skipping
Got Django 1.8.14.
 

M.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/00a3a16a-aab8-4edb-9627-961e27258134%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: UTF-8 name of a staticfiles test is breaking Django installation

Berker Peksağ
On Tue, Aug 2, 2016 at 6:14 PM, Tim Graham <[hidden email]> wrote:
> A similar report was https://code.djangoproject.com/ticket/24761. I'm not
> sure if this is a bug in those install tools or if it's something we should
> try to fix in Django. You can use `git blame
> tests/staticfiles_tests/apps/test/static/test/⊗.txt` to find the commit
> where the file was introduced to learn the reason for it.

It looks like this is a bug in setuptools:
https://github.com/pypa/setuptools/issues/709

--Berker

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAF4280%2BXA9%3DKDAE7CKrgCjY4ttq%2BkxG5VdXZk2p6KdUn5k8VWQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: UTF-8 name of a staticfiles test is breaking Django installation

Aymeric Augustin
Hello,

Django used to create this file dynamically while running tests, but that got in the way of test parallelization. To avoid race conditions, the git checkout needs to be treated as readonly when running tests in parallel.

That’s why I switched to committing it to the git repository when I implemented test parallelization. I assumed all relevant systems for Django would support utf-8 by now… I was too optimistic!

Can we consider “update setuptools” to be an adequate resolution for this issue?

--
Aymeric.

> On 02 Aug 2016, at 18:03, Berker Peksağ <[hidden email]> wrote:
>
> On Tue, Aug 2, 2016 at 6:14 PM, Tim Graham <[hidden email]> wrote:
>> A similar report was https://code.djangoproject.com/ticket/24761. I'm not
>> sure if this is a bug in those install tools or if it's something we should
>> try to fix in Django. You can use `git blame
>> tests/staticfiles_tests/apps/test/static/test/⊗.txt` to find the commit
>> where the file was introduced to learn the reason for it.
>
> It looks like this is a bug in setuptools:
> https://github.com/pypa/setuptools/issues/709
>
> --Berker
>
> --
> You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> To post to this group, send email to [hidden email].
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAF4280%2BXA9%3DKDAE7CKrgCjY4ttq%2BkxG5VdXZk2p6KdUn5k8VWQ%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/3FC5B5B0-681F-40F4-A0A6-6915F043C2E7%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: UTF-8 name of a staticfiles test is breaking Django installation

René Fleschenberg

Hi everyone,

 

> Can we consider “update setuptools” to be an adequate resolution for this

> issue?

 

I realize this is probably not the most substantial argument, but I want to mention it anyway: Considering Django's popularity, requiring a recent version of setuptools might be a positive side-effect on the larger Python ecosystem. The prevalence of old setuptools versions can be rather annoying. For example: https://hynek.me/articles/conditional-python-dependencies/

 

--

René Fleschenberg

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/5063025.TKyiib2vOM%40rex.
For more options, visit https://groups.google.com/d/optout.