Should django File wrapper support .next()?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Should django File wrapper support .next()?

Cristiano Coelho
Hello,

I have recently found an interesting issue, using a project that relies on different storage backends, when switching from a custom one to django's file system storage, I found that existing code that would iterate files with the next() call would start to fail, since although django's File is iterable, it doesn't define the next method.

Now I'm wondering if this is on purpose, or a bug. It's odd that every stream (or almost) from the python library such as everything from the .io module or simply the object returned by open() (python 2) supports the next call but the django File wrapper doesn't.

This happened with django 1.10 and I believe it wasn't changed with Django 1.11

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/09ffad76-e473-4593-ac84-4bca7f76e92c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Should django File wrapper support .next()?

Adam Johnson-2
The next() method is gone from the return value of open() in Python 3 and I don't think it was every necessary. As per the iterator protcool ( https://docs.python.org/2/library/stdtypes.html#iterator-types ), __iter__ is meant to return an iterator over the object, and that is what the next() method - __next__ on Python 3 - should be attached to. Django's File.__iter__ is a generator function, thus it returns a generator, with comes with a next()/__next__().

I think you should just try iterating your files by calling iter() on them first, it's Python 3 compatible too.

On 27 July 2017 at 17:24, Cristiano Coelho <[hidden email]> wrote:
Hello,

I have recently found an interesting issue, using a project that relies on different storage backends, when switching from a custom one to django's file system storage, I found that existing code that would iterate files with the next() call would start to fail, since although django's File is iterable, it doesn't define the next method.

Now I'm wondering if this is on purpose, or a bug. It's odd that every stream (or almost) from the python library such as everything from the .io module or simply the object returned by open() (python 2) supports the next call but the django File wrapper doesn't.

This happened with django 1.10 and I believe it wasn't changed with Django 1.11

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/09ffad76-e473-4593-ac84-4bca7f76e92c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Adam

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM394-B26RzXpfhCqb8MYJPwpu%2B_eELpttFDqk6Zfn4Rvg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Should django File wrapper support .next()?

Cristiano Coelho
I forgot about django dropping support for Python2 so I guess you are right this doesn't make much sense if next() is removed, but for Python2 it would make sense for django's File object to have the same api as the object returned by open().


El martes, 15 de agosto de 2017, 19:21:08 (UTC-3), Adam Johnson escribió:
The next() method is gone from the return value of open() in Python 3 and I don't think it was every necessary. As per the iterator protcool ( <a href="https://docs.python.org/2/library/stdtypes.html#iterator-types" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fdocs.python.org%2F2%2Flibrary%2Fstdtypes.html%23iterator-types\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGohaosTxpPlCHKeXbLb4Bwma6Ybw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fdocs.python.org%2F2%2Flibrary%2Fstdtypes.html%23iterator-types\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGohaosTxpPlCHKeXbLb4Bwma6Ybw&#39;;return true;">https://docs.python.org/2/library/stdtypes.html#iterator-types ), __iter__ is meant to return an iterator over the object, and that is what the next() method - __next__ on Python 3 - should be attached to. Django's File.__iter__ is a generator function, thus it returns a generator, with comes with a next()/__next__().

I think you should just try iterating your files by calling iter() on them first, it's Python 3 compatible too.

On 27 July 2017 at 17:24, Cristiano Coelho <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="Ge311CtoCAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">cristia...@...> wrote:
Hello,

I have recently found an interesting issue, using a project that relies on different storage backends, when switching from a custom one to django's file system storage, I found that existing code that would iterate files with the next() call would start to fail, since although django's File is iterable, it doesn't define the next method.

Now I'm wondering if this is on purpose, or a bug. It's odd that every stream (or almost) from the python library such as everything from the .io module or simply the object returned by open() (python 2) supports the next call but the django File wrapper doesn't.

This happened with django 1.10 and I believe it wasn't changed with Django 1.11

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="Ge311CtoCAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">django-develop...@googlegroups.com.
To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="Ge311CtoCAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">django-d...@googlegroups.com.
Visit this group at <a href="https://groups.google.com/group/django-developers" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/group/django-developers&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/group/django-developers&#39;;return true;">https://groups.google.com/group/django-developers.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/django-developers/09ffad76-e473-4593-ac84-4bca7f76e92c%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/django-developers/09ffad76-e473-4593-ac84-4bca7f76e92c%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/django-developers/09ffad76-e473-4593-ac84-4bca7f76e92c%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/django-developers/09ffad76-e473-4593-ac84-4bca7f76e92c%40googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.



--
Adam

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/cb7c4945-5932-44a0-8eb8-4571813a0ba3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Should django File wrapper support .next()?

Adam Johnson-2
I looked a bit more and the reason file objects have next() in Py2 is because "A file object is its own iterator" ( https://docs.python.org/2/library/stdtypes.html#file.next ). This is still the case in Py3 - "IOBase (and its subclasses) supports the iterator protocol" ( https://docs.python.org/3/library/io.html#io.IOBase ).

So yes there is a disparity in Django's File. After a quick look at the code I'm not sure if it's possible to modify it to match though because of all the logic embedded in __iter__ but it's worth having a try.

On 16 August 2017 at 13:55, Cristiano Coelho <[hidden email]> wrote:
I forgot about django dropping support for Python2 so I guess you are right this doesn't make much sense if next() is removed, but for Python2 it would make sense for django's File object to have the same api as the object returned by open().


El martes, 15 de agosto de 2017, 19:21:08 (UTC-3), Adam Johnson escribió:
The next() method is gone from the return value of open() in Python 3 and I don't think it was every necessary. As per the iterator protcool ( https://docs.python.org/2/library/stdtypes.html#iterator-types ), __iter__ is meant to return an iterator over the object, and that is what the next() method - __next__ on Python 3 - should be attached to. Django's File.__iter__ is a generator function, thus it returns a generator, with comes with a next()/__next__().

I think you should just try iterating your files by calling iter() on them first, it's Python 3 compatible too.

On 27 July 2017 at 17:24, Cristiano Coelho <[hidden email]> wrote:
Hello,

I have recently found an interesting issue, using a project that relies on different storage backends, when switching from a custom one to django's file system storage, I found that existing code that would iterate files with the next() call would start to fail, since although django's File is iterable, it doesn't define the next method.

Now I'm wondering if this is on purpose, or a bug. It's odd that every stream (or almost) from the python library such as everything from the .io module or simply the object returned by open() (python 2) supports the next call but the django File wrapper doesn't.

This happened with django 1.10 and I believe it wasn't changed with Django 1.11

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/09ffad76-e473-4593-ac84-4bca7f76e92c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Adam

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/cb7c4945-5932-44a0-8eb8-4571813a0ba3%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
Adam

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM0arRQc-5kJiDq1FcxiH_%3DxN04jWGNq4RAp0rxMHSDv2A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.