ValueError: I/O operation on closed file. with python3

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

ValueError: I/O operation on closed file. with python3

Adam Mercer
Hi

I'm trying to update one of my scripts so that it runs under python2
and python3, but I'm running into an issue that the following example
illustrates:

$ cat test.py
try:
  # python-2.x
  from urllib2 import urlopen
  from ConfigParser import ConfigParser
except ImportError:
  # python-3.x
  from urllib.request import urlopen
  from configparser import ConfigParser

server='http://www.lsc-group.phys.uwm.edu/~ram/files'

fp = urlopen('%s/latest.ini' % server).fp
cp = ConfigParser()
cp.readfp(fp)
print(cp.get('version', '10.8'))
$

This works as expected when using python2:

$ python2.7 test.py
5.2.10
$

but when using python3 I receive the following error:

$ python3.3 test.py
Traceback (most recent call last):
  File "test.py", line 14, in <module>
    cp.readfp(fp)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/configparser.py",
line 753, in readfp
    self.read_file(fp, source=filename)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/configparser.py",
line 708, in read_file
    self._read(f, source)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/configparser.py",
line 1010, in _read
    for lineno, line in enumerate(fp, start=1):
ValueError: I/O operation on closed file.
$

Is there a way to get this working in both python2 and python3?

This is a small script and I'm starting to have some users wanting to
use python3 and others sticking to python2 so I'd like to accommodate
them both if possible.

Cheers

Adam


Reply | Threaded
Open this post in threaded view
|

ValueError: I/O operation on closed file. with python3

Peter Otten
Adam Mercer wrote:

> Hi
>
> I'm trying to update one of my scripts so that it runs under python2
> and python3, but I'm running into an issue that the following example
> illustrates:
>
> $ cat test.py
> try:
>   # python-2.x
>   from urllib2 import urlopen
>   from ConfigParser import ConfigParser
> except ImportError:
>   # python-3.x
>   from urllib.request import urlopen
>   from configparser import ConfigParser
>
> server='http://www.lsc-group.phys.uwm.edu/~ram/files'
>
> fp = urlopen('%s/latest.ini' % server).fp
> cp = ConfigParser()
> cp.readfp(fp)
> print(cp.get('version', '10.8'))
> $
>
> This works as expected when using python2:
>
> $ python2.7 test.py
> 5.2.10
> $
>
> but when using python3 I receive the following error:
>
> $ python3.3 test.py
> Traceback (most recent call last):
>   File "test.py", line 14, in <module>
>     cp.readfp(fp)
>   File
>  
"/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/configparser.py",
> line 753, in readfp
>     self.read_file(fp, source=filename)
>   File
>  
"/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/configparser.py",
> line 708, in read_file
>     self._read(f, source)
>   File
>  
"/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/configparser.py",

> line 1010, in _read
>     for lineno, line in enumerate(fp, start=1):
> ValueError: I/O operation on closed file.
> $
>
> Is there a way to get this working in both python2 and python3?
>
> This is a small script and I'm starting to have some users wanting to
> use python3 and others sticking to python2 so I'd like to accommodate
> them both if possible.

Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 30 2012, 14:49:00)
[GCC 4.6.1] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from urllib.request import urlopen
>>> url = "http://www.lsc-group.phys.uwm.edu/~ram/files/latest.ini"
>>> fp = urlopen(url).fp
>>> fp.read()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: read of closed file
>>> resp = urlopen(url)
>>> resp.fp.read()
b'[version]\n10.5 = 5.2.4\n10.6 = 5.2.10\n10.7 = 5.2.10\n10.8 = 5.2.10\n'

I don't know whether this behaviour is intentional or accidental (I assume
the latter).

I then ran into another problem:

>>> from configparser import ConfigParser
>>> p = ConfigParser()
>>> resp = urlopen(url)
>>> p.readfp(resp.fp)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.3/configparser.py", line 754, in readfp
    self.read_file(fp, source=filename)
  File "/usr/local/lib/python3.3/configparser.py", line 709, in read_file
    self._read(f, source)
  File "/usr/local/lib/python3.3/configparser.py", line 1011, in _read
    if line.strip().startswith(prefix):
TypeError: startswith first arg must be bytes or a tuple of bytes, not str

So ConfigParser.readfp() (which has been deprecated in favour of
read_file(), btw.) expects a text file:

>>> from io import TextIOWrapper
>>> resp = urlopen(url)
>>> p.readfp(TextIOWrapper(resp.fp))
>>> p.get("version", "10.8")
'5.2.10'

Applying these findings to your script:

from contextlib import contextmanager
try:
  # python-2.x
  from urllib2 import urlopen
  from ConfigParser import ConfigParser

  @contextmanager
  def my_urlopen(url):
      yield urlopen(url).fp

except ImportError:
  # python-3.x
  from urllib.request import urlopen
  from configparser import ConfigParser
  import io

  @contextmanager
  def my_urlopen(url):
      resp = urlopen(url)
      yield io.TextIOWrapper(resp.fp)

server='http://www.lsc-group.phys.uwm.edu/~ram/files'

cp = ConfigParser()
with my_urlopen('%s/latest.ini' % server) as fp:
    cp.readfp(fp)

print(cp.get('version', '10.8'))

I've run it with 2.6, 2.7, 3.2, and 3.3.




Reply | Threaded
Open this post in threaded view
|

ValueError: I/O operation on closed file. with python3

Adam Mercer
On Wed, Jun 12, 2013 at 2:26 AM, Peter Otten <__peter__ at web.de> wrote:

> Applying these findings to your script:
>
> from contextlib import contextmanager
> try:
>   # python-2.x
>   from urllib2 import urlopen
>   from ConfigParser import ConfigParser
>
>   @contextmanager
>   def my_urlopen(url):
>       yield urlopen(url).fp
>
> except ImportError:
>   # python-3.x
>   from urllib.request import urlopen
>   from configparser import ConfigParser
>   import io
>
>   @contextmanager
>   def my_urlopen(url):
>       resp = urlopen(url)
>       yield io.TextIOWrapper(resp.fp)
>
> server='http://www.lsc-group.phys.uwm.edu/~ram/files'
>
> cp = ConfigParser()
> with my_urlopen('%s/latest.ini' % server) as fp:
>     cp.readfp(fp)
>
> print(cp.get('version', '10.8'))
>
> I've run it with 2.6, 2.7, 3.2, and 3.3.

Thanks that's very helpful, I hadn't realised that .readfp() had been
deprecated.

Cheers

Adam


Reply | Threaded
Open this post in threaded view
|

ValueError: I/O operation on closed file. with python3

Serhiy Storchaka-2
In reply to this post by Peter Otten
12.06.13 10:26, Peter Otten ???????(??):
>    @contextmanager
>    def my_urlopen(url):
>        resp = urlopen(url)
>        yield io.TextIOWrapper(resp.fp)

         with urlopen(url) as resp:
             yield io.TextIOWrapper(resp)

Note that last bugfix releases (i.e. 3.3.1) are needed. There was a bug
with io.TextIOWrapper and HTTPResponse (http://bugs.python.org/issue16723).