Returning a custom file object (Python 3)

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Returning a custom file object (Python 3)

Steven D'Aprano-8
I'd like to return a custom file object, say my own subclass. I can easily
subclass the file object:


from io import TextIOWrapper
class MyFile(TextIOWrapper):
    pass


but how do I tell open() to use MyFile?


Answers for Python 3, thanks.




--
Steven



Reply | Threaded
Open this post in threaded view
|

Returning a custom file object (Python 3)

Ben Finney-10
Steven D'Aprano <steve at pearwood.info> writes:

> from io import TextIOWrapper
> class MyFile(TextIOWrapper):
>     pass
>
> but how do I tell open() to use MyFile?

I haven't used it, but does the ?opener? parameter do what you want?

    open(file, mode='r', buffering=-1, encoding=None,
             errors=None, newline=None, closefd=True, opener=None) ->
             file object

    [?]

    A custom opener can be used by passing a callable as *opener*. The
    underlying file descriptor for the file object is then obtained by
    calling *opener* with (*file*, *flags*). *opener* must return an
    open file descriptor (passing os.open as *opener* results in
    functionality similar to passing None).

--
 \          ?Our products just aren't engineered for security.? ?Brian |
  `\             Valentine, senior vice-president of Microsoft Windows |
_o__)                                                      development |
Ben Finney



Reply | Threaded
Open this post in threaded view
|

Returning a custom file object (Python 3)

Ben Finney-10
Ben Finney <ben+python at benfinney.id.au> writes:

> Steven D'Aprano <steve at pearwood.info> writes:
>
> > but how do I tell open() to use MyFile?
>
> I haven't used it, but does the ?opener? parameter do what you want?

No, it doesn't; the ?opener? parameter doesn't have any say in the type
of object returned from ?open?.

It seems the existing ?open? implementation doesn't allow you to
override the type of object returned. You may have to resort to
monkey-patching the ?io? attributes to put your preferred classes there.

--
 \       ?Science shows that belief in God is not only obsolete. It is |
  `\                        also incoherent.? ?Victor J. Stenger, 2001 |
_o__)                                                                  |
Ben Finney



Reply | Threaded
Open this post in threaded view
|

Returning a custom file object (Python 3)

Marko Rauhamaa
In reply to this post by Ben Finney-10
Ben Finney <ben+python at benfinney.id.au>:

> It seems the existing ?open? implementation doesn't allow you to
> override the type of object returned.

The question is, can you assign to the builtin namespace. I'm guessing
you can't.

Within a module, you can simply do:

   open = MyFile

Also, in other modules, you can:

   from myfile import open


Marko


Reply | Threaded
Open this post in threaded view
|

Returning a custom file object (Python 3)

Chris Angelico
On Thu, May 28, 2015 at 3:29 PM, Marko Rauhamaa <marko at pacujo.net> wrote:

> Ben Finney <ben+python at benfinney.id.au>:
>
>> It seems the existing ?open? implementation doesn't allow you to
>> override the type of object returned.
>
> The question is, can you assign to the builtin namespace. I'm guessing
> you can't.
>
> Within a module, you can simply do:
>
>    open = MyFile
>
> Also, in other modules, you can:
>
>    from myfile import open

Well, you can...

>>> import builtins
>>> builtins.open
<built-in function open>
>>> builtins.open = "not your grandfather's open() function"
>>> open
"not your grandfather's open() function"

... but I don't think replacing all of open() is what Steven has in
mind; it's a function that does a lot of work, most of which is what's
wanted.

Depending on how brutal you want to be, though, you _could_ hack around it.

>>> from io import TextIOWrapper
>>> class MyFile(TextIOWrapper):
...     def demo(self): print("Hello, world!")
...
>>> f = open("/tmp/dummy", "w", encoding="utf-8")
>>> f
<_io.TextIOWrapper name='/tmp/dummy' mode='w' encoding='utf-8'>
>>> f.__class__
<class '_io.TextIOWrapper'>
>>> f.__class__ = MyFile
>>> f.demo()
Hello, world!

This does appear to work. Whether or not it's a good idea is a
separate question. And of course, you could replace open() with
something like this:

>>> _orig_open = open
>>> def open(*args, **kw):
...   cls = kw.pop("use_class", None)
...   f = _orig_open(*args, **kw)
...   if cls is not None: f.__class__ = cls
...   return f
...
>>> f = open("/tmp/dummy", "w", encoding="utf-8", use_class=MyFile)
>>> f
<_io.TextIOWrapper name='/tmp/dummy' mode='w' encoding='utf-8'>
>>> f.demo()
Hello, world!

But again, I'm really not sure this is a good way to do things.

ChrisA


Reply | Threaded
Open this post in threaded view
|

Returning a custom file object (Python 3)

Gary Herron-2
In reply to this post by Marko Rauhamaa
On 05/27/2015 10:29 PM, Marko Rauhamaa wrote:
> Ben Finney <ben+python at benfinney.id.au>:
>
>> It seems the existing ?open? implementation doesn't allow you to
>> override the type of object returned.
> The question is, can you assign to the builtin namespace. I'm guessing
> you can't.

Of course you can.

Python2:
 >>> __builtins__.open = "whatever"
 >>>

Python3:
 >>> import builtins
 >>> builtins.open = "whatever"
 >>>

Of course doing so is like shooting yourself in the foot:  Any
subsequent pain is your own fault and probably well deserved.

Gary Herron



>
> Within a module, you can simply do:
>
>     open = MyFile
>
> Also, in other modules, you can:
>
>     from myfile import open
>
>
> Marko


--
Dr. Gary Herron
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418




Reply | Threaded
Open this post in threaded view
|

Returning a custom file object (Python 3)

Steven D'Aprano-11
In reply to this post by Marko Rauhamaa
On Thursday 28 May 2015 15:56, Gary Herron wrote:

> On 05/27/2015 10:29 PM, Marko Rauhamaa wrote:
>> Ben Finney <ben+python at benfinney.id.au>:
>>
>>> It seems the existing ?open? implementation doesn't allow you to
>>> override the type of object returned.
>> The question is, can you assign to the builtin namespace. I'm guessing
>> you can't.
>
> Of course you can.
>
> Python2:
>  >>> __builtins__.open = "whatever"


Don't use __builtins__ with an "s". That's an implementation of CPython and
may not exist in the future, and doesn't exist in other implementations.

Instead, you should import __builtin__ with no "s". Confusing? It certainly
is, which is why Python 3 renamed __builtin__ to builtins.




--
Steve



Reply | Threaded
Open this post in threaded view
|

Returning a custom file object (Python 3)

Steven D'Aprano-11
In reply to this post by Marko Rauhamaa
On Thursday 28 May 2015 15:49, Chris Angelico wrote:

> ... but I don't think replacing all of open() is what Steven has in
> mind; it's a function that does a lot of work, most of which is what's
> wanted.

Correct. If I wanted to replace open(), I would have just shadowed it, or
monkey-patched it, or just used myopen().

I want open() to return my own subclass instead of the standard one. And I
think you have the solution I was looking for:


> Depending on how brutal you want to be, though, you _could_ hack around
> it.
>
>>>> from io import TextIOWrapper
>>>> class MyFile(TextIOWrapper):
> ...     def demo(self): print("Hello, world!")
> ...
>>>> f = open("/tmp/dummy", "w", encoding="utf-8")
>>>> f
> <_io.TextIOWrapper name='/tmp/dummy' mode='w' encoding='utf-8'>
>>>> f.__class__
> <class '_io.TextIOWrapper'>
>>>> f.__class__ = MyFile
>>>> f.demo()
> Hello, world!
>
> This does appear to work. Whether or not it's a good idea is a
> separate question.


And this is EXACTLY the sort of use-case that having __class__ be writable
is intended to solve. So this is exactly the solution I was after, thank
you. Er... except for one little problem... in Python 3.3:

py> f = open("/tmp/a", "r")
py> f.__class__ = MyFile
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __class__ assignment: only for heap types


So it doesn't work for me. What version of Python are you using?



--
Steve



Reply | Threaded
Open this post in threaded view
|

Returning a custom file object (Python 3)

Chris Angelico
On Thu, May 28, 2015 at 5:04 PM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:

>> This does appear to work. Whether or not it's a good idea is a
>> separate question.
>
>
> And this is EXACTLY the sort of use-case that having __class__ be writable
> is intended to solve. So this is exactly the solution I was after, thank
> you. Er... except for one little problem... in Python 3.3:
>
> py> f = open("/tmp/a", "r")
> py> f.__class__ = MyFile
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: __class__ assignment: only for heap types
>
>
> So it doesn't work for me. What version of Python are you using?

Huh, didn't even think to check other versions.

$ python3
Python 3.5.0b1+ (default:7255af1a1c50+, May 26 2015, 00:39:06)
[GCC 4.9.2] on linux

It's a build fresh from source control around about this weekend (some
time after feature freeze, obviously, but beyond that all I know is
what you see there). I guess this is pretty new; trying the same thing
on 3.4.2 doesn't work. But hey! 3.5 will be out soon...

ChrisA


Reply | Threaded
Open this post in threaded view
|

Returning a custom file object (Python 3)

Oscar Benjamin-2
In reply to this post by Steven D'Aprano-8
On 28 May 2015 at 03:16, Steven D'Aprano <steve at pearwood.info> wrote:

> I'd like to return a custom file object, say my own subclass. I can easily
> subclass the file object:
>
>
> from io import TextIOWrapper
> class MyFile(TextIOWrapper):
>     pass
>
>
> but how do I tell open() to use MyFile?

Does the below do what you want?

#!/usr/bin/env python3

class Wrapper:
    def __init__(self, wrapped):
        self._wrapped = wrapped
    def __getattr__(self, attrname):
        return getattr(self._wrapped, attrname)
    # Special methods are not resolved by __getattr__
    def __iter__(self):
        return self._wrapped.__iter__()
    def __enter__(self):
        return self._wrapped.__enter__()
    def __exit__(self, *args):
        return self._wrapped.__exit__(*args)

class WrapFile(Wrapper):
    def write(self):
        return "Writing..."

f = WrapFile(open('wrap.py'))
print(f.readline())
with f:
    print(len(list(f)))
print(f.write())

>
> Answers for Python 3, thanks.

Tested on 3.2.


--
Oscar