jython vs test_descr

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

jython vs test_descr

Khalid Zuberi
I'm attempting to run test_descr.py from cpython 2.3.5 against current
jython cvs, to get an idea of what needs fixing. Its a long test
script and not surprisingly doesn't get too far in jython. So the
approach i'm taking is to pick out a few (hopefully approachable)
errors, spin off a separate little test script, and patch what i can
to get it to pass.

The first round of this addresses some dict and list bugs that mostly
show up when subclassing the built-in types. For example:

    class C(dict):
        def __getitem__(self, key):
            return self.get(key)

    d = C()
    print d[1]

should print None but seems to get jython recursing in __getitem__()
until the stack is exhausted. I've uploaded attempted fixes to the
tracker:

  http://www.python.org/sf/1455153

but thought i'd mention it here for feedback.

Thanks,
- kz

(By the way, a little typo made at the jython prompt:

  x = dict[1,])

ouch!)


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
<a href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid0944&bid$1720&dat1642">http://sel.as-us.falkag.net/sel?cmd=lnk&kid0944&bid$1720&dat1642
_______________________________________________
Jython-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-dev
Reply | Threaded
Open this post in threaded view
|

Re: jython vs test_descr

Samuele Pedroni
Khalid Zuberi wrote:

> I'm attempting to run test_descr.py from cpython 2.3.5 against current
> jython cvs, to get an idea of what needs fixing. Its a long test
> script and not surprisingly doesn't get too far in jython. So the
> approach i'm taking is to pick out a few (hopefully approachable)
> errors, spin off a separate little test script, and patch what i can
> to get it to pass.
>
> The first round of this addresses some dict and list bugs that mostly
> show up when subclassing the built-in types. For example:
>
>     class C(dict):
>         def __getitem__(self, key):
>             return self.get(key)
>

I see, notice that while being as similar as possible to CPython
is of course worthwhile, in general the delegation
relationships of the various methods of builtin types
is undefined in Python. So in some sense the test is testing too much.
The problem is I'm not sure that there are tests that check for all
possible combinations. Just pointing out that one at some point
one needs to draw a line.

Indeed is arguable whether the fact
that subclassing "__getitem__" change "get" too is a feature or a bug.
Jython historically delegated internally much more than
CPython.

Anyway looking at what had to be changed for PyPy

http://codespeak.net/svn/pypy/dist/lib-python/modified-2.4.1/test/test_descr.py

may help in this task.



>     d = C()
>     print d[1]
>
> should print None but seems to get jython recursing in __getitem__()
> until the stack is exhausted. I've uploaded attempted fixes to the
> tracker:
>
>   http://www.python.org/sf/1455153
>
> but thought i'd mention it here for feedback.
>
> Thanks,
> - kz
>
> (By the way, a little typo made at the jython prompt:
>
>   x = dict[1,])
>
> ouch!)
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting language
> that extends applications into web and mobile media. Attend the live webcast
> and join the prime developer group breaking into this new coding territory!
> http://sel.as-us.falkag.net/sel?cmd_______________________________________________
> Jython-dev mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/jython-dev



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
Jython-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-dev
Reply | Threaded
Open this post in threaded view
|

Re: jython vs test_descr

Kent Johnson
In reply to this post by Khalid Zuberi
Khalid Zuberi wrote:

> I'm attempting to run test_descr.py from cpython 2.3.5 against current
> jython cvs, to get an idea of what needs fixing. Its a long test
> script and not surprisingly doesn't get too far in jython. So the
> approach i'm taking is to pick out a few (hopefully approachable)
> errors, spin off a separate little test script, and patch what i can
> to get it to pass.
>
> The first round of this addresses some dict and list bugs that mostly
> show up when subclassing the built-in types. For example:
>
>     class C(dict):
>         def __getitem__(self, key):
>             return self.get(key)
>
>     d = C()
>     print d[1]
>
> should print None but seems to get jython recursing in __getitem__()
> until the stack is exhausted.

Is that really taken from the tests? I would expect it to give a stack
overflow because self.get() must be implemented on top of
self.__getitem__. What happens if you change it to call the inherited get()?

     class C(dict):
         def __getitem__(self, key):
             return dict.get(self, key)

Hmm, I tried your example in Python 2.4.2 and it does return None. A
test shows that dict.get() *doesn't* call __getitem__() in CPython. It
doesn't call __contains__() either. This seems like a bug in CPython to me.

I looked at the source for dict (dictobject.c) and dict.get() and dict[]
both call a more primitive lookup method, so I guess Jython should do
the same. Seems pretty strange to me though.

Kent



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
Jython-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-dev
Reply | Threaded
Open this post in threaded view
|

Re: jython vs test_descr

Samuele Pedroni
Kent Johnson wrote:

> Khalid Zuberi wrote:
>
>> I'm attempting to run test_descr.py from cpython 2.3.5 against current
>> jython cvs, to get an idea of what needs fixing. Its a long test
>> script and not surprisingly doesn't get too far in jython. So the
>> approach i'm taking is to pick out a few (hopefully approachable)
>> errors, spin off a separate little test script, and patch what i can
>> to get it to pass.
>>
>> The first round of this addresses some dict and list bugs that mostly
>> show up when subclassing the built-in types. For example:
>>
>>     class C(dict):
>>         def __getitem__(self, key):
>>             return self.get(key)
>>
>>     d = C()
>>     print d[1]
>>
>> should print None but seems to get jython recursing in __getitem__()
>> until the stack is exhausted.
>
>
> Is that really taken from the tests? I would expect it to give a stack
> overflow because self.get() must be implemented on top of
> self.__getitem__. What happens if you change it to call the inherited
> get()?
>
>     class C(dict):
>         def __getitem__(self, key):
>             return dict.get(self, key)
>
> Hmm, I tried your example in Python 2.4.2 and it does return None. A
> test shows that dict.get() *doesn't* call __getitem__() in CPython. It
> doesn't call __contains__() either. This seems like a bug in CPython to me.
>
> I looked at the source for dict (dictobject.c) and dict.get() and dict[]
> both call a more primitive lookup method, so I guess Jython should do
> the same. Seems pretty strange to me though.
>
>

the internals of builtin types in CPython are of course older than the
type/class unification, and indeed were not designed with overriding and
subclassing in mind, that's why is agreed that the exact delegation
details are undefined, so implementations are "free" to do better and
from that point of view the test is over-testing. OTOH I'm not sure
Jython as it is offers a coherent picture. There are too many builtin
methods and ways to implement or not them on top of each other.

Basically the lesson, unless you control and know that a only restricted
subset of all the methods will be called on your subclassed builtin,
either don't subclass or be prepared to rewrite all methods (sort of
like the old UserDict etc were doing).




-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
Jython-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-dev
Reply | Threaded
Open this post in threaded view
|

Re: jython vs test_descr

Khalid Zuberi
In reply to this post by Kent Johnson
On 3/21/06, Kent Johnson <[hidden email]> wrote:

> Is that really taken from the tests? I would expect it to give a stack
> overflow because self.get() must be implemented on top of
> self.__getitem__.
>

Its distilled from pydicts() in test_descr. Also, the same appears in
the pypy version version of the test suite (thanks for the link
Samuele), so pypy must be following the cpython behaviour here. Now i
haven't tried ironpython, but would be a bit surprised if it diverged
from this (anyone want to confirm?).

Rather than guess what the implementation should do, i'm just
deferring to the cpython implementation, especially when the behaviour
is enshrined in a test case.

Note the patch i linked to also addresses some more hum-drum things
like jython not accepting certain arg variations in
list.__getslice__()/__setslice__() (the patches are small and broken
out, but i'm not sure if correct, hence the request for review).

Thanks for the comments,
- kz


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
<a href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid0944&bid$1720&dat1642">http://sel.as-us.falkag.net/sel?cmd=lnk&kid0944&bid$1720&dat1642
_______________________________________________
Jython-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-dev
Reply | Threaded
Open this post in threaded view
|

RE: jython vs test_descr

cupdike
In reply to this post by Khalid Zuberi
> Rather than guess what the implementation should do, i'm just
> deferring to the cpython implementation, especially when the
> behaviour is enshrined in a test case.

At one point, Brian Zimmer discussed with some cpython folks the
notion of separating the cpython implementation-specific
test behaviors from the generic Python tests.  I don't think
anything came of it though.

-Clark


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
<a href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid0944&bid$1720&dat1642">http://sel.as-us.falkag.net/sel?cmd=lnk&kid0944&bid$1720&dat1642
_______________________________________________
Jython-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-dev
Reply | Threaded
Open this post in threaded view
|

Re: jython vs test_descr

fwierzbicki@gmail.com
> At one point, Brian Zimmer discussed with some cpython folks the
> notion of separating the cpython implementation-specific
> test behaviors from the generic Python tests.  I don't think
> anything came of it though.
>
> -Clark

Perhaps we at Jython can start dividing tests up this way.  It might
even be a good idea to coordinate with other implementations like PyPy
or even IronPython, since I'm sure they have this problem as well.

-Frank


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
<a href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid0944&bid$1720&dat1642">http://sel.as-us.falkag.net/sel?cmd=lnk&kid0944&bid$1720&dat1642
_______________________________________________
Jython-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-dev
Reply | Threaded
Open this post in threaded view
|

Re: jython vs test_descr

Khalid Zuberi
On 3/21/06, Frank Wierzbicki <[hidden email]> wrote:

> > At one point, Brian Zimmer discussed with some cpython folks the
> > notion of separating the cpython implementation-specific
> > test behaviors from the generic Python tests.  I don't think
> > anything came of it though.
> >
> > -Clark
>
> Perhaps we at Jython can start dividing tests up this way.  It might
> even be a good idea to coordinate with other implementations like PyPy
> or even IronPython, since I'm sure they have this problem as well.
>
> -Frank
>

Interestingly, this has just come up on the python-3000 list. If you
guys have some concrete proposals that would help for jython, you
might want to chime in:

  http://mail.python.org/pipermail/python-3000/2006-April/000725.html

- kz


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
<a href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid0944&bid$1720&dat1642">http://sel.as-us.falkag.net/sel?cmd=lnk&kid0944&bid$1720&dat1642
_______________________________________________
Jython-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-dev