Exceptions in comparison operators

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Exceptions in comparison operators

Mark Shannon-3
Comparing two objects (of the same type for simplicity)
involves a three stage lookup:
The class has the operator C.__eq__
It can be applied to operator (descriptor protocol): C().__eq__
and it produces a result: C().__eq__(C())

Exceptions can be raised in all 3 phases,
but an exception in the first phase is not really an error,
its just says the operation is not supported.
E.g.

class C: pass

C() == C() is False, rather than raising an Exception.

If an exception is raised in the 3rd stage, then it is propogated,
as follows:

class C:
    def __eq__(self, other):
        raise Exception("I'm incomparable")

C() == C()  raises an exception

However, if an exception is raised in the second phase (descriptor)
then it is silenced:

def no_eq(self):
     raise Exception("I'm incomparable")

class C:
    __eq__ = property(no_eq)

C() == C() is False.

But should it raise an exception?

The behaviour for arithmetic is different.

def no_add(self):
     raise Exception("I don't add up")

class C:
    __add__ = property(no_add)

C() + C() raises an exception.

So what is the "correct" behaviour?
It is my opinion that comparisons should behave like arithmetic
and raise an exception.

Cheers,
Mark
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Exceptions in comparison operators

Guido van Rossum
On Mon, Mar 5, 2012 at 4:41 AM, Mark Shannon <[hidden email]> wrote:

> Comparing two objects (of the same type for simplicity)
> involves a three stage lookup:
> The class has the operator C.__eq__
> It can be applied to operator (descriptor protocol): C().__eq__
> and it produces a result: C().__eq__(C())
>
> Exceptions can be raised in all 3 phases,
> but an exception in the first phase is not really an error,
> its just says the operation is not supported.
> E.g.
>
> class C: pass
>
> C() == C() is False, rather than raising an Exception.
>
> If an exception is raised in the 3rd stage, then it is propogated,
> as follows:
>
> class C:
>   def __eq__(self, other):
>       raise Exception("I'm incomparable")
>
> C() == C()  raises an exception
>
> However, if an exception is raised in the second phase (descriptor)
> then it is silenced:
>
> def no_eq(self):
>    raise Exception("I'm incomparable")
>
> class C:
>   __eq__ = property(no_eq)
>
> C() == C() is False.
>
> But should it raise an exception?
>
> The behaviour for arithmetic is different.
>
> def no_add(self):
>    raise Exception("I don't add up")
>
> class C:
>   __add__ = property(no_add)
>
> C() + C() raises an exception.
>
> So what is the "correct" behaviour?
> It is my opinion that comparisons should behave like arithmetic
> and raise an exception.

I think you're probably right. This is one of those edge cases that
are so rare (and always considered a bug in the user code) that we
didn't define carefully what should happen. There are probably some
implementation-specific reasons why it was done this way (comparisons
use a very different code path from regular binary operators) but that
doesn't sound like a very good reason.

OTOH there *is* a difference: as you say, C() == C() is False when the
class doesn't define __eq__, whereas C() + C() raises an exception if
it doesn't define __add__. Still, this is more likely to have favored
the wrong outcome for (2) by accident than by design.

You'll have to dig through the CPython implementation and find out
exactly what code needs to be changed before I could be sure though --
sometimes seeing the code jogs my memory.

But I think of x==y as roughly equivalent to

r = NotImplemented
if hasattr(x, '__eq__'):
  r = x.__eq__(y)
if r is NotImplemented and hasattr(y, '__eq__'):
  r = y.__eq__(x)
if r is NotImplemented:
  r = False

which would certainly suggest that (2) should raise an exception. A
possibility is that the code looking for the __eq__ attribute
suppresses *all* exceptions instead of just AttributeError. If you
change no_eq() to return 42, for example, the comparison raises the
much more reasonable TypeError: 'int' object is not callable.

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Exceptions in comparison operators

Guido van Rossum
Mark, did you do anything with my reply?

On Mon, Mar 5, 2012 at 10:41 AM, Guido van Rossum <[hidden email]> wrote:

> On Mon, Mar 5, 2012 at 4:41 AM, Mark Shannon <[hidden email]> wrote:
>> Comparing two objects (of the same type for simplicity)
>> involves a three stage lookup:
>> The class has the operator C.__eq__
>> It can be applied to operator (descriptor protocol): C().__eq__
>> and it produces a result: C().__eq__(C())
>>
>> Exceptions can be raised in all 3 phases,
>> but an exception in the first phase is not really an error,
>> its just says the operation is not supported.
>> E.g.
>>
>> class C: pass
>>
>> C() == C() is False, rather than raising an Exception.
>>
>> If an exception is raised in the 3rd stage, then it is propogated,
>> as follows:
>>
>> class C:
>>   def __eq__(self, other):
>>       raise Exception("I'm incomparable")
>>
>> C() == C()  raises an exception
>>
>> However, if an exception is raised in the second phase (descriptor)
>> then it is silenced:
>>
>> def no_eq(self):
>>    raise Exception("I'm incomparable")
>>
>> class C:
>>   __eq__ = property(no_eq)
>>
>> C() == C() is False.
>>
>> But should it raise an exception?
>>
>> The behaviour for arithmetic is different.
>>
>> def no_add(self):
>>    raise Exception("I don't add up")
>>
>> class C:
>>   __add__ = property(no_add)
>>
>> C() + C() raises an exception.
>>
>> So what is the "correct" behaviour?
>> It is my opinion that comparisons should behave like arithmetic
>> and raise an exception.
>
> I think you're probably right. This is one of those edge cases that
> are so rare (and always considered a bug in the user code) that we
> didn't define carefully what should happen. There are probably some
> implementation-specific reasons why it was done this way (comparisons
> use a very different code path from regular binary operators) but that
> doesn't sound like a very good reason.
>
> OTOH there *is* a difference: as you say, C() == C() is False when the
> class doesn't define __eq__, whereas C() + C() raises an exception if
> it doesn't define __add__. Still, this is more likely to have favored
> the wrong outcome for (2) by accident than by design.
>
> You'll have to dig through the CPython implementation and find out
> exactly what code needs to be changed before I could be sure though --
> sometimes seeing the code jogs my memory.
>
> But I think of x==y as roughly equivalent to
>
> r = NotImplemented
> if hasattr(x, '__eq__'):
>  r = x.__eq__(y)
> if r is NotImplemented and hasattr(y, '__eq__'):
>  r = y.__eq__(x)
> if r is NotImplemented:
>  r = False
>
> which would certainly suggest that (2) should raise an exception. A
> possibility is that the code looking for the __eq__ attribute
> suppresses *all* exceptions instead of just AttributeError. If you
> change no_eq() to return 42, for example, the comparison raises the
> much more reasonable TypeError: 'int' object is not callable.
>
> --
> --Guido van Rossum (python.org/~guido)



--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com
Reply | Threaded
Open this post in threaded view
|

Re: Exceptions in comparison operators

Mark Shannon-3
Guido van Rossum wrote:
> Mark, did you do anything with my reply?

Not yet.

I noticed the difference when developing my HotPy VM
(latest incarnation thereof) which substitutes a sequence of low-level
bytecodes for the high-level ones when tracing.
(A bit like PyPy but much more Python-specific and amenable to
interpretation, rather than compilation)

I generate all the code sequences for binary ops from a template
and noticed the slight difference when running the test suite.
My implementation of equals follows the same pattern as the arithmetic
operators (which is why I was wondering if that were the correct behaviour).

My definition of op1 == op2:

def surrogate_eq(op1, op2):
     if $overrides(op1, op2, '__eq__'):
         if op2?__eq__:
             result = op2$__eq__(op1)
             if result is not NotImplemented:
                 return result
         if op1?__eq__:
             result = op1$__eq__(op2)
             if result is not NotImplemented:
                 return result
     else:
         if op1?__eq__:
             result = op1$__eq__(op2)
             if result is not NotImplemented:
                 return result
         if op2?__eq__:
             result = op2$__eq__(op1)
             if result is not NotImplemented:
                 return result
     return op1 is op2

Where:

x$__op__ means special lookup (bypassing the instance dictionary):

x?__op__ means has the named special method i.e.
any('__op__' in t.__dict__ for t in type(op).__mro__))

and
$overrides(op1, op2, 'xxx') means that
type(op2) is a proper subtype of type(op1)
*and* type(op1).__dict__['xxx'] != type(op2).__dict__['xxx']


It would appear that the current version is:

def surrogate_eq(op1, op2):
     if is_proper_subtype_of( type(op1), type(op1) ):
         if op2?__eq__:
             result = op2$__eq__(op1)
             if result is not NotImplemented:
                 return result
         if op1?__eq__:
             result = op1$__eq__(op2)
             if result is not NotImplemented:
                 return result
     else:
         if op1?__eq__:
             result = op1$__eq__(op2)
             if result is not NotImplemented:
                 return result
         if op2?__eq__:
             result = op2$__eq__(op1)
             if result is not NotImplemented:
                 return result
     return op1 is op2

Which means that == behaves differently to + for
subtypes which do not override the __eq__ method.
Thus:

class MyValue1:
     def __init__(self, val):
         self.val = val

     def __lt__(self, other):
         print("lt")
         return self.val < other.val

     def __gt__(self, other):
         print("gt")
         return self.val > other.val

     def __add__(self, other):
         print("add")
         return self.val + other.val

     def __radd__(self, other):
         print("radd")
         return self.val + other.val

class MyValue2(MyValue1):
     pass

a = MyValue1(1)
b = MyValue2(2)

print(a + b)
print(a < b)

currently prints the following:

add
3
gt
True

Cheers,
Mark.

>
> On Mon, Mar 5, 2012 at 10:41 AM, Guido van Rossum <[hidden email]> wrote:
>> On Mon, Mar 5, 2012 at 4:41 AM, Mark Shannon <[hidden email]> wrote:
>>> Comparing two objects (of the same type for simplicity)
>>> involves a three stage lookup:
>>> The class has the operator C.__eq__
>>> It can be applied to operator (descriptor protocol): C().__eq__
>>> and it produces a result: C().__eq__(C())
>>>
>>> Exceptions can be raised in all 3 phases,
>>> but an exception in the first phase is not really an error,
>>> its just says the operation is not supported.
>>> E.g.
>>>
>>> class C: pass
>>>
>>> C() == C() is False, rather than raising an Exception.
>>>
>>> If an exception is raised in the 3rd stage, then it is propogated,
>>> as follows:
>>>
>>> class C:
>>>   def __eq__(self, other):
>>>       raise Exception("I'm incomparable")
>>>
>>> C() == C()  raises an exception
>>>
>>> However, if an exception is raised in the second phase (descriptor)
>>> then it is silenced:
>>>
>>> def no_eq(self):
>>>    raise Exception("I'm incomparable")
>>>
>>> class C:
>>>   __eq__ = property(no_eq)
>>>
>>> C() == C() is False.
>>>
>>> But should it raise an exception?
>>>
>>> The behaviour for arithmetic is different.
>>>
>>> def no_add(self):
>>>    raise Exception("I don't add up")
>>>
>>> class C:
>>>   __add__ = property(no_add)
>>>
>>> C() + C() raises an exception.
>>>
>>> So what is the "correct" behaviour?
>>> It is my opinion that comparisons should behave like arithmetic
>>> and raise an exception.
>> I think you're probably right. This is one of those edge cases that
>> are so rare (and always considered a bug in the user code) that we
>> didn't define carefully what should happen. There are probably some
>> implementation-specific reasons why it was done this way (comparisons
>> use a very different code path from regular binary operators) but that
>> doesn't sound like a very good reason.
>>
>> OTOH there *is* a difference: as you say, C() == C() is False when the
>> class doesn't define __eq__, whereas C() + C() raises an exception if
>> it doesn't define __add__. Still, this is more likely to have favored
>> the wrong outcome for (2) by accident than by design.
>>
>> You'll have to dig through the CPython implementation and find out
>> exactly what code needs to be changed before I could be sure though --
>> sometimes seeing the code jogs my memory.
>>
>> But I think of x==y as roughly equivalent to
>>
>> r = NotImplemented
>> if hasattr(x, '__eq__'):
>>  r = x.__eq__(y)
>> if r is NotImplemented and hasattr(y, '__eq__'):
>>  r = y.__eq__(x)
>> if r is NotImplemented:
>>  r = False
>>
>> which would certainly suggest that (2) should raise an exception. A
>> possibility is that the code looking for the __eq__ attribute
>> suppresses *all* exceptions instead of just AttributeError. If you
>> change no_eq() to return 42, for example, the comparison raises the
>> much more reasonable TypeError: 'int' object is not callable.
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>
>
>

_______________________________________________
Python-Dev mailing list
[hidden email]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/lists%2B1324100855712-1801473%40n6.nabble.com