Quantcast

excepthook-like behavior in ipython

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

excepthook-like behavior in ipython

Erik Tollerud-2
Hello all,

I noticed some odd (although I now realize quite sensible) behavior in
IPython, and I'm curious if there's a workaround.  I have an
application log that I want to use to capture unhandled exceptions.
When not using IPython, it's straightforward to just override
sys.excepthook to first sent the exception to our logging system, and
then continue the normal excepthook behavior.  However, IPython seems
to always use only its own excepthook in interactive mode
(TerminalInteractiveShell.excepthook, at least in the standard
terminal).

So my question is: Is there any way to plug anything into the IPython
exception handling system, either with sys.excepthook-style syntax, or
some other way?  As I said, it doesn't have to actually catch the
exceptions - we just want entries placed in a log if an interactive
session hits an unhandled exception, and it could then pass normally
into the IPython formatting/pdb running system.

--
Erik Tollerud
_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: excepthook-like behavior in ipython

MinRK


On Mon, Apr 9, 2012 at 18:39, Erik Tollerud <[hidden email]> wrote:
Hello all,

I noticed some odd (although I now realize quite sensible) behavior in
IPython, and I'm curious if there's a workaround.  I have an
application log that I want to use to capture unhandled exceptions.
When not using IPython, it's straightforward to just override
sys.excepthook to first sent the exception to our logging system, and
then continue the normal excepthook behavior.  However, IPython seems
to always use only its own excepthook in interactive mode
(TerminalInteractiveShell.excepthook, at least in the standard
terminal).

So my question is: Is there any way to plug anything into the IPython
exception handling system, either with sys.excepthook-style syntax, or
some other way?  As I said, it doesn't have to actually catch the
exceptions - we just want entries placed in a log if an interactive
session hits an unhandled exception, and it could then pass normally
into the IPython formatting/pdb running system.

You use shell.set_custom_exc for this:

# define custom exception handler:

def custom_exc(shell, etype, evalue, tb, tb_offset=None):
    # do you own thing:
    print 'hi', etype
    # also do what IPython would have done, if you want:
    shell.showtraceback((etype, evalue, tb), tb_offset=tb_offset)
 
# Tell IPython to use it for any/all Exceptions:
get_ipython().set_custom_exc((Exception,), custom_exc)


--
Erik Tollerud
_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev


_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: excepthook-like behavior in ipython

Erik Tollerud-2
This at first seemed to work, but I've discovered a test case where it
fails.  If I paste exactly what you have here into IPython, and then
do ``raise ValueError('a message')``, it works exactly as expected.
However, if instead I do ``raise Exception('a message')``, I get a
traceback like what I've shown below.  Further investigation has
revealed that it only happens if I try to raise an exception of a
class exactly the same as what's given in the first argument to
set_custom_exc (Exception, in this case), while if I raise
sub-exceptions, everything works just fine.  Is this a bug, or am I
mis-using it somehow?

(if I execute the Min's example, and then do ``raise Exception``):

... the "hi <type 'exceptions.Exception'>" message repeats for the
length of the recursion limit...
hi <type 'exceptions.Exception'>
Custom TB Handler failed, unregistering
Custom TB Handler failed, unregistering
Custom TB Handler failed, unregistering
Custom TB Handler failed, unregistering
ERROR: Internal Python error in the inspect module.
Below is the traceback from this internal error.
hi Traceback (most recent call last):
  File "/home/erik/src/ipython/IPython/core/ultratb.py", line 756, in
structured_traceback
Custom TB Handler failed, unregistering
ERROR: Internal Python error in the inspect module.
Below is the traceback from this internal error.
Traceback (most recent call last):
  File "/home/erik/src/ipython/IPython/core/ultratb.py", line 756, in
structured_traceback
    records = _fixed_getinnerframes(etb, context, tb_offset)
  File "/home/erik/src/ipython/IPython/core/ultratb.py", line 242, in
_fixed_getinnerframes
    records  = fix_frame_records_filenames(inspect.getinnerframes(etb, context))
  File "/usr/lib/python2.7/inspect.py", line 1043, in getinnerframes
    framelist.append((tb.tb_frame,) + getframeinfo(tb, context))
  File "/usr/lib/python2.7/inspect.py", line 1003, in getframeinfo
    filename = getsourcefile(frame) or getfile(frame)
  File "/usr/lib/python2.7/inspect.py", line 444, in getsourcefile
    filename = getfile(object)
  File "/usr/lib/python2.7/inspect.py", line 404, in getfile
    if isclass(object):
  File "/usr/lib/python2.7/inspect.py", line 65, in isclass
    return isinstance(object, (type, types.ClassType))
RuntimeError: maximum recursion depth exceeded while calling a Python object

Unfortunately, your original traceback can not be constructed.

The original exception:
---------------------------------------------------------------------------
(<type 'exceptions.Exception'>, Exception(), <traceback object at
0x1850998>)Traceback (most recent call last)
(<type 'exceptions.Exception'>, Exception(), <traceback object at
0x1850998>): None






On Mon, Apr 9, 2012 at 7:09 PM, MinRK <[hidden email]> wrote:

>
>
> On Mon, Apr 9, 2012 at 18:39, Erik Tollerud <[hidden email]> wrote:
>>
>> Hello all,
>>
>> I noticed some odd (although I now realize quite sensible) behavior in
>> IPython, and I'm curious if there's a workaround.  I have an
>> application log that I want to use to capture unhandled exceptions.
>> When not using IPython, it's straightforward to just override
>> sys.excepthook to first sent the exception to our logging system, and
>> then continue the normal excepthook behavior.  However, IPython seems
>> to always use only its own excepthook in interactive mode
>> (TerminalInteractiveShell.excepthook, at least in the standard
>> terminal).
>>
>> So my question is: Is there any way to plug anything into the IPython
>> exception handling system, either with sys.excepthook-style syntax, or
>> some other way?  As I said, it doesn't have to actually catch the
>> exceptions - we just want entries placed in a log if an interactive
>> session hits an unhandled exception, and it could then pass normally
>> into the IPython formatting/pdb running system.
>
>
> You use shell.set_custom_exc for this:
>
> # define custom exception handler:
>
> def custom_exc(shell, etype, evalue, tb, tb_offset=None):
>     # do you own thing:
>     print 'hi', etype
>     # also do what IPython would have done, if you want:
>     shell.showtraceback((etype, evalue, tb), tb_offset=tb_offset)
>
> # Tell IPython to use it for any/all Exceptions:
> get_ipython().set_custom_exc((Exception,), custom_exc)
>
>>
>> --
>> Erik Tollerud
>> _______________________________________________
>> IPython-dev mailing list
>> [hidden email]
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



--
Erik Tollerud
_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: excepthook-like behavior in ipython

Erik Tollerud-2
In reply to this post by MinRK
Further odd behavior that I'm not sure to classify as a bug or a
feature.  If I make a *script* to do this, the exact same thing
happens.

However, if I make a script that's identical, but raises ValueError
(or any other subclass of Exception), it does *not* trigger the
recursion loop, but also, the "hi" message does *not* appear - it
seems to just use the standard traceback report and ignore the custom
one.  Is this intended?

On Mon, Apr 9, 2012 at 7:09 PM, MinRK <[hidden email]> wrote:

>
>
> On Mon, Apr 9, 2012 at 18:39, Erik Tollerud <[hidden email]> wrote:
>>
>> Hello all,
>>
>> I noticed some odd (although I now realize quite sensible) behavior in
>> IPython, and I'm curious if there's a workaround.  I have an
>> application log that I want to use to capture unhandled exceptions.
>> When not using IPython, it's straightforward to just override
>> sys.excepthook to first sent the exception to our logging system, and
>> then continue the normal excepthook behavior.  However, IPython seems
>> to always use only its own excepthook in interactive mode
>> (TerminalInteractiveShell.excepthook, at least in the standard
>> terminal).
>>
>> So my question is: Is there any way to plug anything into the IPython
>> exception handling system, either with sys.excepthook-style syntax, or
>> some other way?  As I said, it doesn't have to actually catch the
>> exceptions - we just want entries placed in a log if an interactive
>> session hits an unhandled exception, and it could then pass normally
>> into the IPython formatting/pdb running system.
>
>
> You use shell.set_custom_exc for this:
>
> # define custom exception handler:
>
> def custom_exc(shell, etype, evalue, tb, tb_offset=None):
>     # do you own thing:
>     print 'hi', etype
>     # also do what IPython would have done, if you want:
>     shell.showtraceback((etype, evalue, tb), tb_offset=tb_offset)
>
> # Tell IPython to use it for any/all Exceptions:
> get_ipython().set_custom_exc((Exception,), custom_exc)
>
>>
>> --
>> Erik Tollerud
>> _______________________________________________
>> IPython-dev mailing list
>> [hidden email]
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> _______________________________________________
> IPython-dev mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



--
Erik Tollerud
_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: excepthook-like behavior in ipython

Thomas Kluyver-2
In reply to this post by Erik Tollerud-2
On 14 May 2012 10:25, Erik Tollerud <[hidden email]> wrote:
> Further investigation has
> revealed that it only happens if I try to raise an exception of a
> class exactly the same as what's given in the first argument to
> set_custom_exc (Exception, in this case), while if I raise
> sub-exceptions, everything works just fine.  Is this a bug, or am I
> mis-using it somehow?

Having worked out what's going on, I would consider it a bug, although
fixing it changes the API slightly. I've made a pull request for it.

https://github.com/ipython/ipython/pull/1742

Thomas
_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: excepthook-like behavior in ipython

Carl Smith
Hi

I remember hearing that stdin in Notebook was something that you knew
how to implement, but needed time to get done. I was just wondering
what's going on with that.

Recently, I've been playing around on EC2, where having Notebook as a
fully functional, remote shell would be enormously useful. The current
need to have a bunch of stuff installed on your local machine before
you can actually use The Cloud is ridiculous.


On 16 May 2012 13:10, Thomas Kluyver <[hidden email]> wrote:

> On 14 May 2012 10:25, Erik Tollerud <[hidden email]> wrote:
>> Further investigation has
>> revealed that it only happens if I try to raise an exception of a
>> class exactly the same as what's given in the first argument to
>> set_custom_exc (Exception, in this case), while if I raise
>> sub-exceptions, everything works just fine.  Is this a bug, or am I
>> mis-using it somehow?
>
> Having worked out what's going on, I would consider it a bug, although
> fixing it changes the API slightly. I've made a pull request for it.
>
> https://github.com/ipython/ipython/pull/1742
>
> Thomas
> _______________________________________________
> IPython-dev mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/ipython-dev
_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: excepthook-like behavior in ipython

Fernando Perez
On Wed, May 16, 2012 at 7:04 PM, Carl Smith <[hidden email]> wrote:
> I remember hearing that stdin in Notebook was something that you knew
> how to implement, but needed time to get done. I was just wondering
> what's going on with that.

Haven't had any time to work on that, I'm afraid.  It's not super
difficult, but it does require some knowledge of the networking bits.

> Recently, I've been playing around on EC2, where having Notebook as a
> fully functional, remote shell would be enormously useful. The current
> need to have a bunch of stuff installed on your local machine before
> you can actually use The Cloud is ridiculous.

You can *sort of* use it already with

!cmd

the only thing that doesn't work is truly interactive stuff that
requires stdin reading.  For now, in a cloud scenario the only
reasonable solution is to ssh into your notebook node and use

ipython console --existing [json info]

after getting the connection info with %connect_info.  Fortunately
starcluster makes the process of ssh-ing into your notebook node
completely trivial, so it's not a completely absurd solution.

Obviously in the long run we want real stdin in the notebook, I'm just
offering a temporary workaround.  And if you're not using starcluster
for your EC2 setup, switch to it right away: it will save you a LOT of
time and effort.

Cheers,

f
_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: excepthook-like behavior in ipython

Carl Smith
Hi Fernando

Thanks for your input. I've used StarCluster, its really good, but for
what I'm trying to do, I found it easier to just use boto and do
things by hand. I really want lots of small VMs, one per user,
controlled by a single, larger server that exists to manage users and
their VMs. Each user would have complete control over their own
instance, but only via IPython. They wouldn't have/need anything on
their local machine besides a decent browser.

This is meant to be used to learn programming collaboratively.

It's not very difficult to do what I'm looking into doing, but it'd
not work well without stdin. I'm not familiar with IPython internals,
so I guess I'll just have to be patient. It's not especially urgent
anyway.

Again, thanks for you reply and all the hard work.

Carl

On 17 May 2012 03:22, Fernando Perez <[hidden email]> wrote:

> On Wed, May 16, 2012 at 7:04 PM, Carl Smith <[hidden email]> wrote:
>> I remember hearing that stdin in Notebook was something that you knew
>> how to implement, but needed time to get done. I was just wondering
>> what's going on with that.
>
> Haven't had any time to work on that, I'm afraid.  It's not super
> difficult, but it does require some knowledge of the networking bits.
>
>> Recently, I've been playing around on EC2, where having Notebook as a
>> fully functional, remote shell would be enormously useful. The current
>> need to have a bunch of stuff installed on your local machine before
>> you can actually use The Cloud is ridiculous.
>
> You can *sort of* use it already with
>
> !cmd
>
> the only thing that doesn't work is truly interactive stuff that
> requires stdin reading.  For now, in a cloud scenario the only
> reasonable solution is to ssh into your notebook node and use
>
> ipython console --existing [json info]
>
> after getting the connection info with %connect_info.  Fortunately
> starcluster makes the process of ssh-ing into your notebook node
> completely trivial, so it's not a completely absurd solution.
>
> Obviously in the long run we want real stdin in the notebook, I'm just
> offering a temporary workaround.  And if you're not using starcluster
> for your EC2 setup, switch to it right away: it will save you a LOT of
> time and effort.
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/ipython-dev
_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: excepthook-like behavior in ipython

Fernando Perez
On Sat, May 19, 2012 at 10:05 AM, Carl Smith <[hidden email]> wrote:
> It's not very difficult to do what I'm looking into doing, but it'd
> not work well without stdin. I'm not familiar with IPython internals,
> so I guess I'll just have to be patient. It's not especially urgent
> anyway.

Understood (and thanks for the boto pointer, worth keeping in mind).
Hopefully it won't take us too long to get this done...

Best

f
_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev
Loading...