Quantcast

query frontend?

classic Classic list List threaded Threaded
7 messages Options
mqk
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

query frontend?

mqk
Hi

Is there some way to find out from within a python module whether it is being imported from a web notebook rather than from the command line? Something akin to if __name__ == "__main__"?

Thanks,

Mike
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: query frontend?

Brian Granger-3
Given the architecture of IPython, this doesn't even make sense to
ask.  The kernel can have multiple different frontends attached to it,
even simultaneously.  You could have a qtconsole and notebook attached
to a single kernel and the kernel has absolutely no way of knowing
which commands are coming from the qtconsole and which are coming from
the notebook.  That the kernel knows nothing about the frontends is
not an accident, but rather a deliberate design decision that I don't
see us changing anytime soon.

But maybe the better question is this: what exactly are you trying to
do?  Maybe there is a different way of approaching it that would make
more sense.  Or it might be that there are other abstraction IPython
has or might have to help solve this issue.

Cheers,

Brian

On Tue, Apr 10, 2012 at 5:24 PM, mqk <[hidden email]> wrote:

> Hi
>
> Is there some way to find out from within a python module whether it is
> being imported from a web notebook rather than from the command line?
> Something akin to if __name__ == "__main__"?
>
> Thanks,
>
> Mike
>
> --
> View this message in context: http://python.6.n6.nabble.com/query-frontend-tp4790920p4790920.html
> Sent from the IPython - User mailing list archive at Nabble.com.
> _______________________________________________
> IPython-User mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/ipython-user



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
[hidden email] and [hidden email]
_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: query frontend?

Michael Kuhlen
I was afraid my question might not even make sense. Sorry about that.
My understanding of IPython clearly is still woefully incomplete.

> But maybe the better question is this: what exactly are you trying to do?

OK, in an attempt to validate my non-sensical question, let me explain
how it arose for me. Here's an abstracted version of my module (let's
call it my_module.py):

## beginning of module
def func(x,y):
    do stuff to x,y
    return result

def plotit(result):
    make a plot of result

import loadit as l  ### this loads a bunch of data from disk and takes
a long time.
do a whole bunch of manipulations of l.data resulting in x,y
## end of module

I usually use this interactively from an IPython shell, like so:
In [1]: import my_module as mm   ### this takes a long time because mm
imports the module loadit.
In [2]: result = mm.func(mm.x,mm.y)
In [3]: mm.plotit(result)
Fiddle with mm.plotit (or mm.func).
In [4]: reload(mm)  ### this does not take a long time, since loadit
was already imported previously.
In [5]: mm.plotit(result)
and so on.


Now I'm exploring switching to IPython notebooks. I've moved the
contents of loadit.py into its own cell, and then the manipulations
into a second cell. Now in the following cell I would like to do a
"from my_module import func, plotit" and then proceed as before. The
trouble is that that import triggers the import of the module loadit,
which takes a long time and duplicates my memory footprint. So I was
looking for something like

if not __imported_from_notebook__:
    import loadit as l
    do a whole bunch of manipulations of l.data resulting in x,y

in order to prevent the import of loadit when I'm using a notebook. Of
course I could simply get rid the "import loadit as l" and the
manipulations in my_module.py, but in the future I might want to
return to a non-notebook way of doing things, so I wanted to avoid
changing any of my existing modules, but it sounds like that may not
be an option in this case. I'll just go with commenting out those
parts.

Thanks in advance for indulging my idiosyncrasies.

Mike



On Tue, Apr 10, 2012 at 6:33 PM, Brian Granger <[hidden email]> wrote:

> Given the architecture of IPython, this doesn't even make sense to
> ask.  The kernel can have multiple different frontends attached to it,
> even simultaneously.  You could have a qtconsole and notebook attached
> to a single kernel and the kernel has absolutely no way of knowing
> which commands are coming from the qtconsole and which are coming from
> the notebook.  That the kernel knows nothing about the frontends is
> not an accident, but rather a deliberate design decision that I don't
> see us changing anytime soon.
>
> But maybe the better question is this: what exactly are you trying to
> do?  Maybe there is a different way of approaching it that would make
> more sense.  Or it might be that there are other abstraction IPython
> has or might have to help solve this issue.
>
> Cheers,
>
> Brian
>
> On Tue, Apr 10, 2012 at 5:24 PM, mqk <[hidden email]> wrote:
>> Hi
>>
>> Is there some way to find out from within a python module whether it is
>> being imported from a web notebook rather than from the command line?
>> Something akin to if __name__ == "__main__"?
>>
>> Thanks,
>>
>> Mike
>>
>> --
>> View this message in context: http://python.6.n6.nabble.com/query-frontend-tp4790920p4790920.html
>> Sent from the IPython - User mailing list archive at Nabble.com.
>> _______________________________________________
>> IPython-User mailing list
>> [hidden email]
>> http://mail.scipy.org/mailman/listinfo/ipython-user
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> [hidden email] and [hidden email]
> _______________________________________________
> IPython-User mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/ipython-user



--
*********************************************************************
*                                                                   *
*  Dr. Michael Kuhlen              Theoretical Astrophysics Center  *
*  email: [hidden email]   UC Berkeley                      *
*  cell phone: (831) 588-1468      B-116 Hearst Field Annex # 3411  *
*  skype username: mikekuhlen      Berkeley, CA 94720               *
*                                                                   *
*********************************************************************
_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: query frontend?

Brian Granger-3
On Tue, Apr 10, 2012 at 7:55 PM, Michael Kuhlen <[hidden email]> wrote:
> I was afraid my question might not even make sense. Sorry about that.
> My understanding of IPython clearly is still woefully incomplete.

No problem at all, let's see if we can figure something out below...

>> But maybe the better question is this: what exactly are you trying to do?
>
> OK, in an attempt to validate my non-sensical question, let me explain
> how it arose for me. Here's an abstracted version of my module (let's
> call it my_module.py):
>
> ## beginning of module
> def func(x,y):
>    do stuff to x,y
>    return result
>
> def plotit(result):
>    make a plot of result
>
> import loadit as l  ### this loads a bunch of data from disk and takes
> a long time.
> do a whole bunch of manipulations of l.data resulting in x,y
> ## end of module

OK I am following.

> I usually use this interactively from an IPython shell, like so:
> In [1]: import my_module as mm   ### this takes a long time because mm
> imports the module loadit.
> In [2]: result = mm.func(mm.x,mm.y)
> In [3]: mm.plotit(result)
> Fiddle with mm.plotit (or mm.func).
> In [4]: reload(mm)  ### this does not take a long time, since loadit
> was already imported previously.
> In [5]: mm.plotit(result)
> and so on.
>
>
> Now I'm exploring switching to IPython notebooks. I've moved the
> contents of loadit.py into its own cell, and then the manipulations
> into a second cell. Now in the following cell I would like to do a
> "from my_module import func, plotit" and then proceed as before. The
> trouble is that that import triggers the import of the module loadit,
> which takes a long time and duplicates my memory footprint. So I was
> looking for something like
>
> if not __imported_from_notebook__:
>    import loadit as l
>    do a whole bunch of manipulations of l.data resulting in x,y
>
> in order to prevent the import of loadit when I'm using a notebook. Of
> course I could simply get rid the "import loadit as l" and the
> manipulations in my_module.py, but in the future I might want to
> return to a non-notebook way of doing things, so I wanted to avoid
> changing any of my existing modules, but it sounds like that may not
> be an option in this case. I'll just go with commenting out those
> parts.

A few thoughts on this workflow:

* Try making your imports do no computation.  Put a special function
in the loadit module that does the expensive setup and call that
function once the first time you import it.  If you want to be able to
call that function more than once, it could use a global variable to
track whether it has already been called.  That at least decouples the
expensive computation from the import/reload logic.
* The next question is what exactly you want to move to the notebook.
If you move *all* of your code to the notebook, I don't think you will
have any problems.  You can call the setup functions once at the top
and then use them at will below.  This will also avoid reloads, etc.
But you are right that this makes it more difficult to move to a
non-notebook workflow later.
* I think you need to break the chain that interactive code calls
my_module which imports loadit.  Can you make it so my_module doesn't
know about loadit, but instead gets the right things passed to it when
its functions are called.  That would allow you to call loadit once in
a cell and then pass its data to my_module.  You top level code (in
the notebook) would then look like this:

import loadit
loadit.initialize()  # this does the expensive setup and saves the
needed data as x, y, ...

import my_module as mm
mm.func(loadit.x, loadit.y)
mm.plot(loadit.x, loadit.y)

Would something like this work?

Cheers,

Brian

> Thanks in advance for indulging my idiosyncrasies.
>
> Mike
>
>
>
> On Tue, Apr 10, 2012 at 6:33 PM, Brian Granger <[hidden email]> wrote:
>> Given the architecture of IPython, this doesn't even make sense to
>> ask.  The kernel can have multiple different frontends attached to it,
>> even simultaneously.  You could have a qtconsole and notebook attached
>> to a single kernel and the kernel has absolutely no way of knowing
>> which commands are coming from the qtconsole and which are coming from
>> the notebook.  That the kernel knows nothing about the frontends is
>> not an accident, but rather a deliberate design decision that I don't
>> see us changing anytime soon.
>>
>> But maybe the better question is this: what exactly are you trying to
>> do?  Maybe there is a different way of approaching it that would make
>> more sense.  Or it might be that there are other abstraction IPython
>> has or might have to help solve this issue.
>>
>> Cheers,
>>
>> Brian
>>
>> On Tue, Apr 10, 2012 at 5:24 PM, mqk <[hidden email]> wrote:
>>> Hi
>>>
>>> Is there some way to find out from within a python module whether it is
>>> being imported from a web notebook rather than from the command line?
>>> Something akin to if __name__ == "__main__"?
>>>
>>> Thanks,
>>>
>>> Mike
>>>
>>> --
>>> View this message in context: http://python.6.n6.nabble.com/query-frontend-tp4790920p4790920.html
>>> Sent from the IPython - User mailing list archive at Nabble.com.
>>> _______________________________________________
>>> IPython-User mailing list
>>> [hidden email]
>>> http://mail.scipy.org/mailman/listinfo/ipython-user
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> [hidden email] and [hidden email]
>> _______________________________________________
>> IPython-User mailing list
>> [hidden email]
>> http://mail.scipy.org/mailman/listinfo/ipython-user
>
>
>
> --
> *********************************************************************
> *                                                                   *
> *  Dr. Michael Kuhlen              Theoretical Astrophysics Center  *
> *  email: [hidden email]   UC Berkeley                      *
> *  cell phone: (831) 588-1468      B-116 Hearst Field Annex # 3411  *
> *  skype username: mikekuhlen      Berkeley, CA 94720               *
> *                                                                   *
> *********************************************************************
> _______________________________________________
> IPython-User mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/ipython-user



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
[hidden email] and [hidden email]
_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: query frontend?

Fernando Perez
In reply to this post by Michael Kuhlen
Hi Michael,

On Tue, Apr 10, 2012 at 7:55 PM, Michael Kuhlen <[hidden email]> wrote:
> Now I'm exploring switching to IPython notebooks.

Brian gave you some useful tips, I just wanted to add a minor note:
you can keep 'library' code in .py files if you want, and atop a
notebook you can use

%run mymodule.py

to pull all of those utilities in for further interactive use.  Since
%run does a full execfile(), everytime you run it you get the
equivalent of a clean reload(), though your expensive loadit step
won't get done again.

Cheers,

f
_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: query frontend?

Michael Kuhlen
Hi Brian and Fernando

Thanks for your tips and workflow suggestions. Much appreciated.

> you can keep 'library' code in .py files if you want, and atop a
> notebook you can use
>
> %run mymodule.py
>
> to pull all of those utilities in for further interactive use.
> Since %run does a full execfile(), everytime you run it you get the
> equivalent of a clean reload(), though your expensive loadit step
> won't get done again.

That works very well for me. Thanks for the suggestion.

Mike


On Thu, Apr 12, 2012 at 8:55 PM, Fernando Perez <[hidden email]> wrote:

> Hi Michael,
>
> On Tue, Apr 10, 2012 at 7:55 PM, Michael Kuhlen <[hidden email]> wrote:
>> Now I'm exploring switching to IPython notebooks.
>
> Brian gave you some useful tips, I just wanted to add a minor note:
> you can keep 'library' code in .py files if you want, and atop a
> notebook you can use
>
> %run mymodule.py
>
> to pull all of those utilities in for further interactive use.  Since
> %run does a full execfile(), everytime you run it you get the
> equivalent of a clean reload(), though your expensive loadit step
> won't get done again.
>
> Cheers,
>
> f
> _______________________________________________
> IPython-User mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/ipython-user



--
*********************************************************************
*                                                                   *
*  Dr. Michael Kuhlen              Theoretical Astrophysics Center  *
*  email: [hidden email]   UC Berkeley                      *
*  cell phone: (831) 588-1468      B-116 Hearst Field Annex # 3411  *
*  skype username: mikekuhlen      Berkeley, CA 94720               *
*                                                                   *
*********************************************************************
_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: query frontend?

Fernando Perez
On Fri, Apr 13, 2012 at 1:35 PM, Michael Kuhlen <[hidden email]> wrote:
> That works very well for me. Thanks for the suggestion.

Glad to hear that!  BTW, don't hesitate to ping me on campus, a number
of core IPython developers happen to be at Berkeley, and there's also
a strong contingent of python users in astro (J. Bloom, the MIRIAD
folks - http://arxiv.org/abs/1203.0330).

Cheers,

f
_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user
Loading...