[IPython-User] Using DirectView.execute to import modules on worker processes

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[IPython-User] Using DirectView.execute to import modules on worker processes

William Shipman
I have looked through the documentation about importing modules on IPython Parallel worker processes, and it seems imports of the form "import foo as bar" are not supported.  However, I found that using DirectView.execute allows me to get around this.  Is this safe, if not what can go wrong?

Here's the code I'm using to import part of the dateutil library and :

    from IPython.parallel import Client
    ipython_client = Client()
    dview = ipython_client.direct_view()
    dview.execute('from dateutil import parser as dtparser')

I am only using on my local PC, not a cluster.  After executing the above I can happily run code that uses dtparser using dview.map_sync.  I can rewrite my code to avoid doing this but I'd like to know why I shouldn't use this method.

Thanks in advance.

_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user
Reply | Threaded
Open this post in threaded view
|

Re: Using DirectView.execute to import modules on worker processes

MinRK

On Sat, Aug 9, 2014 at 9:20 AM, William Shipman <[hidden email]> wrote:

I have looked through the documentation about importing modules on IPython Parallel worker processes, and it seems imports of the form "import foo as bar" are not supported.  However, I found that using DirectView.execute allows me to get around this.  Is this safe, if not what can go wrong?

Here's the code I'm using to import part of the dateutil library and :

    from IPython.parallel import Client
    ipython_client = Client()
    dview = ipython_client.direct_view()
    dview.execute('from dateutil import parser as dtparser')

I am only using on my local PC, not a cluster.  After executing the above I can happily run code that uses dtparser using dview.map_sync.  I can rewrite my code to avoid doing this but I'd like to know why I shouldn't use this method.

import foo as bar via view.execute("import foo as bar") or %px import foo as bar is totally fine. The only place where this doesn’t work is when you uses the with view.sync_imports() context manager, which uses an import hook. import foo as bar is two operations: import, then assignment. The import hook only knows about the import, not the assignment, so it won’t behave correctly. Executing code as text is not affected by this. I will try to clarify the docs, so they don’t give people this misconception.

-MinRK


Thanks in advance.

_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user


_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user
Reply | Threaded
Open this post in threaded view
|

Re: Using DirectView.execute to import modules on worker processes

William Shipman
Thanks for clarifying that.

Regards,
William.


On 13 August 2014 19:34, MinRK <[hidden email]> wrote:

On Sat, Aug 9, 2014 at 9:20 AM, William Shipman <[hidden email]> wrote:

I have looked through the documentation about importing modules on IPython Parallel worker processes, and it seems imports of the form "import foo as bar" are not supported.  However, I found that using DirectView.execute allows me to get around this.  Is this safe, if not what can go wrong?

Here's the code I'm using to import part of the dateutil library and :

    from IPython.parallel import Client
    ipython_client = Client()
    dview = ipython_client.direct_view()
    dview.execute('from dateutil import parser as dtparser')

I am only using on my local PC, not a cluster.  After executing the above I can happily run code that uses dtparser using dview.map_sync.  I can rewrite my code to avoid doing this but I'd like to know why I shouldn't use this method.

import foo as bar via view.execute("import foo as bar") or %px import foo as bar is totally fine. The only place where this doesn’t work is when you uses the with view.sync_imports() context manager, which uses an import hook. import foo as bar is two operations: import, then assignment. The import hook only knows about the import, not the assignment, so it won’t behave correctly. Executing code as text is not affected by this. I will try to clarify the docs, so they don’t give people this misconception.

-MinRK


Thanks in advance.

_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user


_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user



_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user
Reply | Threaded
Open this post in threaded view
|

[IPython-User] Slideshow on github page

justin
Hello all,

To share my Ipython Notebook slideshows, I have been using

ipython nbconvert --to slides blah.ipynb

Then I take the outputted blah.slides.html file and push it to a
gh-pages branch of a github repo.  Then I can just share the link.

However, after upgrading to Ipy 2.2, when I push the slides.html files
to github, It no longer renders as slides.  Interestingly, the old
slides.html notebooks that I have pushed to github still render as slides.

Also, when i simply do ipython nbconvert blah.ipynb --to slides --post
serve it *does* render as slides.

Any idea on what I might be doing wrong?

Thanks,

Justin

FWIW when I used the old Ipython and NBconvert, it came back as slides
_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user
Reply | Threaded
Open this post in threaded view
|

Re: Slideshow on github page

Damián Avila
You have to do something like this:

ipython nbconvert blah.ipynb --to slides --reveal-prefix "http://cdn.jsdelivr.net/reveal.js/2.5.0"

So, the resulting slide will point to the cdn instead of a local reveal.js.

Damian.


2014-08-19 16:11 GMT-03:00 Justin <[hidden email]>:
Hello all,

To share my Ipython Notebook slideshows, I have been using

ipython nbconvert --to slides blah.ipynb

Then I take the outputted blah.slides.html file and push it to a
gh-pages branch of a github repo.  Then I can just share the link.

However, after upgrading to Ipy 2.2, when I push the slides.html files
to github, It no longer renders as slides.  Interestingly, the old
slides.html notebooks that I have pushed to github still render as slides.

Also, when i simply do ipython nbconvert blah.ipynb --to slides --post
serve it *does* render as slides.

Any idea on what I might be doing wrong?

Thanks,

Justin

FWIW when I used the old Ipython and NBconvert, it came back as slides
_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user



--
Damián

_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user
Reply | Threaded
Open this post in threaded view
|

Re: Slideshow on github page

Zoltán Vörös-2
In reply to this post by justin
Hi Justin,

I think you should post your question on ipython-dev, because
ipython-user has virtually no traffic now.

Cheers,
Zoltán


On 08/19/2014 09:11 PM, Justin wrote:

> Hello all,
>
> To share my Ipython Notebook slideshows, I have been using
>
> ipython nbconvert --to slides blah.ipynb
>
> Then I take the outputted blah.slides.html file and push it to a
> gh-pages branch of a github repo.  Then I can just share the link.
>
> However, after upgrading to Ipy 2.2, when I push the slides.html files
> to github, It no longer renders as slides.  Interestingly, the old
> slides.html notebooks that I have pushed to github still render as slides.
>
> Also, when i simply do ipython nbconvert blah.ipynb --to slides --post
> serve it *does* render as slides.
>
> Any idea on what I might be doing wrong?
>
> Thanks,
>
> Justin
>
> FWIW when I used the old Ipython and NBconvert, it came back as slides
> _______________________________________________
> IPython-User mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/ipython-user

_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user
Reply | Threaded
Open this post in threaded view
|

Re: Slideshow on github page

justin
In reply to this post by Damián Avila
Easy as that !  Thanks Damian. 
On 08/20/2014 04:41 PM, Damián Avila wrote:
You have to do something like this:

ipython nbconvert blah.ipynb --to slides --reveal-prefix "http://cdn.jsdelivr.net/reveal.js/2.5.0"

So, the resulting slide will point to the cdn instead of a local reveal.js.

Damian.


2014-08-19 16:11 GMT-03:00 Justin <[hidden email]>:
Hello all,

To share my Ipython Notebook slideshows, I have been using

ipython nbconvert --to slides blah.ipynb

Then I take the outputted blah.slides.html file and push it to a
gh-pages branch of a github repo.  Then I can just share the link.

However, after upgrading to Ipy 2.2, when I push the slides.html files
to github, It no longer renders as slides.  Interestingly, the old
slides.html notebooks that I have pushed to github still render as slides.

Also, when i simply do ipython nbconvert blah.ipynb --to slides --post
serve it *does* render as slides.

Any idea on what I might be doing wrong?

Thanks,

Justin

FWIW when I used the old Ipython and NBconvert, it came back as slides
_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user



--
Damián


_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user


_______________________________________________
IPython-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-user