WebGL example in IPython notebook based on three.js

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

WebGL example in IPython notebook based on three.js

David Powell
Hello all,

I recently managed to get 3D vector plots working in the IPython notebook, by statically producing three.js output. I thought I'd share it here in case this example is useful for anyone else, or if anyone has any suggestions as to how I could do things better.

The advantage of the static approach is that the output survives the nbconvert process, which you can see in this notebook here (which also compares with a couple of other unsatisfactory ways of visualising 3D vector plots).


If you are interested to see how this works, you can check out the python source:


As well as the javascript:


and finally the HTML:


By the way, I am aware of the efforts to make a proper three.js IPython widget (https://github.com/jasongrout/pythreejs/), and will probably switch over to this once it is stable, and when the output can survive ipython's nbconvert process. Is this something we can expect for widgets in IPython 3.0?

regards
David


_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|

Re: WebGL example in IPython notebook based on three.js

Soumith Chintala
This is fantastic! I was going to do some of this work myself over the next month for iTorch (https://github.com/facebook/iTorch), and you saved me some trouble.
I will take a look at your code over the next couple of days, I am wondering if you would have any issues if I directly adapted your javascript side of things in iTorch rather than rewriting from scratch.
Soumith

From: David Powell <[hidden email]>
Reply-To: IPython developers list <[hidden email]>
Date: Sunday, January 18, 2015 at 10:47 PM
To: IPython developers list <[hidden email]>
Subject: [IPython-dev] WebGL example in IPython notebook based on three.js

Hello all,

I recently managed to get 3D vector plots working in the IPython notebook, by statically producing three.js output. I thought I'd share it here in case this example is useful for anyone else, or if anyone has any suggestions as to how I could do things better.

The advantage of the static approach is that the output survives the nbconvert process, which you can see in this notebook here (which also compares with a couple of other unsatisfactory ways of visualising 3D vector plots).


If you are interested to see how this works, you can check out the python source:


As well as the javascript:


and finally the HTML:


By the way, I am aware of the efforts to make a proper three.js IPython widget (https://github.com/jasongrout/pythreejs/), and will probably switch over to this once it is stable, and when the output can survive ipython's nbconvert process. Is this something we can expect for widgets in IPython 3.0?

regards
David


_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|

Re: WebGL example in IPython notebook based on three.js

Matthias Bussonnier
Hi all, 
Le 19 janv. 2015 à 07:50, Soumith Chintala a écrit :

This is fantastic! I was going to do some of this work myself over the next month for iTorch (https://github.com/facebook/iTorch), and you saved me some trouble.
I will take a look at your code over the next couple of days, I am wondering if you would have any issues if I directly adapted your javascript side of things in iTorch rather than rewriting from scratch.

We still haven't really figured it out how to share javascript in between packages of different languages. 
(well technically in 3.0 yo can install into $NBEXTENSION directory, but install process is not perfect) 
If you can figured that out, and that all kernels agree on a set of relatively common JS "plugins"  that
are often used we might be able to work on making  theses plugin available on nbviewer.

Long term planning of course, but I'd like people to keep that in mind. 

-- 
M


_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|

Re: WebGL example in IPython notebook based on three.js

rossant
Thanks for sharing this!

I'd like to add that there's also work being done currently for
bringing WebGL visualization to the notebook via VisPy. We don't use
three.js but custom WebGL code generated by Python. The WebGL backend
currently requires a live Python server, but a standalone version is
also in the pipes.

2015-01-19 12:08 GMT+01:00 Matthias BUSSONNIER <[hidden email]>:

> Hi all,
> Le 19 janv. 2015 à 07:50, Soumith Chintala a écrit :
>
> This is fantastic! I was going to do some of this work myself over the next
> month for iTorch (https://github.com/facebook/iTorch), and you saved me some
> trouble.
> I will take a look at your code over the next couple of days, I am wondering
> if you would have any issues if I directly adapted your javascript side of
> things in iTorch rather than rewriting from scratch.
>
>
> We still haven't really figured it out how to share javascript in between
> packages of different languages.
> (well technically in 3.0 yo can install into $NBEXTENSION directory, but
> install process is not perfect)
> If you can figured that out, and that all kernels agree on a set of
> relatively common JS "plugins"  that
> are often used we might be able to work on making  theses plugin available
> on nbviewer.
>
> Long term planning of course, but I'd like people to keep that in mind.
>
> --
> M
>
>
> _______________________________________________
> IPython-dev mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|

Re: WebGL example in IPython notebook based on three.js

Andrew Payne
In reply to this post by David Powell

I recently managed to get 3D vector plots working in the IPython notebook, by statically producing three.js output. I thought I'd share it here in case this example is useful for anyone else, or if anyone has any suggestions as to how I could do things better.

Great work!  Thanks for sharing that.

Did you consider or explore using require.js to load the three.js library?  You can reference and load the library once, and then not have to inline it within each cell.

Aside for folks fiddling with Javscript libraries & the Web notebook:   the obvious approach of dropping a <script> reference in an HTML cell or result isn't robust.  When the cell is reexecuted, the browser will load the library again, overwriting the original library name space.  However, the first instance of the library will likely still be there because of lingering references.  So you'll have a variable "foo" of type X in the old library instance, and you'll pass it into a function call (new library instance) and things will fail in strange ways because all the type tests in the new library instance fail.  And you will drive yourself crazy because you'll be in the JS console and wonder why foo is clearly of type "X", but "isinstance X" is giving False.

That's why David's JS code has this test:


if ( camera instanceof THREE.Camera === false || ! document.body.contains(three_container)) {
            console.log("Animation loop failed: stopping");
            return;
        }


The advantage of the static approach is that the output survives the nbconvert process, which you can see in this notebook here (which also compares with a couple of other unsatisfactory ways of visualising 3D vector plots).


If you are interested to see how this works, you can check out the python source:


As well as the javascript:


and finally the HTML:


By the way, I am aware of the efforts to make a proper three.js IPython widget (https://github.com/jasongrout/pythreejs/), and will probably switch over to this once it is stable, and when the output can survive ipython's nbconvert process. Is this something we can expect for widgets in IPython 3.0?

regards
David


_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev



_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|

Re: WebGL example in IPython notebook based on three.js

Andrew Payne
[clicked send accidentally, please ignore my previous incomplete email]

I recently managed to get 3D vector plots working in the IPython notebook, by statically producing three.js output. I thought I'd share it here in case this example is useful for anyone else, or if anyone has any suggestions as to how I could do things better.

Great work!  Thanks for sharing that.

Did you consider or explore using require.js to load the three.js library?  You can reference and load the library once, and then not have to inline it within each cell.

Aside for folks fiddling with Javscript libraries & the Web notebook:   the obvious approach of dropping a <script> reference in an HTML cell or result isn't robust.  When the cell is reexecuted, the browser will load the library again, overwriting the original library name space.  However, the first instance of the library will likely still be there because of lingering references.  So you'll have a variable "foo" of type X in the old library instance, and you'll pass it into a function call (new library instance) and things will fail in strange ways because all the type tests in the new library instance fail.  And you will drive yourself crazy because you'll be in the JS console and wonder why foo is clearly of type "X", but "isinstanceof X" is giving False.

That's why David's JS code has this test:

https://github.com/DavidPowell/OpenModes/blob/master/openmodes/static/three_js_plot.js#L175

       if ( camera instanceof THREE.Camera === false || ! document.body.contains(three_container)) {
            console.log("Animation loop failed: stopping");
            return;
        }



For the dev team, this raises the issue of:  what's the best idiom for JS in a cell to access the output area?  (See SO question:  http://stackoverflow.com/questions/20019690/how-to-access-current-cell-output-area-with-javascript-in-ipython-web-notebook)

On a "live" notebook, the Javascript element variable is fine.  But as you can see with my notebook (above) vs David's, the element variable approach doesn't survive nbconvert -- my Javascript examples break because element doesn't exist in an nbconvert(ed) notebook.

David made things work by generating (effectively) a UID as part of his HTML cell result that the corresponding Javascript can pick up.

As we see Javascript used more with the Web notebook, folks are going to consistently run into this pattern.  I think the notebook could provide some "standard" way to reference the output area for the cell, that survives nbconvert.  

What do you think?

-andy


_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|

Re: WebGL example in IPython notebook based on three.js

Antonio González Peña
In reply to this post by David Powell
Hello,

Related to this topic. We maintain Emperor
(http://biocore.github.io/emperor/), a package that uses python to
create very simple THREE.js code that can be embedded as a web page to
visualize and manipulate scatter plots + their metadata. We have
successfully been able to include Emperor plots into IPython notebooks
using _html_repr_. Note that in this case resources are stored in a
server, but we should really be making use of the $NBEXTENSION.

Anyway, as a side note we encountered a small issue with
nbviewer+Emperor: https://github.com/jupyter/nbviewer/issues/316,
which you perhaps might encounter.

Cheers,


On Sun, Jan 18, 2015 at 11:47 PM, David Powell
<[hidden email]> wrote:

> Hello all,
>
> I recently managed to get 3D vector plots working in the IPython notebook,
> by statically producing three.js output. I thought I'd share it here in case
> this example is useful for anyone else, or if anyone has any suggestions as
> to how I could do things better.
>
> The advantage of the static approach is that the output survives the
> nbconvert process, which you can see in this notebook here (which also
> compares with a couple of other unsatisfactory ways of visualising 3D vector
> plots).
>
> http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/How%20to%20create%203D%20plots.ipynb
>
> Another example can be seen at:
>
> http://nbviewer.ipython.org/github/DavidPowell/openmodes-examples/blob/master/Using%20and%20creating%20geometric%20shapes.ipynb
>
> If you are interested to see how this works, you can check out the python
> source:
>
> https://github.com/DavidPowell/OpenModes/blob/master/openmodes/ipython.py
>
> As well as the javascript:
>
> https://github.com/DavidPowell/OpenModes/blob/master/openmodes/static/three_js_plot.js
>
> and finally the HTML:
>
> https://github.com/DavidPowell/OpenModes/blob/master/openmodes/templates/three_js_plot.html
>
> By the way, I am aware of the efforts to make a proper three.js IPython
> widget (https://github.com/jasongrout/pythreejs/), and will probably switch
> over to this once it is stable, and when the output can survive ipython's
> nbconvert process. Is this something we can expect for widgets in IPython
> 3.0?
>
> regards
> David
>
>
> _______________________________________________
> IPython-dev mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



--
Antonio
_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|

Re: WebGL example in IPython notebook based on three.js

David Powell
In reply to this post by Soumith Chintala

Hello Soumith,
My code is GPL3, but if you tell me which files you want to use, I will release them under a BSD license to be compatible with itorch.
Regards,
David

This is fantastic! I was going to do some of this work myself over the next month for iTorch (https://github.com/facebook/iTorch), and you saved me some trouble.
I will take a look at your code over the next couple of days, I am wondering if you would have any issues if I directly adapted your javascript side of things in iTorch rather than rewriting from scratch.
Soumith

From: David Powell <[hidden email]>
Reply-To: IPython developers list <[hidden email]>
Date: Sunday, January 18, 2015 at 10:47 PM
To: IPython developers list <[hidden email]>
Subject: [IPython-dev] WebGL example in IPython notebook based on three.js

Hello all,

I recently managed to get 3D vector plots working in the IPython notebook, by statically producing three.js output. I thought I'd share it here in case this example is useful for anyone else, or if anyone has any suggestions as to how I could do things better.

The advantage of the static approach is that the output survives the nbconvert process, which you can see in this notebook here (which also compares with a couple of other unsatisfactory ways of visualising 3D vector plots).


If you are interested to see how this works, you can check out the python source:


As well as the javascript:


and finally the HTML:


By the way, I am aware of the efforts to make a proper three.js IPython widget (https://github.com/jasongrout/pythreejs/), and will probably switch over to this once it is stable, and when the output can survive ipython's nbconvert process. Is this something we can expect for widgets in IPython 3.0?

regards
David


_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev


_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|

Re: WebGL example in IPython notebook based on three.js

David Powell
In reply to this post by Andrew Payne
Did you consider or explore using require.js to load the three.js library?  You can reference and load the library once, and then not have to inline it within each cell.


My reason for not using require is because I need parts of three.js (e.g orbit controls) which are not part of the main library, but are instead included as examples, and hence are not hosted on CDNs.
Also, my code for injecting javascript libraries is in a separate function which only needs to be called once at the start of the notebook, so it shouldn't need to be included more than once.

 
Aside for folks fiddling with Javscript libraries & the Web notebook:   the obvious approach of dropping a <script> reference in an HTML cell or result isn't robust.  When the cell is reexecuted, the browser will load the library again, overwriting the original library name space.  However, the first instance of the library will likely still be there because of lingering references.  So you'll have a variable "foo" of type X in the old library instance, and you'll pass it into a function call (new library instance) and things will fail in strange ways because all the type tests in the new library instance fail.  And you will drive yourself crazy because you'll be in the JS console and wonder why foo is clearly of type "X", but "isinstanceof X" is giving False.

That's why David's JS code has this test:

https://github.com/DavidPowell/OpenModes/blob/master/openmodes/static/three_js_plot.js#L175

       if ( camera instanceof THREE.Camera === false || ! document.body.contains(three_container)) {
            console.log("Animation loop failed: stopping");
            return;
        }


I agree that it's a horrible hack, I'm definitely looking for a more elegant long-term solution. This hack here isn't strictly needed for the code to work, but if it's missing, then the browser's javascript console will spew out error messages 60 times per second, which makes debugging javascript essentially impossible. I think I minimise some of the problems you describe by injecting the javascript libraries once at the start, rather than with every plot.
 

 I did find your examples online, but had real trouble getting your approach to work with the additional javascript from the three.js examples. So in the end I just went back to this simpler approach of directly injecting code.
 

For the dev team, this raises the issue of:  what's the best idiom for JS in a cell to access the output area?  (See SO question:  http://stackoverflow.com/questions/20019690/how-to-access-current-cell-output-area-with-javascript-in-ipython-web-notebook)

On a "live" notebook, the Javascript element variable is fine.  But as you can see with my notebook (above) vs David's, the element variable approach doesn't survive nbconvert -- my Javascript examples break because element doesn't exist in an nbconvert(ed) notebook.

David made things work by generating (effectively) a UID as part of his HTML cell result that the corresponding Javascript can pick up.

As we see Javascript used more with the Web notebook, folks are going to consistently run into this pattern.  I think the notebook could provide some "standard" way to reference the output area for the cell, that survives nbconvert.  

I agree, you've exactly described the problems I ran into, and the reason why I generate html with a <script> tag rather than javascript output.
 

What do you think?

-andy



_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|

Re: WebGL example in IPython notebook based on three.js

David Powell
In reply to this post by rossant
Hello Cyrille,

Thanks for the info. In case it helps you, the reasons why I didn't go with vispy are
- When I started this coding, the IPython notebook examples for vispy weren't readily available (I vagely remember failing to find them with google even though something was mentioned on the home page)
- The idea that you *have* to write low-level GLSL code seems a bit intimidating (with three.js you can, but don't need to)
- three.js includes nice controls to pan, zoom etc. out of the box
- Because I'm trying to attract users to my software, I think it's absolutely essential to be able to use the nbconverted output to show nice interactive plots to potential users who haven't installed my program yet (and who may not even be familiar with the scipy ecosystem), as a kind of "marketing" tool.

However, I'll definitely keep an eye out for progress on vispy, it sounds like some cool features are planned. The one thing I don't like about three.js is that it is very much fixed around the idea of always animating at 60 frames/sec for games etc. This means that it is chewing up significant CPU, even if the figure if not updating.

regards
David

On 19 January 2015 at 23:30, Cyrille Rossant <[hidden email]> wrote:
Thanks for sharing this!

I'd like to add that there's also work being done currently for
bringing WebGL visualization to the notebook via VisPy. We don't use
three.js but custom WebGL code generated by Python. The WebGL backend
currently requires a live Python server, but a standalone version is
also in the pipes.

2015-01-19 12:08 GMT+01:00 Matthias BUSSONNIER <[hidden email]>:
> Hi all,
> Le 19 janv. 2015 à 07:50, Soumith Chintala a écrit :
>
> This is fantastic! I was going to do some of this work myself over the next
> month for iTorch (https://github.com/facebook/iTorch), and you saved me some
> trouble.
> I will take a look at your code over the next couple of days, I am wondering
> if you would have any issues if I directly adapted your javascript side of
> things in iTorch rather than rewriting from scratch.
>
>
> We still haven't really figured it out how to share javascript in between
> packages of different languages.
> (well technically in 3.0 yo can install into $NBEXTENSION directory, but
> install process is not perfect)
> If you can figured that out, and that all kernels agree on a set of
> relatively common JS "plugins"  that
> are often used we might be able to work on making  theses plugin available
> on nbviewer.
>
> Long term planning of course, but I'd like people to keep that in mind.
>
> --
> M
>
>
> _______________________________________________
> IPython-dev mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev


_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|

Re: WebGL example in IPython notebook based on three.js

Sylvain Corlay
@David, PyThreejs works pretty well with the current dev version of IPython (3.0dev). Older versions should not be supported as it uses the new API with the javascript promises. (Ping Jason Grout). 

Sylvain

On Mon, Jan 19, 2015 at 9:44 PM, David Powell <[hidden email]> wrote:
Hello Cyrille,

Thanks for the info. In case it helps you, the reasons why I didn't go with vispy are
- When I started this coding, the IPython notebook examples for vispy weren't readily available (I vagely remember failing to find them with google even though something was mentioned on the home page)
- The idea that you *have* to write low-level GLSL code seems a bit intimidating (with three.js you can, but don't need to)
- three.js includes nice controls to pan, zoom etc. out of the box
- Because I'm trying to attract users to my software, I think it's absolutely essential to be able to use the nbconverted output to show nice interactive plots to potential users who haven't installed my program yet (and who may not even be familiar with the scipy ecosystem), as a kind of "marketing" tool.

However, I'll definitely keep an eye out for progress on vispy, it sounds like some cool features are planned. The one thing I don't like about three.js is that it is very much fixed around the idea of always animating at 60 frames/sec for games etc. This means that it is chewing up significant CPU, even if the figure if not updating.

regards
David

On 19 January 2015 at 23:30, Cyrille Rossant <[hidden email]> wrote:
Thanks for sharing this!

I'd like to add that there's also work being done currently for
bringing WebGL visualization to the notebook via VisPy. We don't use
three.js but custom WebGL code generated by Python. The WebGL backend
currently requires a live Python server, but a standalone version is
also in the pipes.

2015-01-19 12:08 GMT+01:00 Matthias BUSSONNIER <[hidden email]>:
> Hi all,
> Le 19 janv. 2015 à 07:50, Soumith Chintala a écrit :
>
> This is fantastic! I was going to do some of this work myself over the next
> month for iTorch (https://github.com/facebook/iTorch), and you saved me some
> trouble.
> I will take a look at your code over the next couple of days, I am wondering
> if you would have any issues if I directly adapted your javascript side of
> things in iTorch rather than rewriting from scratch.
>
>
> We still haven't really figured it out how to share javascript in between
> packages of different languages.
> (well technically in 3.0 yo can install into $NBEXTENSION directory, but
> install process is not perfect)
> If you can figured that out, and that all kernels agree on a set of
> relatively common JS "plugins"  that
> are often used we might be able to work on making  theses plugin available
> on nbviewer.
>
> Long term planning of course, but I'd like people to keep that in mind.
>
> --
> M
>
>
> _______________________________________________
> IPython-dev mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev


_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev



_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|

Re: WebGL example in IPython notebook based on three.js

rossant
In reply to this post by David Powell
Thank you for your feedback. VisPy/WebGL is definitely not yet ready
for production, I was essentially mentioning that there was work in
progress in that direction!

> - When I started this coding, the IPython notebook examples for vispy
> weren't readily available (I vagely remember failing to find them with
> google even though something was mentioned on the home page)

It's still experimental at this point, but a more robust version will
be available in the coming weeks.

> - The idea that you *have* to write low-level GLSL code seems a bit
> intimidating (with three.js you can, but don't need to)

That's actually not the case in the latest version, see for example
<http://vispy.org/examples/basics/scene/surface_plot.html>. We do have
high-level interfaces that don't require users to write GLSL. Only the
most advanced users will be interested in writing custom GLSL.

> - three.js includes nice controls to pan, zoom etc. out of the box

Idem, in VisPy we do have interactive "cameras" that provide controls
to pan, zoom, etc.

> - Because I'm trying to attract users to my software, I think it's
> absolutely essential to be able to use the nbconverted output to show nice
> interactive plots to potential users who haven't installed my program yet
> (and who may not even be familiar with the scipy ecosystem), as a kind of
> "marketing" tool.

That's a good point and that's definitely something on our radar.

> However, I'll definitely keep an eye out for progress on vispy, it sounds
> like some cool features are planned. The one thing I don't like about
> three.js is that it is very much fixed around the idea of always animating
> at 60 frames/sec for games etc. This means that it is chewing up significant
> CPU, even if the figure if not updating.

VisPy is indeed more efficient in that the scene is only updated when
needed (pan, zoom, etc.). But you can also have timer-based
animations.

Cyrille
_______________________________________________
IPython-dev mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/ipython-dev