ipython1 and mpi

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

ipython1 and mpi

Ondrej Marsalek-2
hi,

i would like to test ipython1 together with mpi and one thing is a bit
confusing for me. the faq says:

"In fact, IPython Engines can be started as MPI processes (they can
call MPI_Init) and you can then use MPI however you want."

does that mean that it is my option to call mpi_init by hand or does
it somehow happen behind the scenes? i understand that i have to use
mpirun to launch the engines.

i would like to use mpi from within an extension wrapped using swig
but i suppose that this is not a problem.

regards,
ondrej
_______________________________________________
IPython-dev mailing list
[hidden email]
http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
Reply | Threaded
Open this post in threaded view
|

Re: ipython1 and mpi

Brian Granger-2
Ondrej,

Here is a broad picture of how mpi integration works:

1) When the engines start, the look for a configuration variable
called mpiImportStatement.  If it is set, that strings gets exec'd the
first thing when the engine starts.  This statement should be a python
statement that causes mpi_init to be called.  A typically example is:

mpiImportStatement = "from mpi4py import MPI as mpi"

The docstrings for mpiImportStatement are here:

http://projects.scipy.org/ipython/ipython/browser/ipython/branches/saw/ipython1/config/objects.py

To set this variable, use the file mpirc.py as a template and drop it
into your ~/.ipython directory.

> does that mean that it is my option to call mpi_init by hand or does
> it somehow happen behind the scenes? i understand that i have to use
> mpirun to launch the engines.

2) Then you must start the engines using

To use mpi from within swig wrapped code there are a few things you
will have to figure out:

1) Does that wrapped code call mpi_init, or does it expect that
mpi_init has already been called?

2) Which mpi implementation are you using?  With a modern mpi
implementation like openmpi, things are fairly easy.  If you are using
an older version like mpich1, like becomes much more complicated
because of how mpich starts processes.  I can show you how to get that
going if you need to.

Here is how I would go about things:

* Use openmpi and install the mpi4py python bindings to mpi

* Set the mpiImportStatement to import mpi4py.  This will cause
mpi_init to be called.

* Make sure that your wrapped code doesn't call mpi_init again.

With that, things should just work.  Let me know how this goes - I
realize all of this is still somewhat complicated.

Brian


>
> regards,
> ondrej
> _______________________________________________
> IPython-dev mailing list
> [hidden email]
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>
_______________________________________________
IPython-dev mailing list
[hidden email]
http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev