ipyparallel: local controller / remote engines

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

ipyparallel: local controller / remote engines

jeromeku
Hi,

I'm trying to set up a distributed ipyparallel / ipython cluster.  I'd like to run the controller locally on my laptop with engines running remotely on Amazon ec2 (without resorting to Starcluster).

I've read the latest documentation on setting up ssh ipython with local / remote hosts and tried the following:
-Generated 'parallel' profile using "ipython create profile --parallel..."
-In ipcluster_config.py:
  -set SSHEngineSetLauncher.engines dictionary to my ec2 instance address and engine count, as well as set the SSHEngineLaunch.user name appropriately.
  -created an alias to launch ipengine via ssh on the ec2 instance and set the SSHEngineSetLauncher.engine_cmd accordingly.
-In ipcontroller_config.py, set HubFactory.ip to '*' as well as in ipengine_config.py.
-Set up password-less ssh login to my ec2 instance

Launching ipcluster from my laptop with this configuration and setting log-level to debug, I see that my local ipython profile is successfully transferred to my ec2 instance (along with the necessary .json security files) and that the ipengine command is successfully executed remotely.  However, this results in a "Registration timeout”, as the ip address in the transferred ipcontroller_engine.json file is set to the local (as opposed to public) IP address of my laptop.

When I manually set HubFactory.ip in my ipcluster_config.py to the public IP of my laptop and re-launch ipcluster, I get a ZeroMQ error "Can't assign requested address.”

My question is, what’s the proper way to set the controller IP / location such so that remote engines (on AWS ec2, different LAN) can connect?

Best,
Jerome
Reply | Threaded
Open this post in threaded view
|

Re: ipyparallel: local controller / remote engines

MinRK

On Tue, Feb 21, 2017 at 8:37 PM, jeromeku <[hidden email]> wrote:

Hi,

I'm trying to set up a distributed ipyparallel / ipython cluster.  I'd like
to run the controller locally on my laptop with engines running remotely on
Amazon ec2 (without resorting to Starcluster).

I've read the latest documentation on setting up ssh ipython with local /
remote hosts and tried the following:
-Generated 'parallel' profile using "ipython create profile --parallel..."
-In ipcluster_config.py:
  -set SSHEngineSetLauncher.engines dictionary to my ec2 instance address
and engine count, as well as set the SSHEngineLaunch.user name
appropriately.
  -created an alias to launch ipengine via ssh on the ec2 instance and set
the SSHEngineSetLauncher.engine_cmd accordingly.
-In ipcontroller_config.py, set HubFactory.ip to '*' as well as in
ipengine_config.py.
-Set up password-less ssh login to my ec2 instance

Launching ipcluster from my laptop with this configuration and setting
log-level to debug, I see that my local ipython profile is successfully
transferred to my ec2 instance (along with the necessary .json security
files) and that the ipengine command is successfully executed remotely.
However, this results in a "Registration timeout”, as the ip address in the
transferred ipcontroller_engine.json file is set to the local (as opposed to
public) IP address of my laptop.

When I manually set HubFactory.ip in my ipcluster_config.py to the public IP
of my laptop and re-launch ipcluster, I get a ZeroMQ error "Can't assign
requested address.”

My question is, what’s the proper way to set the controller IP / location
such so that remote engines (on AWS ec2, different LAN) can connect?

To connect engines to a controller over ssh you need the engines to be able to SSH to the controller (or at least its LAN), not just the other way around. That’s probably going to be trickiest bit if you want your controller on your laptop. Technically, the connection could be done with an ssh reverse tunnel, but IPython doesn’t have an option to do this for you automatically.

In general, when engines connect over ssh, you need this config:

# ipcontroller_config.py
# Tell the engines to connect over SSH:
c.IPControllerApp.engine_ssh_server = '[user@]host[:port]'
# If engines can only ssh to the controller's LAN, but not all the way to the controller's host,
# it must also listen on external interfaces:
c.HubFactory.ip = '*' # or LAN ip on the controller

It would probably be easiest, if it is acceptable, to run the controller on the EC2 server as well. There would be a lot fewer tunnelled connections this way. Plus, the SSHProxy launchers can work quite a bit better than plain SSH for multiple engines on a single remote host:

# ipcluster_config.py
ssh_target = 'user@ec2host'

c.IPClusterStart.controller_launcher_class = 'SSH'
c.SSHControllerLauncher.controller_args.append('--ssh=' + ssh_target)
c.SSHControllerLauncher.location = ssh_target
c.IPClusterEngines.engine_launcher_class = 'SSHProxy'
c.SSHProxyEngineSetLauncher.location = ssh_target

-Min


Best,
Jerome



--
View this message in context: http://python.6.x6.nabble.com/ipyparallel-local-controller-remote-engines-tp5209969.html
Sent from the IPython - User mailing list archive at Nabble.com.
_______________________________________________
IPython-User mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/ipython-user


_______________________________________________
IPython-User mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/ipython-user