[IPython-User] Getting started: Call another function in parallel

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[IPython-User] Getting started: Call another function in parallel

Florian Lindner
Hello,

I'm a bit lost on how to get started using the parallel ipython stuff:

I have some code:

from ipyparallel import Client

c = Client()
lview = c.load_balanced_view()

def g(x):
    return x + 4

@lview.parallel()
def f(x):
    import g
    return 5 + g(x)

res = f.map(range(5))

print(res)
print(res.get_dict())


but this prints:

<AsyncMapResult: f>
Traceback (most recent call last):
  File "iparallel.py", line 22, in <module>
    print(res.get_dict())
  File "/usr/lib/python3.5/site-packages/ipyparallel/client/asyncresult.py", line 256, in get_dict
    results = self.get(timeout)
  File "/usr/lib/python3.5/site-packages/ipyparallel/client/asyncresult.py", line 166, in get
    raise self.exception()
  File "/usr/lib/python3.5/site-packages/ipyparallel/client/asyncresult.py", line 225, in _resolve_result
    results = error.collect_exceptions(results, self._fname)
  File "/usr/lib/python3.5/site-packages/ipyparallel/error.py", line 233, in collect_exceptions
    raise e
  File "/usr/lib/python3.5/site-packages/ipyparallel/error.py", line 231, in collect_exceptions
    raise CompositeError(msg, elist)
ipyparallel.error.CompositeError: one or more exceptions from call to method: f
[2:apply]: ImportError: No module named 'g'
[3:apply]: ImportError: No module named 'g'
[0:apply]: ImportError: No module named 'g'
[1:apply]: ImportError: No module named 'g'
.... 1 more exceptions ...


How can I access other functions in my module?

Thanks,
Florian
_______________________________________________
IPython-User mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/ipython-user
Reply | Threaded
Open this post in threaded view
|

Re: Getting started: Call another function in parallel

Thomas Kluyver-2
Hi Florian,

'import' only works with modules. Have a look at the docs here for ideas on how to get your function to the workers:

http://ipyparallel.readthedocs.io/en/latest/multiengine.html#moving-python-objects-around

Thomas

On 12 December 2016 at 08:33, Florian Lindner <[hidden email]> wrote:
Hello,

I'm a bit lost on how to get started using the parallel ipython stuff:

I have some code:

from ipyparallel import Client

c = Client()
lview = c.load_balanced_view()

def g(x):
    return x + 4

@lview.parallel()
def f(x):
    import g
    return 5 + g(x)

res = f.map(range(5))

print(res)
print(res.get_dict())


but this prints:

<AsyncMapResult: f>
Traceback (most recent call last):
  File "iparallel.py", line 22, in <module>
    print(res.get_dict())
  File "/usr/lib/python3.5/site-packages/ipyparallel/client/asyncresult.py", line 256, in get_dict
    results = self.get(timeout)
  File "/usr/lib/python3.5/site-packages/ipyparallel/client/asyncresult.py", line 166, in get
    raise self.exception()
  File "/usr/lib/python3.5/site-packages/ipyparallel/client/asyncresult.py", line 225, in _resolve_result
    results = error.collect_exceptions(results, self._fname)
  File "/usr/lib/python3.5/site-packages/ipyparallel/error.py", line 233, in collect_exceptions
    raise e
  File "/usr/lib/python3.5/site-packages/ipyparallel/error.py", line 231, in collect_exceptions
    raise CompositeError(msg, elist)
ipyparallel.error.CompositeError: one or more exceptions from call to method: f
[2:apply]: ImportError: No module named 'g'
[3:apply]: ImportError: No module named 'g'
[0:apply]: ImportError: No module named 'g'
[1:apply]: ImportError: No module named 'g'
.... 1 more exceptions ...


How can I access other functions in my module?

Thanks,
Florian
_______________________________________________
IPython-User mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/ipython-user


_______________________________________________
IPython-User mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/ipython-user