xyzpy.gen.ray_executor
#
Module Contents#
Classes#
Basic |
|
Basic |
|
A |
Functions#
|
|
Allows passing futures by reference - takes e.g. args and kwargs and |
|
|
Cached retrieval of remote function. |
|
Alternative for 'non-function' callables - e.g. partial |
Attributes#
- class xyzpy.gen.ray_executor.RayFuture(obj)[source]#
Basic
concurrent.futures
like future wrapping a rayObjectRef
.- __slots__ = ('_obj', '_cancelled')#
- xyzpy.gen.ray_executor._unpack_dispatch#
- xyzpy.gen.ray_executor._unpack_futures(x)[source]#
Allows passing futures by reference - takes e.g. args and kwargs and replaces all
RayFuture
objects with their underylingObjectRef
within all nested tuples, lists and dicts.[Subclassing
ObjectRef
might avoid needing this.]
- xyzpy.gen.ray_executor.get_remote_fn(fn, **remote_opts)[source]#
Cached retrieval of remote function.
- xyzpy.gen.ray_executor.get_deploy(**remote_opts)[source]#
Alternative for ‘non-function’ callables - e.g. partial functions - pass the callable object too.
- class xyzpy.gen.ray_executor.RayExecutor(*args, default_remote_opts=None, **kwargs)[source]#
Basic
concurrent.futures
like interface usingray
.Example usage:
from xyzpy import RayExecutor # create a pool that by default requests a single gpu per task pool = RayExecutor( num_cpus=4, num_gpus=4, default_remote_opts={"num_gpus": 1}, )
- _maybe_inject_remote_opts(remote_opts=None)[source]#
Return the default remote options, possibly overriding some with those supplied by a
submit call
.
- submit(fn, *args, pure=False, remote_opts=None, **kwargs)[source]#
Remotely run
fn(*args, **kwargs)
, returning aRayFuture
.
- class xyzpy.gen.ray_executor.RayGPUExecutor(*args, gpus_per_task=1, **kwargs)[source]#
Bases:
RayExecutor
A
RayExecutor
that by default requests a single gpu per task.