Spilling configuration with ray up and ray in Kubernetes

Ray Core
1

Dear all

I am configuring a ray cluster using a YAML file in two ways: either native (bare metal)
or as set of kubernetes PODs.
I have a question regarding spilling configuration, especially the path, in this context.

When I create a ray program without such a cluster, I can pass object spilling configuration
to ray.init(), such as type (filesystem) and spilling directory.

How can I specify spilling type (filesystem) and directory to use, when I create
the cluster using a YAML file and ray up?

How can I specify spilling directory within the container, when I create the ray
cluster using a YAML file and Kubernetes, as describend in the documentation?

I believe this would need to be specified somehow in the YAML file as well,
instead of for each program in ray.init().

Thanks a lot
Adrian

2

You need to specify the json-serialized string to ray start. For example,

ray start --num-cpus=0 --head --port=6379 --object-manager-port=8076 --object-store-memory=85899345920 --autoscaling-config=~/ray_bootstrap_config.yaml --system-config='{"automatic_object_spilling_enabled":true,"max_io_workers":1,"object_spilling_config":"{\"type\":\"filesystem\",\"params\":{\"directory_path\":\"/obj-data\"}}"}'
3

just to make it super clear here –
you can specify the above command in the yaml’s ray_start_commands.

1 Like
4

Thanks a lot! This helps!

Thanks
Adrian