How to securely allow user-provided low-end computers to join a Ray cluster as workers only (not as device/drivers)?
Body:
I have the following requirement and some security concerns:
We would like to enable our users to contribute their idle, low-spec computers to our Ray cluster as worker nodes, so they can participate in distributed computation tasks. These user devices should only be passive workers—meaning, they only execute tasks dispatched by our platform. They should not be able to access other cluster resources on their own, nor should they be able to register as device/driver nodes. Essentially, we want users to only provide compute power.
Our ideal setup is that user computers can automatically and securely join our cluster over the public Internet (possibly behind NAT/firewalls). The platform centrally schedules and assigns tasks; users only contribute compute, and we compensate them based on their resource contribution.
The challenge:
If we package the cluster’s IP, port, and connection password (e.g., the Redis password) into the client application for users to download, it is trivial for users to extract these credentials (via reverse engineering or sniffing traffic). This means they could bypass our platform and use Ray’s Python SDK to connect directly to the cluster, potentially registering as device/driver nodes or accessing additional resources. This is obviously insecure.
So, my main question is:
How can we design a secure mechanism for onboarding these user-provided workers, ensuring they can only join as passive workers, and cannot escalate privileges or become device/driver nodes? For example:
Restrict node role/permissions at the Ray cluster level
Strengthen communication encryption and node authentication
Any best practices for securely onboarding external, untrusted worker nodes?
Is there any way to separate the control plane and data plane, or use an intermediate proxy for further isolation?
I’d really appreciate any community insights or recommendations for mature solutions in this area. Thanks in advance!