Using ray.cloudpickle methods in the serve deployment - error

As soon as I use ray.cloudpickle.dumps/loads methods, the error about Ray Client is not connected is reported.

Ray Client is not connected. Please connect by calling `ray.init`.

Env: a ray cluster (head with 2 workers) on k3s installed with ray operator. Code is run from a juypyter notebook on the ray head.

In contrast, running similar code on a local machine ray does not have issue using ray.cloudpickle methods.

Here’s the serve deployment. In the current state, it produces the error. If I comment out the lines with cloudpickle and run the ones with ray.get/put - no problems.

import ray
from ray import serve
import json

# Connect to cluster
    # runtime_env = {"pip":["ruptures","pathos"]}
    ray_client = ray.init(f"ray://localhost:10001", namespace="serve")
except Exception as e:
    print("Ray serve started")
    client = serve.start(detached=True, http_options={"host": ""})

class LastChunker:
    async def __call__(self, request):
            raw_json = await request.json()
            id_str = raw_json.get('store_id')
            id_pkld = base64.b64decode(id_str.encode())
            id_restored = ray.cloudpickle.loads(id_pkld)
            ray_data = ray.get(id_restored
            # one = ray.put(1)
            return {"ray_data": ray_data}
            # return {"ray_data": ray.get(one)}
        except Exception as ex:
            return {"error": ex}


Here is the calling code:

import requests
import base64

f = ray.put("foo")
print(f"f: {f}")
fpikl = ray.cloudpickle.dumps(f)
print(f"fpikl: {fpikl}")

b64fpkl = base64.b64encode(fpikl).decode()
print(f"b64fpkl: {b64fpkl}")

data='{"store_id": "' + f"{b64fpkl}" +'"}'
print(f"data: {data}")

id_str = json.loads(data).get('store_id')
print (f"id_str: {id_str}")

id_pkld = base64.b64decode(id_str.encode())
print(f"id_pkld: {id_pkld}")

id_restored = ray.cloudpickle.loads(id_pkld)
print(f"id_restored: {id_restored}")

resp = requests.get("http://localhost:8000/lastchunker", data=data)

I was able to get past the Ray is not connected error by calling ray.shutdown() after deploying the model and then doing ray.init() in the calling code.

The ray.cloudpickle.get() call works, but the ray.get() is not able to find the object, but rather now have the error

(raylet, ip= [2022-03-04 10:07:57,747 E 127 127] 
Couldn't send pull request from 
of object ffffffffffffffffffffffffffffffffffffffff0100000001000000 , 
setup rpc connection failed.

I think I figured out why the objects cannot be found… SHHH… Don’t tell anyone. The python that was running the client code had not called ray start to connect to the cluster.