Serve deploy app support custom router with runtime_env

Ray Serve
1

1. Severity of the issue: (select one)
Medium: Significantly affects my productivity but can find a workaround.

2. Environment:

  • Ray version:2.50.1
  • Python version:3.12
  • OS:ubuntu
  • Cloud/Infrastructure:gcp
  • Other libs/tools (if relevant):

3. What happened vs. what you expected:

  • Expected:

In my app, one of the deployments uses a request_router_config. The application code is packaged into a zip file and stored on GCS. The runtime_env is configured with the remote code URI for this package: gs://xxxxx. The config.yaml used to deploy the Serve application is roughly as follows:

name: g2
route_prefix: /g2
import_path: main:main
runtime_env:
  env_vars:
    PYTHONPATH: .:../proto
    RAY_APP_NAME: g2
  py_executable: uv run --directory ./app
  working_dir: gs://ai/build/test-26/g2.zip
deployments:
  - name: FastAPIWrapper
    num_replicas: 1
    ray_actor_options:
      num_cpus: 1
    max_replicas_per_node: 1

deploymen code:

@serve.deployment(
    max_ongoing_requests=settings.MAX_ONGOING_REQUESTS,
    request_router_config=RequestRouterConfig(
        request_router_class=ConversateRouter
    ),
    user_config={},
)
class MyDeployment(DeploymentMixin):
    ...

I hope it can be deploy

  • Actual:

The following error occurs because the ServeController runs as a separate process, and its Python search path doesn’t contain the path to the APP code.

INFO 2025-10-29 12:17:39,798 controller 5329 – Imported and built app ‘g2’ successfully.
ERROR 2025-10-29 12:17:39,814 controller 5329 – Unexpected error occurred while applying config for application ‘g2’:
Traceback (most recent call last):
File “/usr/local/lib/python3.12/dist-packages/ray/serve/_private/application_state.py”, line 705, in _reconcile_build_app_task
params[“deployment_name”]: deploy_args_to_deployment_info(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.12/dist-packages/ray/serve/_private/deploy_utils.py”, line 66, in deploy_args_to_deployment_info
deployment_config = DeploymentConfig.from_proto_bytes(deployment_config_proto_bytes)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.12/dist-packages/ray/serve/_private/config.py”, line 358, in from_proto_bytes
return cls.from_proto(proto)
^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.12/dist-packages/ray/serve/_private/config.py”, line 325, in from_proto
data[“request_router_config”] = RequestRouterConfig(
^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.12/dist-packages/ray/serve/config.py”, line 136, in init
self._serialize_request_router_cls()
File “/usr/local/lib/python3.12/dist-packages/ray/serve/config.py”, line 152, in _serialize_request_router_cls
request_router_class = import_attr(request_router_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.12/dist-packages/ray/_common/utils.py”, line 45, in import_attr
module = importlib.import_module(module_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3.12/importlib/init.py”, line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “”, line 1387, in _gcd_import
File “”, line 1360, in _find_and_load
File “”, line 1310, in _find_and_load_unlocked
File “”, line 488, in _call_with_frames_removed
File “”, line 1387, in _gcd_import
File “”, line 1360, in _find_and_load
File “”, line 1310, in _find_and_load_unlocked
File “”, line 488, in _call_with_frames_removed
File “”, line 1387, in _gcd_import
File “”, line 1360, in _find_and_load
File “”, line 1324, in _find_and_load_unlocked
ModuleNotFoundError: No module named ‘business’

While setting the PYTHONPATH environment variable in the container or copying the code into the image during its build could solve this issue, I’d rather not do so. It would defeat the core principle of runtime_env, which is designed for flexible multi-app deployment and environment isolation.

2

This is a known limitation: when using request_router_config with a custom class, Ray Serve’s controller process must be able to import the class at config application time, but the controller does not run in the deployment’s runtime_env, so it cannot see code only present in the remote working_dir (e.g., GCS zip) or custom PYTHONPATH. This results in ModuleNotFoundError, as you observed. The Ray team has discussed this issue and is working on safer import mechanisms, but currently, the only reliable workaround is to ensure the custom router class is importable by the controller—either by including it in the container image or making it available in the controller’s PYTHONPATH. Relying solely on runtime_env for code isolation does not work for request_router_config at this time. See the discussion and technical details in this Ray PR.

Would you like a step-by-step explanation of why this happens and possible workarounds?

Sources:

Hint: Mention @RunLLM in the post for followups.