I am looking at Ray Execution DAG for Functions. So does this IR node work effectively when you know your input values in advance? Does static compilation and IR of node depend on knowing the input values in advance? I have an offline inference workflow where I am creating a ray actor that initializes vLLM instance and does the inference upon receiving input. Most the values will be available at run time when someone invokes that inference workflow. Can I use compiled graphs in this case?
You do not need to know your input values in advance to use Ray DAGs or Compiled Graphs. The IR (intermediate representation) nodes created by .bind() on ray.remote functions or actors define the computation graph statically, but the actual input values are provided at execution time via dag_node.execute()—the graph structure is fixed, but the data can vary per execution. This is well-suited for inference workflows where inputs arrive at runtime, such as your vLLM actor scenario. Compiled Graphs further optimize repeated execution of the same static graph, but do not require static input values—just a static computation structure (Ray DAG docs, Ray Compiled Graph quickstart).
In your offline inference workflow, you can define the actor and its method as part of a Ray DAG, compile it, and then execute it with different inputs as they arrive. The static compilation is about the computation graph, not the data; so your use case is supported and will benefit from the performance optimizations of compiled graphs (Ray Compiled Graph API).
Would you like a step-by-step example for your vLLM inference workflow?
@RunLLM So if my Actor takes 5 params during init method and having different types like str, List, dict etc, how does it work with InputNode() as mentioned in the following code.
with InputNode() as dag_input:
a_ref = a.bind(dag_input)