Use list of DAGNode objects to create Dataset

Ray Libraries (Data, Train, Tune, Serve) Ray Data
1

Is there an API similar to ds = ray.data.from_pandas_refs(refs) but for a list of DAGNode objects (each returning pandas df)?

2

We don’t have an explicit API, but something like this should work:

pandas_df_refs = [node.execute() for node in nodes_list]
ds = ray.data.from_pandas_refs(pandas_df_refs)
3

Yeah but this kills the purpose of using DAGNodes as I want the data frames to be loaded lazily and execute corresponding DAGNode only if it’s needed.