Currently, it is known that the request is obtained through TaskSpecification::GetDependencyIds(), and TaskSpecification is the wrapper of the RPC message of TaskSpec. However, where is the RPC request sent? So where is this dependency stored? Who maintained it? Is it the ownership table?
Most of the information is stored in the local worker that is submitting the task, the latest whitepaper contains the most-to-date description regarding object ownership, which also reveals the call patterns. Hope that helps answer the questions.
Ownership
Task scheduling (object transfer example)
@ClarenceNg
Thanks! That is to say, the objects on which the task depends are managed by the ownership table of the owner. While scheduling a task, only need to query the local ownership table to obtain the object on which the task depends and wait for whether the object already exists. So, why do we need the RPC request, is it because the worker process is different from the owner processes and needs RPC to implement inter-process communication? Btw, does the Object Table stored on GCS?
The table is a distributed table, and might not be on the same node the query sent.
Besides, there is also reference counting to track the object’s lifecycle.
ObjectTable are stored in raylet (object store) and ownership table is stored in worker.