Open telemetry integration with ray

How severe does this issue affect your experience of using Ray?

  • Low: It annoys or frustrates me for a moment.

I integrated ray with open telemetry as stated in Tracing — Ray 1.12.1

I would like to know how to do the following:

  1. is there a possibility to define exporter in environment, and not in code (tracing startup hook)?
    for my other processes I can put the following
  1. How can I get the context to be propagated from the ray client to the automatic instrumented traces in ray? The traces that I see don’t seem to have a parent.

  2. I’ve added manual spans in my actors, in order to see the span in zipkin I need to make sure the actor is alive at least 4 seconds after the function has returned a results. Does it make any sense?

Hi @shiranbi,

Thanks for trying it out!

  1. is there a possibility to define exporter in environment, and not in code (tracing startup hook)?

You can do that by implementing your own hook and read the config file or environment variable. In fact, we would even appreciate a PR to Ray if you have this kind of general purpose tracing hook working.

  1. How can I get the context to be propagated from the ray client to the automatic instrumented traces in ray? The traces that I see don’t seem to have a parent.

Unfortunately we did not include Ray Client in scope for tracing when we originally designed this. I’m grateful that you have tried this out. Can you open a feature request in Ray repo? Please also describe how critical this feature is for you.

I’ve added manual spans in my actors, in order to see the span in zipkin I need to make sure the actor is alive at least 4 seconds after the function has returned a results.

I believe this has to do with OpenTelemetry implementation (either in the SDK level or Zipkin exporter level). We directly sends to the span to OpenTelemetry; there must be some buffering happening within the libraries. I would encourage a bug report to their repo instead.

Thank you,
Simon Mo