Hi All,
I am trying to integrate Datadog to Ray Serve for the monitoring, In Datadog documentation they are asking to add annotation to ray pods, I have tried adding these annotation at service level
apiVersion: ray.io/v1alpha1
kind: RayService
metadata:
name: {{ include "rayservice.fullname" . }}
namespace: {{ .Values.namespace }}
annotations:
ray.io/ft-enabled: "true"
ad.datadoghq.com/ray.checks: |-
{
"ray": {
"instances": [
{
"openmetrics_endpoint": "http://%%host%%:8080"
}
]
}
}
And also in the head node configuration,
headGroupSpec:
# The `rayStartParams` are used to configure the `ray start` command.
# See https://github.com/ray-project/kuberay/blob/master/docs/guidance/rayStartParams.md for the default settings of `rayStartParams` in KubeRay.
# See https://docs.ray.io/en/latest/cluster/cli.html#ray-start for all available options in `rayStartParams`.
rayStartParams:
dashboard-host: '0.0.0.0'
#pod template
template:
metadata:
annotations:
ad.datadoghq.com/ray.checks: |-
{
"ray": {
"instances": [
{
"openmetrics_endpoint": "http://%%host%%:8080"
}
]
}
}
spec:
I might be missing something, or using incorrect way to add annotation to Ray pods, can some help me here in setting up the annotation in a correct way.
Note: The first method is not showing annotation in pod description, whereas the second method does.
Datadog Integration Documentation: Ray