Test coverage for ray serve remote calls

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

When writing tests I like to keep track of my code coverage using pytest-cov (which itself uses coverage). However, any code that gets executed behind a some_deployment.remote-call does not appear in my code coverage (even though I can guarantee it ran) which makes the coverage look atificially low (and dependending on the setup might lead to failed quality gates in my CI/CD-pipeline).

I tried manually starting the coverage as described here and I also tried to manually start the coverage in a worker setup function but none of this worked.

According to this issue one can get proper line coverage by setting local_mode=True. However, this does not work with async actors and also the local_mode is deprecated.

Is there any way to get proper line coverage for tests that do remote calls?

1 Like