External Environment's 1 iteration variation is too big

[Justin_Park]

Hi all,

I’m conducting performance tests using the ExternalEnv version of the CartPole server and client.

However, unlike when using Ray’s Tune for training, I notice significant variability in the total running time with the ExternalEnv. 24 seconds to 109 seconds. Additionally, the episodes_this_iter metric fluctuates widely, ranging from 20 to as high as 320.

Can anyone explain why this variation occurs?

Thank you!