The recommended approach for building a connection pool in Ray is to use a custom Actor class that manages the pool, rather than the ActorPool utility. ActorPool is designed for distributing tasks across a fixed set of actors, not for managing shared state like a connection pool. By implementing your own ConnectionPool as a Ray actor, you can encapsulate connection management logic, ensure thread safety, and expose methods for acquiring and releasing connections as needed in a production environment. This pattern is well-supported and commonly used in Ray for resource management scenarios such as database connection pools, as discussed in this Ray forum thread.
ActorPool is best suited for load-balancing stateless tasks across multiple actors, not for sharing or synchronizing access to a set of resources. For a connection pool, you want a single actor (or one per node) that holds and manages the pool, providing methods to lease and return connections. Would you like a code example of how to implement such a ConnectionPool actor?
Sources:
Hint: Mention @RunLLM in the post for followups.