Replica initialize is too slow with load_model into replica init

Ajay_Kumar · June 28, 2023, 8:53am

Hi All,

Replica initialize too slow for 1st time loading checkpoint into init.
Is Ray cache something for loading checkpoints into replica?
Are any methods available to stop caching?

ray==2.4.0

Topic		Replies	Views
Deployment has taken more than 30s to initialize. This may be caused by a slow __init__ or reconfigure method Ray Serve	2	138	September 16, 2024
Deployment's init function takes too long to load model Ray Serve	4	541	April 12, 2023
Ray Serve Replica taking a lot of memory before requests even come in Ray Serve	3	492	September 29, 2021
Long initialization time to initialize_session with large scale dataset Ray Train	3	31	January 29, 2025
Problem with anything on Ray Ray Data	2	597	April 20, 2022

Replica initialize is too slow with load_model into replica init

Related topics