# Confusing behavior in setting object store size **URL:** \ **Category:** Ray Data\ **Created:** [February 24, 2026, 5:28am UTC](https://discuss.ray.io/t/confusing-behavior-in-setting-object-store-size/23486 "2026-02-24T05:28:23Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![mk6](https://avatars.discourse-cdn.com/v4/letter/m/7993a0/32.png) [@mk6](https://discuss.ray.io/u/mk6)\ **Post date:** [February 24, 2026, 5:28am UTC](https://discuss.ray.io/t/confusing-behavior-in-setting-object-store-size/23486/1 "2026-02-24T05:28:23Z")
**1. Severity of the issue: (select one)** Low: Annoying but doesn’t hinder my work. **2. Environment:** - Ray version: 2.54.0 - Python version: 3.11.10 - OS: Ubuntu 22.04.3 LTS **3. What happened vs. what you expected:** ```auto ray.init( object_store_memory=400 * 1024**3, # Setting 400G ) ``` - Expected: getting 400G for object store - Actual: getting 200G The node has 2T memory, 1008G shm. I tried running `ray data` in a docker container with `"--shm-size=512gb"`. Setting `ray.init(object_store_memory=400 * 1024**3)` only gives me 200G object store. It’s a bit confusing ---
**Author:** ![RunLLM](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/runllm/32/8249_2.png) [@RunLLM](https://discuss.ray.io/u/RunLLM)\ **Post date:** [February 24, 2026, 5:28am UTC](https://discuss.ray.io/t/confusing-behavior-in-setting-object-store-size/23486/2 "2026-02-24T05:28:45Z")
Ray’s object store memory is capped at 200GB by default, even if you request more via `object_store_memory`, unless you explicitly override this limit. This cap is set by the constant `DEFAULT_OBJECT_STORE_MAX_MEMORY_BYTES` (200GB) in Ray’s source code. To allocate more than 200GB, you must set the environment variable `RAY_DEFAULT_OBJECT_STORE_MAX_MEMORY_BYTES` to your desired value before starting Ray, or modify the Ray source code constant directly. Just increasing `object_store_memory` in `ray.init()` is not sufficient if it exceeds the default cap. See [ray\_constants.py](https://github.com/ray-project/ray/blob/master/python/ray/_private/ray_constants.py) and [resource\_spec.py](https://github.com/ray-project/ray/blob/releases/2.37.0/python/ray/_private/resource_spec.py). Additionally, the actual object store memory cannot exceed the available shared memory (`/dev/shm`), so ensure your `--shm-size` is at least as large as your requested object store size. For more details on these caps and how Ray determines the final object store size, see [resource\_spec.py](https://github.com/ray-project/ray/blob/releases/2.37.0/python/ray/_private/resource_spec.py) and [services.py](https://github.com/ray-project/ray/blob/master/python/ray/_private/services.py). Would you like a step-by-step guide to override the 200GB cap? Sources: - [resource\_spec.py](https://github.com/ray-project/ray/blob/releases/2.37.0/python/ray/_private/resource_spec.py) - [services.py](https://github.com/ray-project/ray/blob/master/python/ray/_private/services.py) - [ray\_constants.py](https://github.com/ray-project/ray/blob/master/python/ray/_private/ray_constants.py) Hint: Mention @RunLLM in the post for followups. ---
**Author:** ![mk6](https://avatars.discourse-cdn.com/v4/letter/m/7993a0/32.png) [@mk6](https://discuss.ray.io/u/mk6)\ **Post date:** [February 24, 2026, 5:40am UTC](https://discuss.ray.io/t/confusing-behavior-in-setting-object-store-size/23486/3 "2026-02-24T05:40:56Z")
@RunLLM Setting neither env var works (`DEFAULT_OBJECT_STORE_MAX_MEMORY_BYTES=429496729600` or `RAY_DEFAULT_OBJECT_STORE_MAX_MEMORY_BYTES=429496729600`). still getting 200G ---
**Author:** ![RunLLM](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/runllm/32/8249_2.png) [@RunLLM](https://discuss.ray.io/u/RunLLM)\ **Post date:** [February 24, 2026, 5:41am UTC](https://discuss.ray.io/t/confusing-behavior-in-setting-object-store-size/23486/4 "2026-02-24T05:41:17Z")
Setting the environment variable `RAY_DEFAULT_OBJECT_STORE_MAX_MEMORY_BYTES` should override the 200GB cap, but it must be set in the environment before starting the Python process that launches Ray. If you set it after Python starts, it will not take effect. Also, ensure you use the correct variable name: `RAY_DEFAULT_OBJECT_STORE_MAX_MEMORY_BYTES` (not `DEFAULT_OBJECT_STORE_MAX_MEMORY_BYTES`). See [ray\_constants.py](https://github.com/ray-project/ray/blob/master/python/ray/_private/ray_constants.py). If you are still getting 200GB, double-check that the environment variable is exported in the shell before running your script, e.g., `export RAY_DEFAULT_OBJECT_STORE_MAX_MEMORY_BYTES=429496729600` and then run your Python script in the same shell. Would you like more troubleshooting steps? Sources: - [ray\_constants.py](https://github.com/ray-project/ray/blob/master/python/ray/_private/ray_constants.py) Hint: Mention @RunLLM in the post for followups. ---
**Author:** ![mk6](https://avatars.discourse-cdn.com/v4/letter/m/7993a0/32.png) [@mk6](https://discuss.ray.io/u/mk6)\ **Post date:** [February 24, 2026, 5:43am UTC](https://discuss.ray.io/t/confusing-behavior-in-setting-object-store-size/23486/5 "2026-02-24T05:43:18Z")
@RunLLM I ran `RAY_DEFAULT_OBJECT_STORE_MAX_MEMORY_BYTES=429496729600 python script.py`, a one-liner, which should work ---
**Author:** ![RunLLM](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/runllm/32/8249_2.png) [@RunLLM](https://discuss.ray.io/u/RunLLM)\ **Post date:** [February 24, 2026, 5:43am UTC](https://discuss.ray.io/t/confusing-behavior-in-setting-object-store-size/23486/6 "2026-02-24T05:43:33Z")
Your usage is correct, but Ray’s cap may still apply if the environment variable is not read early enough or if another limit (like `/dev/shm` size or Docker’s `--shm-size`) is lower. Also, Ray may cap the object store memory to the available shared memory or other internal checks, regardless of the environment variable. See [services.py](https://github.com/ray-project/ray/blob/master/python/ray/_private/services.py) and [ray\_constants.py](https://github.com/ray-project/ray/blob/master/python/ray/_private/ray_constants.py). If you have set `--shm-size` to at least 400GB and the environment variable before the Python process, but still see 200GB, it may be a hard-coded cap or a bug. Would you like a step-by-step debug checklist? Sources: - [services.py](https://github.com/ray-project/ray/blob/master/python/ray/_private/services.py) - [ray\_constants.py](https://github.com/ray-project/ray/blob/master/python/ray/_private/ray_constants.py) Hint: Mention @RunLLM in the post for followups. ---
**Author:** ![mk6](https://avatars.discourse-cdn.com/v4/letter/m/7993a0/32.png) [@mk6](https://discuss.ray.io/u/mk6)\ **Post date:** [February 24, 2026, 6:00am UTC](https://discuss.ray.io/t/confusing-behavior-in-setting-object-store-size/23486/7 "2026-02-24T06:00:28Z")
@RunLLM yes give me a checklist ---
**Author:** ![RunLLM](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/runllm/32/8249_2.png) [@RunLLM](https://discuss.ray.io/u/RunLLM)\ **Post date:** [February 24, 2026, 6:00am UTC](https://discuss.ray.io/t/confusing-behavior-in-setting-object-store-size/23486/8 "2026-02-24T06:00:55Z")
Here’s a checklist to debug why Ray object store memory is capped at 200GB: 1. **Set Environment Variable Before Python** : Ensure you run your script as 2. **Check `/dev/shm` Size** : Confirm your Docker container or host has `/dev/shm` at least as large as your requested object store (e.g., `df -h /dev/shm`). If not, increase with `--shm-size=400gb` ([services.py](https://github.com/ray-project/ray/blob/master/python/ray/_private/services.py)). 3. **Check Ray Version** : Make sure you are using Ray \>=2.37.0, as older versions may not respect the environment variable ([resource\_spec.py](https://github.com/ray-project/ray/blob/releases/2.37.0/python/ray/_private/resource_spec.py)). 4. **Check for Warnings/Logs** : Look for log messages about object store memory capping or warnings about `/dev/shm` in your Ray logs or stdout ([services.py](https://github.com/ray-project/ray/blob/master/python/ray/_private/services.py)). 5. **Verify in Python** : Print the value inside your script to confirm Ray sees the variable: 6. **Check Actual Allocation** : After `ray.init()`, check the reported object store memory: 7. **Check for Other Limits** : Ray may still cap the object store to the minimum of the environment variable, `/dev/shm`, and internal defaults ([resource\_spec.py](https://github.com/ray-project/ray/blob/releases/2.37.0/python/ray/_private/resource_spec.py)). If all steps are correct and it still caps at 200GB, it may be a Ray bug or a hard-coded limit in your version. Sources: - [services.py](https://github.com/ray-project/ray/blob/master/python/ray/_private/services.py) - [resource\_spec.py](https://github.com/ray-project/ray/blob/releases/2.37.0/python/ray/_private/resource_spec.py) Hint: Mention @RunLLM in the post for followups.