# Docker. Using /tmp instead of /dev/shm because /dev/shm has only 31457280000 bytes available **URL:** \ **Category:** Ray Core\ **Created:** [March 26, 2021, 10:29am UTC](https://discuss.ray.io/t/docker-using-tmp-instead-of-dev-shm-because-dev-shm-has-only-31457280000-bytes-available/1404 "2021-03-26T10:29:01Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![eaakimov](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/eaakimov/32/664_2.png) [@eaakimov](https://discuss.ray.io/u/eaakimov)\ **Post date:** [March 26, 2021, 10:29am UTC](https://discuss.ray.io/t/docker-using-tmp-instead-of-dev-shm-because-dev-shm-has-only-31457280000-bytes-available/1404/1 "2021-03-26T10:29:01Z")
Hi again, so, yes only 31Gb available (as set by --shm-size=Xgb). Any ideas? Running on GCP > WARNING: The object store is using /tmp instead of /dev/shm because /dev/shm has only 31457280000 bytes available. This will harm performance! You may be able to free up space by deleting files in /dev/shm. If you are inside a Docker container, you can increase /dev/shm size by passing ‘–shm-size=Xgb’ to ‘docker run’ (or add it to the run\_options list in a Ray cluster config). Make sure to set this to more than 2gb. ---
**Author:** ![eoakes](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/eoakes/32/7825_2.png) [@eoakes](https://discuss.ray.io/u/eoakes)\ **Post date:** [March 26, 2021, 1:21pm UTC](https://discuss.ray.io/t/docker-using-tmp-instead-of-dev-shm-because-dev-shm-has-only-31457280000-bytes-available/1404/2 "2021-03-26T13:21:32Z")
I think this might be because Ray is mis-detecting the available memory trying to set the object store size to \> 31GiB. Can you try setting the object store size explicitly to less than that? e.g., `ray start --object-store-memory=10000000000` or `ray.init(object_store_memory=10000000000)` ---
**Author:** ![eaakimov](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/eaakimov/32/664_2.png) [@eaakimov](https://discuss.ray.io/u/eaakimov)\ **Post date:** [March 26, 2021, 9:06pm UTC](https://discuss.ray.io/t/docker-using-tmp-instead-of-dev-shm-because-dev-shm-has-only-31457280000-bytes-available/1404/3 "2021-03-26T21:06:34Z")
Hi, thanks for helping out. this setup results in this error: > ray.init( num\_cpus = self.tuner\_params[‘cpus’], > num\_gpus = self.tuner\_params[‘gpus’], > \_memory = 38 x1024 x 10e6, > object\_store\_memory =7 x 1024 x 10e6 ) and Docker run > docker run --shm-size=60G cogad3 Docker file: > FROM tensorflow/tensorflow:nightly > > RUN apt-get update && apt-get install --no-install-recommends --no-install-suggests -y curl > RUN apt-get install unzip > RUN apt-get -y install rsync grsync > RUN apt-get -y install python3 > RUN apt-get -y install python3-pip > RUN pip install --upgrade pip > > COPY . . > RUN pip install -r requirements.txt > > EXPOSE 5000 > > ENTRYPOINT [“python3”,“cogad3.py”] ---
**Author:** ![sangcho](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sangcho/32/425_2.png) [@sangcho](https://discuss.ray.io/u/sangcho)\ **Post date:** [March 27, 2021, 10:23pm UTC](https://discuss.ray.io/t/docker-using-tmp-instead-of-dev-shm-because-dev-shm-has-only-31457280000-bytes-available/1404/4 "2021-03-27T22:23:14Z")
cc @ijrsvt Maybe the same issue as the one we found before; Can you check? ---
**Author:** ![cama](https://avatars.discourse-cdn.com/v4/letter/c/f07891/32.png) [@cama](https://discuss.ray.io/u/cama)\ **Post date:** [July 25, 2022, 12:08pm UTC](https://discuss.ray.io/t/docker-using-tmp-instead-of-dev-shm-because-dev-shm-has-only-31457280000-bytes-available/1404/5 "2022-07-25T12:08:26Z")
I am using GCP as well, and I get the same error message. Is it possible to configure ray to use another folder entirely for shared memory? E.g. I want to use the folder $HOME$/NFS/dev/shared? I am working in a system where the machine nodes have limited amounts of memory (~10GB max) and the users have very small personal harddisks (~5GB). Therefore I would like for ray to use the mounted NFS drive for it’s shared memory, is this possible? ---
**Author:** ![jjyao](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/jjyao/32/1799_2.png) [@jjyao](https://discuss.ray.io/u/jjyao)\ **Post date:** [August 15, 2022, 4:49pm UTC](https://discuss.ray.io/t/docker-using-tmp-instead-of-dev-shm-because-dev-shm-has-only-31457280000-bytes-available/1404/6 "2022-08-15T16:49:23Z")
Hi @cama, Sorry for the late reply. You can use the `--plasma-directory` option for that purpose. (e.g. ray start --plasma-directory=NFS/dev/shared) ---
**Author:** ![cama](https://avatars.discourse-cdn.com/v4/letter/c/f07891/32.png) [@cama](https://discuss.ray.io/u/cama)\ **Post date:** [September 6, 2022, 1:11pm UTC](https://discuss.ray.io/t/docker-using-tmp-instead-of-dev-shm-because-dev-shm-has-only-31457280000-bytes-available/1404/7 "2022-09-06T13:11:21Z")
Hello jjyao! Thank you for the answer! However, I am right now using the Pool function, is there a way to pass in the plasma\_directory argument or the object\_store\_memory to the pool function call? ```auto from ray.multiprocessing import Pool with Pool(4) as p: # Do Something ```