# Module not found when in tuning jons

**URL:** <https://discuss.ray.io/t/module-not-found-when-in-tuning-jons/5813>\
**Category:** Ray Tune\
**Created:** [April 15, 2022, 6:38am UTC](https://discuss.ray.io/t/module-not-found-when-in-tuning-jons/5813 "2022-04-15T06:38:48Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![pamparana](https://avatars.discourse-cdn.com/v4/letter/p/df788c/32.png) [@pamparana](https://discuss.ray.io/u/pamparana)\
**Post date:** [April 15, 2022, 6:38am UTC](https://discuss.ray.io/t/module-not-found-when-in-tuning-jons/5813/1 "2022-04-15T06:38:48Z")

</div>

I am using ray tune for optimizing some deep learning model.

I am currently getting an error like:

```auto
TemporaryActor pid=90906) Traceback (most recent call last):
(TemporaryActor pid=90906) File "/Users/luca/opt/anaconda3/envs/mlmod/lib/python3.9/site-packages/ray/_private/function_manager.py", line 594, in _load_actor_class_from_gcs
(TemporaryActor pid=90906) actor_class = pickle.loads(pickled_class)
(TemporaryActor pid=90906) ModuleNotFoundError: No module named 'mlmod'

```

`mlmod` is the package module. I had had similar setup before with optimizing time series models and that always worked.

So, my code is something like:

```auto
ray.init(ignore_reinit_error=True)
result = tune.run(
        tune.with_parameters(train_model, data=data, hydra_config=config, hydra_state=state),
        resources_per_trial=resources_per_trial,
        config=search_config,
        num_samples=num_samples,
        metric="loss",
        mode="min",
        scheduler=scheduler,
        # TODO: We will probably need to add this if we run ray on the cloud.
        # sync_config=tune.SyncConfig(upload_dir="s3://something"),
        resume="AUTO",
    )

def train_model(ray_config, data, hydra_config: DictConfig, hydra_state: Any):
    # required to avoid https://github.com/facebookresearch/hydra/issues/903
    Singleton.set_state(hydra_state)
    # map ray tune parameters to hydra parameters
    for param, value in ray_config.items():
        OmegaConf.update(hydra_config, param, value, merge=False)

    
    from mlmod.apps.train import train
    loss = train(hydra_config, None)
    tune.report(loss=loss)

```

and the called train function at the moment, just does:

// file: mlmod/apps/train.py  
def train(config: DictConfig, datamodule: LightningDataModule) → None:  
import numpy as np

```
return np.random.random()

```

I do not quite understand what is being serialized here and why this issue is happening. I am at a loss now what I can try and how to debug this.

---

<div class="post-metadata">

**Author:** ![amogkam](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/amogkam/32/17_2.png) [@amogkam](https://discuss.ray.io/u/amogkam)\
**Post date:** [April 18, 2022, 9:02pm UTC](https://discuss.ray.io/t/module-not-found-when-in-tuning-jons/5813/2 "2022-04-18T21:02:15Z")

</div>

Hey @pamparana thanks for raising the issue! Can you tell me a bit more about your setup? Is this being run on multiple nodes? Is `mlmod` installed on every single node?

---

<div class="post-metadata">

**Author:** ![amogkam](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/amogkam/32/17_2.png) [@amogkam](https://discuss.ray.io/u/amogkam)\
**Post date:** [April 18, 2022, 9:15pm UTC](https://discuss.ray.io/t/module-not-found-when-in-tuning-jons/5813/3 "2022-04-18T21:15:52Z")

</div>

Here are some other threads which might provide some useful information

> <https://github.com/ray-project/ray/issues/10067>
>
> \### What is the problem?
> \### ModuleNotFoundError: No module named 'functions'
> …
> Does it seem the ray\_tune.py file must stay in the same directory with data/ ??? Otherwise, it doesn't work even if I put \`\`\`'../'\`\`\` in the data loader function...
> Specifically, I run the same ray\_tune.py in the parent folder and child folder respectively, but only got success when I run under the parent folder.
> 
> Cannot provide reproducible code here, because I used my own dataset.
> This issue occurred whenever I switched the ray\_tune.py to a child directory (has \`\`\`../\`\`\`)...
> 
> \*Ray version and other system information (Python version, TensorFlow version, OS):\*
> Ray 0.8.5
> Python 3.7.4
> Pytorch 1.6
> TensorFlow: 2.2.0
> OS linux
> 
> !\[image\](https://user-images.githubusercontent.com/29363464/89991868-0937d000-dcb7-11ea-9637-e5bdf15f847b.png)
> 
> 
> \### Reproduction (REQUIRED)
> Please provide a script that can be run to reproduce the issue. The script should have \*\*no external library dependencies\*\* (i.e., use fake or mock data / environments):
> 
> If we cannot run your script, we cannot fix your issue.
> 
> \- \[x\] I have verified my script runs in a clean environment and reproduces the issue.
> \- \[\] I have verified the issue also occurs with the \[latest wheels\](https://docs.ray.io/en/latest/installation.html).

> <https://github.com/ray-project/ray/issues/5635>
>
> \<!--
> General questions should be asked on the mailing list ray-dev@googlegroups….com.
> Questions about how to use Ray should be asked on
> \[StackOverflow\](https://stackoverflow.com/questions/tagged/ray).
> 
> Before submitting an issue, please fill out the following form.
> \--\>
> 
> \### System information
> \- \*\*OS Platform and Distribution\*\*: Ubuntu 16.04.2 LTS
> \- \*\*Ray installed from (source or binary)\*\*: Binary
> \- \*\*Ray version\*\*: 0.7.2
> \- \*\*Python version\*\*: 3.6.8
> 
> I am trying to build a manual cluster of the machines with IP Addresses. However, When I tried to run the PPO algorithm on the cluster I got an error message from one of the workers complaining about ModuleNotFoundError: No module named "v2i". Here the main module is my custom gym environment. It looks like ray could not able to sync the files between different nodes.
> Here is the complete traceback. \*\*wsl\*\* is my worker hostname.
> \`\`\`
> 
> Traceback (most recent call last):
> File "/home/mayank/miniconda3/envs/v2i/lib/python3.6/site-packages/ray/tune/trial\_runner.py", line 436, in \_process\_trial
> result = self.trial\_executor.fetch\_result(trial)
> File "/home/mayank/miniconda3/envs/v2i/lib/python3.6/site-packages/ray/tune/ray\_trial\_executor.py", line 323, in fetch\_result
> result = ray.get(trial\_future\[0\])
> File "/home/mayank/miniconda3/envs/v2i/lib/python3.6/site-packages/ray/worker.py", line 2195, in get
> raise value
> ray.exceptions.RayTaskError: \[36mray\_PPO:train()\[39m (pid=30729, host=rlmac)
> File "/home/mayank/miniconda3/envs/v2i/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 364, in train
> raise e
> File "/home/mayank/miniconda3/envs/v2i/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 353, in train
> result = Trainable.train(self)
> File "/home/mayank/miniconda3/envs/v2i/lib/python3.6/site-packages/ray/tune/trainable.py", line 150, in train
> result = self.\_train()
> File "/home/mayank/miniconda3/envs/v2i/lib/python3.6/site-packages/ray/rllib/agents/trainer\_template.py", line 126, in \_train
> fetches = self.optimizer.step()
> File "/home/mayank/miniconda3/envs/v2i/lib/python3.6/site-packages/ray/rllib/optimizers/multi\_gpu\_optimizer.py", line 130, in step
> self.num\_envs\_per\_worker, self.train\_batch\_size)
> File "/home/mayank/miniconda3/envs/v2i/lib/python3.6/site-packages/ray/rllib/optimizers/rollout.py", line 29, in collect\_samples
> next\_sample = ray\_get\_and\_free(fut\_sample)
> File "/home/mayank/miniconda3/envs/v2i/lib/python3.6/site-packages/ray/rllib/utils/memory.py", line 33, in ray\_get\_and\_free
> result = ray.get(object\_ids)
> ray.exceptions.RayTaskError: \[36mray\_RolloutWorker:sample()\[39m (pid=11974, host=wsl)
> File "pyarrow/serialization.pxi", line 461, in pyarrow.lib.deserialize
> File "pyarrow/serialization.pxi", line 424, in pyarrow.lib.deserialize\_from
> File "pyarrow/serialization.pxi", line 275, in pyarrow.lib.SerializedPyObject.deserialize
> File "pyarrow/serialization.pxi", line 174, in pyarrow.lib.SerializationContext.\_deserialize\_callback
> File "/media/win/MayankPal/miniconda3/envs/v2i/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle.py", line 965, in subimport
> \_\_import\_\_(name)
> ModuleNotFoundError: No module named 'v2i'
> \`\`\`
> \<!--
> You can obtain the Ray version with
> 
> python -c "import ray; print(ray.\_\_version\_\_)"
> \--\>
> 
> \### Describe the problem
> 
> 
> \### Source code / logs
> 
> 
> \* First start the ray head
> \`ray start --head --redis-port=6666 --num-cpus=22 --num-gpus=1\`
> \* Start ray on worker machine with above redis address
> \`ray start --redis-address=xxx.xxx.xxx.xxx:6666\`
> \* Start PPO training
> \`python train.py\`

In general, it is recommended to not rely on relative paths/imports with ray tune since the working directory of the training function will be changed and is not the same as what’s on the driver.
