# Using ray's python code execution problem

**URL:** https://discuss.ray.io/t/using-rays-python-code-execution-problem/8781
**Category:** Ray Core
**Created:** [December 25, 2022, 10:10pm UTC](https://discuss.ray.io/t/using-rays-python-code-execution-problem/8781 "2022-12-25T22:10:30Z")
**Posts on this page:** 16
**Page:** 1

<div class="post-metadata">

### Author: ![more](https://avatars.discourse-cdn.com/v4/letter/m/2bfe46/32.png) [@more](https://discuss.ray.io/u/more)
#### Post date: [December 25, 2022, 10:10pm UTC](https://discuss.ray.io/t/using-rays-python-code-execution-problem/8781/1 "2022-12-25T22:10:30Z")

</div>

**How severe does this issue affect your experience of using Ray?**

- High: It blocks me to complete my task.

I’m implementing some requirements locally on window 11 with the help of ray and I’m having very strange problems:  
I’ve been editing the code in pycharm and it works fine on my windows 11 laptop, but if I execute my py file from the command line, I can’t achieve the same result as in pycharm, it also prompts that local ray is on and there are no errors, but after a few seconds it just prompts that the process is finished. I use python’s own idle with this same exception.

If I remove the .idea folder from the pycharm project folder, then the above problem also occurs, but if I set a breakpoint for the program and execute it step by step afterwards, it is normal again! It’s so strange!  
I tried ubuntu with the same problem! It doesn’t matter if it’s python 3.9 or python 3.10!

---

<div class="post-metadata">

### Author: ![theo](https://avatars.discourse-cdn.com/v4/letter/t/a87d85/32.png) [@theo](https://discuss.ray.io/u/theo)
#### Post date: [December 26, 2022, 8:59am UTC](https://discuss.ray.io/t/using-rays-python-code-execution-problem/8781/2 "2022-12-26T08:59:19Z")

</div>

Just as a sanity check, are you using a venv? I believe pycharm sets one up be default. If you are, are you running from the venv when using the command line or just using the python3.9 command from the project’s directory?

---

<div class="post-metadata">

### Author: ![more](https://avatars.discourse-cdn.com/v4/letter/m/2bfe46/32.png) [@more](https://discuss.ray.io/u/more)
#### Post date: [December 26, 2022, 9:57am UTC](https://discuss.ray.io/t/using-rays-python-code-execution-problem/8781/4 "2022-12-26T09:57:08Z")

</div>

THANKS REPLY, I don’t use the venu, I just juse the system interpreter, hi, I upload two pics what have code and error detail in there.

---

<div class="post-metadata">

### Author: ![more](https://avatars.discourse-cdn.com/v4/letter/m/2bfe46/32.png) [@more](https://discuss.ray.io/u/more)
#### Post date: [December 26, 2022, 10:00am UTC](https://discuss.ray.io/t/using-rays-python-code-execution-problem/8781/5 "2022-12-26T10:00:12Z")

</div>

![IMG_20221226_175810](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/4/41756b99d4c97b30c9bf41a73eef8d2bd63183d5.jpeg)  
 ![IMG_20221226_175913](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/e/e8bff4f14c86c9618f06447de87d9deef69a7bb6.jpeg)

---

<div class="post-metadata">

### Author: ![theo](https://avatars.discourse-cdn.com/v4/letter/t/a87d85/32.png) [@theo](https://discuss.ray.io/u/theo)
#### Post date: [December 26, 2022, 10:23am UTC](https://discuss.ray.io/t/using-rays-python-code-execution-problem/8781/6 "2022-12-26T10:23:58Z")

</div>

Could you post these as text as well as the code you’re running to get the error (or something to reproduce it)

---

<div class="post-metadata">

### Author: ![more](https://avatars.discourse-cdn.com/v4/letter/m/2bfe46/32.png) [@more](https://discuss.ray.io/u/more)
#### Post date: [December 26, 2022, 10:42am UTC](https://discuss.ray.io/t/using-rays-python-code-execution-problem/8781/7 "2022-12-26T10:42:50Z")

</div>

import ray

from selenium import webdriver

ray.init()

@ray.remote  
class test:  
br = webdriver.Chrome()

w = [test.remote() for \_ in range(3)]

---

<div class="post-metadata">

### Author: ![more](https://avatars.discourse-cdn.com/v4/letter/m/2bfe46/32.png) [@more](https://discuss.ray.io/u/more)
#### Post date: [December 26, 2022, 10:43am UTC](https://discuss.ray.io/t/using-rays-python-code-execution-problem/8781/8 "2022-12-26T10:43:39Z")

</div>

Hi, I just upload the text code!

---

<div class="post-metadata">

### Author: ![more](https://avatars.discourse-cdn.com/v4/letter/m/2bfe46/32.png) [@more](https://discuss.ray.io/u/more)
#### Post date: [January 3, 2023, 9:20pm UTC](https://discuss.ray.io/t/using-rays-python-code-execution-problem/8781/9 "2023-01-03T21:20:36Z")

</div>

**UPDATE: The problem is thread/process related, but deep down I haven’t understood what’s going on.**

---

<div class="post-metadata">

### Author: ![sangcho](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sangcho/32/425_2.png) [@sangcho](https://discuss.ray.io/u/sangcho)
#### Post date: [January 6, 2023, 1:17am UTC](https://discuss.ray.io/t/using-rays-python-code-execution-problem/8781/10 "2023-01-06T01:17:18Z")

</div>

If you import `from selenium import webdriver` inside the remote task, is it solved?

```auto
ray.init()

@ray.remote
class test:
    from selenium import webdriver
    br = webdriver.Chrome()

```

---

<div class="post-metadata">

### Author: ![sangcho](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sangcho/32/425_2.png) [@sangcho](https://discuss.ray.io/u/sangcho)
#### Post date: [January 6, 2023, 1:17am UTC](https://discuss.ray.io/t/using-rays-python-code-execution-problem/8781/11 "2023-01-06T01:17:38Z")

</div>

If the library has a global config, importing it from the driver could cause some random issues

---

<div class="post-metadata">

### Author: ![more](https://avatars.discourse-cdn.com/v4/letter/m/2bfe46/32.png) [@more](https://discuss.ray.io/u/more)
#### Post date: [January 8, 2023, 9:06am UTC](https://discuss.ray.io/t/using-rays-python-code-execution-problem/8781/12 "2023-01-08T09:06:13Z")

</div>

Thanks for the reply edit! I’ve tried and still haven’t been able to fix it, I’m basically sure it’s thread related now, but I don’t know the deeper cause.

---

<div class="post-metadata">

### Author: ![sangcho](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sangcho/32/425_2.png) [@sangcho](https://discuss.ray.io/u/sangcho)
#### Post date: [January 8, 2023, 4:39pm UTC](https://discuss.ray.io/t/using-rays-python-code-execution-problem/8781/13 "2023-01-08T16:39:09Z")

</div>

Hmm the error message indicates (cannot pickle thread.lock) that you capture the lock object to the remote task. This happens when you do something like this

```auto
a = threading.lock()
@ray.remote
def f():
    print(a) # lock object is implicitly captured, and you cannot pickle lock

```

Are you sure import selenium package inside a remote task doesn’t work? For me this seems to be the only possible cause… Is it possible to reproduce the issue with your script?

---

<div class="post-metadata">

### Author: ![more](https://avatars.discourse-cdn.com/v4/letter/m/2bfe46/32.png) [@more](https://discuss.ray.io/u/more)
#### Post date: [January 8, 2023, 5:32pm UTC](https://discuss.ray.io/t/using-rays-python-code-execution-problem/8781/14 "2023-01-08T17:32:41Z")

</div>

```auto
import ray
ray.init()

@ray.remote
class test:
    from selenium import webdriver
    br = webdriver.Chrome()

w = [test.remote() for _ in range(3)]

```

Thanks for the code, it gave me an accurate idea of the cause, I tried it and when I hit run I could get the first chrome browser to pop up, but after that it reported an error when I was using [test.remote() for \_ in range(3)] to generate 3 objects in a row, the error image was as follows：

 ![image](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/b/bb0a332ce60e617081526644b4f368f37de0f7c3.png)

```auto
import ray
ray.init()

@ray.remote
class test:
    from selenium import webdriver
    
    def __int__ (self):
        br = self.webdriver.Chrome()

w = [test.remote() for _ in range(3)]

```

If I follow the documentation and put the lock object into the method, when I hit run, no more chrome browser will pop up and the following error will be reported (there is no active effort to use the keyboard to stop the program):

 ![image](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/9/99f1adda067ca7aa66295cf289b89dca03cc2edd.png)

---

<div class="post-metadata">

### Author: ![more](https://avatars.discourse-cdn.com/v4/letter/m/2bfe46/32.png) [@more](https://discuss.ray.io/u/more)
#### Post date: [January 8, 2023, 5:40pm UTC](https://discuss.ray.io/t/using-rays-python-code-execution-problem/8781/15 "2023-01-08T17:40:30Z")

</div>

Thank you! I just upload my test code and imgs.

---

<div class="post-metadata">

### Author: ![sangcho](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sangcho/32/425_2.png) [@sangcho](https://discuss.ray.io/u/sangcho)
#### Post date: [January 10, 2023, 1:46am UTC](https://discuss.ray.io/t/using-rays-python-code-execution-problem/8781/16 "2023-01-10T01:46:51Z")

</div>

```plaintext
import ray
ray.init()

@ray.remote
class test:
    
    def __init__ (self):
        from selenium import webdriver
        br = webdriver.Chrome()

w = [test.remote() for _ in range(3)]

```

Can you try this instead?

---

<div class="post-metadata">

### Author: ![sangcho](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sangcho/32/425_2.png) [@sangcho](https://discuss.ray.io/u/sangcho)
#### Post date: [January 10, 2023, 1:50am UTC](https://discuss.ray.io/t/using-rays-python-code-execution-problem/8781/17 "2023-01-10T01:50:33Z")

</div>

When ray starts an actor, it pickles (serializes) the actor class and deserializes it when it is initialized in other processes.

When Python pickles objects, and if the object contains the module (e.g., Chrome for example), it stores the reference to the module. When deserialization happens, it imports the module to the process.

So if you import webdriver, it is imported again when the actor (`class test`) is created. My guess is the webdriver module has some global lock, which cannot be pickled.

The workaround is to import it within ` __init__ `. In this case, you can defer the module import until the actor is created, so lock object is not included when you pickle the test class.

Alternatively, I feel like this could also work.

```auto
import ray
import selenium
ray.init()

@ray.remote
class test:
    
    def __init__ (self):
        from selenium import webdriver
        br = selenium.webdriver.Chrome()

w = [test.remote() for _ in range(3)]

```
