Policy Client connection refused from inside Vmware

RLlib
1

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

Hi All,

I am trying to run one some PPO policy_client from inside VMware (with the policy_server on the host machine), and get the following error:

Traceback (most recent call last):
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\site-packages\
urllib3\connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\site-packages\
urllib3\connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\site-packages\
urllib3\connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\http\client.py
", line 1344, in getresponse
    response.begin()
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\http\client.py
", line 307, in begin
    version, status, reason = self._read_status()
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\http\client.py
", line 276, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\site-packages\
requests\adapters.py", line 439, in send
    resp = conn.urlopen(
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\site-packages\
urllib3\connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\site-packages\
urllib3\util\retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\site-packages\
urllib3\packages\six.py", line 734, in reraise
    raise value.with_traceback(tb)
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\site-packages\
urllib3\connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\site-packages\
urllib3\connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\site-packages\
urllib3\connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\http\client.py
", line 1344, in getresponse
    response.begin()
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\http\client.py
", line 307, in begin
    version, status, reason = self._read_status()
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\http\client.py
", line 276, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Re
mote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "polivery_client.py", line 67, in <module>
    episode_id = client.start_episode()
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\site-packages\
ray\rllib\env\policy_client.py", line 106, in start_episode
    return self._send(
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\site-packages\
ray\rllib\env\policy_client.py", line 245, in _send
    response = requests.post(self.address, data=payload)
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\site-packages\
requests\api.py", line 119, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\site-packages\
requests\api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\site-packages\
requests\sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\site-packages\
requests\sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\denys\AppData\Local\Programs\Python\Python38\lib\site-packages\
requests\adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected(
'Remote end closed connection without response'))

I can run it no problem if the client and server are both on the host, or if the client is from external network (port-forwarding configured on host).

Any ideas where I could look further?

2

Hi @Denys_Ashikhin,
Not an export on Vmware, but can you try and post by hand? I’ve in the past found " Simple WebSocket Client" on Chrome to be quite helpful for such things. You can also filter and log network traffic with TShark on Linux. Just to check what’s actually arriving where.
Because I’m pretty sure that this is not an RLlib-related issue.

3

Hi @arturn,
Thanks for the response, I’d love to do that but not sure how? I found the extension you quoted, but not sure what url/parameters I should paste in

1 Like
4

It has been a while since I’ve used them and tbh I can not provide support on them. You’ll have to go through 1-2 tutorials here maybe. ip/port/protocol is obviously according to the settings of your machine and PolicyServer setup!

5

HI, i meet the same problem. Have you resolved the problem already?

6

@Denys_Ashikhin and @Grossbier,

Can you configure VMWare to use “bridged networking”? If so does that solve issue?

7

Unfortunately not… but I’m glad to see other’s are having the same issue! Means it’s not just me lol

8

Sorry for dissapearing like that, personal life kicked and kept me busy (good kind of busy so don’t worry lol). And I tried every kind of bridging - no dice.

Only thing I can say is that wasn’t always the case. If you remember when I was messing around with the Underlords AI a year ago, I could have the exact same connection code/logic and it worked without issue (inside vm and host)