Pretrain bc ppo problems in train_w_bc_finetune_w_ppo.py when loading json file

How severe does this issue affect your experience of using Ray?

  • None: Just asking a question out of curiosity
  • Low: It annoys or frustrates me for a moment.
  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.
  • High: It blocks me to complete my task.
    High

I want to use my pretrained model as policy and value network, so I followed the example code in train_w_bc_finetune_w_ppo.py

However, I try many times but never run it correctly, mostly because of the version issues, especially when I set the ray version to the newest 2.9.X version. So could you please give me a requirements.txt to run this script?
When I set ray version to 2.9.0, it occurs error in ray.data.read_json() function as follow:

TypeError: Could not serialize the argument <ray.data.datasource.json_datasource.JSONDatasource object at 0x0000013EF11D46D0> for a task or actor ray.data.read_api._get_datasource_or_legacy_reader:
================================================================================
Checking Serializability of <ray.data.datasource.json_datasource.JSONDatasource object at 0x0000013EF11D46D0>
================================================================================
!!! FAIL serialization: self.options cannot be converted to a Python object for pickling
    Serializing '_FILE_EXTENSIONS' ['json', 'jsonl']...
    Serializing '_NUM_THREADS_PER_TASK' 0...
    Serializing '_WRITE_FILE_PER_ROW' False...
    Serializing '_annotated' JSONDatasource...
    Serializing '_file_metadata_shuffler' None...
    Serializing '_file_sizes' <bound method FileBasedDatasource._file_sizes of <ray.data.datasource.json_datasource.JSONDatasource object at 0x0000013EF11D46D0>>...
    !!! FAIL serialization: self.options cannot be converted to a Python object for pickling
        Serializing '__func__' <function FileBasedDatasource._file_sizes at 0x0000013EC2E40DC0>...
    WARNING: Did not find non-serializable object in <bound method FileBasedDatasource._file_sizes of <ray.data.datasource.json_datasource.JSONDatasource object at 0x0000013EF11D46D0>>. This may be an oversight.
================================================================================
Variable:

        FailTuple(_file_sizes [obj=<bound method FileBasedDatasource._file_sizes of <ray.data.datasource.json_datasource.JSONDatasource object at 0x0000013EF11D46D0>>, parent=<ray.data.datasource.json_datasource.JSONDatasource object at 0x0000013EF11D46D0>])      

was found to be non-serializable. There may be multiple other undetected variables that were non-serializable.
Consider either removing the instantiation/imports of these variables or moving the instantiation into the scope of the function/class.
================================================================================
Check https://docs.ray.io/en/master/ray-core/objects/serialization.html#troubleshooting for more information.
If you have any suggestions on how to improve this error message, please reach out to the Ray developers on github.com/ray-project/ray/issues/
================================================================================

Hi @Jihy,

I am facing a similar issue. I wonder have you fixed yours?

Cheers,