Tensorboard doesn't give any output

RLlib
1

I don’t see any output in tensorboard after running my experiments. The log directory has multiple checkpoints as well data in result.json and progress.csv file.

When I go to tensorboard I get the following messages:

No scalar data was found.
Probable causes:

- You haven’t written any scalar data to your event files.
- TensorBoard can’t find your event files.

and,

No dashboards are active for the current data set.
Probable causes:

- You haven’t written any data to your event files.
- TensorBoard can’t find your event files.

Running the command tensorboad --inspect also gives the following message:

======================================================================
Processing event files... (this can take a few minutes)
======================================================================

No event files found within logdir .

I am not exactly sure where I should look to debug this.

I do see this log when I start my server though:

INFO logger.py:179 -- pip install 'ray[tune]' to see TensorBoard files.
WARNING logger.py:316 -- Could not instantiate TBXLogger: No module named 'torch'.
2

You need to install tensorboard, aka run pip install tensorboardX.

3

I do have it installed already.

4

Hey @aviskarkc10 , could you check, whether you are seeing the events file in your results dir?
The same dir as you are seeing the results.json in.

This could also be a tune issue? It seems to require torch. …

1 Like
5

This import here is failing in your environment:

from tensorboardX import SummaryWriter

… explaining the INFO logger.py:179 -- pip install 'ray[tune]' to see TensorBoard files. message you
are seeing.

Could you try the above import statement in your python env?

6

This are all the files in my results dir:

checkpoint_000001  checkpoint_000003  params.pkl    result.json
checkpoint_000002  params.json        progress.csv

Here is the error message I get when I try the import statement:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/curizmo/.local/lib/python3.8/site-packages/tensorboardX/__init__.py", line 5, in <module>
    from .torchvis import TorchVis
  File "/home/curizmo/.local/lib/python3.8/site-packages/tensorboardX/torchvis.py", line 11, in <module>
    from .writer import SummaryWriter
  File "/home/curizmo/.local/lib/python3.8/site-packages/tensorboardX/writer.py", line 17, in <module>
    from .comet_utils import CometLogger
  File "/home/curizmo/.local/lib/python3.8/site-packages/tensorboardX/comet_utils.py", line 13, in <module>
    import torch
ModuleNotFoundError: No module named 'torch'

So do you think this will be fixed if I install torch?

7

I installed torch and got rid of the error. But I am still facing the issue with tensorboard. I don’t have any events files

8

Hey @aviskarkc10 , ok, cool, but do you still get the:

Could not instantiate TBXLogger ...

error from tune? This is caused by from tensorboardX import SummaryWriter (could you try this in a python command line?).

Another direction: Did you actually do pip install ray[rllib]?

9

Yeah I got rid of that error Could not instantiate TBXLogger ... after installing torch.

I think I have installed ray[rllib] but I since I can’t tell for sure, I will give it another shot.

Edit: Yes verified that pip install ray[rllib] was done