I don’t see any output in tensorboard after running my experiments. The log directory has multiple checkpoints as well data in result.json
and progress.csv
file.
When I go to tensorboard I get the following messages:
No scalar data was found.
Probable causes:
- You haven’t written any scalar data to your event files.
- TensorBoard can’t find your event files.
and,
No dashboards are active for the current data set.
Probable causes:
- You haven’t written any data to your event files.
- TensorBoard can’t find your event files.
Running the command tensorboad --inspect
also gives the following message:
======================================================================
Processing event files... (this can take a few minutes)
======================================================================
No event files found within logdir .
I am not exactly sure where I should look to debug this.
I do see this log when I start my server though:
INFO logger.py:179 -- pip install 'ray[tune]' to see TensorBoard files.
WARNING logger.py:316 -- Could not instantiate TBXLogger: No module named 'torch'.
You need to install tensorboard, aka run pip install tensorboardX
.
I do have it installed already.
Hey @aviskarkc10 , could you check, whether you are seeing the events
file in your results dir?
The same dir as you are seeing the results.json in.
This could also be a tune issue? It seems to require torch
. …
1 Like
This import here is failing in your environment:
from tensorboardX import SummaryWriter
… explaining the INFO logger.py:179 -- pip install 'ray[tune]' to see TensorBoard files.
message you
are seeing.
Could you try the above import statement in your python env?
This are all the files in my results dir:
checkpoint_000001 checkpoint_000003 params.pkl result.json
checkpoint_000002 params.json progress.csv
Here is the error message I get when I try the import statement:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/curizmo/.local/lib/python3.8/site-packages/tensorboardX/__init__.py", line 5, in <module>
from .torchvis import TorchVis
File "/home/curizmo/.local/lib/python3.8/site-packages/tensorboardX/torchvis.py", line 11, in <module>
from .writer import SummaryWriter
File "/home/curizmo/.local/lib/python3.8/site-packages/tensorboardX/writer.py", line 17, in <module>
from .comet_utils import CometLogger
File "/home/curizmo/.local/lib/python3.8/site-packages/tensorboardX/comet_utils.py", line 13, in <module>
import torch
ModuleNotFoundError: No module named 'torch'
So do you think this will be fixed if I install torch
?
I installed torch
and got rid of the error. But I am still facing the issue with tensorboard. I don’t have any events files
Hey @aviskarkc10 , ok, cool, but do you still get the:
Could not instantiate TBXLogger ...
error from tune? This is caused by from tensorboardX import SummaryWriter
(could you try this in a python command line?).
Another direction: Did you actually do pip install ray[rllib]
?
Yeah I got rid of that error Could not instantiate TBXLogger ...
after installing torch
.
I think I have installed ray[rllib]
but I since I can’t tell for sure, I will give it another shot.
Edit: Yes verified that pip install ray[rllib]
was done