Writing ray dataset to .csv. without headers

Ray Libraries (Data, Train, Tune, Serve)
1

I’m trying to write a dataset to csv files, and I really don’t want headers in the top of my .csv files.

Per the documentation, I should be able to do this with **arrow_csv_args, which means I should use the kwarg write_options=WriteOptions(includer_header=False) (per the arrow documentation).

But when I try to do this, I get some weird serialization errors. Here’s a minimum-reproducible example:

import ray
from pyarrow.csv import WriteOptions
ds = ray.data.from_items([{'lo': i, 'hi': 2 * i} for i in range(10)])
ds.write_csv('test_headless_csv', write_options=WriteOptions(include_header=False))

Thanks!