I am getting this error while converting dataset to pandas using
ds.to_pandas. Is there any way to overcome it?
The dataset has more than the given limit of 100000 records. Use d.limit(N).topandas()
@pratap123 This is to prevent you from loading too much Datasets data into a Pandas DataFrame on your client machine, since it could cause it to crash with an Out Of Memory error.
Have you tried doing as the error message says, and truncating your Dataset to a smaller number of rows before converting it to a Pandas DataFrame?
And if you’re sure that your entire Dataset will fit into memory, you can bump the limit in the