Check failed: instance_ The core worker process is not initialized yet or already shutdown

Ray Core
1

I have 2 remote functions defined.
func1: connect to SQL db to get data
func2: transform a CSV file

But the functions are independent but take **kwargs as the argument. When I run the script for a date range that has few records (ex: 01.01.2021) the process completes without any issue. But if I run the same script for a weekday (ex: 04.09.2021) I’m getting the error during the run. I tried googling this error but could not find any info.

Core Dump:
(gdb) core core.22400
[New LWP 22514]
[New LWP 22538]
[New LWP 22545]
[New LWP 22400]
[New LWP 22461]
[New LWP 22498]
[New LWP 22463]
[New LWP 22507]
[New LWP 22459]
Core was generated by `ray::IDLE '.
Program terminated with signal 6, Aborted.
#0 0x00007f867af57207 in ?? ()

Error Message:
(pid=19735) WARNING: Logging before InitGoogleLogging() is written to STDERR
(pid=19735) F0410 16:21:41.787758 19735 19860 core_worker.cc:165] Check failed: instance_ The core worker process is not initialized yet or already shutdown.
(pid=19735) *** Check failure stack trace: ***
(pid=19735) @ 0x7ff1c5df210d google::LogMessage::Fail()
(pid=19735) @ 0x7ff1c5df357c google::LogMessage::SendToLog()
(pid=19735) @ 0x7ff1c5df1de9 google::LogMessage::Flush()
(pid=19735) @ 0x7ff1c5df2001 google::LogMessage::~LogMessage()
(pid=19732) WARNING: Logging before InitGoogleLogging() is written to STDERR
(pid=19732) F0410 16:21:41.787273 19732 19861 core_worker.cc:165] Check failed: instance_ The core worker process is not initialized yet or already shutdown.
(pid=19732) *** Check failure stack trace: ***
(pid=19732) @ 0x7fdcf283110d google::LogMessage::Fail()
(pid=19732) @ 0x7fdcf283257c google::LogMessage::SendToLog()
(pid=19732) @ 0x7fdcf2830de9 google::LogMessage::Flush()
(pid=19732) @ 0x7fdcf2831001 google::LogMessage::~LogMessage()
(pid=19733) WARNING: Logging before InitGoogleLogging() is written to STDERR
(pid=19733) F0410 16:21:41.787277 19733 19867 core_worker.cc:165] Check failed: instance_ The core worker process is not initialized yet or already shutdown.
(pid=19733) *** Check failure stack trace: ***
(pid=19733) @ 0x7ff1e03d810d google::LogMessage::Fail()
(pid=19733) @ 0x7ff1e03d957c google::LogMessage::SendToLog()
(pid=19733) @ 0x7ff1e03d7de9 google::LogMessage::Flush()
(pid=19733) @ 0x7ff1e03d8001 google::LogMessage::~LogMessage()
(pid=19735) @ 0x7ff1c5da5479 ray::RayLog::~RayLog()
(pid=19732) @ 0x7fdcf27e4479 ray::RayLog::~RayLog()
(pid=19733) @ 0x7ff1e038b479 ray::RayLog::~RayLog()

Box:
Lang: Python
OS: Linux
CPU(s): 32
ray: 1.1.0

Code:

import ray
ray.init(num_cpus=4, num_gpus=1, include_dashboard=False)
def run_ray(**kwargs):
ray.wait([ray_data.remote(**kwargs), ray_shape.remote(**kwargs)])
ray.shutdown()

@ray.remote
def ray_data(**kwargs):
–function queries SQL db

@ray.remote
def ray_shape(**kwargs):
–function transforms a large CSV file