I’m interested in hiring a Ray expert to help me add parallel processing into a Python forecasting tool that I’ve built.
Note – I did read the TOS / Acceptable Use first thing, and didn’t see anything prohibiting or discouraging this kind of post. I’ll promptly remove it if there’s any concerns from the admins or community.
Ray looks like a superior choice for this kind of work, and, I haven’t been able to find specific experts on general sites, so I’m hoping this group may be the right place to ask.
Down the road, could this be added as a category of its own?
Here’s the gist:
I have a couple hundred optimization rules that run in sequence right now as part of a forecast tool.
Each rule starts with the “core data”, optimizes a piece, and then pushes its findings back into the “core data”.
I want to shift this to a pattern where rules can run in parallel on different cores at the same time.
The tricky bit:
Whenever a rule is completed, I want it to push the findings back to the “core dataset” just like it does today. Any NEW rules that are later started should use the new data state, not the original.
Rule processing order is not important, so whatever rules finish first can influence the next ones. This is sort of a multi-dimensional grid search where I iteratively wind my way to the global solution.
Is this something that an expert on this site could help me with?