Hey this is a general feasibility question (careful: Noob warning). This is the Scenario:
At the place where I work we have PC1 which has a licensed simulation software installed, that I can’t install on my private PC2. We basically all have to share PC1 so everyone can run their sims. So I’d like to use my limited time efficiently.
How it works is:
- I remote log into PC1 (from shared VPN).
- execute a batch script which starts the simulation with the correct params (it can use only one core per sim). Each sim takes roughly 20mins.
- When its done, it saves the output to a specified location, each resultfile is around 1GB.
I will need roughly 5000 sims in total.
PC1 specs:
-Windows
-8 cores
PC2 specs
-Windows
-12 cores
First idea was to use a basic ray parallel for loop, to first of all at least use all 8 cores.
Basicall like this:
@ray.remote
def f(input_pars):
log = check_output(["start_sim.bat", input_name_pars], shell=True)
print(log)
return log
result_ids = []
for i in range(1,5000):
result_ids.append(f.remote(input_pars[i]))
Now my question is: Would it make sense to create a cluster over vpn with PC1 and PC2 to have 20 cores total and thus reduce the computation time.
I appreciate your help!