# Overriding resources per worker in ray-llm
**URL:** https://discuss.ray.io/t/overriding-resources-per-worker-in-ray-llm/13505
**Category:** Ray Core
**Created:** [January 22, 2024, 6:25pm UTC](https://discuss.ray.io/t/overriding-resources-per-worker-in-ray-llm/13505 "2024-01-22T18:25:43Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@asharma](https://discuss.ray.io/u/asharma)
#### Post date: [January 22, 2024, 6:25pm UTC](https://discuss.ray.io/t/overriding-resources-per-worker-in-ray-llm/13505/1 "2024-01-22T18:25:43Z")
Hello:
Looking to experiment with ray-llm, I created a ray cluster with a pod template that referred to
```auto
#pod template
template:
spec:
containers:
- name: ray-head
image: anyscale/ray-llm:latest
```
However, I noticed that this image already has a particular accelerator type configured [here](https://github.com/ray-project/ray-llm/blob/master/models/continuous_batching/meta-llama--Llama-2-13b-chat-hf.yaml).
Since I have a different accelerator, I’d like to change that parameter. Is doing so using rayStartParams possible or do I need to rebuild the ray-llm with a different set of params?
I’m using the RayService CRD using kuberay.
---
### Author:  [@Akshay\_Malik](https://discuss.ray.io/u/Akshay_Malik)
#### Post date: [January 22, 2024, 10:13pm UTC](https://discuss.ray.io/t/overriding-resources-per-worker-in-ray-llm/13505/2 "2024-01-22T22:13:20Z")
You’d need to do 2 things - 1. set the correct resources when using raystart. 2. update the yaml to point to the new resources you want to use. You can pass in the updated yaml either through the docker image or through runtime environments.
---
### Author:  [@asharma](https://discuss.ray.io/u/asharma)
#### Post date: [January 22, 2024, 10:46pm UTC](https://discuss.ray.io/t/overriding-resources-per-worker-in-ray-llm/13505/3 "2024-01-22T22:46:11Z")
Do you have an example I can look at?
```auto
workerGroupSpecs:
# the pod replicas in this group typed worker
- replicas: 1
minReplicas: 0
maxReplicas: 1
# logical group name, for this called small-group, also can be functional
groupName: gpu-group
rayStartParams:
num-gpus: "4"
resources: '"{\"accelerator_type_xxx\": 2}"'
```
This didn’t work.
Also, `ray stop` will stop the entire cluster? Is it possible to stop just one job and restart it with a different set of parameters from the head node as opposed to doing it in the kuberay CRD/pod-template?
---
### Author:  [@Sihan\_Wang](https://discuss.ray.io/u/Sihan_Wang)
#### Post date: [January 23, 2024, 5:12am UTC](https://discuss.ray.io/t/overriding-resources-per-worker-in-ray-llm/13505/4 "2024-01-23T05:12:35Z")
Hi @asharma , your template looks correct! can you paste what error did you get? And also ssh into the pod, and paste the output of cli `ray status`.
---
### Author:  [@asharma](https://discuss.ray.io/u/asharma)
#### Post date: [January 25, 2024, 7:20pm UTC](https://discuss.ray.io/t/overriding-resources-per-worker-in-ray-llm/13505/5 "2024-01-25T19:20:05Z")
`1Error: No available node types can fulfill resource request defaultdict(, {'CPU': 9.0, 'accelerator_type_a10': 0.02, 'GPU': 1.0}). Add suitable node types to this cluster`
That’s the error I see on the ray dashboard.
```auto
$ ray status
======== Autoscaler status: 2024-01-25 11:18:33.165916 ========
Node status
---------------------------------------------------------------
Active:
1 node_e247692229da368a65ec11cda51ed67cb797042d033088513145c768
Pending:
(no pending nodes)
Recent failures:
(no failures)
Resources
---------------------------------------------------------------
Usage:
2.0/2.0 CPU
0.0/2.0 accelerator_type_cpu
0B/8.00GiB memory
44B/2.17GiB object_store_memory
Demands:
{'CPU': 1.0, 'accelerator_type_a10': 0.01}: 1+ pending tasks/actors (1+ using placement groups)
{'accelerator_type_a10': 0.01, 'CPU': 1.0} * 1, {'CPU': 8.0, 'accelerator_type_a10': 0.01, 'GPU': 1.0} * 1 (STRICT_PACK): 1+ pending placement groups
```
---
### Author:  [@asharma](https://discuss.ray.io/u/asharma)
#### Post date: [January 25, 2024, 7:28pm UTC](https://discuss.ray.io/t/overriding-resources-per-worker-in-ray-llm/13505/6 "2024-01-25T19:28:25Z")
On the head node:
```auto
(base) ray@ray-llm-raycluster-9s78h-head-8m4lw:~$ ray health-check --address ray-llm-raycluster-9s78h-head-svc.default.svc.cluster.local:6379
(base) ray@ray-llm-raycluster-9s78h-head-8m4lw:~$ echo $?
0
```
But the GPU worker machine is stuck in “waiting\_for\_gcs” state. It seems to be the same problem described here:
[https://docs.ray.io/en/latest/cluster/kubernetes/troubleshooting/troubleshooting.html#disable-the-init-container-injection](https://docs.ray.io/en/latest/cluster/kubernetes/troubleshooting/troubleshooting.html#disable-the-init-container-injection)
Even if I work around the init container problem, it’s not clear how I can fix the GPU type a10 mismatch. Do you have instructions to stop a ray application and restart it with a different parameter/gpu\_type without having to stop/restart a ray cluster via kuberay?
---
### Author:  [@asharma](https://discuss.ray.io/u/asharma)
#### Post date: [January 30, 2024, 10:01pm UTC](https://discuss.ray.io/t/overriding-resources-per-worker-in-ray-llm/13505/7 "2024-01-30T22:01:52Z")
@Sihan_Wang I was able to add a GPU worker node to the cluster. The only issue is that it’s not a a10.
My question is:
How do I stop the existing application “router” and start it with the right GPU type? Is it possible to to edit the deployment config of an application in the “DEPLOYING” state?
Stopping the application and restarting with a different parameter is also fine. But I can’t seem to find docs on how to do that from the head node using the CLI.
---
### Author:  [@asharma](https://discuss.ray.io/u/asharma)
#### Post date: [January 30, 2024, 11:02pm UTC](https://discuss.ray.io/t/overriding-resources-per-worker-in-ray-llm/13505/8 "2024-01-30T23:02:45Z")
Please consider this resolved. The sequence I was looking for:
```auto
serve shutdown
edit
serve start
```
I was looking for this functionality in the `ray` CLI, but it’s in a separate tool.
Would love to be able to do this on a per application basis, instead of shutting down all of them.