# Unable to increase the number of cores for head node **URL:** \ **Category:** Ray Clusters\ **Created:** [June 10, 2022, 3:13pm UTC](https://discuss.ray.io/t/unable-to-increase-the-number-of-cores-for-head-node/6463 "2022-06-10T15:13:08Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![blshao84](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/blshao84/32/1010_2.png) [@blshao84](https://discuss.ray.io/u/blshao84)\ **Post date:** [June 10, 2022, 3:13pm UTC](https://discuss.ray.io/t/unable-to-increase-the-number-of-cores-for-head-node/6463/1 "2022-06-10T15:13:08Z")
Hi, I’m running a ray cluster using k8s and try to assign more cpu cores to the head node. I use the official chart configurations for deployment and here’s my values.yaml ![截屏2022-06-10 23.06.55](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/d/d4bb5be8e4e5d85c41a97bd250d0233c58d9ebc1.png) However, after the cluster is started, from the head node’s pod yaml configuration, the request CPU is some wired number that doesn’t appear anywhere in my configs. ![截屏2022-06-10 23.09.09](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/8/8b7c2cc264cbaf436b58c333a2e1a45ead4950e3.png) As a result, there’s only one 1 core assigned to ray head. I don’t know what else might control this config and hope people could shed some lights. Thanks, -BS ---
**Author:** ![Dmitri](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/dmitri/32/657_2.png) [@Dmitri](https://discuss.ray.io/u/Dmitri)\ **Post date:** [June 10, 2022, 3:22pm UTC](https://discuss.ray.io/t/unable-to-increase-the-number-of-cores-for-head-node/6463/2 "2022-06-10T15:22:01Z")
That’s weird. I’ll take a look. ---
**Author:** ![blshao84](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/blshao84/32/1010_2.png) [@blshao84](https://discuss.ray.io/u/blshao84)\ **Post date:** [June 10, 2022, 3:24pm UTC](https://discuss.ray.io/t/unable-to-increase-the-number-of-cores-for-head-node/6463/3 "2022-06-10T15:24:45Z")
Thanks Dmitri. Just for your information, I’m using ray 1.9 for now and please let me know if you need more information. ---
**Author:** ![Dmitri](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/dmitri/32/657_2.png) [@Dmitri](https://discuss.ray.io/u/Dmitri)\ **Post date:** [June 10, 2022, 3:26pm UTC](https://discuss.ray.io/t/unable-to-increase-the-number-of-cores-for-head-node/6463/4 "2022-06-10T15:26:42Z")
Is the chart pulled from the Ray master branch? ---
**Author:** ![Dmitri](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/dmitri/32/657_2.png) [@Dmitri](https://discuss.ray.io/u/Dmitri)\ **Post date:** [June 10, 2022, 3:41pm UTC](https://discuss.ray.io/t/unable-to-increase-the-number-of-cores-for-head-node/6463/5 "2022-06-10T15:41:26Z")
I was not able to reproduce this on my first attempt. The chart’s logic sets limits equal to requests: > so this is very strange. The Helm chart configures a “RayCluster” custom resource which is then processed by an operator. Maybe we can take a look at the intermediate RayCluster object first. After installing the chart, could you `kubectl -n get raycluster -o yaml` and see what the requests and limits look like in that configuration? ---
**Author:** ![blshao84](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/blshao84/32/1010_2.png) [@blshao84](https://discuss.ray.io/u/blshao84)\ **Post date:** [June 10, 2022, 3:44pm UTC](https://discuss.ray.io/t/unable-to-increase-the-number-of-cores-for-head-node/6463/6 "2022-06-10T15:44:18Z")
'kubectl get raycluster -o yaml ’ shows the correct cpu ‘request’ and ‘limit’ (which is 2 in my case). Only the started head pod’s yaml somehow has a weird request number ---
**Author:** ![blshao84](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/blshao84/32/1010_2.png) [@blshao84](https://discuss.ray.io/u/blshao84)\ **Post date:** [June 10, 2022, 3:45pm UTC](https://discuss.ray.io/t/unable-to-increase-the-number-of-cores-for-head-node/6463/7 "2022-06-10T15:45:58Z")
How should I take a look at ‘RayCluster’ object? ---
**Author:** ![blshao84](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/blshao84/32/1010_2.png) [@blshao84](https://discuss.ray.io/u/blshao84)\ **Post date:** [June 10, 2022, 3:47pm UTC](https://discuss.ray.io/t/unable-to-increase-the-number-of-cores-for-head-node/6463/8 "2022-06-10T15:47:45Z")
Yes. We use the chart from ray-1.9.0 release. I did a file by file diff (ray-1.9.0 v.s ours): ![截屏2022-06-10 23.40.42](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/e/e13788c9dce755ee23f2da0d03c36b9aae8d009c.png) ![截屏2022-06-10 23.41.45](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/e/ed59d6eabdd799502f40c3a2f6ba2e5135358dce.png) ![截屏2022-06-10 23.41.05](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/d/d13967cb1335734e25919c77dc7fe2741cb320b5.png) ![截屏2022-06-10 23.40.55](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/1/17cbb4b7e5b97e4f4e0f73142cd526015b4563ff.png) ![截屏2022-06-10 23.40.48](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/8/83aa1238687a8c7e3d309468d33d225b931035a9.png) ---
**Author:** ![Dmitri](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/dmitri/32/657_2.png) [@Dmitri](https://discuss.ray.io/u/Dmitri)\ **Post date:** [June 10, 2022, 3:49pm UTC](https://discuss.ray.io/t/unable-to-increase-the-number-of-cores-for-head-node/6463/9 "2022-06-10T15:49:05Z")
One theory is that something in your K8s environment is mutating the requested pod. What kind of K8s environment are you running in? Thanks for the config details! ---
**Author:** ![Dmitri](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/dmitri/32/657_2.png) [@Dmitri](https://discuss.ray.io/u/Dmitri)\ **Post date:** [June 10, 2022, 3:52pm UTC](https://discuss.ray.io/t/unable-to-increase-the-number-of-cores-for-head-node/6463/10 "2022-06-10T15:52:14Z")
The operator image should also be pinned to Ray 1.9.0. (`operatorImage: rayproject/ray:1.9.0`). Is that the case in your configs? Just as a sanity check, what happens if you try to create a scratch pod (say, with a busybox image), with cpu requests=limits=2? ---
**Author:** ![blshao84](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/blshao84/32/1010_2.png) [@blshao84](https://discuss.ray.io/u/blshao84)\ **Post date:** [June 10, 2022, 4:03pm UTC](https://discuss.ray.io/t/unable-to-increase-the-number-of-cores-for-head-node/6463/11 "2022-06-10T16:03:57Z")
double check with the operator image, it’s indeed ray:1.9.0. And a scratch pod seems running fine (with correct number of requested cpu) and we have a bunch of other services/pods running under this k8s platform. Regarding to the k8s environment, it’s a vendor K8S platform, TKE: [GitHub - tkestack/tke: Native Kubernetes container management platform supporting multi-tenant and multi-cluster](https://github.com/tkestack/tke) ---
**Author:** ![blshao84](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/blshao84/32/1010_2.png) [@blshao84](https://discuss.ray.io/u/blshao84)\ **Post date:** [June 10, 2022, 4:04pm UTC](https://discuss.ray.io/t/unable-to-increase-the-number-of-cores-for-head-node/6463/12 "2022-06-10T16:04:56Z")
Interestingly, we just tried set request cpu to 4, and in this case, the head pod’s request cpu becomes 1 … not 664m … ---
**Author:** ![Dmitri](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/dmitri/32/657_2.png) [@Dmitri](https://discuss.ray.io/u/Dmitri)\ **Post date:** [June 10, 2022, 7:13pm UTC](https://discuss.ray.io/t/unable-to-increase-the-number-of-cores-for-head-node/6463/13 "2022-06-10T19:13:54Z")
I understand you might not be able to upgrade to a newer Ray for your application, but what happens if you use base Ray 1.13.0 images instead? ---
**Author:** ![blshao84](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/blshao84/32/1010_2.png) [@blshao84](https://discuss.ray.io/u/blshao84)\ **Post date:** [June 14, 2022, 1:37am UTC](https://discuss.ray.io/t/unable-to-increase-the-number-of-cores-for-head-node/6463/14 "2022-06-14T01:37:41Z")
Thanks Dmitri, after some investigation, it turned out to be an issue of our k8s platform (they put some hard-coded limit on the cpu resource for test env) … Appreciate your help anyway！！ ---
**Author:** ![Dmitri](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/dmitri/32/657_2.png) [@Dmitri](https://discuss.ray.io/u/Dmitri)\ **Post date:** [June 14, 2022, 2:47am UTC](https://discuss.ray.io/t/unable-to-increase-the-number-of-cores-for-head-node/6463/15 "2022-06-14T02:47:19Z")
Ok, thanks for letting me know. This is good for my sanity 🙂