# Model Parallel training with failed training actor

**URL:** <https://discuss.ray.io/t/model-parallel-training-with-failed-training-actor/3143>\
**Category:** Ray Core\
**Created:** [August 5, 2021, 10:38pm UTC](https://discuss.ray.io/t/model-parallel-training-with-failed-training-actor/3143 "2021-08-05T22:38:48Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![asm582](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/asm582/32/669_2.png) [@asm582](https://discuss.ray.io/u/asm582)\
**Post date:** [August 5, 2021, 10:38pm UTC](https://discuss.ray.io/t/model-parallel-training-with-failed-training-actor/3143/1 "2021-08-05T22:38:48Z")

</div>

Hello,

I think I understand that ray re-executes failed tasks and actors based on lineage.

If users train deep learning models in model-parallel fashion and if the training actor dies, will the entire training job fail or just the current epoch where the training actor dies?

Thanks

---

<div class="post-metadata">

**Author:** ![sangcho](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sangcho/32/425_2.png) [@sangcho](https://discuss.ray.io/u/sangcho)\
**Post date:** [August 9, 2021, 10:18pm UTC](https://discuss.ray.io/t/model-parallel-training-with-failed-training-actor/3143/2 "2021-08-09T22:18:03Z")

</div>

cc @kai can you answer his question? I think it is about general fault tolerance of actors.