DreamCompute does not support nested virtualization
DreamHost has given us a "ceph" tenant in their OpenStack environment (DreamCompute). After some stability problems, we moved almost all the workloads to OVH's OpenStack instead a few years ago. I have been using this tenant for some testing recently, and I found that some of our old compute nodes were stuck in "Error" state. I asked DreamHost support to delete these, and they deleted them today. It sounds like we will hit this problem again if we try to use nested virtualization on DreamCompute. This might have been a result of running the ceph-ansible Vagrant tests there a long time ago. On Mon, Apr 20, 2020 at 2:43 AM DreamHost Customer Support Team wrote:
Okay! After much hard work by our DreamCompute engineers (they had to completely evacuate each of the hypervisors those instances were on of all other instances so that they could be rebooted), those instances have been successfully deleted! They also asked me to pass along a request that you and your team not try to do nested virtualization on DreamCompute at this time as it currently leads to problems like this (where a complete hypervisor reboot is required). If you have any additional questions, please let us know.
- Ken
That explains why those Jenkins slaves locked up and went offline almost immediately after I added them to Jenkins and they ran their first job. On 4/20/20 12:52 PM, Ken Dreyer wrote:
DreamHost has given us a "ceph" tenant in their OpenStack environment (DreamCompute). After some stability problems, we moved almost all the workloads to OVH's OpenStack instead a few years ago.
I have been using this tenant for some testing recently, and I found that some of our old compute nodes were stuck in "Error" state. I asked DreamHost support to delete these, and they deleted them today.
It sounds like we will hit this problem again if we try to use nested virtualization on DreamCompute. This might have been a result of running the ceph-ansible Vagrant tests there a long time ago.
On Mon, Apr 20, 2020 at 2:43 AM DreamHost Customer Support Team wrote:
Okay! After much hard work by our DreamCompute engineers (they had to completely evacuate each of the hypervisors those instances were on of all other instances so that they could be rebooted), those instances have been successfully deleted! They also asked me to pass along a request that you and your team not try to do nested virtualization on DreamCompute at this time as it currently leads to problems like this (where a complete hypervisor reboot is required). If you have any additional questions, please let us know.
- Ken
Right! I asked if they have plans to support nested virt, and they did not have any public info to share about this. - Ken On Mon, Apr 20, 2020 at 11:39 AM David Galloway <dgallowa@redhat.com> wrote:
That explains why those Jenkins slaves locked up and went offline almost immediately after I added them to Jenkins and they ran their first job.
On 4/20/20 12:52 PM, Ken Dreyer wrote:
DreamHost has given us a "ceph" tenant in their OpenStack environment (DreamCompute). After some stability problems, we moved almost all the workloads to OVH's OpenStack instead a few years ago.
I have been using this tenant for some testing recently, and I found that some of our old compute nodes were stuck in "Error" state. I asked DreamHost support to delete these, and they deleted them today.
It sounds like we will hit this problem again if we try to use nested virtualization on DreamCompute. This might have been a result of running the ceph-ansible Vagrant tests there a long time ago.
On Mon, Apr 20, 2020 at 2:43 AM DreamHost Customer Support Team wrote:
Okay! After much hard work by our DreamCompute engineers (they had to completely evacuate each of the hypervisors those instances were on of all other instances so that they could be rebooted), those instances have been successfully deleted! They also asked me to pass along a request that you and your team not try to do nested virtualization on DreamCompute at this time as it currently leads to problems like this (where a complete hypervisor reboot is required). If you have any additional questions, please let us know.
- Ken
Sepia mailing list -- sepia@ceph.io To unsubscribe send an email to sepia-leave@ceph.io
participants (2)
-
David Galloway
-
Ken Dreyer