"Ceph is ready" covers a lot of territory. It's more like, "How can I delay util Ceph is available for the particular service I need? I've been taking a systemd-bsaed approach. Since I don't actually care bout Ceph in the abstract, but I'm actually looking for the Ceph or Ceph NFS shares, I create dependencies in my systemd setup such that the dependent VM or container (quadlet) requires its ceph mounts to be online before that service can start. I'm still evaluating how well that works. I'll let you know after the next hurricane hits. But so far, so good. Tim On Fri, 2024-08-30 at 11:02 -0300, Alfredo Rezinovsky wrote:
steps 2 to 4 are exactly my idea.
In step 1 All I will check is all active (o non inactive or unknown). Clean is not necessary, I can boot VMs with degraded.as long they are active.
El jue, 29 ago 2024 a las 8:03, Bogdan Adrian Velica (<vbogdan@gmail.com>) escribió:
Hi,
It's a hacky idea but create a script that checks if the Ceph RBD pool is fully "active+clean" to ensure it's ready before starting Proxmox VMs. Something like this...
1. Bash Script: - Write a script that checks if the Ceph RBD pool is in the "active+clean" state using ceph pg stat or ceph -s. - The script should run until the pool is ready before exiting. A loop or something...
2. Systemd Service for Ceph Readiness: - Create a systemd service unit file (something like: ceph- ready.service) that runs the script at startup. - Ensure the service waits until the Ceph cluster is ready before proceeding.
3. Systemd Service for Proxmox VMs: - Modify an existing Proxmox service or create a new one to depend on the ceph-ready.service. - Use After=ceph-ready.service in the unit file to ensure Proxmox VMs start only after Ceph is ready.
4. Enable and Start the Services: - Enable both systemd services to start at boot with systemctl enable. - Reboot to ensure that Proxmox waits for Ceph before starting VMs.
Just an idea...
Thank you, Bogdan Velica croit.io
On Thu, Aug 29, 2024 at 1:53 PM Alfredo Rezinovsky <alfrenovsky@gmail.com> wrote:
I have a proxmox cluster using an external CEPH cluster.
Sometimes due to blackouts the servers need to restart. If proxmox starts before CEPH is ready the VMs fail to boot.
I want to add a dependency in proxmox to wait for ceph to be ready.
I can work with a HEALTH_WARN as long the RBD pool is usable.
ceph status exit status doesn´t helps
Should I grep for "pgs not active" in ceph status or for "inactive" pgs in ceph health or is there something more direct to know if everything is alright?
-- Alfrenovsky _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Alfrenovsky _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io