We are still working on restoring full service. There is a period of time where build information may have been sent to an old copy of the chacra databases. We’re working on resolving any split brain scenarios. If you’re getting 404s or 502s in your teuthology runs or Jenkins builds anytime after this e-mail, please ping me or Dan on Slack.

From: David Galloway via Sepia <sepia@ceph.io>
Date: Friday, July 10, 2026 at 11:25 AM
To: David Galloway via Sepia <sepia@ceph.io>
Subject: [EXTERNAL] [sepia] OVH Maintenance

This Message Is From an External Sender

This message came from outside your organization.

Report Suspicious

OVH is in the middle of a large maintenance window. Our Jenkins and chacra instances were restarted about 6 hours ago. This has resulted, so far, in:

Jenkins builds failing because either a chacra instance went down in the middle of a build or because Jenkins was restarted without a JAVA_ARGS setting we were relying on
chacra repos being incomplete (404s in teuthology jobs)
Jenkins builds getting delayed

There were a handful of failed ceph-dev-pipeline jobs due to #1. I’ve retriggered all of those.

I will see about getting the chacra repos rebuilt but there may still be some with missing packages. Unfortunately, those will require a full rebuild. Ping me if you find a repo with missing packages and I’ll be happy to retrigger the build.

I’ve subscribed to email notifications from OVH so we have a better heads up about this in the future.

David Galloway

Ceph Engineering Labs – Infrastructure Architect

david.galloway@ibm.com

IBM