reef 18.2.2 (hot-fix) QE validation status
Details of this release are summarized here: https://tracker.ceph.com/issues/64721#note-1 Release Notes - TBD LRC upgrade - TBD Seeking approvals/reviews for: smoke - in progress rados - Radek, Laura? quincy-x - in progress Also need approval from Travis, Redouane for Prometheus fix testing.
Looks great to me, Redo has tested this thoroughly. Thanks! Travis On Tue, Mar 5, 2024 at 8:48 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/64721#note-1 Release Notes - TBD LRC upgrade - TBD
Seeking approvals/reviews for:
smoke - in progress rados - Radek, Laura? quincy-x - in progress
Also need approval from Travis, Redouane for Prometheus fix testing. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Looks good to me. Testing went OK without any issues. Thanks, Redo. On Tue, Mar 5, 2024 at 5:22 PM Travis Nielsen <tnielsen@redhat.com> wrote:
Looks great to me, Redo has tested this thoroughly.
Thanks! Travis
On Tue, Mar 5, 2024 at 8:48 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/64721#note-1 Release Notes - TBD LRC upgrade - TBD
Seeking approvals/reviews for:
smoke - in progress rados - Radek, Laura? quincy-x - in progress
Also need approval from Travis, Redouane for Prometheus fix testing. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
Only suits below need approval: smoke - Radek, Laura? rados - Radek, Laura? We are also in the process of upgrading gibba and then LRC into 18.2.2 RC On Tue, Mar 5, 2024 at 7:47 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/64721#note-1 Release Notes - TBD LRC upgrade - TBD
Seeking approvals/reviews for:
smoke - in progress rados - Radek, Laura? quincy-x - in progress
Also need approval from Travis, Redouane for Prometheus fix testing.
Here are the rados and smoke suite summaries. @Radoslaw Zarzynski <rzarzyns@redhat.com>, @Adam King <adking@redhat.com>, @Nizamudeen A <nia@redhat.com>, mind having a look to ensure the results from the rados suite look good to you? @Venky Shankar <vshankar@redhat.com> mind having a look at the smoke suite? There was a resurgence of https://tracker.ceph.com/issues/57206. I don't see this as a blocker to the hotfix release, but LMK your thoughts. *rados* - https://pulpito.ceph.com/yuriw-2024-03-04_20:52:58-rados-reef-release-distro... - https://pulpito.ceph.com/yuriw-2024-03-05_01:29:36-rados-reef-release-distro... Failures, unrelated: 1. https://tracker.ceph.com/issues/64725 -- new tracker, but known issue 2. https://tracker.ceph.com/issues/61774 3. https://tracker.ceph.com/issues/49287 4. https://tracker.ceph.com/issues/55141 5. https://tracker.ceph.com/issues/59142 6. https://tracker.ceph.com/issues/64726 -- new tracker 7. https://tracker.ceph.com/issues/62992 8. https://tracker.ceph.com/issues/64208 Details: 1. rados/singleton: application not enabled on pool 'rbd' - Ceph - RADOS 2. centos 9 testing reveals rocksdb "Leak_StillReachable" memory leak in mons - Ceph - RADOS 3. podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope not found - Ceph - Orchestrator 4. thrashers/fastread: assertion failure: rollback_info_trimmed_to == head - Ceph - RADOS 5. mgr/dashboard: fix e2e for dashboard v3 - Ceph - Mgr - Dashboard 6. LibRadosAioEC.MultiWritePP hang and pkill - Ceph - RADOS 7. Heartbeat crash in reset_timeout and clear_timeout - Ceph - RADOS 8. test_cephadm.sh: Container version mismatch causes job to fail. - Ceph - Orchestrator *smoke* - https://pulpito.ceph.com/yuriw-2024-03-05_15:31:54-smoke-reef-release-distro... Failures, unrelated: 1. https://tracker.ceph.com/issues/52624 2. https://tracker.ceph.com/issues/57206 3. https://tracker.ceph.com/issues/64727 -- new tracker Details: 1. qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" - Ceph 2. ceph_test_libcephfs_reclaim crashes during test - Ceph - CephFS 3. suites/dbench.sh: Socket exception: No route to host (113) - Infrastructure On Tue, Mar 5, 2024 at 3:05 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Only suits below need approval:
smoke - Radek, Laura? rados - Radek, Laura?
We are also in the process of upgrading gibba and then LRC into 18.2.2 RC
On Tue, Mar 5, 2024 at 7:47 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/64721#note-1 Release Notes - TBD LRC upgrade - TBD
Seeking approvals/reviews for:
smoke - in progress rados - Radek, Laura? quincy-x - in progress
Also need approval from Travis, Redouane for Prometheus fix testing.
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
for dashboard, I see 1 failure, 2 dead and 2 passed jobs. The failed e2e is something we fixed a while ago. not sure why it's broken again. but if it's recurring, we'll have a look. In any case it's not a blocker. On Wed, Mar 6, 2024 at 4:53 AM Laura Flores <lflores@redhat.com> wrote:
Here are the rados and smoke suite summaries.
@Radoslaw Zarzynski <rzarzyns@redhat.com>, @Adam King <adking@redhat.com> , @Nizamudeen A <nia@redhat.com>, mind having a look to ensure the results from the rados suite look good to you?
@Venky Shankar <vshankar@redhat.com> mind having a look at the smoke suite? There was a resurgence of https://tracker.ceph.com/issues/57206. I don't see this as a blocker to the hotfix release, but LMK your thoughts.
*rados* - https://pulpito.ceph.com/yuriw-2024-03-04_20:52:58-rados-reef-release-distro... - https://pulpito.ceph.com/yuriw-2024-03-05_01:29:36-rados-reef-release-distro...
Failures, unrelated: 1. https://tracker.ceph.com/issues/64725 -- new tracker, but known issue 2. https://tracker.ceph.com/issues/61774 3. https://tracker.ceph.com/issues/49287 4. https://tracker.ceph.com/issues/55141 5. https://tracker.ceph.com/issues/59142 6. https://tracker.ceph.com/issues/64726 -- new tracker 7. https://tracker.ceph.com/issues/62992 8. https://tracker.ceph.com/issues/64208
Details: 1. rados/singleton: application not enabled on pool 'rbd' - Ceph - RADOS 2. centos 9 testing reveals rocksdb "Leak_StillReachable" memory leak in mons - Ceph - RADOS 3. podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope not found - Ceph - Orchestrator 4. thrashers/fastread: assertion failure: rollback_info_trimmed_to == head - Ceph - RADOS 5. mgr/dashboard: fix e2e for dashboard v3 - Ceph - Mgr - Dashboard 6. LibRadosAioEC.MultiWritePP hang and pkill - Ceph - RADOS 7. Heartbeat crash in reset_timeout and clear_timeout - Ceph - RADOS 8. test_cephadm.sh: Container version mismatch causes job to fail. - Ceph - Orchestrator
*smoke* - https://pulpito.ceph.com/yuriw-2024-03-05_15:31:54-smoke-reef-release-distro...
Failures, unrelated: 1. https://tracker.ceph.com/issues/52624 2. https://tracker.ceph.com/issues/57206 3. https://tracker.ceph.com/issues/64727 -- new tracker
Details: 1. qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" - Ceph 2. ceph_test_libcephfs_reclaim crashes during test - Ceph - CephFS 3. suites/dbench.sh: Socket exception: No route to host (113) - Infrastructure
On Tue, Mar 5, 2024 at 3:05 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Only suits below need approval:
smoke - Radek, Laura? rados - Radek, Laura?
We are also in the process of upgrading gibba and then LRC into 18.2.2 RC
On Tue, Mar 5, 2024 at 7:47 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/64721#note-1 Release Notes - TBD LRC upgrade - TBD
Seeking approvals/reviews for:
smoke - in progress rados - Radek, Laura? quincy-x - in progress
Also need approval from Travis, Redouane for Prometheus fix testing.
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
Hi Laura, On Wed, Mar 6, 2024 at 4:53 AM Laura Flores <lflores@redhat.com> wrote:
Here are the rados and smoke suite summaries.
@Radoslaw Zarzynski <rzarzyns@redhat.com>, @Adam King <adking@redhat.com> , @Nizamudeen A <nia@redhat.com>, mind having a look to ensure the results from the rados suite look good to you?
@Venky Shankar <vshankar@redhat.com> mind having a look at the smoke suite? There was a resurgence of https://tracker.ceph.com/issues/57206. I don't see this as a blocker to the hotfix release, but LMK your thoughts.
Yeh, that's not a blocker. The backport fix can be included for the next release.
*rados* - https://pulpito.ceph.com/yuriw-2024-03-04_20:52:58-rados-reef-release-distro... - https://pulpito.ceph.com/yuriw-2024-03-05_01:29:36-rados-reef-release-distro...
Failures, unrelated: 1. https://tracker.ceph.com/issues/64725 -- new tracker, but known issue 2. https://tracker.ceph.com/issues/61774 3. https://tracker.ceph.com/issues/49287 4. https://tracker.ceph.com/issues/55141 5. https://tracker.ceph.com/issues/59142 6. https://tracker.ceph.com/issues/64726 -- new tracker 7. https://tracker.ceph.com/issues/62992 8. https://tracker.ceph.com/issues/64208
Details: 1. rados/singleton: application not enabled on pool 'rbd' - Ceph - RADOS 2. centos 9 testing reveals rocksdb "Leak_StillReachable" memory leak in mons - Ceph - RADOS 3. podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope not found - Ceph - Orchestrator 4. thrashers/fastread: assertion failure: rollback_info_trimmed_to == head - Ceph - RADOS 5. mgr/dashboard: fix e2e for dashboard v3 - Ceph - Mgr - Dashboard 6. LibRadosAioEC.MultiWritePP hang and pkill - Ceph - RADOS 7. Heartbeat crash in reset_timeout and clear_timeout - Ceph - RADOS 8. test_cephadm.sh: Container version mismatch causes job to fail. - Ceph - Orchestrator
*smoke* - https://pulpito.ceph.com/yuriw-2024-03-05_15:31:54-smoke-reef-release-distro...
Failures, unrelated: 1. https://tracker.ceph.com/issues/52624 2. https://tracker.ceph.com/issues/57206 3. https://tracker.ceph.com/issues/64727 -- new tracker
Details: 1. qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" - Ceph 2. ceph_test_libcephfs_reclaim crashes during test - Ceph - CephFS 3. suites/dbench.sh: Socket exception: No route to host (113) - Infrastructure
On Tue, Mar 5, 2024 at 3:05 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Only suits below need approval:
smoke - Radek, Laura? rados - Radek, Laura?
We are also in the process of upgrading gibba and then LRC into 18.2.2 RC
On Tue, Mar 5, 2024 at 7:47 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/64721#note-1 Release Notes - TBD LRC upgrade - TBD
Seeking approvals/reviews for:
smoke - in progress rados - Radek, Laura? quincy-x - in progress
Also need approval from Travis, Redouane for Prometheus fix testing.
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
-- Cheers, Venky
+Patrick Donnelly On Tue, Mar 5, 2024 at 9:18 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/64721#note-1 Release Notes - TBD LRC upgrade - TBD
Seeking approvals/reviews for:
smoke - in progress rados - Radek, Laura? quincy-x - in progress
I think https://github.com/ceph/ceph/pull/55669 was supposed to be included in this hotfix (I recall Patrick mentioning this in last week's CLT). The change was merged into reef head last week.
Also need approval from Travis, Redouane for Prometheus fix testing. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Cheers, Venky
On Wed, Mar 6, 2024 at 2:55 AM Venky Shankar <vshankar@redhat.com> wrote:
+Patrick Donnelly
On Tue, Mar 5, 2024 at 9:18 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/64721#note-1 Release Notes - TBD LRC upgrade - TBD
Seeking approvals/reviews for:
smoke - in progress rados - Radek, Laura? quincy-x - in progress
I think
https://github.com/ceph/ceph/pull/55669
was supposed to be included in this hotfix (I recall Patrick mentioning this in last week's CLT). The change was merged into reef head last week.
That's fixing a bug in reef HEAD not v18.2.1 so no need to get into this hotfix. -- Patrick Donnelly, Ph.D. He / Him / His Red Hat Partner Engineer IBM, Inc. GPG: 19F28A586F808C2402351B93C3301A3E258DD79D
Went over the rados results with Radek. All looks good for the hotfix, and we are ready to upgrade the LRC. Rados approved! Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804 On Wed, Mar 6, 2024 at 9:20 AM Patrick Donnelly <pdonnell@redhat.com> wrote:
On Wed, Mar 6, 2024 at 2:55 AM Venky Shankar <vshankar@redhat.com> wrote:
+Patrick Donnelly
On Tue, Mar 5, 2024 at 9:18 PM Yuri Weinstein <yweinste@redhat.com>
wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/64721#note-1 Release Notes - TBD LRC upgrade - TBD
Seeking approvals/reviews for:
smoke - in progress rados - Radek, Laura? quincy-x - in progress
I think
https://github.com/ceph/ceph/pull/55669
was supposed to be included in this hotfix (I recall Patrick mentioning this in last week's CLT). The change was merged into reef head last week.
That's fixing a bug in reef HEAD not v18.2.1 so no need to get into this hotfix.
-- Patrick Donnelly, Ph.D. He / Him / His Red Hat Partner Engineer IBM, Inc. GPG: 19F28A586F808C2402351B93C3301A3E258DD79D _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
The LRC was upgraded. All is in order for v18.2.2. On Wed, Mar 6, 2024 at 1:18 PM Laura Flores <lflores@redhat.com> wrote:
Went over the rados results with Radek. All looks good for the hotfix, and we are ready to upgrade the LRC.
Rados approved!
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
On Wed, Mar 6, 2024 at 9:20 AM Patrick Donnelly <pdonnell@redhat.com> wrote:
On Wed, Mar 6, 2024 at 2:55 AM Venky Shankar <vshankar@redhat.com> wrote:
+Patrick Donnelly
On Tue, Mar 5, 2024 at 9:18 PM Yuri Weinstein <yweinste@redhat.com>
wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/64721#note-1 Release Notes - TBD LRC upgrade - TBD
Seeking approvals/reviews for:
smoke - in progress rados - Radek, Laura? quincy-x - in progress
I think
https://github.com/ceph/ceph/pull/55669
was supposed to be included in this hotfix (I recall Patrick mentioning this in last week's CLT). The change was merged into reef head last week.
That's fixing a bug in reef HEAD not v18.2.1 so no need to get into this hotfix.
-- Patrick Donnelly, Ph.D. He / Him / His Red Hat Partner Engineer IBM, Inc. GPG: 19F28A586F808C2402351B93C3301A3E258DD79D _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
participants (7)
-
Laura Flores
-
Nizamudeen A
-
Patrick Donnelly
-
Redouane Kachach
-
Travis Nielsen
-
Venky Shankar
-
Yuri Weinstein