quincy v17.2.6 QE Validation status
Details of this release are summarized here: https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD The reruns were in the queue for 4 days because of some slowness issues. The core team (Neha, Radek, Laura, and others) are trying to narrow down the root cause. Seeking approvals/reviews for: rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test and merge at least one PR https://github.com/ceph/ceph/pull/50575 for the core) rgw - Casey fs - Venky (the fs suite has an unusually high amount of failed jobs, any reason to suspect it in the observed slowness?) orch - Adam King rbd - Ilya krbd - Ilya upgrade/octopus-x - Laura is looking into failures upgrade/pacific-x - Laura is looking into failures upgrade/quincy-p2p - Laura is looking into failures client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman is looking into it powercycle - Brad ceph-volume - needs a rerun on merged https://github.com/ceph/ceph-ansible/pull/7409 Please reply to this email with approval and/or trackers of known issues/PRs to address them. Also, share any findings or hypnosis about the slowness in the execution of the suite. Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals. Thx YuriW
I reviewed the upgrade tests. I opened two new trackers: 1. https://tracker.ceph.com/issues/59121 - "No upgrade in progress" during upgrade tests - Ceph - Orchestrator 2. https://tracker.ceph.com/issues/59124 - "Health check failed: 1/3 mons down, quorum b,c (MON_DOWN)" during quincy p2p upgrade test - Ceph - RADOS Starred (*) trackers occurred frequently throughout the suites. https://pulpito.ceph.com/yuriw-2023-03-14_21:33:13-upgrade:octopus-x-quincy-... Failures: 1. *https://tracker.ceph.com/issues/59121 <https://tracker.ceph.com/issues/59121> *** 2. *https://tracker.ceph.com/issues/56393 <https://tracker.ceph.com/issues/56393> *** 3. https://tracker.ceph.com/issues/53615 Details: 1.* "No upgrade in progress" during upgrade tests - Ceph - Orchestrator* 2. *thrash-erasure-code-big: failed to complete snap trimming before timeout - Ceph - RADOS* 3. qa: upgrade test fails with "timeout expired in wait_until_healthy" - Ceph - CephFS https://pulpito.ceph.com/yuriw-2023-03-15_21:14:59-upgrade:pacific-x-quincy-... Failures: 1. *https://tracker.ceph.com/issues/56393 <https://tracker.ceph.com/issues/56393> *** 2. https://tracker.ceph.com/issues/58914 3. *https://tracker.ceph.com/issues/59121 <https://tracker.ceph.com/issues/59121> *** 4. https://tracker.ceph.com/issues/59123 Details: 1. *thrash-erasure-code-big: failed to complete snap trimming before timeout - Ceph - RADOS* 2. [ FAILED ] TestClsRbd.group_snap_list_max_read in upgrade:quincy-x-reef - Ceph - RBD 3. *"No upgrade in progress" during upgrade tests* 4. Timeout opening channel - Tools - Teuthology https://pulpito.ceph.com/yuriw-2023-03-14_21:36:24-upgrade:quincy-p2p-quincy... Failures: 1. https://tracker.ceph.com/issues/59124 Details: 1. "Health check failed: 1/3 mons down, quorum b,c (MON_DOWN)" during quincy p2p upgrade test - Ceph - RADOS https://pulpito.ceph.com/yuriw-2023-03-15_15:25:50-upgrade-clients:client-up... 1 test failed from failure to fetch package https://pulpito.ceph.com/yuriw-2023-03-15_15:26:37-upgrade-clients:client-up... All green On Tue, Mar 21, 2023 at 3:06 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The reruns were in the queue for 4 days because of some slowness issues. The core team (Neha, Radek, Laura, and others) are trying to narrow down the root cause.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test and merge at least one PR https://github.com/ceph/ceph/pull/50575 for the core) rgw - Casey fs - Venky (the fs suite has an unusually high amount of failed jobs, any reason to suspect it in the observed slowness?) orch - Adam King rbd - Ilya krbd - Ilya upgrade/octopus-x - Laura is looking into failures upgrade/pacific-x - Laura is looking into failures upgrade/quincy-p2p - Laura is looking into failures client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman is looking into it powercycle - Brad ceph-volume - needs a rerun on merged https://github.com/ceph/ceph-ansible/pull/7409
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Also, share any findings or hypnosis about the slowness in the execution of the suite.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
Thx YuriW _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
On Tue, Mar 21, 2023 at 4:06 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The reruns were in the queue for 4 days because of some slowness issues. The core team (Neha, Radek, Laura, and others) are trying to narrow down the root cause.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test and merge at least one PR https://github.com/ceph/ceph/pull/50575 for the core) rgw - Casey
there were some java_s3test failures related to https://tracker.ceph.com/issues/58554. i've added the fix to https://github.com/ceph/java_s3tests/commits/ceph-quincy, so a rerun should resolve those failures there were also some 'Failed to fetch package version' failures in the rerun that warranted another rerun anyway there's also an urgent priority bug fix in https://github.com/ceph/ceph/pull/50625 that i'd really like to add to this release; sorry for the late notice
fs - Venky (the fs suite has an unusually high amount of failed jobs, any reason to suspect it in the observed slowness?) orch - Adam King rbd - Ilya krbd - Ilya upgrade/octopus-x - Laura is looking into failures upgrade/pacific-x - Laura is looking into failures upgrade/quincy-p2p - Laura is looking into failures client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman is looking into it powercycle - Brad ceph-volume - needs a rerun on merged https://github.com/ceph/ceph-ansible/pull/7409
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Also, share any findings or hypnosis about the slowness in the execution of the suite.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
Thx YuriW _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
On Wed, Mar 22, 2023 at 9:27 AM Casey Bodley <cbodley@redhat.com> wrote:
On Tue, Mar 21, 2023 at 4:06 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The reruns were in the queue for 4 days because of some slowness issues. The core team (Neha, Radek, Laura, and others) are trying to narrow down the root cause.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test and merge at least one PR https://github.com/ceph/ceph/pull/50575 for the core) rgw - Casey
there were some java_s3test failures related to https://tracker.ceph.com/issues/58554. i've added the fix to https://github.com/ceph/java_s3tests/commits/ceph-quincy, so a rerun should resolve those failures there were also some 'Failed to fetch package version' failures in the rerun that warranted another rerun anyway
there's also an urgent priority bug fix in https://github.com/ceph/ceph/pull/50625 that i'd really like to add to this release; sorry for the late notice
this fix merged, so rgw is now approved. thanks Yuri
fs - Venky (the fs suite has an unusually high amount of failed jobs, any reason to suspect it in the observed slowness?) orch - Adam King rbd - Ilya krbd - Ilya upgrade/octopus-x - Laura is looking into failures upgrade/pacific-x - Laura is looking into failures upgrade/quincy-p2p - Laura is looking into failures client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman is looking into it powercycle - Brad ceph-volume - needs a rerun on merged https://github.com/ceph/ceph-ansible/pull/7409
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Also, share any findings or hypnosis about the slowness in the execution of the suite.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
Thx YuriW _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
We are still working on core PRs: https://github.com/ceph/ceph/pull/50549 https://github.com/ceph/ceph/pull/50625 - merged https://github.com/ceph/ceph/pull/50575 Will update as soon as we are ready for the next steps. On Thu, Mar 23, 2023 at 10:34 AM Casey Bodley <cbodley@redhat.com> wrote:
On Wed, Mar 22, 2023 at 9:27 AM Casey Bodley <cbodley@redhat.com> wrote:
On Tue, Mar 21, 2023 at 4:06 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The reruns were in the queue for 4 days because of some slowness issues. The core team (Neha, Radek, Laura, and others) are trying to narrow down the root cause.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test and merge at least one PR https://github.com/ceph/ceph/pull/50575 for the core) rgw - Casey
there were some java_s3test failures related to https://tracker.ceph.com/issues/58554. i've added the fix to https://github.com/ceph/java_s3tests/commits/ceph-quincy, so a rerun should resolve those failures there were also some 'Failed to fetch package version' failures in the rerun that warranted another rerun anyway
there's also an urgent priority bug fix in https://github.com/ceph/ceph/pull/50625 that i'd really like to add to this release; sorry for the late notice
this fix merged, so rgw is now approved. thanks Yuri
fs - Venky (the fs suite has an unusually high amount of failed jobs, any reason to suspect it in the observed slowness?) orch - Adam King rbd - Ilya krbd - Ilya upgrade/octopus-x - Laura is looking into failures upgrade/pacific-x - Laura is looking into failures upgrade/quincy-p2p - Laura is looking into failures client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman is looking into it powercycle - Brad ceph-volume - needs a rerun on merged https://github.com/ceph/ceph-ansible/pull/7409
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Also, share any findings or hypnosis about the slowness in the execution of the suite.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
Thx YuriW _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
https://github.com/ceph/ceph/pull/50575 was also merged. On Thu, Mar 23, 2023 at 12:36 PM Yuri Weinstein <yweinste@redhat.com> wrote:
We are still working on core PRs:
https://github.com/ceph/ceph/pull/50549 https://github.com/ceph/ceph/pull/50625 - merged https://github.com/ceph/ceph/pull/50575
Will update as soon as we are ready for the next steps.
On Thu, Mar 23, 2023 at 10:34 AM Casey Bodley <cbodley@redhat.com> wrote:
On Wed, Mar 22, 2023 at 9:27 AM Casey Bodley <cbodley@redhat.com> wrote:
On Tue, Mar 21, 2023 at 4:06 PM Yuri Weinstein <yweinste@redhat.com>
wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The reruns were in the queue for 4 days because of some slowness
issues.
The core team (Neha, Radek, Laura, and others) are trying to narrow down the root cause.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test and merge at least one PR https://github.com/ceph/ceph/pull/50575 for the core) rgw - Casey
there were some java_s3test failures related to https://tracker.ceph.com/issues/58554. i've added the fix to https://github.com/ceph/java_s3tests/commits/ceph-quincy, so a rerun should resolve those failures there were also some 'Failed to fetch package version' failures in the rerun that warranted another rerun anyway
there's also an urgent priority bug fix in https://github.com/ceph/ceph/pull/50625 that i'd really like to add to this release; sorry for the late notice
this fix merged, so rgw is now approved. thanks Yuri
fs - Venky (the fs suite has an unusually high amount of failed jobs, any reason to suspect it in the observed slowness?) orch - Adam King rbd - Ilya krbd - Ilya upgrade/octopus-x - Laura is looking into failures upgrade/pacific-x - Laura is looking into failures upgrade/quincy-p2p - Laura is looking into failures client-upgrade-octopus-quincy-quincy - missing packages, Adam
Kraitman
is looking into it powercycle - Brad ceph-volume - needs a rerun on merged https://github.com/ceph/ceph-ansible/pull/7409
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Also, share any findings or hypnosis about the slowness in the execution of the suite.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
Thx YuriW _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
We are all good on the Core end of things. https://github.com/ceph/ceph/pull/50549 is needed for downstream, but it should not block upstream. On Thu, Mar 23, 2023 at 12:59 PM Laura Flores <lflores@redhat.com> wrote:
https://github.com/ceph/ceph/pull/50575 was also merged.
On Thu, Mar 23, 2023 at 12:36 PM Yuri Weinstein <yweinste@redhat.com> wrote:
We are still working on core PRs:
https://github.com/ceph/ceph/pull/50549 https://github.com/ceph/ceph/pull/50625 - merged https://github.com/ceph/ceph/pull/50575
Will update as soon as we are ready for the next steps.
On Thu, Mar 23, 2023 at 10:34 AM Casey Bodley <cbodley@redhat.com> wrote:
On Wed, Mar 22, 2023 at 9:27 AM Casey Bodley <cbodley@redhat.com>
wrote:
On Tue, Mar 21, 2023 at 4:06 PM Yuri Weinstein <yweinste@redhat.com>
wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The reruns were in the queue for 4 days because of some slowness
issues.
The core team (Neha, Radek, Laura, and others) are trying to narrow down the root cause.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test and merge at least one PR https://github.com/ceph/ceph/pull/50575 for the core) rgw - Casey
there were some java_s3test failures related to https://tracker.ceph.com/issues/58554. i've added the fix to https://github.com/ceph/java_s3tests/commits/ceph-quincy, so a rerun should resolve those failures there were also some 'Failed to fetch package version' failures in the rerun that warranted another rerun anyway
there's also an urgent priority bug fix in https://github.com/ceph/ceph/pull/50625 that i'd really like to add to this release; sorry for the late notice
this fix merged, so rgw is now approved. thanks Yuri
fs - Venky (the fs suite has an unusually high amount of failed
jobs,
any reason to suspect it in the observed slowness?) orch - Adam King rbd - Ilya krbd - Ilya upgrade/octopus-x - Laura is looking into failures upgrade/pacific-x - Laura is looking into failures upgrade/quincy-p2p - Laura is looking into failures client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman is looking into it powercycle - Brad ceph-volume - needs a rerun on merged https://github.com/ceph/ceph-ansible/pull/7409
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Also, share any findings or hypnosis about the slowness in the execution of the suite.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
Thx YuriW _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
-- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
I will have a review of the rados suite ready soon. On Thu, Mar 23, 2023 at 1:44 PM Laura Flores <lflores@redhat.com> wrote:
We are all good on the Core end of things. https://github.com/ceph/ceph/pull/50549 is needed for downstream, but it should not block upstream.
On Thu, Mar 23, 2023 at 12:59 PM Laura Flores <lflores@redhat.com> wrote:
https://github.com/ceph/ceph/pull/50575 was also merged.
On Thu, Mar 23, 2023 at 12:36 PM Yuri Weinstein <yweinste@redhat.com> wrote:
We are still working on core PRs:
https://github.com/ceph/ceph/pull/50549 https://github.com/ceph/ceph/pull/50625 - merged https://github.com/ceph/ceph/pull/50575
Will update as soon as we are ready for the next steps.
On Thu, Mar 23, 2023 at 10:34 AM Casey Bodley <cbodley@redhat.com> wrote:
On Wed, Mar 22, 2023 at 9:27 AM Casey Bodley <cbodley@redhat.com>
On Tue, Mar 21, 2023 at 4:06 PM Yuri Weinstein <yweinste@redhat.com>
wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The reruns were in the queue for 4 days because of some slowness
issues.
The core team (Neha, Radek, Laura, and others) are trying to narrow down the root cause.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test and merge at least one PR https://github.com/ceph/ceph/pull/50575 for the core) rgw - Casey
there were some java_s3test failures related to https://tracker.ceph.com/issues/58554. i've added the fix to https://github.com/ceph/java_s3tests/commits/ceph-quincy, so a rerun should resolve those failures there were also some 'Failed to fetch package version' failures in
wrote: the
rerun that warranted another rerun anyway
there's also an urgent priority bug fix in https://github.com/ceph/ceph/pull/50625 that i'd really like to add to this release; sorry for the late notice
this fix merged, so rgw is now approved. thanks Yuri
fs - Venky (the fs suite has an unusually high amount of failed
jobs,
any reason to suspect it in the observed slowness?) orch - Adam King rbd - Ilya krbd - Ilya upgrade/octopus-x - Laura is looking into failures upgrade/pacific-x - Laura is looking into failures upgrade/quincy-p2p - Laura is looking into failures client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman is looking into it powercycle - Brad ceph-volume - needs a rerun on merged https://github.com/ceph/ceph-ansible/pull/7409
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Also, share any findings or hypnosis about the slowness in the execution of the suite.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
Thx YuriW _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
-- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
How much more time do we need to get PR 50549 in if we delayed v17.2.6? - Ken On Thu, Mar 23, 2023 at 2:44 PM Laura Flores <lflores@redhat.com> wrote:
We are all good on the Core end of things. https://github.com/ceph/ceph/pull/50549 is needed for downstream, but it should not block upstream.
On Thu, Mar 23, 2023 at 12:59 PM Laura Flores <lflores@redhat.com> wrote:
https://github.com/ceph/ceph/pull/50575 was also merged.
On Thu, Mar 23, 2023 at 12:36 PM Yuri Weinstein <yweinste@redhat.com> wrote:
We are still working on core PRs:
https://github.com/ceph/ceph/pull/50549 https://github.com/ceph/ceph/pull/50625 - merged https://github.com/ceph/ceph/pull/50575
Will update as soon as we are ready for the next steps.
On Thu, Mar 23, 2023 at 10:34 AM Casey Bodley <cbodley@redhat.com> wrote:
On Wed, Mar 22, 2023 at 9:27 AM Casey Bodley <cbodley@redhat.com>
On Tue, Mar 21, 2023 at 4:06 PM Yuri Weinstein <yweinste@redhat.com>
wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The reruns were in the queue for 4 days because of some slowness
issues.
The core team (Neha, Radek, Laura, and others) are trying to narrow down the root cause.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test and merge at least one PR https://github.com/ceph/ceph/pull/50575 for the core) rgw - Casey
there were some java_s3test failures related to https://tracker.ceph.com/issues/58554. i've added the fix to https://github.com/ceph/java_s3tests/commits/ceph-quincy, so a rerun should resolve those failures there were also some 'Failed to fetch package version' failures in
wrote: the
rerun that warranted another rerun anyway
there's also an urgent priority bug fix in https://github.com/ceph/ceph/pull/50625 that i'd really like to add to this release; sorry for the late notice
this fix merged, so rgw is now approved. thanks Yuri
fs - Venky (the fs suite has an unusually high amount of failed
jobs,
any reason to suspect it in the observed slowness?) orch - Adam King rbd - Ilya krbd - Ilya upgrade/octopus-x - Laura is looking into failures upgrade/pacific-x - Laura is looking into failures upgrade/quincy-p2p - Laura is looking into failures client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman is looking into it powercycle - Brad ceph-volume - needs a rerun on merged https://github.com/ceph/ceph-ansible/pull/7409
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Also, share any findings or hypnosis about the slowness in the execution of the suite.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
Thx YuriW _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
On Tue, Mar 21, 2023 at 9:06 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The reruns were in the queue for 4 days because of some slowness issues. The core team (Neha, Radek, Laura, and others) are trying to narrow down the root cause.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test and merge at least one PR https://github.com/ceph/ceph/pull/50575 for the core) rgw - Casey fs - Venky (the fs suite has an unusually high amount of failed jobs, any reason to suspect it in the observed slowness?) orch - Adam King rbd - Ilya krbd - Ilya
rbd and krbd approved. Thanks, Ilya
orch approved. After reruns, the only failed jobs in the orch run were orch/rook tests (which are broken currently) and 2 instances of "Test failure: test_non_existent_cluster". That failure is just a command expecting a zero return code and an error message instead of a nonzero return code in a failure case. I think the test got backported without the change of the error code, either way it's not a big deal. I also took a brief look at the orchestrator failure from the upgrade tests (https://tracker.ceph.com/issues/59121) that Laura saw. In the instance of it I looked at, It seems like the test is running "orch upgrade start" and then not running "orch upgrade pause" until about 20 minutes later, at which point the upgrade has already completed (and I can see all the daemons got upgraded to the new image in the logs). It looks like it was waiting on a loop to see a minority of the mons had been upgraded before pausing the upgrade, but even starting that loop took over 2 minutes, despite the only actions in between being a "ceph orch ps" call and echoing out a value. Really not sure why it was so slow in running those commands or why it happened 3 times in the initial run but never in the reruns, but the failure came from that, and the upgrade itself seems to still work fine. - Adam King
On Wed, Mar 22, 2023 at 1:36 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The reruns were in the queue for 4 days because of some slowness issues. The core team (Neha, Radek, Laura, and others) are trying to narrow down the root cause.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test and merge at least one PR https://github.com/ceph/ceph/pull/50575 for the core) rgw - Casey fs - Venky (the fs suite has an unusually high amount of failed jobs, any reason to suspect it in the observed slowness?)
Some of those failures can be attributed to slowness, but not all. Can we have a re-run?
orch - Adam King rbd - Ilya krbd - Ilya upgrade/octopus-x - Laura is looking into failures upgrade/pacific-x - Laura is looking into failures upgrade/quincy-p2p - Laura is looking into failures client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman is looking into it powercycle - Brad ceph-volume - needs a rerun on merged https://github.com/ceph/ceph-ansible/pull/7409
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Also, share any findings or hypnosis about the slowness in the execution of the suite.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
Thx YuriW _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Cheers, Venky
Details of this release are updated here: https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD The slowness we experienced seemed to be self-cured. Neha, Radek, and Laura please provide any findings if you have them. Seeking approvals/reviews for: rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 with PRs merged on top of quincy-release) rgw - Casey (rerun on Build 2 with PRs merged on top of quincy-release) fs - Venky upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any updates?) upgrade/pacific-x - Neha, Laura, Ilya see https://tracker.ceph.com/issues/58914 upgrade/quincy-p2p - Neha, Laura client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue Adam Kraitman any updates?) powercycle - Brad Please reply to this email with approval and/or trackers of known issues/PRs to address them. Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals. On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The reruns were in the queue for 4 days because of some slowness issues. The core team (Neha, Radek, Laura, and others) are trying to narrow down the root cause.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test and merge at least one PR https://github.com/ceph/ceph/pull/50575 for the core) rgw - Casey fs - Venky (the fs suite has an unusually high amount of failed jobs, any reason to suspect it in the observed slowness?) orch - Adam King rbd - Ilya krbd - Ilya upgrade/octopus-x - Laura is looking into failures upgrade/pacific-x - Laura is looking into failures upgrade/quincy-p2p - Laura is looking into failures client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman is looking into it powercycle - Brad ceph-volume - needs a rerun on merged https://github.com/ceph/ceph-ansible/pull/7409
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Also, share any findings or hypnosis about the slowness in the execution of the suite.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
Thx YuriW
On Sat, Mar 25, 2023 at 5:46 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are updated here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The slowness we experienced seemed to be self-cured. Neha, Radek, and Laura please provide any findings if you have them.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 with PRs merged on top of quincy-release) rgw - Casey (rerun on Build 2 with PRs merged on top of quincy-release) fs - Venky
upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any updates?) upgrade/pacific-x - Neha, Laura, Ilya see https://tracker.ceph.com/issues/58914 upgrade/quincy-p2p - Neha, Laura client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue Adam Kraitman any updates?) powercycle - Brad
Still the selinux ping issues I've been trying to pin down. Approved.
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The reruns were in the queue for 4 days because of some slowness issues. The core team (Neha, Radek, Laura, and others) are trying to narrow down the root cause.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test and merge at least one PR https://github.com/ceph/ceph/pull/50575 for the core) rgw - Casey fs - Venky (the fs suite has an unusually high amount of failed jobs, any reason to suspect it in the observed slowness?) orch - Adam King rbd - Ilya krbd - Ilya upgrade/octopus-x - Laura is looking into failures upgrade/pacific-x - Laura is looking into failures upgrade/quincy-p2p - Laura is looking into failures client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman is looking into it powercycle - Brad ceph-volume - needs a rerun on merged https://github.com/ceph/ceph-ansible/pull/7409
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Also, share any findings or hypnosis about the slowness in the execution of the suite.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
Thx YuriW
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
-- Cheers, Brad
Dashboard LGTM! On Sat, Mar 25, 2023 at 1:16 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are updated here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The slowness we experienced seemed to be self-cured. Neha, Radek, and Laura please provide any findings if you have them.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 with PRs merged on top of quincy-release) rgw - Casey (rerun on Build 2 with PRs merged on top of quincy-release) fs - Venky
upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any updates?) upgrade/pacific-x - Neha, Laura, Ilya see https://tracker.ceph.com/issues/58914 upgrade/quincy-p2p <https://tracker.ceph.com/issues/58914upgrade/quincy-p2p> - Neha, Laura client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue Adam Kraitman any updates?) powercycle - Brad
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The reruns were in the queue for 4 days because of some slowness issues. The core team (Neha, Radek, Laura, and others) are trying to narrow down the root cause.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test and merge at least one PR https://github.com/ceph/ceph/pull/50575 for the core) rgw - Casey fs - Venky (the fs suite has an unusually high amount of failed jobs, any reason to suspect it in the observed slowness?) orch - Adam King rbd - Ilya krbd - Ilya upgrade/octopus-x - Laura is looking into failures upgrade/pacific-x - Laura is looking into failures upgrade/quincy-p2p - Laura is looking into failures client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman is looking into it powercycle - Brad ceph-volume - needs a rerun on merged https://github.com/ceph/ceph-ansible/pull/7409
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Also, share any findings or hypnosis about the slowness in the execution of the suite.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
Thx YuriW
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
On Sat, Mar 25, 2023 at 1:17 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are updated here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The slowness we experienced seemed to be self-cured. Neha, Radek, and Laura please provide any findings if you have them.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 with PRs merged on top of quincy-release) rgw - Casey (rerun on Build 2 with PRs merged on top of quincy-release) fs - Venky
fs approved.
upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any updates?) upgrade/pacific-x - Neha, Laura, Ilya see https://tracker.ceph.com/issues/58914 upgrade/quincy-p2p - Neha, Laura client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue Adam Kraitman any updates?) powercycle - Brad
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The reruns were in the queue for 4 days because of some slowness issues. The core team (Neha, Radek, Laura, and others) are trying to narrow down the root cause.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test and merge at least one PR https://github.com/ceph/ceph/pull/50575 for the core) rgw - Casey fs - Venky (the fs suite has an unusually high amount of failed jobs, any reason to suspect it in the observed slowness?) orch - Adam King rbd - Ilya krbd - Ilya upgrade/octopus-x - Laura is looking into failures upgrade/pacific-x - Laura is looking into failures upgrade/quincy-p2p - Laura is looking into failures client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman is looking into it powercycle - Brad ceph-volume - needs a rerun on merged https://github.com/ceph/ceph-ansible/pull/7409
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Also, share any findings or hypnosis about the slowness in the execution of the suite.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
Thx YuriW
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Cheers, Venky
On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are updated here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The slowness we experienced seemed to be self-cured. Neha, Radek, and Laura please provide any findings if you have them.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 with PRs merged on top of quincy-release) rgw - Casey (rerun on Build 2 with PRs merged on top of quincy-release)
rgw approved
fs - Venky
upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any updates?) upgrade/pacific-x - Neha, Laura, Ilya see https://tracker.ceph.com/issues/58914 upgrade/quincy-p2p - Neha, Laura client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue Adam Kraitman any updates?) powercycle - Brad
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The reruns were in the queue for 4 days because of some slowness issues. The core team (Neha, Radek, Laura, and others) are trying to narrow down the root cause.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test and merge at least one PR https://github.com/ceph/ceph/pull/50575 for the core) rgw - Casey fs - Venky (the fs suite has an unusually high amount of failed jobs, any reason to suspect it in the observed slowness?) orch - Adam King rbd - Ilya krbd - Ilya upgrade/octopus-x - Laura is looking into failures upgrade/pacific-x - Laura is looking into failures upgrade/quincy-p2p - Laura is looking into failures client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman is looking into it powercycle - Brad ceph-volume - needs a rerun on merged https://github.com/ceph/ceph-ansible/pull/7409
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Also, share any findings or hypnosis about the slowness in the execution of the suite.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
Thx YuriW
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
Rados review, second round: Failures: 1. https://tracker.ceph.com/issues/58560 2. https://tracker.ceph.com/issues/58476 3. https://tracker.ceph.com/issues/58475 -- pending Q backport 4. https://tracker.ceph.com/issues/49287 5. https://tracker.ceph.com/issues/58585 Details: 1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo - Infrastructure 2. test_non_existent_cluster: cluster does not exist - Ceph - Orchestrator 3. test_dashboard_e2e.sh: Conflicting peer dependency: postcss@8.4.21 - Ceph - Mgr - Dashboard 4. podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope not found - Ceph - Orchestrator 5. rook: failed to pull kubelet image - Ceph - Orchestrator @Radoslaw Zarzynski <rzarzyns@redhat.com> will give final approval for rados. On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley <cbodley@redhat.com> wrote:
On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are updated here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The slowness we experienced seemed to be self-cured. Neha, Radek, and Laura please provide any findings if you have them.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 with PRs merged on top of quincy-release) rgw - Casey (rerun on Build 2 with PRs merged on top of quincy-release)
rgw approved
fs - Venky
upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any updates?) upgrade/pacific-x - Neha, Laura, Ilya see https://tracker.ceph.com/issues/58914 upgrade/quincy-p2p - Neha, Laura client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue Adam Kraitman any updates?) powercycle - Brad
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The reruns were in the queue for 4 days because of some slowness
issues.
The core team (Neha, Radek, Laura, and others) are trying to narrow down the root cause.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test and merge at least one PR https://github.com/ceph/ceph/pull/50575 for the core) rgw - Casey fs - Venky (the fs suite has an unusually high amount of failed jobs, any reason to suspect it in the observed slowness?) orch - Adam King rbd - Ilya krbd - Ilya upgrade/octopus-x - Laura is looking into failures upgrade/pacific-x - Laura is looking into failures upgrade/quincy-p2p - Laura is looking into failures client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman is looking into it powercycle - Brad ceph-volume - needs a rerun on merged https://github.com/ceph/ceph-ansible/pull/7409
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Also, share any findings or hypnosis about the slowness in the execution of the suite.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
Thx YuriW
Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
-- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
rados: approved! On Mon, Mar 27, 2023 at 7:02 PM Laura Flores <lflores@redhat.com> wrote:
Rados review, second round:
Failures: 1. https://tracker.ceph.com/issues/58560 2. https://tracker.ceph.com/issues/58476 3. https://tracker.ceph.com/issues/58475 -- pending Q backport 4. https://tracker.ceph.com/issues/49287 5. https://tracker.ceph.com/issues/58585
Details: 1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo - Infrastructure 2. test_non_existent_cluster: cluster does not exist - Ceph - Orchestrator 3. test_dashboard_e2e.sh: Conflicting peer dependency: postcss@8.4.21 - Ceph - Mgr - Dashboard 4. podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope not found - Ceph - Orchestrator 5. rook: failed to pull kubelet image - Ceph - Orchestrator
@Radoslaw Zarzynski <rzarzyns@redhat.com> will give final approval for rados.
On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley <cbodley@redhat.com> wrote:
On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are updated here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The slowness we experienced seemed to be self-cured. Neha, Radek, and Laura please provide any findings if you have them.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 with PRs merged on top of quincy-release) rgw - Casey (rerun on Build 2 with PRs merged on top of quincy-release)
rgw approved
fs - Venky
upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any updates?) upgrade/pacific-x - Neha, Laura, Ilya see https://tracker.ceph.com/issues/58914 upgrade/quincy-p2p - Neha, Laura client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue Adam Kraitman any updates?) powercycle - Brad
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The reruns were in the queue for 4 days because of some slowness
issues.
The core team (Neha, Radek, Laura, and others) are trying to narrow down the root cause.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test and merge at least one PR https://github.com/ceph/ceph/pull/50575 for the core) rgw - Casey fs - Venky (the fs suite has an unusually high amount of failed jobs, any reason to suspect it in the observed slowness?) orch - Adam King rbd - Ilya krbd - Ilya upgrade/octopus-x - Laura is looking into failures upgrade/pacific-x - Laura is looking into failures upgrade/quincy-p2p - Laura is looking into failures client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman is looking into it powercycle - Brad ceph-volume - needs a rerun on merged https://github.com/ceph/ceph-ansible/pull/7409
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Also, share any findings or hypnosis about the slowness in the execution of the suite.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
Thx YuriW
Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
upgrades approved! Thanks, Neha On Tue, Mar 28, 2023 at 12:09 PM Radoslaw Zarzynski <rzarzyns@redhat.com> wrote:
rados: approved!
On Mon, Mar 27, 2023 at 7:02 PM Laura Flores <lflores@redhat.com> wrote:
Rados review, second round:
Failures: 1. https://tracker.ceph.com/issues/58560 2. https://tracker.ceph.com/issues/58476 3. https://tracker.ceph.com/issues/58475 -- pending Q backport 4. https://tracker.ceph.com/issues/49287 5. https://tracker.ceph.com/issues/58585
Details: 1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo - Infrastructure 2. test_non_existent_cluster: cluster does not exist - Ceph - Orchestrator 3. test_dashboard_e2e.sh: Conflicting peer dependency: postcss@8.4.21 - Ceph - Mgr - Dashboard 4. podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope not found - Ceph - Orchestrator 5. rook: failed to pull kubelet image - Ceph - Orchestrator
@Radoslaw Zarzynski <rzarzyns@redhat.com> will give final approval for rados.
On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley <cbodley@redhat.com> wrote:
On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are updated here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The slowness we experienced seemed to be self-cured. Neha, Radek, and Laura please provide any findings if you have them.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 with PRs merged on top of quincy-release) rgw - Casey (rerun on Build 2 with PRs merged on top of quincy-release)
rgw approved
fs - Venky
upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any updates?) upgrade/pacific-x - Neha, Laura, Ilya see https://tracker.ceph.com/issues/58914 upgrade/quincy-p2p - Neha, Laura client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue Adam Kraitman any updates?) powercycle - Brad
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The reruns were in the queue for 4 days because of some slowness
issues.
The core team (Neha, Radek, Laura, and others) are trying to narrow down the root cause.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test and merge at least one PR https://github.com/ceph/ceph/pull/50575 for the core) rgw - Casey fs - Venky (the fs suite has an unusually high amount of failed jobs, any reason to suspect it in the observed slowness?) orch - Adam King rbd - Ilya krbd - Ilya upgrade/octopus-x - Laura is looking into failures upgrade/pacific-x - Laura is looking into failures upgrade/quincy-p2p - Laura is looking into failures client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman is looking into it powercycle - Brad ceph-volume - needs a rerun on merged https://github.com/ceph/ceph-ansible/pull/7409
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Also, share any findings or hypnosis about the slowness in the execution of the suite.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
Thx YuriW
Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
The release has been approved. And the gibba cluster upgraded. We are awaiting the LRC upgrade and then/or in parallel will publish an RC for testing. ETA for the release publishing is 04/07/23 On Tue, Mar 28, 2023 at 2:59 PM Neha Ojha <nojha@redhat.com> wrote:
upgrades approved!
Thanks, Neha
On Tue, Mar 28, 2023 at 12:09 PM Radoslaw Zarzynski <rzarzyns@redhat.com> wrote:
rados: approved!
On Mon, Mar 27, 2023 at 7:02 PM Laura Flores <lflores@redhat.com> wrote:
Rados review, second round:
Failures: 1. https://tracker.ceph.com/issues/58560 2. https://tracker.ceph.com/issues/58476 3. https://tracker.ceph.com/issues/58475 -- pending Q backport 4. https://tracker.ceph.com/issues/49287 5. https://tracker.ceph.com/issues/58585
Details: 1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo - Infrastructure 2. test_non_existent_cluster: cluster does not exist - Ceph - Orchestrator 3. test_dashboard_e2e.sh: Conflicting peer dependency: postcss@8.4.21 - Ceph - Mgr - Dashboard 4. podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope not found - Ceph - Orchestrator 5. rook: failed to pull kubelet image - Ceph - Orchestrator
@Radoslaw Zarzynski <rzarzyns@redhat.com> will give final approval for rados.
On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley <cbodley@redhat.com> wrote:
On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are updated here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The slowness we experienced seemed to be self-cured. Neha, Radek, and Laura please provide any findings if you have them.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 with PRs merged on top of quincy-release) rgw - Casey (rerun on Build 2 with PRs merged on top of
quincy-release)
rgw approved
fs - Venky
upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any updates?) upgrade/pacific-x - Neha, Laura, Ilya see https://tracker.ceph.com/issues/58914 upgrade/quincy-p2p - Neha, Laura client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue Adam Kraitman any updates?) powercycle - Brad
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The reruns were in the queue for 4 days because of some slowness
issues.
The core team (Neha, Radek, Laura, and others) are trying to narrow down the root cause.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test and merge at least one PR https://github.com/ceph/ceph/pull/50575 for the core) rgw - Casey fs - Venky (the fs suite has an unusually high amount of failed jobs, any reason to suspect it in the observed slowness?) orch - Adam King rbd - Ilya krbd - Ilya upgrade/octopus-x - Laura is looking into failures upgrade/pacific-x - Laura is looking into failures upgrade/quincy-p2p - Laura is looking into failures client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman is looking into it powercycle - Brad ceph-volume - needs a rerun on merged https://github.com/ceph/ceph-ansible/pull/7409
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Also, share any findings or hypnosis about the slowness in the execution of the suite.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
Thx YuriW
Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
Josh, the release is ready for your review and approval. Adam, can you please update the LRC upgrade to 17.2.6 RC? Thx On Wed, Mar 29, 2023 at 3:07 PM Yuri Weinstein <yweinste@redhat.com> wrote:
The release has been approved.
And the gibba cluster upgraded.
We are awaiting the LRC upgrade and then/or in parallel will publish an RC for testing.
ETA for the release publishing is 04/07/23
On Tue, Mar 28, 2023 at 2:59 PM Neha Ojha <nojha@redhat.com> wrote:
upgrades approved!
Thanks, Neha
On Tue, Mar 28, 2023 at 12:09 PM Radoslaw Zarzynski <rzarzyns@redhat.com> wrote:
rados: approved!
On Mon, Mar 27, 2023 at 7:02 PM Laura Flores <lflores@redhat.com> wrote:
Rados review, second round:
Failures: 1. https://tracker.ceph.com/issues/58560 2. https://tracker.ceph.com/issues/58476 3. https://tracker.ceph.com/issues/58475 -- pending Q backport 4. https://tracker.ceph.com/issues/49287 5. https://tracker.ceph.com/issues/58585
Details: 1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo - Infrastructure 2. test_non_existent_cluster: cluster does not exist - Ceph - Orchestrator 3. test_dashboard_e2e.sh: Conflicting peer dependency: postcss@8.4.21 - Ceph - Mgr - Dashboard 4. podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope not found - Ceph - Orchestrator 5. rook: failed to pull kubelet image - Ceph - Orchestrator
@Radoslaw Zarzynski <rzarzyns@redhat.com> will give final approval for rados.
On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley <cbodley@redhat.com> wrote:
On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are updated here:
https://tracker.ceph.com/issues/59070#note-1 Release Notes - TBD
The slowness we experienced seemed to be self-cured. Neha, Radek, and Laura please provide any findings if you have them.
Seeking approvals/reviews for:
rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2
with
PRs merged on top of quincy-release) rgw - Casey (rerun on Build 2 with PRs merged on top of quincy-release)
rgw approved
fs - Venky
upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any updates?) upgrade/pacific-x - Neha, Laura, Ilya see https://tracker.ceph.com/issues/58914 upgrade/quincy-p2p - Neha, Laura client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue Adam Kraitman any updates?) powercycle - Brad
Please reply to this email with approval and/or trackers of known issues/PRs to address them.
Josh, Neha - gibba and LRC upgrades pending major suites approvals. RC release - pending major suites approvals.
On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein <yweinste@redhat.com> wrote: > > Details of this release are summarized here: > > https://tracker.ceph.com/issues/59070#note-1 > Release Notes - TBD > > The reruns were in the queue for 4 days because of some slowness issues. > The core team (Neha, Radek, Laura, and others) are trying to narrow > down the root cause. > > Seeking approvals/reviews for: > > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test > and merge at least one PR https://github.com/ceph/ceph/pull/50575 for > the core) > rgw - Casey > fs - Venky (the fs suite has an unusually high amount of failed jobs, > any reason to suspect it in the observed slowness?) > orch - Adam King > rbd - Ilya > krbd - Ilya > upgrade/octopus-x - Laura is looking into failures > upgrade/pacific-x - Laura is looking into failures > upgrade/quincy-p2p - Laura is looking into failures > client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman > is looking into it > powercycle - Brad > ceph-volume - needs a rerun on merged > https://github.com/ceph/ceph-ansible/pull/7409 > > Please reply to this email with approval and/or trackers of known > issues/PRs to address them. > > Also, share any findings or hypnosis about the slowness in the > execution of the suite. > > Josh, Neha - gibba and LRC upgrades pending major suites approvals. > RC release - pending major suites approvals. > > Thx > YuriW _______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
The LRC upgraded with no problems, so this release is good to go! Josh On Mon, Apr 3, 2023 at 3:36 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Josh, the release is ready for your review and approval.
Adam, can you please update the LRC upgrade to 17.2.6 RC?
Thx
On Wed, Mar 29, 2023 at 3:07 PM Yuri Weinstein <yweinste@redhat.com> wrote:
The release has been approved.
And the gibba cluster upgraded.
We are awaiting the LRC upgrade and then/or in parallel will publish an RC for testing.
ETA for the release publishing is 04/07/23
On Tue, Mar 28, 2023 at 2:59 PM Neha Ojha <nojha@redhat.com> wrote:
upgrades approved!
Thanks, Neha
On Tue, Mar 28, 2023 at 12:09 PM Radoslaw Zarzynski <rzarzyns@redhat.com> wrote:
rados: approved!
On Mon, Mar 27, 2023 at 7:02 PM Laura Flores <lflores@redhat.com> wrote:
Rados review, second round:
Failures: 1. https://tracker.ceph.com/issues/58560 2. https://tracker.ceph.com/issues/58476 3. https://tracker.ceph.com/issues/58475 -- pending Q backport 4. https://tracker.ceph.com/issues/49287 5. https://tracker.ceph.com/issues/58585
Details: 1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo - Infrastructure 2. test_non_existent_cluster: cluster does not exist - Ceph - Orchestrator 3. test_dashboard_e2e.sh: Conflicting peer dependency: postcss@8.4.21 - Ceph - Mgr - Dashboard 4. podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope not found - Ceph - Orchestrator 5. rook: failed to pull kubelet image - Ceph - Orchestrator
@Radoslaw Zarzynski <rzarzyns@redhat.com> will give final approval for rados.
On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley <cbodley@redhat.com> wrote:
On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein <yweinste@redhat.com> wrote: > > Details of this release are updated here: > > https://tracker.ceph.com/issues/59070#note-1 > Release Notes - TBD > > The slowness we experienced seemed to be self-cured. > Neha, Radek, and Laura please provide any findings if you have them. > > Seeking approvals/reviews for: > > rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 with > PRs merged on top of quincy-release) > rgw - Casey (rerun on Build 2 with PRs merged on top of quincy-release)
rgw approved
> fs - Venky > > upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any updates?) > upgrade/pacific-x - Neha, Laura, Ilya see https://tracker.ceph.com/issues/58914 > upgrade/quincy-p2p - Neha, Laura > client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue Adam > Kraitman any updates?) > powercycle - Brad > > Please reply to this email with approval and/or trackers of known > issues/PRs to address them. > > Josh, Neha - gibba and LRC upgrades pending major suites approvals. > RC release - pending major suites approvals. > > On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein <yweinste@redhat.com> wrote: > > > > Details of this release are summarized here: > > > > https://tracker.ceph.com/issues/59070#note-1 > > Release Notes - TBD > > > > The reruns were in the queue for 4 days because of some slowness issues. > > The core team (Neha, Radek, Laura, and others) are trying to narrow > > down the root cause. > > > > Seeking approvals/reviews for: > > > > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to test > > and merge at least one PR https://github.com/ceph/ceph/pull/50575 for > > the core) > > rgw - Casey > > fs - Venky (the fs suite has an unusually high amount of failed jobs, > > any reason to suspect it in the observed slowness?) > > orch - Adam King > > rbd - Ilya > > krbd - Ilya > > upgrade/octopus-x - Laura is looking into failures > > upgrade/pacific-x - Laura is looking into failures > > upgrade/quincy-p2p - Laura is looking into failures > > client-upgrade-octopus-quincy-quincy - missing packages, Adam Kraitman > > is looking into it > > powercycle - Brad > > ceph-volume - needs a rerun on merged > > https://github.com/ceph/ceph-ansible/pull/7409 > > > > Please reply to this email with approval and/or trackers of known > > issues/PRs to address them. > > > > Also, share any findings or hypnosis about the slowness in the > > execution of the suite. > > > > Josh, Neha - gibba and LRC upgrades pending major suites approvals. > > RC release - pending major suites approvals. > > > > Thx > > YuriW > _______________________________________________ > Dev mailing list -- dev@ceph.io > To unsubscribe send an email to dev-leave@ceph.io _______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
Sorry for interfereing, but: Woooohoooo!! Thank you so much for the great work. Can't wait for the release with a good chance to get access to my data again. On 05.04.23 16:15, Josh Durgin wrote:
The LRC upgraded with no problems, so this release is good to go!
Josh
On Mon, Apr 3, 2023 at 3:36 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Josh, the release is ready for your review and approval.
Adam, can you please update the LRC upgrade to 17.2.6 RC?
Thx
On Wed, Mar 29, 2023 at 3:07 PM Yuri Weinstein <yweinste@redhat.com> wrote:
The release has been approved.
And the gibba cluster upgraded.
We are awaiting the LRC upgrade and then/or in parallel will publish an RC for testing.
ETA for the release publishing is 04/07/23
On Tue, Mar 28, 2023 at 2:59 PM Neha Ojha <nojha@redhat.com> wrote:
upgrades approved!
Thanks, Neha
On Tue, Mar 28, 2023 at 12:09 PM Radoslaw Zarzynski <rzarzyns@redhat.com> wrote:
rados: approved!
On Mon, Mar 27, 2023 at 7:02 PM Laura Flores <lflores@redhat.com> wrote:
Rados review, second round:
Failures: 1. https://tracker.ceph.com/issues/58560 2. https://tracker.ceph.com/issues/58476 3. https://tracker.ceph.com/issues/58475 -- pending Q backport 4. https://tracker.ceph.com/issues/49287 5. https://tracker.ceph.com/issues/58585
Details: 1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo - Infrastructure 2. test_non_existent_cluster: cluster does not exist - Ceph - Orchestrator 3. test_dashboard_e2e.sh: Conflicting peer dependency: postcss@8.4.21 - Ceph - Mgr - Dashboard 4. podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope not found - Ceph - Orchestrator 5. rook: failed to pull kubelet image - Ceph - Orchestrator
@Radoslaw Zarzynski <rzarzyns@redhat.com> will give final approval for rados.
On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley <cbodley@redhat.com> wrote:
> On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein <yweinste@redhat.com> > wrote: >> >> Details of this release are updated here: >> >> https://tracker.ceph.com/issues/59070#note-1 >> Release Notes - TBD >> >> The slowness we experienced seemed to be self-cured. >> Neha, Radek, and Laura please provide any findings if you have them. >> >> Seeking approvals/reviews for: >> >> rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 > with >> PRs merged on top of quincy-release) >> rgw - Casey (rerun on Build 2 with PRs merged on top of > quincy-release) > > rgw approved > >> fs - Venky >> >> upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any > updates?) >> upgrade/pacific-x - Neha, Laura, Ilya see > https://tracker.ceph.com/issues/58914 >> upgrade/quincy-p2p - Neha, Laura >> client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue > Adam >> Kraitman any updates?) >> powercycle - Brad >> >> Please reply to this email with approval and/or trackers of known >> issues/PRs to address them. >> >> Josh, Neha - gibba and LRC upgrades pending major suites approvals. >> RC release - pending major suites approvals. >> >> On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein <yweinste@redhat.com> > wrote: >>> >>> Details of this release are summarized here: >>> >>> https://tracker.ceph.com/issues/59070#note-1 >>> Release Notes - TBD >>> >>> The reruns were in the queue for 4 days because of some slowness > issues. >>> The core team (Neha, Radek, Laura, and others) are trying to > narrow >>> down the root cause. >>> >>> Seeking approvals/reviews for: >>> >>> rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to > test >>> and merge at least one PR https://github.com/ceph/ceph/pull/50575 > for >>> the core) >>> rgw - Casey >>> fs - Venky (the fs suite has an unusually high amount of failed > jobs, >>> any reason to suspect it in the observed slowness?) >>> orch - Adam King >>> rbd - Ilya >>> krbd - Ilya >>> upgrade/octopus-x - Laura is looking into failures >>> upgrade/pacific-x - Laura is looking into failures >>> upgrade/quincy-p2p - Laura is looking into failures >>> client-upgrade-octopus-quincy-quincy - missing packages, Adam > Kraitman >>> is looking into it >>> powercycle - Brad >>> ceph-volume - needs a rerun on merged >>> https://github.com/ceph/ceph-ansible/pull/7409 >>> >>> Please reply to this email with approval and/or trackers of known >>> issues/PRs to address them. >>> >>> Also, share any findings or hypnosis about the slowness in the >>> execution of the suite. >>> >>> Josh, Neha - gibba and LRC upgrades pending major suites > approvals. >>> RC release - pending major suites approvals. >>> >>> Thx >>> YuriW >> _______________________________________________ >> Dev mailing list -- dev@ceph.io >> To unsubscribe send an email to dev-leave@ceph.io > _______________________________________________ > Dev mailing list -- dev@ceph.io > To unsubscribe send an email to dev-leave@ceph.io >
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
dev@ceph.io stop ________________________________ From: Thomas Widhalm <widhalmt@widhalm.or.at> Sent: Wednesday, April 5, 2023 7:26 AM To: ceph-users@ceph.io <ceph-users@ceph.io> Subject: [ceph-users] Re: quincy v17.2.6 QE Validation status Sorry for interfereing, but: Woooohoooo!! Thank you so much for the great work. Can't wait for the release with a good chance to get access to my data again. On 05.04.23 16:15, Josh Durgin wrote:
The LRC upgraded with no problems, so this release is good to go!
Josh
On Mon, Apr 3, 2023 at 3:36 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Josh, the release is ready for your review and approval.
Adam, can you please update the LRC upgrade to 17.2.6 RC?
Thx
On Wed, Mar 29, 2023 at 3:07 PM Yuri Weinstein <yweinste@redhat.com> wrote:
The release has been approved.
And the gibba cluster upgraded.
We are awaiting the LRC upgrade and then/or in parallel will publish an RC for testing.
ETA for the release publishing is 04/07/23
On Tue, Mar 28, 2023 at 2:59 PM Neha Ojha <nojha@redhat.com> wrote:
upgrades approved!
Thanks, Neha
On Tue, Mar 28, 2023 at 12:09 PM Radoslaw Zarzynski <rzarzyns@redhat.com> wrote:
rados: approved!
On Mon, Mar 27, 2023 at 7:02 PM Laura Flores <lflores@redhat.com> wrote:
Rados review, second round:
Failures: 1. https://tracker.ceph.com/issues/58560 2. https://tracker.ceph.com/issues/58476 3. https://tracker.ceph.com/issues/58475 -- pending Q backport 4. https://tracker.ceph.com/issues/49287 5. https://tracker.ceph.com/issues/58585
Details: 1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo - Infrastructure 2. test_non_existent_cluster: cluster does not exist - Ceph - Orchestrator 3. test_dashboard_e2e.sh: Conflicting peer dependency: postcss@8.4.21 - Ceph - Mgr - Dashboard 4. podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope not found - Ceph - Orchestrator 5. rook: failed to pull kubelet image - Ceph - Orchestrator
@Radoslaw Zarzynski <rzarzyns@redhat.com> will give final approval for rados.
On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley <cbodley@redhat.com> wrote:
> On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein <yweinste@redhat.com> > wrote: >> >> Details of this release are updated here: >> >> https://tracker.ceph.com/issues/59070#note-1 >> Release Notes - TBD >> >> The slowness we experienced seemed to be self-cured. >> Neha, Radek, and Laura please provide any findings if you have them. >> >> Seeking approvals/reviews for: >> >> rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 > with >> PRs merged on top of quincy-release) >> rgw - Casey (rerun on Build 2 with PRs merged on top of > quincy-release) > > rgw approved > >> fs - Venky >> >> upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any > updates?) >> upgrade/pacific-x - Neha, Laura, Ilya see > https://tracker.ceph.com/issues/58914 >> upgrade/quincy-p2p - Neha, Laura >> client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue > Adam >> Kraitman any updates?) >> powercycle - Brad >> >> Please reply to this email with approval and/or trackers of known >> issues/PRs to address them. >> >> Josh, Neha - gibba and LRC upgrades pending major suites approvals. >> RC release - pending major suites approvals. >> >> On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein <yweinste@redhat.com> > wrote: >>> >>> Details of this release are summarized here: >>> >>> https://tracker.ceph.com/issues/59070#note-1 >>> Release Notes - TBD >>> >>> The reruns were in the queue for 4 days because of some slowness > issues. >>> The core team (Neha, Radek, Laura, and others) are trying to > narrow >>> down the root cause. >>> >>> Seeking approvals/reviews for: >>> >>> rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to > test >>> and merge at least one PR https://github.com/ceph/ceph/pull/50575 > for >>> the core) >>> rgw - Casey >>> fs - Venky (the fs suite has an unusually high amount of failed > jobs, >>> any reason to suspect it in the observed slowness?) >>> orch - Adam King >>> rbd - Ilya >>> krbd - Ilya >>> upgrade/octopus-x - Laura is looking into failures >>> upgrade/pacific-x - Laura is looking into failures >>> upgrade/quincy-p2p - Laura is looking into failures >>> client-upgrade-octopus-quincy-quincy - missing packages, Adam > Kraitman >>> is looking into it >>> powercycle - Brad >>> ceph-volume - needs a rerun on merged >>> https://github.com/ceph/ceph-ansible/pull/7409 >>> >>> Please reply to this email with approval and/or trackers of known >>> issues/PRs to address them. >>> >>> Also, share any findings or hypnosis about the slowness in the >>> execution of the suite. >>> >>> Josh, Neha - gibba and LRC upgrades pending major suites > approvals. >>> RC release - pending major suites approvals. >>> >>> Thx >>> YuriW >> _______________________________________________ >> Dev mailing list -- dev@ceph.io >> To unsubscribe send an email to dev-leave@ceph.io > _______________________________________________ > Dev mailing list -- dev@ceph.io > To unsubscribe send an email to dev-leave@ceph.io >
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users@ceph.io stop ________________________________ From: Thomas Widhalm <widhalmt@widhalm.or.at> Sent: Wednesday, April 5, 2023 7:26 AM To: ceph-users@ceph.io <ceph-users@ceph.io> Subject: [ceph-users] Re: quincy v17.2.6 QE Validation status Sorry for interfereing, but: Woooohoooo!! Thank you so much for the great work. Can't wait for the release with a good chance to get access to my data again. On 05.04.23 16:15, Josh Durgin wrote:
The LRC upgraded with no problems, so this release is good to go!
Josh
On Mon, Apr 3, 2023 at 3:36 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Josh, the release is ready for your review and approval.
Adam, can you please update the LRC upgrade to 17.2.6 RC?
Thx
On Wed, Mar 29, 2023 at 3:07 PM Yuri Weinstein <yweinste@redhat.com> wrote:
The release has been approved.
And the gibba cluster upgraded.
We are awaiting the LRC upgrade and then/or in parallel will publish an RC for testing.
ETA for the release publishing is 04/07/23
On Tue, Mar 28, 2023 at 2:59 PM Neha Ojha <nojha@redhat.com> wrote:
upgrades approved!
Thanks, Neha
On Tue, Mar 28, 2023 at 12:09 PM Radoslaw Zarzynski <rzarzyns@redhat.com> wrote:
rados: approved!
On Mon, Mar 27, 2023 at 7:02 PM Laura Flores <lflores@redhat.com> wrote:
Rados review, second round:
Failures: 1. https://tracker.ceph.com/issues/58560 2. https://tracker.ceph.com/issues/58476 3. https://tracker.ceph.com/issues/58475 -- pending Q backport 4. https://tracker.ceph.com/issues/49287 5. https://tracker.ceph.com/issues/58585
Details: 1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo - Infrastructure 2. test_non_existent_cluster: cluster does not exist - Ceph - Orchestrator 3. test_dashboard_e2e.sh: Conflicting peer dependency: postcss@8.4.21 - Ceph - Mgr - Dashboard 4. podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope not found - Ceph - Orchestrator 5. rook: failed to pull kubelet image - Ceph - Orchestrator
@Radoslaw Zarzynski <rzarzyns@redhat.com> will give final approval for rados.
On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley <cbodley@redhat.com> wrote:
> On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein <yweinste@redhat.com> > wrote: >> >> Details of this release are updated here: >> >> https://tracker.ceph.com/issues/59070#note-1 >> Release Notes - TBD >> >> The slowness we experienced seemed to be self-cured. >> Neha, Radek, and Laura please provide any findings if you have them. >> >> Seeking approvals/reviews for: >> >> rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 > with >> PRs merged on top of quincy-release) >> rgw - Casey (rerun on Build 2 with PRs merged on top of > quincy-release) > > rgw approved > >> fs - Venky >> >> upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any > updates?) >> upgrade/pacific-x - Neha, Laura, Ilya see > https://tracker.ceph.com/issues/58914 >> upgrade/quincy-p2p - Neha, Laura >> client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue > Adam >> Kraitman any updates?) >> powercycle - Brad >> >> Please reply to this email with approval and/or trackers of known >> issues/PRs to address them. >> >> Josh, Neha - gibba and LRC upgrades pending major suites approvals. >> RC release - pending major suites approvals. >> >> On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein <yweinste@redhat.com> > wrote: >>> >>> Details of this release are summarized here: >>> >>> https://tracker.ceph.com/issues/59070#note-1 >>> Release Notes - TBD >>> >>> The reruns were in the queue for 4 days because of some slowness > issues. >>> The core team (Neha, Radek, Laura, and others) are trying to > narrow >>> down the root cause. >>> >>> Seeking approvals/reviews for: >>> >>> rados - Neha, Radek, Travis, Ernesto, Adam King (we still have to > test >>> and merge at least one PR https://github.com/ceph/ceph/pull/50575 > for >>> the core) >>> rgw - Casey >>> fs - Venky (the fs suite has an unusually high amount of failed > jobs, >>> any reason to suspect it in the observed slowness?) >>> orch - Adam King >>> rbd - Ilya >>> krbd - Ilya >>> upgrade/octopus-x - Laura is looking into failures >>> upgrade/pacific-x - Laura is looking into failures >>> upgrade/quincy-p2p - Laura is looking into failures >>> client-upgrade-octopus-quincy-quincy - missing packages, Adam > Kraitman >>> is looking into it >>> powercycle - Brad >>> ceph-volume - needs a rerun on merged >>> https://github.com/ceph/ceph-ansible/pull/7409 >>> >>> Please reply to this email with approval and/or trackers of known >>> issues/PRs to address them. >>> >>> Also, share any findings or hypnosis about the slowness in the >>> execution of the suite. >>> >>> Josh, Neha - gibba and LRC upgrades pending major suites > approvals. >>> RC release - pending major suites approvals. >>> >>> Thx >>> YuriW >> _______________________________________________ >> Dev mailing list -- dev@ceph.io >> To unsubscribe send an email to dev-leave@ceph.io > _______________________________________________ > Dev mailing list -- dev@ceph.io > To unsubscribe send an email to dev-leave@ceph.io >
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Thx, Josh! We will start releasing now. Release notes - https://github.com/ceph/ceph/pull/50721 On Wed, Apr 5, 2023 at 7:16 AM Josh Durgin <jdurgin@redhat.com> wrote:
The LRC upgraded with no problems, so this release is good to go!
Josh
On Mon, Apr 3, 2023 at 3:36 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Josh, the release is ready for your review and approval.
Adam, can you please update the LRC upgrade to 17.2.6 RC?
Thx
On Wed, Mar 29, 2023 at 3:07 PM Yuri Weinstein <yweinste@redhat.com> wrote:
The release has been approved.
And the gibba cluster upgraded.
We are awaiting the LRC upgrade and then/or in parallel will publish an RC for testing.
ETA for the release publishing is 04/07/23
On Tue, Mar 28, 2023 at 2:59 PM Neha Ojha <nojha@redhat.com> wrote:
upgrades approved!
Thanks, Neha
On Tue, Mar 28, 2023 at 12:09 PM Radoslaw Zarzynski < rzarzyns@redhat.com> wrote:
rados: approved!
On Mon, Mar 27, 2023 at 7:02 PM Laura Flores <lflores@redhat.com> wrote:
Rados review, second round:
Failures: 1. https://tracker.ceph.com/issues/58560 2. https://tracker.ceph.com/issues/58476 3. https://tracker.ceph.com/issues/58475 -- pending Q backport 4. https://tracker.ceph.com/issues/49287 5. https://tracker.ceph.com/issues/58585
Details: 1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo - Infrastructure 2. test_non_existent_cluster: cluster does not exist - Ceph - Orchestrator 3. test_dashboard_e2e.sh: Conflicting peer dependency: postcss@8.4.21 - Ceph - Mgr - Dashboard 4. podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope not found - Ceph - Orchestrator 5. rook: failed to pull kubelet image - Ceph - Orchestrator
@Radoslaw Zarzynski <rzarzyns@redhat.com> will give final approval for rados.
On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley <cbodley@redhat.com> wrote:
> On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein <yweinste@redhat.com> > wrote: > > > > Details of this release are updated here: > > > > https://tracker.ceph.com/issues/59070#note-1 > > Release Notes - TBD > > > > The slowness we experienced seemed to be self-cured. > > Neha, Radek, and Laura please provide any findings if you have > them. > > > > Seeking approvals/reviews for: > > > > rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 > with > > PRs merged on top of quincy-release) > > rgw - Casey (rerun on Build 2 with PRs merged on top of > quincy-release) > > rgw approved > > > fs - Venky > > > > upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any > updates?) > > upgrade/pacific-x - Neha, Laura, Ilya see > https://tracker.ceph.com/issues/58914 > > upgrade/quincy-p2p - Neha, Laura > > client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue > Adam > > Kraitman any updates?) > > powercycle - Brad > > > > Please reply to this email with approval and/or trackers of known > > issues/PRs to address them. > > > > Josh, Neha - gibba and LRC upgrades pending major suites approvals. > > RC release - pending major suites approvals. > > > > On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein < > yweinste@redhat.com> wrote: > > > > > > Details of this release are summarized here: > > > > > > https://tracker.ceph.com/issues/59070#note-1 > > > Release Notes - TBD > > > > > > The reruns were in the queue for 4 days because of some slowness > issues. > > > The core team (Neha, Radek, Laura, and others) are trying to > narrow > > > down the root cause. > > > > > > Seeking approvals/reviews for: > > > > > > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have > to test > > > and merge at least one PR > https://github.com/ceph/ceph/pull/50575 for > > > the core) > > > rgw - Casey > > > fs - Venky (the fs suite has an unusually high amount of failed > jobs, > > > any reason to suspect it in the observed slowness?) > > > orch - Adam King > > > rbd - Ilya > > > krbd - Ilya > > > upgrade/octopus-x - Laura is looking into failures > > > upgrade/pacific-x - Laura is looking into failures > > > upgrade/quincy-p2p - Laura is looking into failures > > > client-upgrade-octopus-quincy-quincy - missing packages, Adam > Kraitman > > > is looking into it > > > powercycle - Brad > > > ceph-volume - needs a rerun on merged > > > https://github.com/ceph/ceph-ansible/pull/7409 > > > > > > Please reply to this email with approval and/or trackers of known > > > issues/PRs to address them. > > > > > > Also, share any findings or hypnosis about the slowness in the > > > execution of the suite. > > > > > > Josh, Neha - gibba and LRC upgrades pending major suites > approvals. > > > RC release - pending major suites approvals. > > > > > > Thx > > > YuriW > > _______________________________________________ > > Dev mailing list -- dev@ceph.io > > To unsubscribe send an email to dev-leave@ceph.io > _______________________________________________ > Dev mailing list -- dev@ceph.io > To unsubscribe send an email to dev-leave@ceph.io >
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
Update: The release is ready to be published, but we have problems with the signing VM. That VM is being actively fixed as we speak (thank you, Dan, Ken, and Zack!) However, we need to delay the publishing till early next week. Thx and sorry for the delay. YuriW On Wed, Apr 5, 2023 at 7:16 AM Josh Durgin <jdurgin@redhat.com> wrote:
The LRC upgraded with no problems, so this release is good to go!
Josh
On Mon, Apr 3, 2023 at 3:36 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Josh, the release is ready for your review and approval.
Adam, can you please update the LRC upgrade to 17.2.6 RC?
Thx
On Wed, Mar 29, 2023 at 3:07 PM Yuri Weinstein <yweinste@redhat.com> wrote:
The release has been approved.
And the gibba cluster upgraded.
We are awaiting the LRC upgrade and then/or in parallel will publish an RC for testing.
ETA for the release publishing is 04/07/23
On Tue, Mar 28, 2023 at 2:59 PM Neha Ojha <nojha@redhat.com> wrote:
upgrades approved!
Thanks, Neha
On Tue, Mar 28, 2023 at 12:09 PM Radoslaw Zarzynski < rzarzyns@redhat.com> wrote:
rados: approved!
On Mon, Mar 27, 2023 at 7:02 PM Laura Flores <lflores@redhat.com> wrote:
Rados review, second round:
Failures: 1. https://tracker.ceph.com/issues/58560 2. https://tracker.ceph.com/issues/58476 3. https://tracker.ceph.com/issues/58475 -- pending Q backport 4. https://tracker.ceph.com/issues/49287 5. https://tracker.ceph.com/issues/58585
Details: 1. test_envlibrados_for_rocksdb.sh failed to subscribe to repo - Infrastructure 2. test_non_existent_cluster: cluster does not exist - Ceph - Orchestrator 3. test_dashboard_e2e.sh: Conflicting peer dependency: postcss@8.4.21 - Ceph - Mgr - Dashboard 4. podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope not found - Ceph - Orchestrator 5. rook: failed to pull kubelet image - Ceph - Orchestrator
@Radoslaw Zarzynski <rzarzyns@redhat.com> will give final approval for rados.
On Mon, Mar 27, 2023 at 10:02 AM Casey Bodley <cbodley@redhat.com> wrote:
> On Fri, Mar 24, 2023 at 3:46 PM Yuri Weinstein <yweinste@redhat.com> > wrote: > > > > Details of this release are updated here: > > > > https://tracker.ceph.com/issues/59070#note-1 > > Release Notes - TBD > > > > The slowness we experienced seemed to be self-cured. > > Neha, Radek, and Laura please provide any findings if you have > them. > > > > Seeking approvals/reviews for: > > > > rados - Neha, Radek, Travis, Ernesto, Adam King (rerun on Build 2 > with > > PRs merged on top of quincy-release) > > rgw - Casey (rerun on Build 2 with PRs merged on top of > quincy-release) > > rgw approved > > > fs - Venky > > > > upgrade/octopus-x - Neha, Laura (package issue Adam Kraitman any > updates?) > > upgrade/pacific-x - Neha, Laura, Ilya see > https://tracker.ceph.com/issues/58914 > > upgrade/quincy-p2p - Neha, Laura > > client-upgrade-octopus-quincy-quincy - Neha, Laura (package issue > Adam > > Kraitman any updates?) > > powercycle - Brad > > > > Please reply to this email with approval and/or trackers of known > > issues/PRs to address them. > > > > Josh, Neha - gibba and LRC upgrades pending major suites approvals. > > RC release - pending major suites approvals. > > > > On Tue, Mar 21, 2023 at 1:04 PM Yuri Weinstein < > yweinste@redhat.com> wrote: > > > > > > Details of this release are summarized here: > > > > > > https://tracker.ceph.com/issues/59070#note-1 > > > Release Notes - TBD > > > > > > The reruns were in the queue for 4 days because of some slowness > issues. > > > The core team (Neha, Radek, Laura, and others) are trying to > narrow > > > down the root cause. > > > > > > Seeking approvals/reviews for: > > > > > > rados - Neha, Radek, Travis, Ernesto, Adam King (we still have > to test > > > and merge at least one PR > https://github.com/ceph/ceph/pull/50575 for > > > the core) > > > rgw - Casey > > > fs - Venky (the fs suite has an unusually high amount of failed > jobs, > > > any reason to suspect it in the observed slowness?) > > > orch - Adam King > > > rbd - Ilya > > > krbd - Ilya > > > upgrade/octopus-x - Laura is looking into failures > > > upgrade/pacific-x - Laura is looking into failures > > > upgrade/quincy-p2p - Laura is looking into failures > > > client-upgrade-octopus-quincy-quincy - missing packages, Adam > Kraitman > > > is looking into it > > > powercycle - Brad > > > ceph-volume - needs a rerun on merged > > > https://github.com/ceph/ceph-ansible/pull/7409 > > > > > > Please reply to this email with approval and/or trackers of known > > > issues/PRs to address them. > > > > > > Also, share any findings or hypnosis about the slowness in the > > > execution of the suite. > > > > > > Josh, Neha - gibba and LRC upgrades pending major suites > approvals. > > > RC release - pending major suites approvals. > > > > > > Thx > > > YuriW > > _______________________________________________ > > Dev mailing list -- dev@ceph.io > > To unsubscribe send an email to dev-leave@ceph.io > _______________________________________________ > Dev mailing list -- dev@ceph.io > To unsubscribe send an email to dev-leave@ceph.io >
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
participants (14)
-
Adam King
-
Brad Hubbard
-
Casey Bodley
-
Crown Upholstery
-
Ilya Dryomov
-
Josh Durgin
-
Ken Dreyer
-
Laura Flores
-
Neha Ojha
-
Nizamudeen A
-
Radoslaw Zarzynski
-
Thomas Widhalm
-
Venky Shankar
-
Yuri Weinstein