squid 19.2.0 QE validation status
Details of this release are summarized here: https://tracker.ceph.com/issues/67779#note-1 Release Notes - TBD Gibba upgrade -TBD LRC upgrade - TBD It was decided and agreed upon that there would be limited testing for this release, given it is based on 19.1.1 rather than a full rebase. Seeking approvals/reviews for: (some reruns are still in progress) rgw - Eric, Adam E fs - Venky orch - Adam King quincy-x - Laura, Neha reef-x - Laura, Neha Pls let me know if any tests were missed from this list.
On Sat, Aug 31, 2024 at 12:43 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/67779#note-1
Release Notes - TBD Gibba upgrade -TBD LRC upgrade - TBD
It was decided and agreed upon that there would be limited testing for this release, given it is based on 19.1.1 rather than a full rebase.
Seeking approvals/reviews for: (some reruns are still in progress)
rgw - Eric, Adam E fs - Venky orch - Adam King
quincy-x - Laura, Neha reef-x - Laura, Neha
If possible it might be a good idea to run the reef-x tests again with a backport of https://github.com/ceph/ceph/pull/59568 (assuming it gets merged in main).
Pls let me know if any tests were missed from this list. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Cheers, Brad
Hi Yuri, (cc Rachana) On Fri, Aug 30, 2024 at 8:13 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/67779#note-1
Release Notes - TBD Gibba upgrade -TBD LRC upgrade - TBD
It was decided and agreed upon that there would be limited testing for this release, given it is based on 19.1.1 rather than a full rebase.
Seeking approvals/reviews for: (some reruns are still in progress)
rgw - Eric, Adam E fs - Venky
fs approved. Failures are noted here https://tracker.ceph.com/projects/cephfs/wiki/Squid#squid-1920
orch - Adam King
quincy-x - Laura, Neha reef-x - Laura, Neha
Pls let me know if any tests were missed from this list. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Cheers, Venky
Still looking at the rgw failures. One caught in rgw testing looks to be in core, a valgrind mismatched delete[] in libceph, and I think this squid PR is addressing: https://github.com/ceph/ceph/pull/58991 Here's the valgrind error: https://qa-proxy.ceph.com/teuthology/yuriw-2024-08-29_20:04:55-rgw-squid-rel... Eric (he/him)
On Aug 30, 2024, at 10:42 AM, Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/67779#note-1
Release Notes - TBD Gibba upgrade -TBD LRC upgrade - TBD
It was decided and agreed upon that there would be limited testing for this release, given it is based on 19.1.1 rather than a full rebase.
Seeking approvals/reviews for: (some reruns are still in progress)
rgw - Eric, Adam E fs - Venky orch - Adam King
quincy-x - Laura, Neha reef-x - Laura, Neha
Pls let me know if any tests were missed from this list. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
rgw — approved Eric (he/him)
On Aug 30, 2024, at 10:42 AM, Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/67779#note-1
Release Notes - TBD Gibba upgrade -TBD LRC upgrade - TBD
It was decided and agreed upon that there would be limited testing for this release, given it is based on 19.1.1 rather than a full rebase.
Seeking approvals/reviews for: (some reruns are still in progress)
rgw - Eric, Adam E fs - Venky orch - Adam King
quincy-x - Laura, Neha reef-x - Laura, Neha
Pls let me know if any tests were missed from this list. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
orch approved, failures are known issues and not release blockers On Fri, Aug 30, 2024 at 10:43 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/67779#note-1
Release Notes - TBD Gibba upgrade -TBD LRC upgrade - TBD
It was decided and agreed upon that there would be limited testing for this release, given it is based on 19.1.1 rather than a full rebase.
Seeking approvals/reviews for: (some reruns are still in progress)
rgw - Eric, Adam E fs - Venky orch - Adam King
quincy-x - Laura, Neha reef-x - Laura, Neha
Pls let me know if any tests were missed from this list. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Upgrade suites approved: https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-httpstrackercephcom... On Wed, Sep 4, 2024 at 12:02 PM Adam King <adking@redhat.com> wrote:
orch approved, failures are known issues and not release blockers
On Fri, Aug 30, 2024 at 10:43 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/67779#note-1
Release Notes - TBD Gibba upgrade -TBD LRC upgrade - TBD
It was decided and agreed upon that there would be limited testing for this release, given it is based on 19.1.1 rather than a full rebase.
Seeking approvals/reviews for: (some reruns are still in progress)
rgw - Eric, Adam E fs - Venky orch - Adam King
quincy-x - Laura, Neha reef-x - Laura, Neha
Pls let me know if any tests were missed from this list. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
We have added one more core PR to the release, which was deemed a blocker ( https://github.com/ceph/ceph/pull/59492). Rados, upgrade, and smoke will need to be reapproved. On Wed, Sep 4, 2024 at 3:59 PM Laura Flores <lflores@redhat.com> wrote:
Upgrade suites approved: https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-httpstrackercephcom...
On Wed, Sep 4, 2024 at 12:02 PM Adam King <adking@redhat.com> wrote:
orch approved, failures are known issues and not release blockers
On Fri, Aug 30, 2024 at 10:43 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/67779#note-1
Release Notes - TBD Gibba upgrade -TBD LRC upgrade - TBD
It was decided and agreed upon that there would be limited testing for this release, given it is based on 19.1.1 rather than a full rebase.
Seeking approvals/reviews for: (some reruns are still in progress)
rgw - Eric, Adam E fs - Venky orch - Adam King
quincy-x - Laura, Neha reef-x - Laura, Neha
Pls let me know if any tests were missed from this list. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
-- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
Hi All ! PR 59492 <https://github.com/ceph/ceph/pull/59492> was cherry-picked on top of the last built. RADOS, upgrade and smoke test suits were executed. Seeking approvals/reviews for following reruns RADOS Test Result <https://pulpito.ceph.com/skanta-2024-09-07_00:00:30-rados-squid-release-distro-default-smithi/>- Neha Upgrade Test result <https://pulpito.ceph.com/sangadi-2024-09-06_06:58:25-upgrade-squid-release-distro-default-smithi/>- Neha, Laura Smoke Test Result <https://pulpito.ceph.com/sangadi-2024-09-06_06:53:40-smoke-squid-release-distro-default-smithi/> On Fri, Sep 6, 2024 at 9:57 AM Laura Flores <lflores@redhat.com> wrote:
We have added one more core PR to the release, which was deemed a blocker ( https://github.com/ceph/ceph/pull/59492). Rados, upgrade, and smoke will need to be reapproved.
On Wed, Sep 4, 2024 at 3:59 PM Laura Flores <lflores@redhat.com> wrote:
Upgrade suites approved:
https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-httpstrackercephcom...
On Wed, Sep 4, 2024 at 12:02 PM Adam King <adking@redhat.com> wrote:
orch approved, failures are known issues and not release blockers
On Fri, Aug 30, 2024 at 10:43 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/67779#note-1
Release Notes - TBD Gibba upgrade -TBD LRC upgrade - TBD
It was decided and agreed upon that there would be limited testing for this release, given it is based on 19.1.1 rather than a full rebase.
Seeking approvals/reviews for: (some reruns are still in progress)
rgw - Eric, Adam E fs - Venky orch - Adam King
quincy-x - Laura, Neha reef-x - Laura, Neha
Pls let me know if any tests were missed from this list. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804 _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Thanks, Rachana
Hey Rachana, Thanks for these links. Currently reviewing them and will update the thread soon. Thanks, Laura On Mon, Sep 9, 2024 at 1:06 AM Rachana Patel <racpatel@redhat.com> wrote:
Hi All ! PR 59492 <https://github.com/ceph/ceph/pull/59492> was cherry-picked on top of the last built. RADOS, upgrade and smoke test suits were executed.
Seeking approvals/reviews for following reruns
RADOS Test Result <https://pulpito.ceph.com/skanta-2024-09-07_00:00:30-rados-squid-release-distro-default-smithi/>- Neha Upgrade Test result <https://pulpito.ceph.com/sangadi-2024-09-06_06:58:25-upgrade-squid-release-distro-default-smithi/>- Neha, Laura Smoke Test Result <https://pulpito.ceph.com/sangadi-2024-09-06_06:53:40-smoke-squid-release-distro-default-smithi/>
On Fri, Sep 6, 2024 at 9:57 AM Laura Flores <lflores@redhat.com> wrote:
We have added one more core PR to the release, which was deemed a blocker ( https://github.com/ceph/ceph/pull/59492). Rados, upgrade, and smoke will need to be reapproved.
On Wed, Sep 4, 2024 at 3:59 PM Laura Flores <lflores@redhat.com> wrote:
Upgrade suites approved:
https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-httpstrackercephcom...
On Wed, Sep 4, 2024 at 12:02 PM Adam King <adking@redhat.com> wrote:
orch approved, failures are known issues and not release blockers
On Fri, Aug 30, 2024 at 10:43 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/67779#note-1
Release Notes - TBD Gibba upgrade -TBD LRC upgrade - TBD
It was decided and agreed upon that there would be limited testing
for
this release, given it is based on 19.1.1 rather than a full rebase.
Seeking approvals/reviews for: (some reruns are still in progress)
rgw - Eric, Adam E fs - Venky orch - Adam King
quincy-x - Laura, Neha reef-x - Laura, Neha
Pls let me know if any tests were missed from this list. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804 _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
--
Thanks, Rachana
-- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
Rados reviewed here: https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstracke... I have asked @Radoslaw Zarzynski <rzarzyns@redhat.com> to take a look at the summary and confirm there are no blockers. In progress reviewing upgrade and smoke; will provide another update for those soon as well. On Mon, Sep 9, 2024 at 9:49 AM Laura Flores <lflores@redhat.com> wrote:
Hey Rachana,
Thanks for these links. Currently reviewing them and will update the thread soon.
Thanks, Laura
On Mon, Sep 9, 2024 at 1:06 AM Rachana Patel <racpatel@redhat.com> wrote:
Hi All ! PR 59492 <https://github.com/ceph/ceph/pull/59492> was cherry-picked on top of the last built. RADOS, upgrade and smoke test suits were executed.
Seeking approvals/reviews for following reruns
RADOS Test Result <https://pulpito.ceph.com/skanta-2024-09-07_00:00:30-rados-squid-release-distro-default-smithi/>- Neha Upgrade Test result <https://pulpito.ceph.com/sangadi-2024-09-06_06:58:25-upgrade-squid-release-distro-default-smithi/>- Neha, Laura Smoke Test Result <https://pulpito.ceph.com/sangadi-2024-09-06_06:53:40-smoke-squid-release-distro-default-smithi/>
On Fri, Sep 6, 2024 at 9:57 AM Laura Flores <lflores@redhat.com> wrote:
We have added one more core PR to the release, which was deemed a blocker ( https://github.com/ceph/ceph/pull/59492). Rados, upgrade, and smoke will need to be reapproved.
On Wed, Sep 4, 2024 at 3:59 PM Laura Flores <lflores@redhat.com> wrote:
Upgrade suites approved:
https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-httpstrackercephcom...
On Wed, Sep 4, 2024 at 12:02 PM Adam King <adking@redhat.com> wrote:
orch approved, failures are known issues and not release blockers
On Fri, Aug 30, 2024 at 10:43 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release are summarized here:
https://tracker.ceph.com/issues/67779#note-1
Release Notes - TBD Gibba upgrade -TBD LRC upgrade - TBD
It was decided and agreed upon that there would be limited testing
for
this release, given it is based on 19.1.1 rather than a full rebase.
Seeking approvals/reviews for: (some reruns are still in progress)
rgw - Eric, Adam E fs - Venky orch - Adam King
quincy-x - Laura, Neha reef-x - Laura, Neha
Pls let me know if any tests were missed from this list. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804 _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
--
Thanks, Rachana
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
-- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
I have finished reviewing the upgrade and smoke suites. Most failures are already known/tracked: https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstracke... *Upgrade: Pending check from FS team* Two fs issues stood out. @Gregory Farnum <gfarnum@redhat.com> @Venky Shankar <vshankar@redhat.com> can you confirm these are not blockers? - https://tracker.ceph.com/issues/67999 - upgrade/quincy-x/filestore-remove-check: Expected string "FileStore has been deprecated and is no longer supported" is absent from osd log - https://tracker.ceph.com/issues/68002 - upgrade/cephfs/mds_upgrade_sequence: fsstress coredumps When this is confirmed, the upgrade suite can be approved. *Smoke: APPROVED* *Rados: Pending check from core team* On Tue, Sep 10, 2024 at 8:00 AM Laura Flores <lflores@redhat.com> wrote:
Rados reviewed here: https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstracke...
I have asked @Radoslaw Zarzynski <rzarzyns@redhat.com> to take a look at the summary and confirm there are no blockers.
In progress reviewing upgrade and smoke; will provide another update for those soon as well.
On Mon, Sep 9, 2024 at 9:49 AM Laura Flores <lflores@redhat.com> wrote:
Hey Rachana,
Thanks for these links. Currently reviewing them and will update the thread soon.
Thanks, Laura
On Mon, Sep 9, 2024 at 1:06 AM Rachana Patel <racpatel@redhat.com> wrote:
Hi All ! PR 59492 <https://github.com/ceph/ceph/pull/59492> was cherry-picked on top of the last built. RADOS, upgrade and smoke test suits were executed.
Seeking approvals/reviews for following reruns
RADOS Test Result <https://pulpito.ceph.com/skanta-2024-09-07_00:00:30-rados-squid-release-distro-default-smithi/>- Neha Upgrade Test result <https://pulpito.ceph.com/sangadi-2024-09-06_06:58:25-upgrade-squid-release-distro-default-smithi/>- Neha, Laura Smoke Test Result <https://pulpito.ceph.com/sangadi-2024-09-06_06:53:40-smoke-squid-release-distro-default-smithi/>
On Fri, Sep 6, 2024 at 9:57 AM Laura Flores <lflores@redhat.com> wrote:
We have added one more core PR to the release, which was deemed a blocker ( https://github.com/ceph/ceph/pull/59492). Rados, upgrade, and smoke will need to be reapproved.
On Wed, Sep 4, 2024 at 3:59 PM Laura Flores <lflores@redhat.com> wrote:
Upgrade suites approved:
https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-httpstrackercephcom...
On Wed, Sep 4, 2024 at 12:02 PM Adam King <adking@redhat.com> wrote:
orch approved, failures are known issues and not release blockers
On Fri, Aug 30, 2024 at 10:43 AM Yuri Weinstein <yweinste@redhat.com
wrote:
> Details of this release are summarized here: > > https://tracker.ceph.com/issues/67779#note-1 > > Release Notes - TBD > Gibba upgrade -TBD > LRC upgrade - TBD > > It was decided and agreed upon that there would be limited testing
for
> this release, given it is based on 19.1.1 rather than a full rebase. > > Seeking approvals/reviews for: > (some reruns are still in progress) > > rgw - Eric, Adam E > fs - Venky > orch - Adam King > > quincy-x - Laura, Neha > reef-x - Laura, Neha > > Pls let me know if any tests were missed from this list. > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-leave@ceph.io > > _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804 _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
--
Thanks, Rachana
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
-- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
Rados approved On Tue, Sep 10, 2024 at 1:43 PM Laura Flores <lflores@redhat.com> wrote:
I have finished reviewing the upgrade and smoke suites. Most failures are already known/tracked: https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstracke...
*Upgrade: Pending check from FS team* Two fs issues stood out. @Gregory Farnum <gfarnum@redhat.com> @Venky Shankar <vshankar@redhat.com> can you confirm these are not blockers?
- https://tracker.ceph.com/issues/67999 - upgrade/quincy-x/filestore-remove-check: Expected string "FileStore has been deprecated and is no longer supported" is absent from osd log - https://tracker.ceph.com/issues/68002 - upgrade/cephfs/mds_upgrade_sequence: fsstress coredumps
When this is confirmed, the upgrade suite can be approved.
*Smoke: APPROVED*
*Rados: Pending check from core team*
On Tue, Sep 10, 2024 at 8:00 AM Laura Flores <lflores@redhat.com> wrote:
Rados reviewed here: https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstracke...
I have asked @Radoslaw Zarzynski <rzarzyns@redhat.com> to take a look at the summary and confirm there are no blockers.
In progress reviewing upgrade and smoke; will provide another update for those soon as well.
On Mon, Sep 9, 2024 at 9:49 AM Laura Flores <lflores@redhat.com> wrote:
Hey Rachana,
Thanks for these links. Currently reviewing them and will update the thread soon.
Thanks, Laura
On Mon, Sep 9, 2024 at 1:06 AM Rachana Patel <racpatel@redhat.com> wrote:
Hi All ! PR 59492 <https://github.com/ceph/ceph/pull/59492> was cherry-picked on top of the last built. RADOS, upgrade and smoke test suits were executed.
Seeking approvals/reviews for following reruns
RADOS Test Result <https://pulpito.ceph.com/skanta-2024-09-07_00:00:30-rados-squid-release-distro-default-smithi/>- Neha Upgrade Test result <https://pulpito.ceph.com/sangadi-2024-09-06_06:58:25-upgrade-squid-release-distro-default-smithi/>- Neha, Laura Smoke Test Result <https://pulpito.ceph.com/sangadi-2024-09-06_06:53:40-smoke-squid-release-distro-default-smithi/>
On Fri, Sep 6, 2024 at 9:57 AM Laura Flores <lflores@redhat.com> wrote:
We have added one more core PR to the release, which was deemed a blocker ( https://github.com/ceph/ceph/pull/59492). Rados, upgrade, and smoke will need to be reapproved.
On Wed, Sep 4, 2024 at 3:59 PM Laura Flores <lflores@redhat.com> wrote:
Upgrade suites approved:
https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-httpstrackercephcom...
On Wed, Sep 4, 2024 at 12:02 PM Adam King <adking@redhat.com> wrote:
> orch approved, failures are known issues and not release blockers > > On Fri, Aug 30, 2024 at 10:43 AM Yuri Weinstein <
yweinste@redhat.com>
> wrote: > > > Details of this release are summarized here: > > > > https://tracker.ceph.com/issues/67779#note-1 > > > > Release Notes - TBD > > Gibba upgrade -TBD > > LRC upgrade - TBD > > > > It was decided and agreed upon that there would be limited testing for > > this release, given it is based on 19.1.1 rather than a full rebase. > > > > Seeking approvals/reviews for: > > (some reruns are still in progress) > > > > rgw - Eric, Adam E > > fs - Venky > > orch - Adam King > > > > quincy-x - Laura, Neha > > reef-x - Laura, Neha > > > > Pls let me know if any tests were missed from this list. > > _______________________________________________ > > ceph-users mailing list -- ceph-users@ceph.io > > To unsubscribe send an email to ceph-users-leave@ceph.io > > > > > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-leave@ceph.io >
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804 _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
--
Thanks, Rachana
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
-- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
Hey Laura, On Wed, Sep 11, 2024 at 12:13 AM Laura Flores <lflores@redhat.com> wrote:
I have finished reviewing the upgrade and smoke suites. Most failures are already known/tracked: https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstracke...
*Upgrade: Pending check from FS team* Two fs issues stood out. @Gregory Farnum <gfarnum@redhat.com> @Venky Shankar <vshankar@redhat.com> can you confirm these are not blockers?
- https://tracker.ceph.com/issues/67999 - upgrade/quincy-x/filestore-remove-check: Expected string "FileStore has been deprecated and is no longer supported" is absent from osd log
How is this cephfs related?
- - https://tracker.ceph.com/issues/68002 - upgrade/cephfs/mds_upgrade_sequence: fsstress coredumps
I left my findings in the tracker. See https://tracker.ceph.com/issues/68002#note-2 The fsstress crash is due to SIGBUS and there are a flurry of OSD related heartbeat_check messages from a node where there is a cephfs kernel driver traceback in the kernel ring buffer. I am running the tracker through Xiubo.
-
When this is confirmed, the upgrade suite can be approved.
*Smoke: APPROVED*
*Rados: Pending check from core team*
On Tue, Sep 10, 2024 at 8:00 AM Laura Flores <lflores@redhat.com> wrote:
Rados reviewed here: https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstracke...
I have asked @Radoslaw Zarzynski <rzarzyns@redhat.com> to take a look at the summary and confirm there are no blockers.
In progress reviewing upgrade and smoke; will provide another update for those soon as well.
On Mon, Sep 9, 2024 at 9:49 AM Laura Flores <lflores@redhat.com> wrote:
Hey Rachana,
Thanks for these links. Currently reviewing them and will update the thread soon.
Thanks, Laura
On Mon, Sep 9, 2024 at 1:06 AM Rachana Patel <racpatel@redhat.com> wrote:
Hi All ! PR 59492 <https://github.com/ceph/ceph/pull/59492> was cherry-picked on top of the last built. RADOS, upgrade and smoke test suits were executed.
Seeking approvals/reviews for following reruns
RADOS Test Result <https://pulpito.ceph.com/skanta-2024-09-07_00:00:30-rados-squid-release-distro-default-smithi/>- Neha Upgrade Test result <https://pulpito.ceph.com/sangadi-2024-09-06_06:58:25-upgrade-squid-release-distro-default-smithi/>- Neha, Laura Smoke Test Result <https://pulpito.ceph.com/sangadi-2024-09-06_06:53:40-smoke-squid-release-distro-default-smithi/>
On Fri, Sep 6, 2024 at 9:57 AM Laura Flores <lflores@redhat.com> wrote:
We have added one more core PR to the release, which was deemed a blocker ( https://github.com/ceph/ceph/pull/59492). Rados, upgrade, and smoke will need to be reapproved.
On Wed, Sep 4, 2024 at 3:59 PM Laura Flores <lflores@redhat.com> wrote:
Upgrade suites approved:
https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-httpstrackercephcom...
On Wed, Sep 4, 2024 at 12:02 PM Adam King <adking@redhat.com> wrote:
> orch approved, failures are known issues and not release blockers > > On Fri, Aug 30, 2024 at 10:43 AM Yuri Weinstein <
yweinste@redhat.com>
> wrote: > > > Details of this release are summarized here: > > > > https://tracker.ceph.com/issues/67779#note-1 > > > > Release Notes - TBD > > Gibba upgrade -TBD > > LRC upgrade - TBD > > > > It was decided and agreed upon that there would be limited testing for > > this release, given it is based on 19.1.1 rather than a full rebase. > > > > Seeking approvals/reviews for: > > (some reruns are still in progress) > > > > rgw - Eric, Adam E > > fs - Venky > > orch - Adam King > > > > quincy-x - Laura, Neha > > reef-x - Laura, Neha > > > > Pls let me know if any tests were missed from this list. > > _______________________________________________ > > ceph-users mailing list -- ceph-users@ceph.io > > To unsubscribe send an email to ceph-users-leave@ceph.io > > > > > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-leave@ceph.io >
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804 _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
--
Thanks, Rachana
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
-- Cheers, Venky
Thanks @Venky Shankar <vshankar@redhat.com> for looking into https://tracker.ceph.com/issues/68002. As for https://tracker.ceph.com/issues/67999, you are right- it is not cephfs related. Mistake on my part. After looking into this one, it seems to be a test issue, where we check for a "Filestore has been deprecated" warning in all the OSD logs once they've upgraded to Squid. On one of the OSDs, the warning was missing, but that OSD hadn't yet been upgraded to Squid. I believe the test needs to be modified to wait until all OSDs have been upgraded. Thus, it should not be a blocker. @Rachana Patel <racpatel@redhat.com> if Venky + Xiubo confirm that https://tracker.ceph.com/issues/68002 is not a blocker, we are good to proceed with next steps. On Wed, Sep 11, 2024 at 4:46 AM Venky Shankar <vshankar@redhat.com> wrote:
Hey Laura,
On Wed, Sep 11, 2024 at 12:13 AM Laura Flores <lflores@redhat.com> wrote:
I have finished reviewing the upgrade and smoke suites. Most failures are already known/tracked: https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstracke...
*Upgrade: Pending check from FS team* Two fs issues stood out. @Gregory Farnum <gfarnum@redhat.com> @Venky Shankar <vshankar@redhat.com> can you confirm these are not blockers?
- https://tracker.ceph.com/issues/67999 - upgrade/quincy-x/filestore-remove-check: Expected string "FileStore has been deprecated and is no longer supported" is absent from osd log
How is this cephfs related?
- - https://tracker.ceph.com/issues/68002 - upgrade/cephfs/mds_upgrade_sequence: fsstress coredumps
I left my findings in the tracker. See
https://tracker.ceph.com/issues/68002#note-2
The fsstress crash is due to SIGBUS and there are a flurry of OSD related heartbeat_check messages from a node where there is a cephfs kernel driver traceback in the kernel ring buffer. I am running the tracker through Xiubo.
-
When this is confirmed, the upgrade suite can be approved.
*Smoke: APPROVED*
*Rados: Pending check from core team*
On Tue, Sep 10, 2024 at 8:00 AM Laura Flores <lflores@redhat.com> wrote:
Rados reviewed here: https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstracke...
I have asked @Radoslaw Zarzynski <rzarzyns@redhat.com> to take a look at the summary and confirm there are no blockers.
In progress reviewing upgrade and smoke; will provide another update for those soon as well.
On Mon, Sep 9, 2024 at 9:49 AM Laura Flores <lflores@redhat.com> wrote:
Hey Rachana,
Thanks for these links. Currently reviewing them and will update the thread soon.
Thanks, Laura
On Mon, Sep 9, 2024 at 1:06 AM Rachana Patel <racpatel@redhat.com> wrote:
Hi All ! PR 59492 <https://github.com/ceph/ceph/pull/59492> was cherry-picked on top of the last built. RADOS, upgrade and smoke test suits were executed.
Seeking approvals/reviews for following reruns
RADOS Test Result <https://pulpito.ceph.com/skanta-2024-09-07_00:00:30-rados-squid-release-distro-default-smithi/>- Neha Upgrade Test result <https://pulpito.ceph.com/sangadi-2024-09-06_06:58:25-upgrade-squid-release-distro-default-smithi/>- Neha, Laura Smoke Test Result <https://pulpito.ceph.com/sangadi-2024-09-06_06:53:40-smoke-squid-release-distro-default-smithi/>
On Fri, Sep 6, 2024 at 9:57 AM Laura Flores <lflores@redhat.com> wrote:
We have added one more core PR to the release, which was deemed a blocker ( https://github.com/ceph/ceph/pull/59492). Rados, upgrade, and smoke will need to be reapproved.
On Wed, Sep 4, 2024 at 3:59 PM Laura Flores <lflores@redhat.com> wrote:
> Upgrade suites approved: > https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-httpstrackercephcom... > > On Wed, Sep 4, 2024 at 12:02 PM Adam King <adking@redhat.com> wrote: > >> orch approved, failures are known issues and not release blockers >> >> On Fri, Aug 30, 2024 at 10:43 AM Yuri Weinstein < yweinste@redhat.com> >> wrote: >> >> > Details of this release are summarized here: >> > >> > https://tracker.ceph.com/issues/67779#note-1 >> > >> > Release Notes - TBD >> > Gibba upgrade -TBD >> > LRC upgrade - TBD >> > >> > It was decided and agreed upon that there would be limited testing for >> > this release, given it is based on 19.1.1 rather than a full rebase. >> > >> > Seeking approvals/reviews for: >> > (some reruns are still in progress) >> > >> > rgw - Eric, Adam E >> > fs - Venky >> > orch - Adam King >> > >> > quincy-x - Laura, Neha >> > reef-x - Laura, Neha >> > >> > Pls let me know if any tests were missed from this list. >> > _______________________________________________ >> > ceph-users mailing list -- ceph-users@ceph.io >> > To unsubscribe send an email to ceph-users-leave@ceph.io >> > >> > >> _______________________________________________ >> ceph-users mailing list -- ceph-users@ceph.io >> To unsubscribe send an email to ceph-users-leave@ceph.io >> > > > -- > > Laura Flores > > She/Her/Hers > > Software Engineer, Ceph Storage <https://ceph.io> > > Chicago, IL > > lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> > M: +17087388804 > > >
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804 _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
--
Thanks, Rachana
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
-- Cheers, Venky
-- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
On Wed, Sep 11, 2024 at 10:00 PM Laura Flores <lflores@redhat.com> wrote:
Thanks @Venky Shankar <vshankar@redhat.com> for looking into https://tracker.ceph.com/issues/68002.
As for https://tracker.ceph.com/issues/67999, you are right- it is not cephfs related. Mistake on my part. After looking into this one, it seems to be a test issue, where we check for a "Filestore has been deprecated" warning in all the OSD logs once they've upgraded to Squid. On one of the OSDs, the warning was missing, but that OSD hadn't yet been upgraded to Squid. I believe the test needs to be modified to wait until all OSDs have been upgraded.
Thus, it should not be a blocker.
@Rachana Patel <racpatel@redhat.com> if Venky + Xiubo confirm that https://tracker.ceph.com/issues/68002 is not a blocker, we are good to proceed with next steps.
https://tracker.ceph.com/issues/68002 is NOT a blocker either and is a duplicate of https://tracker.ceph.com/issues/64471.
On Wed, Sep 11, 2024 at 4:46 AM Venky Shankar <vshankar@redhat.com> wrote:
Hey Laura,
On Wed, Sep 11, 2024 at 12:13 AM Laura Flores <lflores@redhat.com> wrote:
I have finished reviewing the upgrade and smoke suites. Most failures are already known/tracked: https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstracke...
*Upgrade: Pending check from FS team* Two fs issues stood out. @Gregory Farnum <gfarnum@redhat.com> @Venky Shankar <vshankar@redhat.com> can you confirm these are not blockers?
- https://tracker.ceph.com/issues/67999 - upgrade/quincy-x/filestore-remove-check: Expected string "FileStore has been deprecated and is no longer supported" is absent from osd log
How is this cephfs related?
- - https://tracker.ceph.com/issues/68002 - upgrade/cephfs/mds_upgrade_sequence: fsstress coredumps
I left my findings in the tracker. See
https://tracker.ceph.com/issues/68002#note-2
The fsstress crash is due to SIGBUS and there are a flurry of OSD related heartbeat_check messages from a node where there is a cephfs kernel driver traceback in the kernel ring buffer. I am running the tracker through Xiubo.
-
When this is confirmed, the upgrade suite can be approved.
*Smoke: APPROVED*
*Rados: Pending check from core team*
On Tue, Sep 10, 2024 at 8:00 AM Laura Flores <lflores@redhat.com> wrote:
Rados reviewed here: https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstracke...
I have asked @Radoslaw Zarzynski <rzarzyns@redhat.com> to take a look at the summary and confirm there are no blockers.
In progress reviewing upgrade and smoke; will provide another update for those soon as well.
On Mon, Sep 9, 2024 at 9:49 AM Laura Flores <lflores@redhat.com> wrote:
Hey Rachana,
Thanks for these links. Currently reviewing them and will update the thread soon.
Thanks, Laura
On Mon, Sep 9, 2024 at 1:06 AM Rachana Patel <racpatel@redhat.com> wrote:
Hi All ! PR 59492 <https://github.com/ceph/ceph/pull/59492> was cherry-picked on top of the last built. RADOS, upgrade and smoke test suits were executed.
Seeking approvals/reviews for following reruns
RADOS Test Result <https://pulpito.ceph.com/skanta-2024-09-07_00:00:30-rados-squid-release-distro-default-smithi/>- Neha Upgrade Test result <https://pulpito.ceph.com/sangadi-2024-09-06_06:58:25-upgrade-squid-release-distro-default-smithi/>- Neha, Laura Smoke Test Result <https://pulpito.ceph.com/sangadi-2024-09-06_06:53:40-smoke-squid-release-distro-default-smithi/>
On Fri, Sep 6, 2024 at 9:57 AM Laura Flores <lflores@redhat.com> wrote:
> We have added one more core PR to the release, which was deemed a > blocker ( > https://github.com/ceph/ceph/pull/59492). Rados, upgrade, and smoke > will > need to be reapproved. > > On Wed, Sep 4, 2024 at 3:59 PM Laura Flores <lflores@redhat.com> > wrote: > > > Upgrade suites approved: > > > https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-httpstrackercephcom... > > > > On Wed, Sep 4, 2024 at 12:02 PM Adam King <adking@redhat.com> > wrote: > > > >> orch approved, failures are known issues and not release blockers > >> > >> On Fri, Aug 30, 2024 at 10:43 AM Yuri Weinstein < > yweinste@redhat.com> > >> wrote: > >> > >> > Details of this release are summarized here: > >> > > >> > https://tracker.ceph.com/issues/67779#note-1 > >> > > >> > Release Notes - TBD > >> > Gibba upgrade -TBD > >> > LRC upgrade - TBD > >> > > >> > It was decided and agreed upon that there would be limited > testing for > >> > this release, given it is based on 19.1.1 rather than a full > rebase. > >> > > >> > Seeking approvals/reviews for: > >> > (some reruns are still in progress) > >> > > >> > rgw - Eric, Adam E > >> > fs - Venky > >> > orch - Adam King > >> > > >> > quincy-x - Laura, Neha > >> > reef-x - Laura, Neha > >> > > >> > Pls let me know if any tests were missed from this list. > >> > _______________________________________________ > >> > ceph-users mailing list -- ceph-users@ceph.io > >> > To unsubscribe send an email to ceph-users-leave@ceph.io > >> > > >> > > >> _______________________________________________ > >> ceph-users mailing list -- ceph-users@ceph.io > >> To unsubscribe send an email to ceph-users-leave@ceph.io > >> > > > > > > -- > > > > Laura Flores > > > > She/Her/Hers > > > > Software Engineer, Ceph Storage <https://ceph.io> > > > > Chicago, IL > > > > lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> > > M: +17087388804 > > > > > > > > -- > > Laura Flores > > She/Her/Hers > > Software Engineer, Ceph Storage <https://ceph.io> > > Chicago, IL > > lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> > M: +17087388804 > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-leave@ceph.io >
--
Thanks, Rachana
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
-- Cheers, Venky
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
-- Cheers, Venky
Thanks Venky ! We can now focus on next tasks - - Release Notes https://github.com/ceph/ceph/pull/59539 . Requesting all TLs to review Release notes. - Gibba upgrade - LRC upgrade @Laura Flores <lflores@ibm.com> , Kindly upgrade cluster and share results. On Thu, Sep 12, 2024 at 1:03 AM Venky Shankar <vshankar@redhat.com> wrote:
On Wed, Sep 11, 2024 at 10:00 PM Laura Flores <lflores@redhat.com> wrote:
Thanks @Venky Shankar <vshankar@redhat.com> for looking into https://tracker.ceph.com/issues/68002.
As for https://tracker.ceph.com/issues/67999, you are right- it is not cephfs related. Mistake on my part. After looking into this one, it seems to be a test issue, where we check for a "Filestore has been deprecated" warning in all the OSD logs once they've upgraded to Squid. On one of the OSDs, the warning was missing, but that OSD hadn't yet been upgraded to Squid. I believe the test needs to be modified to wait until all OSDs have been upgraded.
Thus, it should not be a blocker.
@Rachana Patel <racpatel@redhat.com> if Venky + Xiubo confirm that https://tracker.ceph.com/issues/68002 is not a blocker, we are good to proceed with next steps.
https://tracker.ceph.com/issues/68002 is NOT a blocker either and is a duplicate of https://tracker.ceph.com/issues/64471.
On Wed, Sep 11, 2024 at 4:46 AM Venky Shankar <vshankar@redhat.com> wrote:
Hey Laura,
On Wed, Sep 11, 2024 at 12:13 AM Laura Flores <lflores@redhat.com> wrote:
I have finished reviewing the upgrade and smoke suites. Most failures are already known/tracked: https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstracke...
*Upgrade: Pending check from FS team* Two fs issues stood out. @Gregory Farnum <gfarnum@redhat.com> @Venky Shankar <vshankar@redhat.com> can you confirm these are not blockers?
- https://tracker.ceph.com/issues/67999 - upgrade/quincy-x/filestore-remove-check: Expected string "FileStore has been deprecated and is no longer supported" is absent from osd log
How is this cephfs related?
- - https://tracker.ceph.com/issues/68002 - upgrade/cephfs/mds_upgrade_sequence: fsstress coredumps
I left my findings in the tracker. See
https://tracker.ceph.com/issues/68002#note-2
The fsstress crash is due to SIGBUS and there are a flurry of OSD related heartbeat_check messages from a node where there is a cephfs kernel driver traceback in the kernel ring buffer. I am running the tracker through Xiubo.
-
When this is confirmed, the upgrade suite can be approved.
*Smoke: APPROVED*
*Rados: Pending check from core team*
On Tue, Sep 10, 2024 at 8:00 AM Laura Flores <lflores@redhat.com> wrote:
Rados reviewed here: https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstracke...
I have asked @Radoslaw Zarzynski <rzarzyns@redhat.com> to take a look at the summary and confirm there are no blockers.
In progress reviewing upgrade and smoke; will provide another update for those soon as well.
On Mon, Sep 9, 2024 at 9:49 AM Laura Flores <lflores@redhat.com> wrote:
Hey Rachana,
Thanks for these links. Currently reviewing them and will update the thread soon.
Thanks, Laura
On Mon, Sep 9, 2024 at 1:06 AM Rachana Patel <racpatel@redhat.com> wrote:
> Hi All ! > PR 59492 <https://github.com/ceph/ceph/pull/59492> was cherry-picked > on top of the last built. RADOS, upgrade and smoke test suits were > executed. > > Seeking approvals/reviews for following reruns > > RADOS Test Result > <https://pulpito.ceph.com/skanta-2024-09-07_00:00:30-rados-squid-release-distro-default-smithi/>- > Neha > Upgrade Test result > <https://pulpito.ceph.com/sangadi-2024-09-06_06:58:25-upgrade-squid-release-distro-default-smithi/>- > Neha, Laura > Smoke Test Result > <https://pulpito.ceph.com/sangadi-2024-09-06_06:53:40-smoke-squid-release-distro-default-smithi/> > > > > On Fri, Sep 6, 2024 at 9:57 AM Laura Flores <lflores@redhat.com> > wrote: > >> We have added one more core PR to the release, which was deemed a >> blocker ( >> https://github.com/ceph/ceph/pull/59492). Rados, upgrade, and >> smoke will >> need to be reapproved. >> >> On Wed, Sep 4, 2024 at 3:59 PM Laura Flores <lflores@redhat.com> >> wrote: >> >> > Upgrade suites approved: >> > >> https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-httpstrackercephcom... >> > >> > On Wed, Sep 4, 2024 at 12:02 PM Adam King <adking@redhat.com> >> wrote: >> > >> >> orch approved, failures are known issues and not release blockers >> >> >> >> On Fri, Aug 30, 2024 at 10:43 AM Yuri Weinstein < >> yweinste@redhat.com> >> >> wrote: >> >> >> >> > Details of this release are summarized here: >> >> > >> >> > https://tracker.ceph.com/issues/67779#note-1 >> >> > >> >> > Release Notes - TBD >> >> > Gibba upgrade -TBD >> >> > LRC upgrade - TBD >> >> > >> >> > It was decided and agreed upon that there would be limited >> testing for >> >> > this release, given it is based on 19.1.1 rather than a full >> rebase. >> >> > >> >> > Seeking approvals/reviews for: >> >> > (some reruns are still in progress) >> >> > >> >> > rgw - Eric, Adam E >> >> > fs - Venky >> >> > orch - Adam King >> >> > >> >> > quincy-x - Laura, Neha >> >> > reef-x - Laura, Neha >> >> > >> >> > Pls let me know if any tests were missed from this list. >> >> > _______________________________________________ >> >> > ceph-users mailing list -- ceph-users@ceph.io >> >> > To unsubscribe send an email to ceph-users-leave@ceph.io >> >> > >> >> > >> >> _______________________________________________ >> >> ceph-users mailing list -- ceph-users@ceph.io >> >> To unsubscribe send an email to ceph-users-leave@ceph.io >> >> >> > >> > >> > -- >> > >> > Laura Flores >> > >> > She/Her/Hers >> > >> > Software Engineer, Ceph Storage <https://ceph.io> >> > >> > Chicago, IL >> > >> > lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> >> > M: +17087388804 >> > >> > >> > >> >> -- >> >> Laura Flores >> >> She/Her/Hers >> >> Software Engineer, Ceph Storage <https://ceph.io> >> >> Chicago, IL >> >> lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> >> M: +17087388804 >> _______________________________________________ >> ceph-users mailing list -- ceph-users@ceph.io >> To unsubscribe send an email to ceph-users-leave@ceph.io >> > > > -- > > Thanks, > Rachana >
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
-- Cheers, Venky
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
-- Cheers, Venky
-- Thanks, Rachana
Will do, @Rachana Patel <racpatel@redhat.com>. On Thu, Sep 12, 2024 at 3:44 AM Rachana Patel <racpatel@redhat.com> wrote:
Thanks Venky ! We can now focus on next tasks -
- Release Notes https://github.com/ceph/ceph/pull/59539 . Requesting all TLs to review Release notes. - Gibba upgrade - LRC upgrade
@Laura Flores <lflores@ibm.com> , Kindly upgrade cluster and share results.
On Thu, Sep 12, 2024 at 1:03 AM Venky Shankar <vshankar@redhat.com> wrote:
On Wed, Sep 11, 2024 at 10:00 PM Laura Flores <lflores@redhat.com> wrote:
Thanks @Venky Shankar <vshankar@redhat.com> for looking into https://tracker.ceph.com/issues/68002.
As for https://tracker.ceph.com/issues/67999, you are right- it is not cephfs related. Mistake on my part. After looking into this one, it seems to be a test issue, where we check for a "Filestore has been deprecated" warning in all the OSD logs once they've upgraded to Squid. On one of the OSDs, the warning was missing, but that OSD hadn't yet been upgraded to Squid. I believe the test needs to be modified to wait until all OSDs have been upgraded.
Thus, it should not be a blocker.
@Rachana Patel <racpatel@redhat.com> if Venky + Xiubo confirm that https://tracker.ceph.com/issues/68002 is not a blocker, we are good to proceed with next steps.
https://tracker.ceph.com/issues/68002 is NOT a blocker either and is a duplicate of https://tracker.ceph.com/issues/64471.
On Wed, Sep 11, 2024 at 4:46 AM Venky Shankar <vshankar@redhat.com> wrote:
Hey Laura,
On Wed, Sep 11, 2024 at 12:13 AM Laura Flores <lflores@redhat.com> wrote:
I have finished reviewing the upgrade and smoke suites. Most failures are already known/tracked: https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstracke...
*Upgrade: Pending check from FS team* Two fs issues stood out. @Gregory Farnum <gfarnum@redhat.com> @Venky Shankar <vshankar@redhat.com> can you confirm these are not blockers?
- https://tracker.ceph.com/issues/67999 - upgrade/quincy-x/filestore-remove-check: Expected string "FileStore has been deprecated and is no longer supported" is absent from osd log
How is this cephfs related?
- - https://tracker.ceph.com/issues/68002 - upgrade/cephfs/mds_upgrade_sequence: fsstress coredumps
I left my findings in the tracker. See
https://tracker.ceph.com/issues/68002#note-2
The fsstress crash is due to SIGBUS and there are a flurry of OSD related heartbeat_check messages from a node where there is a cephfs kernel driver traceback in the kernel ring buffer. I am running the tracker through Xiubo.
-
When this is confirmed, the upgrade suite can be approved.
*Smoke: APPROVED*
*Rados: Pending check from core team*
On Tue, Sep 10, 2024 at 8:00 AM Laura Flores <lflores@redhat.com> wrote:
Rados reviewed here: https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstracke...
I have asked @Radoslaw Zarzynski <rzarzyns@redhat.com> to take a look at the summary and confirm there are no blockers.
In progress reviewing upgrade and smoke; will provide another update for those soon as well.
On Mon, Sep 9, 2024 at 9:49 AM Laura Flores <lflores@redhat.com> wrote:
> Hey Rachana, > > Thanks for these links. Currently reviewing them and will update the > thread soon. > > Thanks, > Laura > > On Mon, Sep 9, 2024 at 1:06 AM Rachana Patel <racpatel@redhat.com> > wrote: > >> Hi All ! >> PR 59492 <https://github.com/ceph/ceph/pull/59492> was cherry-picked >> on top of the last built. RADOS, upgrade and smoke test suits were >> executed. >> >> Seeking approvals/reviews for following reruns >> >> RADOS Test Result >> <https://pulpito.ceph.com/skanta-2024-09-07_00:00:30-rados-squid-release-distro-default-smithi/>- >> Neha >> Upgrade Test result >> <https://pulpito.ceph.com/sangadi-2024-09-06_06:58:25-upgrade-squid-release-distro-default-smithi/>- >> Neha, Laura >> Smoke Test Result >> <https://pulpito.ceph.com/sangadi-2024-09-06_06:53:40-smoke-squid-release-distro-default-smithi/> >> >> >> >> On Fri, Sep 6, 2024 at 9:57 AM Laura Flores <lflores@redhat.com> >> wrote: >> >>> We have added one more core PR to the release, which was deemed a >>> blocker ( >>> https://github.com/ceph/ceph/pull/59492). Rados, upgrade, and >>> smoke will >>> need to be reapproved. >>> >>> On Wed, Sep 4, 2024 at 3:59 PM Laura Flores <lflores@redhat.com> >>> wrote: >>> >>> > Upgrade suites approved: >>> > >>> https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-httpstrackercephcom... >>> > >>> > On Wed, Sep 4, 2024 at 12:02 PM Adam King <adking@redhat.com> >>> wrote: >>> > >>> >> orch approved, failures are known issues and not release >>> blockers >>> >> >>> >> On Fri, Aug 30, 2024 at 10:43 AM Yuri Weinstein < >>> yweinste@redhat.com> >>> >> wrote: >>> >> >>> >> > Details of this release are summarized here: >>> >> > >>> >> > https://tracker.ceph.com/issues/67779#note-1 >>> >> > >>> >> > Release Notes - TBD >>> >> > Gibba upgrade -TBD >>> >> > LRC upgrade - TBD >>> >> > >>> >> > It was decided and agreed upon that there would be limited >>> testing for >>> >> > this release, given it is based on 19.1.1 rather than a full >>> rebase. >>> >> > >>> >> > Seeking approvals/reviews for: >>> >> > (some reruns are still in progress) >>> >> > >>> >> > rgw - Eric, Adam E >>> >> > fs - Venky >>> >> > orch - Adam King >>> >> > >>> >> > quincy-x - Laura, Neha >>> >> > reef-x - Laura, Neha >>> >> > >>> >> > Pls let me know if any tests were missed from this list. >>> >> > _______________________________________________ >>> >> > ceph-users mailing list -- ceph-users@ceph.io >>> >> > To unsubscribe send an email to ceph-users-leave@ceph.io >>> >> > >>> >> > >>> >> _______________________________________________ >>> >> ceph-users mailing list -- ceph-users@ceph.io >>> >> To unsubscribe send an email to ceph-users-leave@ceph.io >>> >> >>> > >>> > >>> > -- >>> > >>> > Laura Flores >>> > >>> > She/Her/Hers >>> > >>> > Software Engineer, Ceph Storage <https://ceph.io> >>> > >>> > Chicago, IL >>> > >>> > lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> >>> > M: +17087388804 >>> > >>> > >>> > >>> >>> -- >>> >>> Laura Flores >>> >>> She/Her/Hers >>> >>> Software Engineer, Ceph Storage <https://ceph.io> >>> >>> Chicago, IL >>> >>> lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> >>> M: +17087388804 >>> _______________________________________________ >>> ceph-users mailing list -- ceph-users@ceph.io >>> To unsubscribe send an email to ceph-users-leave@ceph.io >>> >> >> >> -- >> >> Thanks, >> Rachana >> > > > -- > > Laura Flores > > She/Her/Hers > > Software Engineer, Ceph Storage <https://ceph.io> > > Chicago, IL > > lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> > M: +17087388804 > > >
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
-- Cheers, Venky
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
-- Cheers, Venky
--
Thanks, Rachana
-- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
The gibba cluster upgrade is complete! All looks good. On Thu, Sep 12, 2024 at 9:54 AM Laura Flores <lflores@redhat.com> wrote:
Will do, @Rachana Patel <racpatel@redhat.com>.
On Thu, Sep 12, 2024 at 3:44 AM Rachana Patel <racpatel@redhat.com> wrote:
Thanks Venky ! We can now focus on next tasks -
- Release Notes https://github.com/ceph/ceph/pull/59539 . Requesting all TLs to review Release notes. - Gibba upgrade - LRC upgrade
@Laura Flores <lflores@ibm.com> , Kindly upgrade cluster and share results.
On Thu, Sep 12, 2024 at 1:03 AM Venky Shankar <vshankar@redhat.com> wrote:
On Wed, Sep 11, 2024 at 10:00 PM Laura Flores <lflores@redhat.com> wrote:
Thanks @Venky Shankar <vshankar@redhat.com> for looking into https://tracker.ceph.com/issues/68002.
As for https://tracker.ceph.com/issues/67999, you are right- it is not cephfs related. Mistake on my part. After looking into this one, it seems to be a test issue, where we check for a "Filestore has been deprecated" warning in all the OSD logs once they've upgraded to Squid. On one of the OSDs, the warning was missing, but that OSD hadn't yet been upgraded to Squid. I believe the test needs to be modified to wait until all OSDs have been upgraded.
Thus, it should not be a blocker.
@Rachana Patel <racpatel@redhat.com> if Venky + Xiubo confirm that https://tracker.ceph.com/issues/68002 is not a blocker, we are good to proceed with next steps.
https://tracker.ceph.com/issues/68002 is NOT a blocker either and is a duplicate of https://tracker.ceph.com/issues/64471.
On Wed, Sep 11, 2024 at 4:46 AM Venky Shankar <vshankar@redhat.com> wrote:
Hey Laura,
On Wed, Sep 11, 2024 at 12:13 AM Laura Flores <lflores@redhat.com> wrote:
I have finished reviewing the upgrade and smoke suites. Most failures are already known/tracked: https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstracke...
*Upgrade: Pending check from FS team* Two fs issues stood out. @Gregory Farnum <gfarnum@redhat.com> @Venky Shankar <vshankar@redhat.com> can you confirm these are not blockers?
- https://tracker.ceph.com/issues/67999 - upgrade/quincy-x/filestore-remove-check: Expected string "FileStore has been deprecated and is no longer supported" is absent from osd log
How is this cephfs related?
- - https://tracker.ceph.com/issues/68002 - upgrade/cephfs/mds_upgrade_sequence: fsstress coredumps
I left my findings in the tracker. See
https://tracker.ceph.com/issues/68002#note-2
The fsstress crash is due to SIGBUS and there are a flurry of OSD related heartbeat_check messages from a node where there is a cephfs kernel driver traceback in the kernel ring buffer. I am running the tracker through Xiubo.
-
When this is confirmed, the upgrade suite can be approved.
*Smoke: APPROVED*
*Rados: Pending check from core team*
On Tue, Sep 10, 2024 at 8:00 AM Laura Flores <lflores@redhat.com> wrote:
> Rados reviewed here: > https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstracke... > > I have asked @Radoslaw Zarzynski <rzarzyns@redhat.com> to take a > look at the summary and confirm there are no blockers. > > In progress reviewing upgrade and smoke; will provide another update > for those soon as well. > > On Mon, Sep 9, 2024 at 9:49 AM Laura Flores <lflores@redhat.com> > wrote: > >> Hey Rachana, >> >> Thanks for these links. Currently reviewing them and will update >> the thread soon. >> >> Thanks, >> Laura >> >> On Mon, Sep 9, 2024 at 1:06 AM Rachana Patel <racpatel@redhat.com> >> wrote: >> >>> Hi All ! >>> PR 59492 <https://github.com/ceph/ceph/pull/59492> was cherry-picked >>> on top of the last built. RADOS, upgrade and smoke test suits were >>> executed. >>> >>> Seeking approvals/reviews for following reruns >>> >>> RADOS Test Result >>> <https://pulpito.ceph.com/skanta-2024-09-07_00:00:30-rados-squid-release-distro-default-smithi/>- >>> Neha >>> Upgrade Test result >>> <https://pulpito.ceph.com/sangadi-2024-09-06_06:58:25-upgrade-squid-release-distro-default-smithi/>- >>> Neha, Laura >>> Smoke Test Result >>> <https://pulpito.ceph.com/sangadi-2024-09-06_06:53:40-smoke-squid-release-distro-default-smithi/> >>> >>> >>> >>> On Fri, Sep 6, 2024 at 9:57 AM Laura Flores <lflores@redhat.com> >>> wrote: >>> >>>> We have added one more core PR to the release, which was deemed a >>>> blocker ( >>>> https://github.com/ceph/ceph/pull/59492). Rados, upgrade, and >>>> smoke will >>>> need to be reapproved. >>>> >>>> On Wed, Sep 4, 2024 at 3:59 PM Laura Flores <lflores@redhat.com> >>>> wrote: >>>> >>>> > Upgrade suites approved: >>>> > >>>> https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-httpstrackercephcom... >>>> > >>>> > On Wed, Sep 4, 2024 at 12:02 PM Adam King <adking@redhat.com> >>>> wrote: >>>> > >>>> >> orch approved, failures are known issues and not release >>>> blockers >>>> >> >>>> >> On Fri, Aug 30, 2024 at 10:43 AM Yuri Weinstein < >>>> yweinste@redhat.com> >>>> >> wrote: >>>> >> >>>> >> > Details of this release are summarized here: >>>> >> > >>>> >> > https://tracker.ceph.com/issues/67779#note-1 >>>> >> > >>>> >> > Release Notes - TBD >>>> >> > Gibba upgrade -TBD >>>> >> > LRC upgrade - TBD >>>> >> > >>>> >> > It was decided and agreed upon that there would be limited >>>> testing for >>>> >> > this release, given it is based on 19.1.1 rather than a full >>>> rebase. >>>> >> > >>>> >> > Seeking approvals/reviews for: >>>> >> > (some reruns are still in progress) >>>> >> > >>>> >> > rgw - Eric, Adam E >>>> >> > fs - Venky >>>> >> > orch - Adam King >>>> >> > >>>> >> > quincy-x - Laura, Neha >>>> >> > reef-x - Laura, Neha >>>> >> > >>>> >> > Pls let me know if any tests were missed from this list. >>>> >> > _______________________________________________ >>>> >> > ceph-users mailing list -- ceph-users@ceph.io >>>> >> > To unsubscribe send an email to ceph-users-leave@ceph.io >>>> >> > >>>> >> > >>>> >> _______________________________________________ >>>> >> ceph-users mailing list -- ceph-users@ceph.io >>>> >> To unsubscribe send an email to ceph-users-leave@ceph.io >>>> >> >>>> > >>>> > >>>> > -- >>>> > >>>> > Laura Flores >>>> > >>>> > She/Her/Hers >>>> > >>>> > Software Engineer, Ceph Storage <https://ceph.io> >>>> > >>>> > Chicago, IL >>>> > >>>> > lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> >>>> > M: +17087388804 >>>> > >>>> > >>>> > >>>> >>>> -- >>>> >>>> Laura Flores >>>> >>>> She/Her/Hers >>>> >>>> Software Engineer, Ceph Storage <https://ceph.io> >>>> >>>> Chicago, IL >>>> >>>> lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> >>>> M: +17087388804 >>>> _______________________________________________ >>>> ceph-users mailing list -- ceph-users@ceph.io >>>> To unsubscribe send an email to ceph-users-leave@ceph.io >>>> >>> >>> >>> -- >>> >>> Thanks, >>> Rachana >>> >> >> >> -- >> >> Laura Flores >> >> She/Her/Hers >> >> Software Engineer, Ceph Storage <https://ceph.io> >> >> Chicago, IL >> >> lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> >> M: +17087388804 >> >> >> > > -- > > Laura Flores > > She/Her/Hers > > Software Engineer, Ceph Storage <https://ceph.io> > > Chicago, IL > > lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> > M: +17087388804 > > >
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
-- Cheers, Venky
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
-- Cheers, Venky
--
Thanks, Rachana
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
-- Laura Flores She/Her/Hers Software Engineer, Ceph Storage <https://ceph.io> Chicago, IL lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
Thanks Laura ! Only 2 pending tasks in the 19.2.0 squid validation phase. 1. LRC upgrade - @Dan Mick <dmick@redhat.com>, kindly share LRC upgrade plan/status. 2. Release Notes - https://github.com/ceph/ceph/pull/59539 . Requesting all TLs to review Release notes. On Thu, Sep 12, 2024 at 1:46 PM Laura Flores <lflores@redhat.com> wrote:
The gibba cluster upgrade is complete! All looks good.
On Thu, Sep 12, 2024 at 9:54 AM Laura Flores <lflores@redhat.com> wrote:
Will do, @Rachana Patel <racpatel@redhat.com>.
On Thu, Sep 12, 2024 at 3:44 AM Rachana Patel <racpatel@redhat.com> wrote:
Thanks Venky ! We can now focus on next tasks -
- Release Notes https://github.com/ceph/ceph/pull/59539 . Requesting all TLs to review Release notes. - Gibba upgrade - LRC upgrade
@Laura Flores <lflores@ibm.com> , Kindly upgrade cluster and share results.
On Thu, Sep 12, 2024 at 1:03 AM Venky Shankar <vshankar@redhat.com> wrote:
On Wed, Sep 11, 2024 at 10:00 PM Laura Flores <lflores@redhat.com> wrote:
Thanks @Venky Shankar <vshankar@redhat.com> for looking into https://tracker.ceph.com/issues/68002.
As for https://tracker.ceph.com/issues/67999, you are right- it is not cephfs related. Mistake on my part. After looking into this one, it seems to be a test issue, where we check for a "Filestore has been deprecated" warning in all the OSD logs once they've upgraded to Squid. On one of the OSDs, the warning was missing, but that OSD hadn't yet been upgraded to Squid. I believe the test needs to be modified to wait until all OSDs have been upgraded.
Thus, it should not be a blocker.
@Rachana Patel <racpatel@redhat.com> if Venky + Xiubo confirm that https://tracker.ceph.com/issues/68002 is not a blocker, we are good to proceed with next steps.
https://tracker.ceph.com/issues/68002 is NOT a blocker either and is a duplicate of https://tracker.ceph.com/issues/64471.
On Wed, Sep 11, 2024 at 4:46 AM Venky Shankar <vshankar@redhat.com> wrote:
Hey Laura,
On Wed, Sep 11, 2024 at 12:13 AM Laura Flores <lflores@redhat.com> wrote:
> I have finished reviewing the upgrade and smoke suites. Most > failures are already known/tracked: > https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstracke... > > *Upgrade: Pending check from FS team* > Two fs issues stood out. @Gregory Farnum <gfarnum@redhat.com> @Venky > Shankar <vshankar@redhat.com> can you confirm these are not > blockers? > > - https://tracker.ceph.com/issues/67999 - > upgrade/quincy-x/filestore-remove-check: Expected string "FileStore has > been deprecated and is no longer supported" is absent from osd log > > How is this cephfs related?
> > - > - https://tracker.ceph.com/issues/68002 - > upgrade/cephfs/mds_upgrade_sequence: fsstress coredumps > > I left my findings in the tracker. See
https://tracker.ceph.com/issues/68002#note-2
The fsstress crash is due to SIGBUS and there are a flurry of OSD related heartbeat_check messages from a node where there is a cephfs kernel driver traceback in the kernel ring buffer. I am running the tracker through Xiubo.
> > - > > When this is confirmed, the upgrade suite can be approved. > > *Smoke: APPROVED* > > *Rados: Pending check from core team* > > > On Tue, Sep 10, 2024 at 8:00 AM Laura Flores <lflores@redhat.com> > wrote: > >> Rados reviewed here: >> https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-build-3-httpstracke... >> >> I have asked @Radoslaw Zarzynski <rzarzyns@redhat.com> to take a >> look at the summary and confirm there are no blockers. >> >> In progress reviewing upgrade and smoke; will provide another >> update for those soon as well. >> >> On Mon, Sep 9, 2024 at 9:49 AM Laura Flores <lflores@redhat.com> >> wrote: >> >>> Hey Rachana, >>> >>> Thanks for these links. Currently reviewing them and will update >>> the thread soon. >>> >>> Thanks, >>> Laura >>> >>> On Mon, Sep 9, 2024 at 1:06 AM Rachana Patel <racpatel@redhat.com> >>> wrote: >>> >>>> Hi All ! >>>> PR 59492 <https://github.com/ceph/ceph/pull/59492> was cherry-picked >>>> on top of the last built. RADOS, upgrade and smoke test suits were >>>> executed. >>>> >>>> Seeking approvals/reviews for following reruns >>>> >>>> RADOS Test Result >>>> <https://pulpito.ceph.com/skanta-2024-09-07_00:00:30-rados-squid-release-distro-default-smithi/>- >>>> Neha >>>> Upgrade Test result >>>> <https://pulpito.ceph.com/sangadi-2024-09-06_06:58:25-upgrade-squid-release-distro-default-smithi/>- >>>> Neha, Laura >>>> Smoke Test Result >>>> <https://pulpito.ceph.com/sangadi-2024-09-06_06:53:40-smoke-squid-release-distro-default-smithi/> >>>> >>>> >>>> >>>> On Fri, Sep 6, 2024 at 9:57 AM Laura Flores <lflores@redhat.com> >>>> wrote: >>>> >>>>> We have added one more core PR to the release, which was deemed >>>>> a blocker ( >>>>> https://github.com/ceph/ceph/pull/59492). Rados, upgrade, and >>>>> smoke will >>>>> need to be reapproved. >>>>> >>>>> On Wed, Sep 4, 2024 at 3:59 PM Laura Flores <lflores@redhat.com> >>>>> wrote: >>>>> >>>>> > Upgrade suites approved: >>>>> > >>>>> https://tracker.ceph.com/projects/rados/wiki/SQUID#v1920-httpstrackercephcom... >>>>> > >>>>> > On Wed, Sep 4, 2024 at 12:02 PM Adam King <adking@redhat.com> >>>>> wrote: >>>>> > >>>>> >> orch approved, failures are known issues and not release >>>>> blockers >>>>> >> >>>>> >> On Fri, Aug 30, 2024 at 10:43 AM Yuri Weinstein < >>>>> yweinste@redhat.com> >>>>> >> wrote: >>>>> >> >>>>> >> > Details of this release are summarized here: >>>>> >> > >>>>> >> > https://tracker.ceph.com/issues/67779#note-1 >>>>> >> > >>>>> >> > Release Notes - TBD >>>>> >> > Gibba upgrade -TBD >>>>> >> > LRC upgrade - TBD >>>>> >> > >>>>> >> > It was decided and agreed upon that there would be limited >>>>> testing for >>>>> >> > this release, given it is based on 19.1.1 rather than a >>>>> full rebase. >>>>> >> > >>>>> >> > Seeking approvals/reviews for: >>>>> >> > (some reruns are still in progress) >>>>> >> > >>>>> >> > rgw - Eric, Adam E >>>>> >> > fs - Venky >>>>> >> > orch - Adam King >>>>> >> > >>>>> >> > quincy-x - Laura, Neha >>>>> >> > reef-x - Laura, Neha >>>>> >> > >>>>> >> > Pls let me know if any tests were missed from this list. >>>>> >> > _______________________________________________ >>>>> >> > ceph-users mailing list -- ceph-users@ceph.io >>>>> >> > To unsubscribe send an email to ceph-users-leave@ceph.io >>>>> >> > >>>>> >> > >>>>> >> _______________________________________________ >>>>> >> ceph-users mailing list -- ceph-users@ceph.io >>>>> >> To unsubscribe send an email to ceph-users-leave@ceph.io >>>>> >> >>>>> > >>>>> > >>>>> > -- >>>>> > >>>>> > Laura Flores >>>>> > >>>>> > She/Her/Hers >>>>> > >>>>> > Software Engineer, Ceph Storage <https://ceph.io> >>>>> > >>>>> > Chicago, IL >>>>> > >>>>> > lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> >>>>> > M: +17087388804 >>>>> > >>>>> > >>>>> > >>>>> >>>>> -- >>>>> >>>>> Laura Flores >>>>> >>>>> She/Her/Hers >>>>> >>>>> Software Engineer, Ceph Storage <https://ceph.io> >>>>> >>>>> Chicago, IL >>>>> >>>>> lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> >>>>> M: +17087388804 >>>>> _______________________________________________ >>>>> ceph-users mailing list -- ceph-users@ceph.io >>>>> To unsubscribe send an email to ceph-users-leave@ceph.io >>>>> >>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> Rachana >>>> >>> >>> >>> -- >>> >>> Laura Flores >>> >>> She/Her/Hers >>> >>> Software Engineer, Ceph Storage <https://ceph.io> >>> >>> Chicago, IL >>> >>> lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> >>> M: +17087388804 >>> >>> >>> >> >> -- >> >> Laura Flores >> >> She/Her/Hers >> >> Software Engineer, Ceph Storage <https://ceph.io> >> >> Chicago, IL >> >> lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> >> M: +17087388804 >> >> >> > > -- > > Laura Flores > > She/Her/Hers > > Software Engineer, Ceph Storage <https://ceph.io> > > Chicago, IL > > lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> > M: +17087388804 > > >
-- Cheers, Venky
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
-- Cheers, Venky
--
Thanks, Rachana
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
--
Laura Flores
She/Her/Hers
Software Engineer, Ceph Storage <https://ceph.io>
Chicago, IL
lflores@ibm.com | lflores@redhat.com <lflores@redhat.com> M: +17087388804
-- Thanks, Rachana
participants (7)
-
Adam King
-
Brad Hubbard
-
J. Eric Ivancich
-
Laura Flores
-
Rachana Patel
-
Venky Shankar
-
Yuri Weinstein