14.2.14 Nautilus validation status
Details of this release summarized here: https://tracker.ceph.com/issues/48200#note-1 Asking dev leads for early approval as we target the release date - early next week. Some suites are still in progress (will try to finish over the weekend) rados - approved Neha? rgw - approved Casey? rbd - approved Jason? krbd - approved Jason, Ilya? fs - approved Patrick? multimds - approved Patrick? ceph-deploy - in progress upgrade/client-upgrade-jewel-nautilus (nautilus) - in progress upgrade/client-upgrade-mimic (nautilus) - in progress upgrade/client-upgrade-luminous-nautilus (nautilus) - in progress upgrade/client-upgrade-nautilus-octopus-octopus (octopus) - in progress upgrade/nautilus-p2 - in progress upgrade/luminous-x (nautilus) - in progress upgrade/mimic-x (nautilus) - in progress upgrade/nautilus-x (octopus) - in progress ceph-volume - in progress (Jan pls see) Thx YuriW
On Fri, Nov 13, 2020 at 2:10 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release summarized here: https://tracker.ceph.com/issues/48200#note-1
Asking dev leads for early approval as we target the release date - early next week. Some suites are still in progress (will try to finish over the weekend)
rados - approved Neha? rgw - approved Casey?
rgw approved
rbd - approved Jason? krbd - approved Jason, Ilya? fs - approved Patrick? multimds - approved Patrick?
ceph-deploy - in progress upgrade/client-upgrade-jewel-nautilus (nautilus) - in progress upgrade/client-upgrade-mimic (nautilus) - in progress upgrade/client-upgrade-luminous-nautilus (nautilus) - in progress upgrade/client-upgrade-nautilus-octopus-octopus (octopus) - in progress upgrade/nautilus-p2 - in progress upgrade/luminous-x (nautilus) - in progress upgrade/mimic-x (nautilus) - in progress upgrade/nautilus-x (octopus) - in progress ceph-volume - in progress (Jan pls see)
Thx YuriW _______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
On Fri, Nov 13, 2020 at 8:10 PM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release summarized here: https://tracker.ceph.com/issues/48200#note-1
Asking dev leads for early approval as we target the release date - early next week. Some suites are still in progress (will try to finish over the weekend)
rados - approved Neha? rgw - approved Casey? rbd - approved Jason? krbd - approved Jason, Ilya?
Approved. Thanks, Ilya
On Fri, Nov 13, 2020 at 11:10 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release summarized here: https://tracker.ceph.com/issues/48200#note-1
Asking dev leads for early approval as we target the release date - early next week. Some suites are still in progress (will try to finish over the weekend)
rados - approved Neha? rgw - approved Casey? rbd - approved Jason? krbd - approved Jason, Ilya? fs - approved Patrick?
We now have a warning in the cluster log about mgr modules failing: "2020-11-12 23:11:43.631135 mon.b (mon.0) 100 : cluster [ERR] Health check failed: 3 mgr modules have failed (MGR_MODULE_ERROR)" in cluster log From: https://pulpito.ceph.com/yuriw-2020-11-12_20:43:15-fs-nautilus-distro-basic-... e.g.: /ceph/teuthology-archive/yuriw-2020-11-12_20:43:15-fs-nautilus-distro-basic-smithi/5617317/remote/smithi045/log/ceph-mgr.x.log.gz I don't see why this warning is happening. The modules in question look like they have loaded successfully. Otherwise the runs look good to me but this warning needs addressed. -- Patrick Donnelly, Ph.D. He / Him / His Principal Software Engineer Red Hat Sunnyvale, CA GPG: 19F28A586F808C2402351B93C3301A3E258DD79D
On Fri, Nov 13, 2020 at 11:50 AM Patrick Donnelly <pdonnell@redhat.com> wrote:
On Fri, Nov 13, 2020 at 11:10 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release summarized here: https://tracker.ceph.com/issues/48200#note-1
Asking dev leads for early approval as we target the release date - early next week. Some suites are still in progress (will try to finish over the weekend)
rados - approved Neha?
approved
rgw - approved Casey? rbd - approved Jason? krbd - approved Jason, Ilya? fs - approved Patrick?
We now have a warning in the cluster log about mgr modules failing:
"2020-11-12 23:11:43.631135 mon.b (mon.0) 100 : cluster [ERR] Health check failed: 3 mgr modules have failed (MGR_MODULE_ERROR)" in cluster log
From: https://pulpito.ceph.com/yuriw-2020-11-12_20:43:15-fs-nautilus-distro-basic-...
e.g.: /ceph/teuthology-archive/yuriw-2020-11-12_20:43:15-fs-nautilus-distro-basic-smithi/5617317/remote/smithi045/log/ceph-mgr.x.log.gz
I don't see why this warning is happening. The modules in question look like they have loaded successfully.
https://tracker.ceph.com/issues/48230 - this is appearing in RADOS too, but I think it is harmless Neha
Otherwise the runs look good to me but this warning needs addressed.
-- Patrick Donnelly, Ph.D. He / Him / His Principal Software Engineer Red Hat Sunnyvale, CA GPG: 19F28A586F808C2402351B93C3301A3E258DD79D
On Fri, Nov 13, 2020 at 11:56 AM Neha Ojha <nojha@redhat.com> wrote:
On Fri, Nov 13, 2020 at 11:50 AM Patrick Donnelly <pdonnell@redhat.com> wrote:
On Fri, Nov 13, 2020 at 11:10 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release summarized here: https://tracker.ceph.com/issues/48200#note-1
Asking dev leads for early approval as we target the release date - early next week. Some suites are still in progress (will try to finish over the weekend)
rados - approved Neha?
approved
rgw - approved Casey? rbd - approved Jason? krbd - approved Jason, Ilya? fs - approved Patrick?
We now have a warning in the cluster log about mgr modules failing:
"2020-11-12 23:11:43.631135 mon.b (mon.0) 100 : cluster [ERR] Health check failed: 3 mgr modules have failed (MGR_MODULE_ERROR)" in cluster log
From: https://pulpito.ceph.com/yuriw-2020-11-12_20:43:15-fs-nautilus-distro-basic-...
e.g.: /ceph/teuthology-archive/yuriw-2020-11-12_20:43:15-fs-nautilus-distro-basic-smithi/5617317/remote/smithi045/log/ceph-mgr.x.log.gz
I don't see why this warning is happening. The modules in question look like they have loaded successfully.
https://tracker.ceph.com/issues/48230 - this is appearing in RADOS too, but I think it is harmless
It does look harmless but do we know why it's happening though? -- Patrick Donnelly, Ph.D. He / Him / His Principal Software Engineer Red Hat Sunnyvale, CA GPG: 19F28A586F808C2402351B93C3301A3E258DD79D
On 11/13/2020 12:18 PM, Patrick Donnelly wrote:
On Fri, Nov 13, 2020 at 11:56 AM Neha Ojha <nojha@redhat.com> wrote:
On Fri, Nov 13, 2020 at 11:50 AM Patrick Donnelly <pdonnell@redhat.com> wrote:
On Fri, Nov 13, 2020 at 11:10 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release summarized here: https://tracker.ceph.com/issues/48200#note-1
Asking dev leads for early approval as we target the release date - early next week. Some suites are still in progress (will try to finish over the weekend)
rados - approved Neha?
approved
rgw - approved Casey? rbd - approved Jason? krbd - approved Jason, Ilya? fs - approved Patrick?
We now have a warning in the cluster log about mgr modules failing:
"2020-11-12 23:11:43.631135 mon.b (mon.0) 100 : cluster [ERR] Health check failed: 3 mgr modules have failed (MGR_MODULE_ERROR)" in cluster log
From: https://pulpito.ceph.com/yuriw-2020-11-12_20:43:15-fs-nautilus-distro-basic-...
e.g.: /ceph/teuthology-archive/yuriw-2020-11-12_20:43:15-fs-nautilus-distro-basic-smithi/5617317/remote/smithi045/log/ceph-mgr.x.log.gz
I don't see why this warning is happening. The modules in question look like they have loaded successfully.
https://tracker.ceph.com/issues/48230 - this is appearing in RADOS too, but I think it is harmless
It does look harmless but do we know why it's happening though?
It's odd, because the mgr log for the job cited in the bug shows a lot of what look like normal status messages from rbd_support (at least)
https://tracker.ceph.com/issues/48230#note-2 is what seems to be the problem, though I don't understand it fully yet, could be related to https://github.com/ceph/ceph/pull/37844. On Fri, Nov 13, 2020 at 12:24 PM Dan Mick <dmick@redhat.com> wrote:
On 11/13/2020 12:18 PM, Patrick Donnelly wrote:
On Fri, Nov 13, 2020 at 11:56 AM Neha Ojha <nojha@redhat.com> wrote:
On Fri, Nov 13, 2020 at 11:50 AM Patrick Donnelly <pdonnell@redhat.com> wrote:
On Fri, Nov 13, 2020 at 11:10 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release summarized here: https://tracker.ceph.com/issues/48200#note-1
Asking dev leads for early approval as we target the release date - early next week. Some suites are still in progress (will try to finish over the weekend)
rados - approved Neha?
approved
rgw - approved Casey? rbd - approved Jason? krbd - approved Jason, Ilya? fs - approved Patrick?
We now have a warning in the cluster log about mgr modules failing:
"2020-11-12 23:11:43.631135 mon.b (mon.0) 100 : cluster [ERR] Health check failed: 3 mgr modules have failed (MGR_MODULE_ERROR)" in cluster log
From: https://pulpito.ceph.com/yuriw-2020-11-12_20:43:15-fs-nautilus-distro-basic-...
e.g.: /ceph/teuthology-archive/yuriw-2020-11-12_20:43:15-fs-nautilus-distro-basic-smithi/5617317/remote/smithi045/log/ceph-mgr.x.log.gz
I don't see why this warning is happening. The modules in question look like they have loaded successfully.
https://tracker.ceph.com/issues/48230 - this is appearing in RADOS too, but I think it is harmless
It does look harmless but do we know why it's happening though?
It's odd, because the mgr log for the job cited in the bug shows a lot of what look like normal status messages from rbd_support (at least)
Ok, looks like we need https://github.com/ceph/ceph/pull/35760 in nautilus as well, it was backported only to octopus. I have updated https://tracker.ceph.com/issues/46224 accordingly. On Fri, Nov 13, 2020 at 1:24 PM Neha Ojha <nojha@redhat.com> wrote:
https://tracker.ceph.com/issues/48230#note-2 is what seems to be the problem, though I don't understand it fully yet, could be related to https://github.com/ceph/ceph/pull/37844.
On Fri, Nov 13, 2020 at 12:24 PM Dan Mick <dmick@redhat.com> wrote:
On 11/13/2020 12:18 PM, Patrick Donnelly wrote:
On Fri, Nov 13, 2020 at 11:56 AM Neha Ojha <nojha@redhat.com> wrote:
On Fri, Nov 13, 2020 at 11:50 AM Patrick Donnelly <pdonnell@redhat.com> wrote:
On Fri, Nov 13, 2020 at 11:10 AM Yuri Weinstein <yweinste@redhat.com> wrote:
Details of this release summarized here: https://tracker.ceph.com/issues/48200#note-1
Asking dev leads for early approval as we target the release date - early next week. Some suites are still in progress (will try to finish over the weekend)
rados - approved Neha?
approved
rgw - approved Casey? rbd - approved Jason? krbd - approved Jason, Ilya? fs - approved Patrick?
We now have a warning in the cluster log about mgr modules failing:
"2020-11-12 23:11:43.631135 mon.b (mon.0) 100 : cluster [ERR] Health check failed: 3 mgr modules have failed (MGR_MODULE_ERROR)" in cluster log
From: https://pulpito.ceph.com/yuriw-2020-11-12_20:43:15-fs-nautilus-distro-basic-...
e.g.: /ceph/teuthology-archive/yuriw-2020-11-12_20:43:15-fs-nautilus-distro-basic-smithi/5617317/remote/smithi045/log/ceph-mgr.x.log.gz
I don't see why this warning is happening. The modules in question look like they have loaded successfully.
https://tracker.ceph.com/issues/48230 - this is appearing in RADOS too, but I think it is harmless
It does look harmless but do we know why it's happening though?
It's odd, because the mgr log for the job cited in the bug shows a lot of what look like normal status messages from rbd_support (at least)
On Fri, Nov 13, 2020 at 01:39:15PM -0800, Neha Ojha wrote:
Ok, looks like we need https://github.com/ceph/ceph/pull/35760 in nautilus as well, it was backported only to octopus. I have updated https://tracker.ceph.com/issues/46224 accordingly.
Backport PR is up: https://github.com/ceph/ceph/pull/38069 Nathan
https://github.com/ceph/ceph/pull/38069 => starting testing in parallel with other tests in progress. On Fri, Nov 13, 2020 at 1:50 PM Nathan Cutler <ncutler@suse.com> wrote:
On Fri, Nov 13, 2020 at 01:39:15PM -0800, Neha Ojha wrote:
Ok, looks like we need https://github.com/ceph/ceph/pull/35760 in nautilus as well, it was backported only to octopus. I have updated https://tracker.ceph.com/issues/46224 accordingly.
Backport PR is up: https://github.com/ceph/ceph/pull/38069
Nathan _______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
https://github.com/ceph/ceph/pull/38076 is a simple fix for a failing test if we can squeeze it in? On Sun, Nov 15, 2020 at 9:29 AM Yuri Weinstein <yweinste@redhat.com> wrote:
https://github.com/ceph/ceph/pull/38069 tested, approved and megred
-- Cheers, Brad
If it's (https://github.com/ceph/ceph/pull/38076) ready as is, we can merge it. Neha, Josh ? On Mon, Nov 16, 2020 at 12:25 AM Brad Hubbard <bhubbard@redhat.com> wrote:
https://github.com/ceph/ceph/pull/38076 is a simple fix for a failing test if we can squeeze it in?
On Sun, Nov 15, 2020 at 9:29 AM Yuri Weinstein <yweinste@redhat.com> wrote:
https://github.com/ceph/ceph/pull/38069 tested, approved and megred
-- Cheers, Brad
Josh, David This is ready for LRC upgrade and final approval. https://tracker.ceph.com/issues/48170#note-1
Yes, it's fine to merge since it's only changing a test. On 11/16/20 8:14 AM, Yuri Weinstein wrote:
If it's (https://github.com/ceph/ceph/pull/38076) ready as is, we can merge it. Neha, Josh ?
On Mon, Nov 16, 2020 at 12:25 AM Brad Hubbard <bhubbard@redhat.com> wrote:
https://github.com/ceph/ceph/pull/38076 is a simple fix for a failing test if we can squeeze it in?
On Sun, Nov 15, 2020 at 9:29 AM Yuri Weinstein <yweinste@redhat.com> wrote:
https://github.com/ceph/ceph/pull/38069 tested, approved and megred
-- Cheers, Brad
See summary https://tracker.ceph.com/issues/48200#note-1 rados - approved Neha? rbd - approved Jason? fs - approved Patrick? multimds - approved Patrick? ceph-deploy - in progress upgrade/nautilus-p2 - in progress upgrade/mimic-x (nautilus) - Neha, seems like a known issue, pls confirm ceph-volume - FAILED, Jan approved?
On Mon, Nov 16, 2020 at 9:11 AM Yuri Weinstein <yweinste@redhat.com> wrote:
See summary https://tracker.ceph.com/issues/48200#note-1
rados - approved Neha?
approved
rbd - approved Jason?
fs - approved Patrick? multimds - approved Patrick?
ceph-deploy - in progress
upgrade/nautilus-p2 - in progress
upgrade/mimic-x (nautilus) - Neha, seems like a known issue, pls confirm
approved
ceph-volume - FAILED, Jan approved?
Hi. Is there any estimate time for nautilus 14.2.14 based on new CVE? On Mon, Nov 16, 2020 at 9:24 PM Neha Ojha <nojha@redhat.com> wrote:
On Mon, Nov 16, 2020 at 9:11 AM Yuri Weinstein <yweinste@redhat.com> wrote:
See summary https://tracker.ceph.com/issues/48200#note-1
rados - approved Neha?
approved
rbd - approved Jason?
fs - approved Patrick? multimds - approved Patrick?
ceph-deploy - in progress
upgrade/nautilus-p2 - in progress
upgrade/mimic-x (nautilus) - Neha, seems like a known issue, pls confirm
approved
ceph-volume - FAILED, Jan approved?
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
Later today for both nautilus and octopus. On 11/18/20 2:08 PM, Seena Fallah wrote:
Hi. Is there any estimate time for nautilus 14.2.14 based on new CVE?
On Mon, Nov 16, 2020 at 9:24 PM Neha Ojha <nojha@redhat.com <mailto:nojha@redhat.com>> wrote:
On Mon, Nov 16, 2020 at 9:11 AM Yuri Weinstein <yweinste@redhat.com <mailto:yweinste@redhat.com>> wrote: > > See summary > https://tracker.ceph.com/issues/48200#note-1 <https://tracker.ceph.com/issues/48200#note-1> > > rados - approved Neha?
approved
> > rbd - approved Jason? > > fs - approved Patrick? > multimds - approved Patrick? > > ceph-deploy - in progress > > upgrade/nautilus-p2 - in progress > > upgrade/mimic-x (nautilus) - Neha, seems like a known issue, pls confirm
approved
> > ceph-volume - FAILED, Jan approved? > _______________________________________________ Dev mailing list -- dev@ceph.io <mailto:dev@ceph.io> To unsubscribe send an email to dev-leave@ceph.io <mailto:dev-leave@ceph.io>
_______________________________________________ Ceph-qa mailing list -- ceph-qa@ceph.io To unsubscribe send an email to ceph-qa-leave@ceph.io
Fo rceph-volume, we really need for https://github.com/ceph/ceph/pull/38046 to make it in 14.2.14, otherwise, both .13 and .14 won't be usable to deploy Ceph or expand on upgraded cluster. This is a major bug. Thanks! ––––––––– Sébastien Han Senior Principal Software Engineer, Storage Architect "Always give 100%. Unless you're giving blood." On Wed, Nov 18, 2020 at 11:17 PM Josh Durgin <jdurgin@redhat.com> wrote:
Later today for both nautilus and octopus.
On 11/18/20 2:08 PM, Seena Fallah wrote:
Hi. Is there any estimate time for nautilus 14.2.14 based on new CVE?
On Mon, Nov 16, 2020 at 9:24 PM Neha Ojha <nojha@redhat.com <mailto:nojha@redhat.com>> wrote:
On Mon, Nov 16, 2020 at 9:11 AM Yuri Weinstein <yweinste@redhat.com <mailto:yweinste@redhat.com>> wrote: > > See summary > https://tracker.ceph.com/issues/48200#note-1 <https://tracker.ceph.com/issues/48200#note-1> > > rados - approved Neha?
approved
> > rbd - approved Jason? > > fs - approved Patrick? > multimds - approved Patrick? > > ceph-deploy - in progress > > upgrade/nautilus-p2 - in progress > > upgrade/mimic-x (nautilus) - Neha, seems like a known issue, pls confirm
approved
> > ceph-volume - FAILED, Jan approved? > _______________________________________________ Dev mailing list -- dev@ceph.io <mailto:dev@ceph.io> To unsubscribe send an email to dev-leave@ceph.io <mailto:dev-leave@ceph.io>
_______________________________________________ Ceph-qa mailing list -- ceph-qa@ceph.io To unsubscribe send an email to ceph-qa-leave@ceph.io
I will ceph-volume tests on it so we can merge soon On Thu, Nov 19, 2020 at 6:18 AM Sebastien Han <shan@redhat.com> wrote:
Fo rceph-volume, we really need for https://github.com/ceph/ceph/pull/38046 to make it in 14.2.14, otherwise, both .13 and .14 won't be usable to deploy Ceph or expand on upgraded cluster. This is a major bug.
Thanks! ––––––––– Sébastien Han Senior Principal Software Engineer, Storage Architect
"Always give 100%. Unless you're giving blood."
On Wed, Nov 18, 2020 at 11:17 PM Josh Durgin <jdurgin@redhat.com> wrote:
Later today for both nautilus and octopus.
On 11/18/20 2:08 PM, Seena Fallah wrote:
Hi. Is there any estimate time for nautilus 14.2.14 based on new CVE?
On Mon, Nov 16, 2020 at 9:24 PM Neha Ojha <nojha@redhat.com <mailto:nojha@redhat.com>> wrote:
On Mon, Nov 16, 2020 at 9:11 AM Yuri Weinstein <yweinste@redhat.com <mailto:yweinste@redhat.com>> wrote: > > See summary > https://tracker.ceph.com/issues/48200#note-1 <https://tracker.ceph.com/issues/48200#note-1> > > rados - approved Neha?
approved
> > rbd - approved Jason? > > fs - approved Patrick? > multimds - approved Patrick? > > ceph-deploy - in progress > > upgrade/nautilus-p2 - in progress > > upgrade/mimic-x (nautilus) - Neha, seems like a known issue, pls confirm
approved
> > ceph-volume - FAILED, Jan approved? > _______________________________________________ Dev mailing list -- dev@ceph.io <mailto:dev@ceph.io> To unsubscribe send an email to dev-leave@ceph.io <mailto:dev-leave@ceph.io>
_______________________________________________ Ceph-qa mailing list -- ceph-qa@ceph.io To unsubscribe send an email to ceph-qa-leave@ceph.io
On Fri, Nov 13, 2020 at 11:56:13AM -0800, Neha Ojha wrote:
rados - approved Neha?
approved
Just wondering, don't we want the fix for [1] to be backported before the next nautilus release? [1] https://tracker.ceph.com/issues/48153 -- Mykola Golub
participants (12)
-
Brad Hubbard
-
Casey Bodley
-
Dan Mick
-
Ilya Dryomov
-
Josh Durgin
-
Mykola Golub
-
Nathan Cutler
-
Neha Ojha
-
Patrick Donnelly
-
Sebastien Han
-
Seena Fallah
-
Yuri Weinstein