It's still in the process of being published.
On Mon, Dec 9, 2019 at 8:52 AM Sasha Litvak
<alexander.v.litvak(a)gmail.com> wrote:
Sorry for nagging, but when will 14.2.5 actually be released?
On Fri, Dec 6, 2019, 10:32 AM Alfredo Deza <adeza(a)redhat.com> wrote:
>
> On Fri, Dec 6, 2019 at 9:52 AM Sage Weil <sweil(a)redhat.com> wrote:
> >
> > On Fri, 6 Dec 2019, Alfredo Deza wrote:
> > > I would like to raise once again the discussion of improving our
> > > release process so that we can minimize these things happening at the
> > > very last minute.
> > >
> > > This isn't the first time we've hit issues that had to be reverted
> > > (luckily packages didn't get to
download.ceph.com), and since we
> > > insist on owning the package and repository creation, there
> > > is no way we can retire a "bad" release that is available
publicly.
> >
> > Doesn't the fact that we own the distribution mean that we *can* yank a
> > bad release? If we were relying on downstreams to do this for us we'd
> > have no control at all. Currently the only reason we can't is because
> > we're chafing against the tools.
> >
> > > A few of the items that I believe could improve are:
> > >
> > > * Start having stricter development and freeze periods
> > > * Add a quiet period (or waiting time) of a week for a release to be
public
> > > * Determine what the SHA to release is, so that testing can be done
> > > against that SHA
> > > * Avoid depending on back and forth emails asking leads if they are OK
> > > with a release or if they need another PR to be merged - let's define
> > > criteria to meet this.
> >
> > I think the main missing piece here is deploying the release candidate
> > into production before actually releasing. Most of these last minute
> > issues were caught when we deployed on the lab cluster and my home
> > cluster. I think this is similar to your 'quiet' period: once qa
passes,
> > we should deploy internally and make sure there aren't things the qa suite
> > missed.
>
> Yes exactly, a release candidate would be great.
>
> >
> > The other problem was discovered in parallel by David and was totally
> > unrelated to the release process/freeze/testing, but was deemed to be a
> > release blocker. I don't think any process can avoid that.
>
> And by no means I want to raise the release issues as something
> specific to this last problem, but something
> that does need to be addressed.
>
> >
> > This release is a bit different, though, because there is a data
> > corruption bug we really want to get out. Perhaps what we should have
> > done here is a minimal release that only has that one bug fix so that our
> > regression risk is basically 0 and we can expedite. Instead we grabbed
> > everything that had already been tested and merged and went with that.
>
> Yes exactly. We've had similar "merge with what was tested and
> release" situations before that have not worked well.
>
> >
> > sage
> >
> >
> > >
> > >
> > > On Thu, Dec 5, 2019 at 1:09 PM Neha Ojha <nojha(a)redhat.com> wrote:
> > > >
> > > >
https://github.com/ceph/ceph/pull/32018 has merged, we should be
ready
> > > > to build 14.2.5 now.
> > > >
> > > > On Wed, Dec 4, 2019 at 5:02 PM Neha Ojha <nojha(a)redhat.com>
wrote:
> > > > >
> > > > > David Zafman has discovered a buggy patch in nautilus, which we
want
> > > > > to revert before releasing 14.2.5. More details in
> > > > >
https://github.com/ceph/ceph/pull/31970#issuecomment-561913632.
The
> > > > > revert PR
https://github.com/ceph/ceph/pull/32018 is being tested
now.
> > > > > We'll need to rebuild 14.2.5 once the revert merges.
> > > > >
> > > > > Sorry about the inconvenience.
> > > > >
> > > > > Neha
> > > > >
> > > > > On Wed, Dec 4, 2019 at 11:28 AM Sage Weil
<sweil(a)redhat.com> wrote:
> > > > > >
> > > > > > On Wed, 4 Dec 2019, Abhishek Lekshmanan wrote:
> > > > > > > Yuri Weinstein <yweinste(a)redhat.com> writes:
> > > > > > >
> > > > > > > > David, assuming Sage is OK with `ceph-deploy` and
overall tests
> > > > > > > > results, this is ready for publishing.
> > > > > > >
> > > > > > > Sage, is this ready to build and start publishing
packages for?
> > > > > >
> > > > > > Yeah, I think we're good to go!
> > > > > >
> > > > > > Thanks everyone-
> > > > > > sage
> > > > > >
> > > > > > > > Abhishek, Nathan FYI
> > > > > > > >
> > > > > > > > On Wed, Nov 27, 2019 at 12:38 PM Neha Ojha
<nojha(a)redhat.com> wrote:
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Wed, Nov 27, 2019 at 8:01 AM Yuri Weinstein
<yweinste(a)redhat.com> wrote:
> > > > > > > >>>
> > > > > > > >>> Outstanding need approval:
> > > > > > > >>>
> > > > > > > >>> ceph-deploy - Sage
> > > > > > > >>> upgrade/luminous-x (nautilus) - Neha, Josh
reviewing
> > > > > > > >>
> > > > > > > >> approved
> > > > > > > >>
> > > > > > > >>>
> > > > > > > >>> upgrade/mimic-x (nautilus) - Neha, Josh
reviewing
> > > > > > > >>
> > > > > > > >> approved, failure tracked in
https://tracker.ceph.com/issues/43048
> > > > > > > >>
> > > > > > > >>>
> > > > > > > >>> On Fri, Nov 22, 2019 at 7:22 AM Yuri
Weinstein <yweinste(a)redhat.com> wrote:
> > > > > > > >>> >
> > > > > > > >>> > (This is an early update, some tests
are still running, as we are
> > > > > > > >>> > trying to release this point next
week before the US holidays, and
> > > > > > > >>> > have more time to review results)
> > > > > > > >>> >
> > > > > > > >>> > Details of this release summarized
here:
> > > > > > > >>> >
https://tracker.ceph.com/issues/42839#note-3
> > > > > > > >>> >
> > > > > > > >>> > rados - approved by Neha
> > > > > > > >>> > rgw - approved by Casey
> > > > > > > >>> > rbd - need approval Jason
> > > > > > > >>> > krbd - need approval Jason, Ilya
> > > > > > > >>> > fs - need approval Patrick, Ramana
> > > > > > > >>> > kcephfs - need approval Patrick,
Ramana
> > > > > > > >>> > multimds - need approval Patrick,
Ramana
> > > > > > > >>> > ceph-deploy - FAILED Sage, Alfredo ?
> > > > > > > >>> > ceph-disk - N/A
> > > > > > > >>> > upgrade/client-upgrade-hammer
(nautilus) - N/A
> > > > > > > >>> > upgrade/client-upgrade-jewel
(nautilus) - PASSED
> > > > > > > >>> > upgrade/client-upgrade-mimic
(nautilus) - FAILED
> > > > > > > >>> > upgrade/luminous-p2p - in progress
> > > > > > > >>> > powercycle - in progress
> > > > > > > >>> > ceph-ansible - Brad is finxing
> > > > > > > >>> > upgrade/luminous-x (nautilus) - in
progress
> > > > > > > >>> > upgrade/mimic-x (nautilus) - in
progress
> > > > > > > >>> > ceph-volume - Jan fixing
> > > > > > > >>> > (please speak up if something is
missing)
> > > > > > > >>> >
> > > > > > > >>> > Thx
> > > > > > > >>> > YuriW
> > > > > > > >>>
> > > > > > > > _______________________________________________
> > > > > > > > Dev mailing list -- dev(a)ceph.io
> > > > > > > > To unsubscribe send an email to dev-leave(a)ceph.io
> > > > > > >
> > > > > > > --
> > > > > > > Abhishek Lekshmanan
> > > > > > > SUSE Software Solutions Germany GmbH
> > > > > > > GF: Felix Imendörffer, Mary Higgins, Sri Rasiah, HRB
21284 (AG Nürnberg)
> > > > > > > _______________________________________________
> > > > > > > Dev mailing list -- dev(a)ceph.io
> > > > > > > To unsubscribe send an email to dev-leave(a)ceph.io
> > > > > > > _______________________________________________
> > > > > > Dev mailing list -- dev(a)ceph.io
> > > > > > To unsubscribe send an email to dev-leave(a)ceph.io
> > > >
> > >
> > >
> _______________________________________________
> Dev mailing list -- dev(a)ceph.io
> To unsubscribe send an email to dev-leave(a)ceph.io