On Mon, Jun 29, 2020 at 11:38 AM Patrick Donnelly <pdonnell(a)redhat.com> wrote:
On Mon, Jun 29, 2020 at 10:35 AM Yuri Weinstein <yweinste(a)redhat.com> wrote:
fs - approved Patrick?
There's a new upgrade test failure:
https://pulpito.ceph.com/yuriw-2020-06-25_22:31:00-fs-octopus-distro-basic-…
2020-06-26T07:13:13.444 INFO:teuthology.orchestra.run.smithi087:> sudo
adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage
timeout 120 ceph --cluster ceph --log-early osd dump --format=json
...
2020-06-26T07:13:13.709
INFO:teuthology.orchestra.run.smithi087.stderr:Error EINVAL: invalid
command
I vaguely recall seeing this before but can't find the tracker ticket.
The last successful run was on the 24th (the day before you scheduled
tests!!):
https://pulpito.ceph.com/teuthology-2020-06-24_04:10:03-fs-octopus-distro-b…
The difference in commits
5c9eee5a7ad1bb8b161f33b4560d5dfc4a000b6d..upstream/heads/octopus does
not shine light on why this would happen.
Any ideas?
Looks like this is caused by an older version of Mimic, v13.2.8. The
successful run from June 24th:
2020-06-25T06:10:56.756 DEBUG:teuthology.packaging:Querying
https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor…
2020-06-25T06:10:56.938 INFO:teuthology.task.install.deb:Pulling from
https://4.chacra.ceph.com/r/ceph/mimic/8e9675a830b7e7a7164659a2a8f8ad08b8f6…
2020-06-25T06:10:56.938 INFO:teuthology.task.install.deb:Package
version is 13.2.10-15-g8e9675a830-1bionic
vs. the failure on June 25th:
2020-06-26T07:11:31.659 DEBUG:teuthology.packaging:Querying
https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor…
2020-06-26T07:11:31.820 INFO:teuthology.task.install.deb:Pulling from
https://chacra.ceph.com/r/ceph/mimic/5579a94fafbc1f9cc913a0f5d362953a5d9c3a…
2020-06-26T07:11:31.821 INFO:teuthology.task.install.deb:Package
version is 13.2.8-1bionic
This is probably caused by the LRC data loss. Looks like we need to
get new builds of Mimic.
--
Patrick Donnelly, Ph.D.
He / Him / His
Senior Software Engineer
Red Hat Sunnyvale, CA
GPG: 19F28A586F808C2402351B93C3301A3E258DD79D