I've added code to ceph-container.git and ceph-build.git [1] to
automatically build a 'daemon-base' container for each branch with a
name beginning with 'wip-', for the CentOS 7 'default' flavor build.
This is the container that is used with Rook.
Each container image is pushed to quay.io/cephci, an organization
created by me but intended for public consumption. The images are
tagged with the name of the ceph wip branch, the 7-digit SHA1 of the
head commit, and the suffix "centos-7-x86_64-devel", so for instance one
of the tags built today was
wip-sage3-testing-2019-09-10-1000-7295ce6-centos-7-x86_64-devel
so a pull of
"quay.io/cephci/daemon-base:wip-sage3-testing-2019-09-10-1000-7295ce6-centos-7-x86_64-devel"
would fetch that image.
You can see the list of tags currently available at
https://quay.io/repository/cephci/daemon-base?tab=tags, or just remember
to browse to "quay.io/cephci" and dig down.
So far no image reaping mechanism is in place; I expect we'll need one
eventually and we'll define a policy then.
Note, again, that the branch must be named "wip-" to allow this
building; this is a side-effect of existing code in ceph-container. If
that becomes too much of a limitation, we can address that with a later
change.
I'll add a description of this mechanism to the Ceph developer docs soon.
Please let me know of any issues with either the build or the resultant
images and I'll help figure out what's up.
-----
[1] https://github.com/ceph/ceph-container/pull/1457 and
https://github.com/ceph/ceph-build/pull/1378
--
Dan Mick
Red Hat, Inc.
Ceph docs: http://ceph.com/docs
Hi all,
I've just merged https://github.com/ceph/ceph/pull/30191 to master, which
adds assertions in the OSD that the result of any successful write is zero
(or a negate error), and the output buffer is empty. This is preparation
for adding support for >0 and non-empty buffers--but first we should
verify no existing code is returning such data inadvertantly.
If this turns up failures in the rgw, rbd, or cephfs suites, it's probably
because a cls method is returning >0 or data for a write that the OSD was
previously clearing out on its behalf. The fix is to just adjust the cls
code to return 0 and no data.
Thanks!
sage
Hi Folks,
Perf meeting is on in ~20 minutes! Today let's talk about Casey's
propsal to avoid write stalls during bucket splitting, bluestore 4K min
alloc size, and competing proposals for small objects in OMAP from Igor
and Yanghonggang. Please feel free to add any other topics to the
etherpad. See you there!
Etherpad:
https://pad.ceph.com/p/performance_weekly
Bluejeans:
https://bluejeans.com/908675367
Thanks,
Mark
Background: In nautilus, bluestore started maintaining usage stats on a
per-pool basis. BlueStore OSDs created before nautilus lack these stats.
Running a ceph-bluestore-tool repair can calculate the usage so that
the OSD can maintain and report them going forward.
There are two options:
- bluestore_warn_on_legacy_statfs (bool, default: true), which makes the
cluster issue a health warning when there are OSDs that have legacy stats.
- bluestore_no_per_pool_stats_tolerance (enum enforce, until_fsck,
until_repair, default: until_repair).
'until_fsck' will tolerate the legacy but fsck will fail
'until_repair' will tolerate the legacy but fsck will pass
'enforce' will tolerate the legacy but disable the warning
The octopus addition of per-pool omap usage tracking presents an identical
problem: a new tracking ability in bluestore that reqires a conversion to
enable after upgrade.
I think that we can simplify these settings and make them less confusing,
still with two options:
- bluestore_fsck_error_on_no_per_pool_omap (bool, default: false). During
fsck, we can either generate a 'warning' about non-per-pool omap, or an
error. Generate a warning by default, which means that the fsck return
code can indicate success.
- bluestore_warn_on_no_per_pool_omap (bool, default: true). At runtime, we
can generate a health warning if the OSD is using the legacy non-per-pool
omap.
The overall default behavior is the same as we have with the
legacy_statfs: OSDs still work, fsck passes, and we generate a health
warning.
Setting bluestore_warn_on_no_per_pool_omap=false is the same, AFAICS, as
setting bluestore_no_per_pool_stats_tolerance=enforce. (Except maybe
repair won't do the conversion? I don't see why we'd ever not want to
do the conversion, though.)
Setting bluestore_fsck_error_on_no_per_pool_omap=true is the same, AFAICS,
as bluestore_no_per_pool_stats_tolerance=until_fsck.
Overall, this seems simpler and easier for a user to understand.
Realistically, the only option I expect a user will ever change is
bluestore_warn_on_no_per_pool_omap=false to make the health warning go
away after an upgrade.
What do you think? Should I convert the legacy_statfs to behave the same
way?
sage