22 Oct
2019
22 Oct
'19
4:50 p.m.
Hi all,
I have 40 nvme drives with about 20G free space each.
Would creating a 10GB partition/lvm on each of the nvmes for an rgw index
pool be a bad idea?
RGW has about about 5 million objects
I don't think space will be an issue but I am worried about the 10G size,
is it just too small for a bluestore OSD?
thx
Frank
23 Oct
23 Oct
3:21 a.m.
I did a set of 30GB OSDs before with extra disk space on my SSDs for the
metadata pool on cephfs and my entire cluster locked up about 3 weeks
later. Some metadata operation was happening, filled some of the 30GB disks
to 100%, and all IO was blocked in the cluster. I did some trickery of
deleting 1 copy of a few PGs on each OSD, such that I still had at least 2
copies of each PG, and was able to backfill the pool back onto my HDDs and
restore cluster functionality. I would say that trying to use that space is
definitely not worth it.
In one of my production clusters I occasionally get a warning state that an
omap object is too large in my buckets.index pool. I could very easily
imagine that stalling the entire cluster if my index pool were on such
small OSDs.
On Tue, Oct 22, 2019, 6:55 PM Frank R <frankaritchie(a)gmail.com> wrote:
Hi all,
I have 40 nvme drives with about 20G free space each.
Would creating a 10GB partition/lvm on each of the nvmes for an rgw index
pool be a bad idea?
RGW has about about 5 million objects
I don't think space will be an issue but I am worried about the 10G size,
is it just too small for a bluestore OSD?
thx
Frank
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
25 Oct
25 Oct
1:53 p.m.
10G should be fine on bluestore the smallest size you can have is about 2GB since LVM
takes up about 1GB of space at that size so at that point it most of the disk is taken up
with LVM. I have seen/recorded performance benefits in some cases when using small OSD
sizes on bluestore instead of larger OSDs. I have a poster outlining the results I have
got that can be found https://github.com/GMW99/CephOnRam
Of course, this will be different for a different setup. However, it is worth looking to
as there are potentially some good performance gains in there.
1656
days inactive
1659
days old
2 comments
3 participants
participants (3)
-
David Turner -
Frank R -
gabryel.mason-williams＠diamond.ac.uk