Anything with an iodepth of 1 is going to be
(relatively) terrible on
RBD.
iodepth=128 results in 300 iops in the same setup. Not much better :)
The forthcoming Octopus release of librbd adds support
for sparse
copy-up writes [1] when your min OSD release is set to Octopus (reads
from the parent image were already sparse-read ops). Using holes was
previously not very practical due to the large allocations sizes on
the OSD, but with the change to 4KiB minimum block sizes, such a
technique would be possible (albeit a breaking change for all older
clients controlled via a new feature bit). You also have the ability
to control the RBD object sizes and use something smaller than the
4MiB default.
That's good news! Is it already in 15.1.1, can I test it?
--
Vitaliy Filippov