we may need a method like CEPH_OSD_OP_FLAG_FADVISE_* to bypass msg2 / bluestore compress.
For objstore, client maybe already compress the data(attribute:
user.rgw.content_encoding). So need recompress on msg layer or bluestore layer.
I very much agree with Sage’s second idea. It like
https://lwn.net/Articles/837816/
(Encode I/O). This can largely reduce network and CPU.