[ceph-users] Mixed FileStore and BlueStore OSDs in Nautilus and beyond

Hi, Our large Luminous cluster still has around 2k FileStore OSDs (35% of OSDs). We haven't had any particular need to move these over to BlueStore yet, as the performance is fine for our use case. Obviously, it would be easiest if we could let the FileStore OSDs stay in the cluster until the hardware generation is removed from the cluster in a few years. While reading around for our upgrade to Nautilus, and I stumbled across a thread [1] where Sage mentioned they had some issues getting FileStore OSDs and PG merging to play nicely. While I'm not planning on trying to merge PGs on this cluster any time soon, it raised a few more general questions about the OSD back ends in general. - How much extra risk are we exposed to by running both types of OSDs in the same pools. So far we have not run into any issues due to this, but I get the feeling it is not a well-tested configuration compared to BlueStore only clusters. - Are FileStore OSDs going to continue to be supported indefinitely? I know they have their uses, but if there is an aspiration to reduce support or stop supporting them entirely in the coming releases then we may need to plan around that. Any thoughts, opinions, or pointers to previous discussions about this would be great. Thanks, Tom [1] https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/GRQOYECPFTO…