Hi,
However as soon as we issue either of the above tell commands, it just hangs. Furthermore when ceph tell hangs, pg are also becoming stuck in "Activating" and "Peering" states.
It seems to be related, as soon as we stop ceph tell (ctrl-c it), a few minutes later the pgs are peered/active.
We can reproduce this problem also with very busy osds, which have been moved to another host - they also do not react to the ceph tell commands.
Does this also happen when you issue a osd specific "tell", i.e. ceph tell osd.13 injectargs '--osd-max-backfills 4' Does this also happen when you loop over it one by one?
Did anyone see this before and/or do you have a hint on how to debug ceph tell as it is not a daemon on its own?
IIRC I have seen this, but not in combination with PGs peering / activating. Has the config change become effective on alls OSDs: verify with ceph daemon osd.13 config get osd_max_backfills (for all OSDs) Gr. Stefan