On 02-04-2024 15:09, Zakhar Kirpichenko wrote:
Hi,
I'm adding a few OSDs to an existing cluster, the cluster is running with `osd noout,noin`:
cluster: id: 3f50555a-ae2a-11eb-a2fc-ffde44714d86 health: HEALTH_WARN noout,noin flag(s) set
Specifically `noin` is documented as "prevents booting OSDs from being marked in". But freshly added OSDs were immediately marked `up` and `in`:
services: ... osd: 96 osds: 96 up (since 5m), 96 in (since 6m); 338 remapped pgs flags noout,noin
# ceph osd tree in | grep -E "osd.11|osd.12|osd.26" 11 hdd 9.38680 osd.11 up 1.00000 1.00000 12 hdd 9.38680 osd.12 up 1.00000 1.00000 26 hdd 9.38680 osd.26 up 1.00000 1.00000
Is this expected behavior? Do I misunderstand the purpose of the `noin` option?
We have "mon_osd_auto_mark_new_in = false" configured for this reason. With this configuration option setting OSDs "IN" becomes a manual operation. If you don't want to have OSDs marked in automatically after they have been marked out you can use this option as well: mon_osd_auto_mark_auto_out_in = false. Gr. Stefan