How to (permanently) disable msgr v1 on Ceph?
Hi, For new clusters one of the first things I do is to disable messenger v1: ceph config set mon ms_bind_msgr1 false However, that is not enough, as a restart of the monitors will leave v1 enabled in the monmap. Why is that? That looks like a bug to me. Therefore I explicitly set the addrs and port of the monitors, i.e.: ceph mon set-addrs stefan-ceph-test01 $ip:3300 Be careful not to make a typo, that will ruin your day, see [1]. This immediately disables v1 and a new monmap is created. All good, however ... When a monitor is removed, say mon1, and added again, the new mon is added with both a v1 and a v2 address. I don't want that, I want to disable legacy v1 permanently. How do I do that? Adding: [mon] ms_bind_msgr1 = false to /etc/ceph/ceph.conf does not help either. When I performed a test this morning on a fresh 18.2.4 cluster (remove a mon, add it back again) to reproduce this issue the v1 was enabled again on two monitors (a third was left with only v2). Repeating the "ceph mon set-addrs" command removes the v1 binding again. But I would like to avoid messing around with this, triggering monmaps / elections, especially when making an error with this brings your cluster to a halt [1]. In what Ceph release will v1 be disabled by default on new clusters? Gr. Stefan [1] https://tracker.ceph.com/issues/54744
Thank you the bug report! Yeah, there was no so much concern about disabling v1 AFAIK.
In what Ceph release will v1 be disabled by default on new clusters?
This a really good question that has many common points with the old discussions on our guarantees toward the compatibility with clients. So far it was mostly about dencoding, feature bits etc, but it applies to the v1 as well. Perhaps we should reiterate on CDM (or CSC at least). CCing Ilya. Regards, Radek
Just a supplement. I'm aware there are different stances on the client compatibility ranging from perpetual to N-3. Yet, being "compatible by default" is somewhat different from having "optional compatibility" (by opt-in). On Fri, Mar 14, 2025 at 11:34 AM Radoslaw Zarzynski <rzarzyns@redhat.com> wrote:
Thank you the bug report! Yeah, there was no so much concern about disabling v1 AFAIK.
In what Ceph release will v1 be disabled by default on new clusters?
This a really good question that has many common points with the old discussions on our guarantees toward the compatibility with clients. So far it was mostly about dencoding, feature bits etc, but it applies to the v1 as well.
Perhaps we should reiterate on CDM (or CSC at least).
CCing Ilya.
Regards, Radek
participants (2)
-
Radoslaw Zarzynski
-
Stefan Kooman