We're working on the migration to cephadm, but it requires some prerequisites that still needs planing. root@host:~# cat /etc/ceph/ceph.conf ; ceph config dump [global] fsid = ... mon_host = [OLD_NETWORK::10], [OLD_NETWORK::11], [OLD_NETWORK::12] #public_network = OLD_NETWORK::/64, NEW_NETWORK::/64 ms_bind_ipv6 = true ms_bind_ipv4 = false auth_cluster_required = none auth_service_required = none auth_client_required = none [client....] ms_mon_client_mode = crc #debug_rgw = 20 rgw_frontends = beast endpoint=[OLD_NETWORK::12]:7480 rgw_region = ... rgw_zone = ... rgw_thread_pool_size = 512 rgw_dns_name = ... rgw_dns_s3website_name = ... [mon....-new] public_addr = fNEW_NETWORK::12 public_bind_addr = NEW_NETWORK::12 WHO MASK LEVEL OPTION VALUE RO global advanced auth_client_required none * global advanced auth_cluster_required none * global advanced auth_service_required none * global advanced mon_allow_pool_size_one true global advanced ms_bind_ipv4 false global advanced ms_bind_ipv6 true global advanced osd_pool_default_pg_autoscale_mode warn global advanced public_network OLD_NETWORK::/64, NEW_NETWORK::/64 * mon advanced auth_allow_insecure_global_id_reclaim false mon advanced mon_allow_pool_delete false mgr advanced mgr/balancer/active true mgr advanced mgr/balancer/mode upmap mgr advanced mgr/cephadm/migration_current 5 * mgr advanced mgr/orchestrator/orchestrator cephadm mgr.0cc47a6df14e basic container_image quay.io/ceph/ceph@sha256:09e527353463993f0441ad3e86be98076c89c34552163e558a8c2f9bfb4a35f5 * mgr.0cc47aad8ce8 basic container_image quay.io/ceph/ceph@sha256:09e527353463993f0441ad3e86be98076c89c34552163e558a8c2f9bfb4a35f5 * osd.0 basic osd_mclock_max_capacity_iops_ssd 13295.404086 osd.1 basic osd_mclock_max_capacity_iops_ssd 14952.522452 osd.2 basic osd_mclock_max_capacity_iops_ssd 13584.113025 osd.3 basic osd_mclock_max_capacity_iops_ssd 16421.770356 osd.4 basic osd_mclock_max_capacity_iops_ssd 15209.375302 osd.5 basic osd_mclock_max_capacity_iops_ssd 15333.697366 Am Mo., 21. Aug. 2023 um 14:20 Uhr schrieb Eugen Block <eblock@nde.ag>:
Hi,
I don't have those configs. The cluster is not maintained via cephadm / orchestrator.
I just assumed that with Quincy it already would be managed by cephadm. So what does the ceph.conf currently look like on an OSD host (mask sensitive data)?
Zitat von Boris Behrens <bb@kervyn.de>:
Hey Eugen, I don't have those configs. The cluster is not maintained via cephadm / orchestrator. The ceph.conf does not have IPaddresses configured. A grep in /var/lib/ceph show only binary matches on the mons
I've restarted the whole host, which also did not work.
Am Mo., 21. Aug. 2023 um 13:18 Uhr schrieb Eugen Block <eblock@nde.ag>:
Hi,
there have been a couple of threads wrt network change, simply restarting OSDs is not sufficient. I still haven't had to do it myself, but did you 'ceph orch reconfig osd' after adding the second public network, then restart them? I'm not sure if the orchestrator works as expected here, last year there was a thread [1] with the same intention. Can you check the local ceph.conf file (/var/lib/ceph/<FSID>/<SERVICE>/config) of the OSDs (or start with one) if it contains both public networks? I (still) expect the orchestrator to update that config as well. Maybe it's worth a bug report? If there's more to it than just updating the monmap I would like to see that added to the docs since moving monitors to a different network is already documented [2].
Regards, Eugen
[1] https://www.spinics.net/lists/ceph-users/msg75162.html [2]
https://docs.ceph.com/en/quincy/cephadm/services/mon/#moving-monitors-to-a-d...
Zitat von Boris Behrens <bb@kervyn.de>:
Hi, I need to migrate a storage cluster to a new network.
I added the new network to the ceph config via: ceph config set global public_network "old_network/64, new_network/64" I've added a set of new mon daemons with IP addresses in the new
network
and they are added to the quorum and seem to work as expected.
But when I restart the OSD daemons, the do not bind to the new addresses. I would have expected that the OSDs try to bind to all networks but they are only bound to the old_network.
The idea was to add the new set of network config to the current storage hosts, bind everything to ip addresses in both networks, shift over workload, and then remove the old network. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend im groüen Saal. _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
-- Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend im groüen Saal.