are the old multisite conf values still in ceph.conf (eg, rgw_zonegroup, rgw_zone, rgw_realm)? ________________________________ From: Adiga, Anantha <anantha.adiga@intel.com> Sent: 08 May 2023 18:27 To: ceph-users@ceph.io <ceph-users@ceph.io> Subject: [ceph-users] rgw service fails to start with zone not found CAUTION: This email originates from outside THG Hi, An existing multisite configuration was removed. But the radosgw services still see the old zone name and fail to start. journalctl -u ceph-d0a3b6e0-d2c3-11ed-be05-a7a3a1d7a87e@rgw.default.default.fl31ca104ja0201.ninovs ........... May 08 16:10:48 fl31ca104ja0201 bash[3964341]: debug 2023-05-08T16:10:48.897+0000 7f2f47634740 0 deferred set uid:gid to 167:167 (ceph:ceph) May 08 16:10:48 fl31ca104ja0201 bash[3964341]: debug 2023-05-08T16:10:48.897+0000 7f2f47634740 0 ceph version 17.2.6 (d7ff0d10654d2280e08f1ab989> May 08 16:10:48 fl31ca104ja0201 bash[3964341]: debug 2023-05-08T16:10:48.897+0000 7f2f47634740 0 framework: beast May 08 16:10:48 fl31ca104ja0201 bash[3964341]: debug 2023-05-08T16:10:48.897+0000 7f2f47634740 0 framework conf key: port, val: 8080 May 08 16:10:48 fl31ca104ja0201 bash[3964341]: debug 2023-05-08T16:10:48.897+0000 7f2f47634740 1 radosgw_Main not setting numa affinity May 08 16:10:48 fl31ca104ja0201 bash[3964341]: debug 2023-05-08T16:10:48.901+0000 7f2f47634740 1 rgw_d3n: rgw_d3n_l1_local_datacache_enabled=0 May 08 16:10:48 fl31ca104ja0201 bash[3964341]: debug 2023-05-08T16:10:48.901+0000 7f2f47634740 1 D3N datacache enabled: 0 May 08 16:10:48 fl31ca104ja0201 bash[3964341]: debug 2023-05-08T16:10:48.917+0000 7f2f47634740 0 rgw main: ERROR: could not find zone (fl2site2) May 08 16:10:48 fl31ca104ja0201 bash[3964341]: debug 2023-05-08T16:10:48.917+0000 7f2f47634740 0 rgw main: ERROR: failed to start notify service> May 08 16:10:48 fl31ca104ja0201 bash[3964341]: debug 2023-05-08T16:10:48.917+0000 7f2f47634740 0 rgw main: ERROR: failed to init services (ret=(> May 08 16:10:48 fl31ca104ja0201 bash[3964341]: debug 2023-05-08T16:10:48.921+0000 7f2f47634740 -1 Couldn't init storage provider (RADOS) May 08 16:10:49 fl31ca104ja0201 systemd[1]: ceph-d0a3b6e0-d2c3-11ed-be05-a7a3a1d7a87e@rgw.default.default.fl31ca104ja0201.ninovs.service: Main pr> May 08 16:10:49 fl31ca104ja0201 systemd[1]: ceph-d0a3b6e0-d2c3-11ed-be05-a7a3a1d7a87e@rgw.default.default.fl31ca104ja0201.ninovs.service: Failed Here is the current configuration root@fl31ca104ja0201:/# radosgw-admin period get { "id": "729f7cef-6340-4750-b3ae-9164177c0df3", "epoch": 1, "predecessor_uuid": "1d124ad5-57b0-41de-8def-823bd40f72aa", "sync_status": [], "period_map": { "id": "729f7cef-6340-4750-b3ae-9164177c0df3", "zonegroups": [ { "id": "21b8306c-be43-4567-a0c5-74ab69937535", "name": "default", "api_name": "default", "is_master": "true", "endpoints": [], "hostnames": [], "hostnames_s3website": [], "master_zone": "ff468492-8a14-414b-a8cf-7c20ab699af3", "zones": [ { "id": "ff468492-8a14-414b-a8cf-7c20ab699af3", "name": "default", "endpoints": [], "log_meta": "false", "log_data": "false", "bucket_index_max_shards": 11, "read_only": "false", "tier_type": "", "sync_from_all": "true", "sync_from": [], "redirect_zone": "" } ], "placement_targets": [ { "name": "default-placement", "tags": [], "storage_classes": [ "STANDARD" ] } ], "default_placement": "default-placement", "realm_id": "8999e04c-17a4-4150-8845-cecd6672d312", "sync_policy": { "groups": [] } } ], "short_zone_ids": [ { "key": "ff468492-8a14-414b-a8cf-7c20ab699af3", "val": 3883136521 } ] }, "master_zonegroup": "21b8306c-be43-4567-a0c5-74ab69937535", "master_zone": "ff468492-8a14-414b-a8cf-7c20ab699af3", "period_config": { "bucket_quota": { "enabled": false, "check_on_raw": false, "max_size": -1, "max_size_kb": 0, "max_objects": -1 }, "user_quota": { "enabled": false, "check_on_raw": false, "max_size": -1, "max_size_kb": 0, "max_objects": -1 }, "user_ratelimit": { "max_read_ops": 0, "max_write_ops": 0, "max_read_bytes": 0, "max_write_bytes": 0, "enabled": false }, "bucket_ratelimit": { "max_read_ops": 0, "max_write_ops": 0, "max_read_bytes": 0, "max_write_bytes": 0, "enabled": false }, "anonymous_ratelimit": { "max_read_ops": 0, "max_write_ops": 0, "max_read_bytes": 0, "max_write_bytes": 0, "enabled": false } }, "realm_id": "8999e04c-17a4-4150-8845-cecd6672d312", "realm_name": "default", "realm_epoch": 2 } root@fl31ca104ja0201:/# radosgw-admin realm list { "default_info": "8999e04c-17a4-4150-8845-cecd6672d312", "realms": [ "default" ] } root@fl31ca104ja0201:/# radosgw-admin zonegroup list { "default_info": "21b8306c-be43-4567-a0c5-74ab69937535", "zonegroups": [ "default" ] } root@fl31ca104ja0201:/# radosgw-admin zone list { "default_info": "", "zones": [ "default" ] } Where is the radosgw service getting the old zone info from? How to fix this configuration? Thank you, Anantha _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io