radosgw multisite unable to initialize site config
Hey all, I've been having some problems with a relatively basic rgw multisite set up. I initially had it working a few weeks ago with a simple config but rebuilt it with ssl enabled for the endpoints and then started having some problems to work through. I eventually had that working but then tore it all down so I could verify the recreation steps but now I'm having problems with the multisite. There are a few different errors.... I set up the master zone without problems, but if I use the rgw module with a token and spec it fails to sync the sync user and the primary zone can't sync back from the secondary zone. A permission denied error for data syncing primary->secondary but secondary-> primary works because it has the credentials. So i used to be able to manually create the user on the secondary zone and get data syncing but the metadata wouldn't sync with the master showing an error message but when I check the mdlog on the primary it looks okay. 2025-09-11T15:15:18.771+0000 7fc29e96c640 1 req 14061042141473454829 0.001000005s failed to read mdlog history: (2) No such file or directory I ended up trying a full clean reset of rgw on both clusters so I deleted the daemons and all related pools. The primary starts up but I still have the problem of the sync user not being created on the secondary and now when I try to create a user on the secondary for syncing or just for the dashboard to work I get errors that I've tracked down to https://github.com/ceph/ceph/blob/62bcf65e8c0995783bb3e368909716346874ad62/s... but am not quite following whats all going on in the source to determine why. # radosgw-admin user create --uid=dashboard --display-name=dashboard --system couldn't init storage provider # radosgw-admin user create --uid=sysuser-asdf --display-name=sysuser-asdf --access-key=xxxxxxxxxxxxx --secret-key=yyyyyyyyyyyyyyy --system --rgw-realm=asdf Unable to initialize site config. Any thoughts? It seems like I may have messed something up in the ceph config database from multiple tear downs and restarts of the multisite setup but I'm not seeing what it could be. ceph 19.2.1 using cephadm on alma 9 Thanks! Kevin
How exactly did you clean up the RGW config? Zitat von Kevin Hrpcek <khrpcek@ucar.edu>:
Hey all,
I've been having some problems with a relatively basic rgw multisite set up. I initially had it working a few weeks ago with a simple config but rebuilt it with ssl enabled for the endpoints and then started having some problems to work through. I eventually had that working but then tore it all down so I could verify the recreation steps but now I'm having problems with the multisite. There are a few different errors....
I set up the master zone without problems, but if I use the rgw module with a token and spec it fails to sync the sync user and the primary zone can't sync back from the secondary zone. A permission denied error for data syncing primary->secondary but secondary-> primary works because it has the credentials.
So i used to be able to manually create the user on the secondary zone and get data syncing but the metadata wouldn't sync with the master showing an error message but when I check the mdlog on the primary it looks okay.
2025-09-11T15:15:18.771+0000 7fc29e96c640 1 req 14061042141473454829 0.001000005s failed to read mdlog history: (2) No such file or directory
I ended up trying a full clean reset of rgw on both clusters so I deleted the daemons and all related pools. The primary starts up but I still have the problem of the sync user not being created on the secondary and now when I try to create a user on the secondary for syncing or just for the dashboard to work I get errors that I've tracked down to https://github.com/ceph/ceph/blob/62bcf65e8c0995783bb3e368909716346874ad62/s... but am not quite following whats all going on in the source to determine why.
# radosgw-admin user create --uid=dashboard --display-name=dashboard --system couldn't init storage provider # radosgw-admin user create --uid=sysuser-asdf --display-name=sysuser-asdf --access-key=xxxxxxxxxxxxx --secret-key=yyyyyyyyyyyyyyy --system --rgw-realm=asdf Unable to initialize site config.
Any thoughts? It seems like I may have messed something up in the ceph config database from multiple tear downs and restarts of the multisite setup but I'm not seeing what it could be.
ceph 19.2.1 using cephadm on alma 9
Thanks! Kevin _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
I went through multiple deletions and rebuilds but the steps I usually took were: - ceph orch rm the rgw daemons - remove secondary realm - commit period - Remove primary realm - Remove zongroup on both - Remove realm on both - Delete realm related pools On the last delete when I wanted to ensure I removed everything and attempt a completely fresh start I also deleted the .rgw.root and any default realm pools (that I never used) on both clusters. Though I believe I skipped a couple steps after the realm removal because I thought deleting the .rgw.root pools would give it a fresh start. Kevin On Sun, Sep 14, 2025 at 2:48 AM Eugen Block <eblock@nde.ag> wrote:
How exactly did you clean up the RGW config?
Zitat von Kevin Hrpcek <khrpcek@ucar.edu>:
Hey all,
I've been having some problems with a relatively basic rgw multisite set up. I initially had it working a few weeks ago with a simple config but rebuilt it with ssl enabled for the endpoints and then started having some problems to work through. I eventually had that working but then tore it all down so I could verify the recreation steps but now I'm having problems with the multisite. There are a few different errors....
I set up the master zone without problems, but if I use the rgw module with a token and spec it fails to sync the sync user and the primary zone can't sync back from the secondary zone. A permission denied error for data syncing primary->secondary but secondary-> primary works because it has the credentials.
So i used to be able to manually create the user on the secondary zone and get data syncing but the metadata wouldn't sync with the master showing an error message but when I check the mdlog on the primary it looks okay.
2025-09-11T15:15:18.771+0000 7fc29e96c640 1 req 14061042141473454829 0.001000005s failed to read mdlog history: (2) No such file or directory
I ended up trying a full clean reset of rgw on both clusters so I deleted the daemons and all related pools. The primary starts up but I still have the problem of the sync user not being created on the secondary and now when I try to create a user on the secondary for syncing or just for the dashboard to work I get errors that I've tracked down to
https://github.com/ceph/ceph/blob/62bcf65e8c0995783bb3e368909716346874ad62/s...
but am not quite following whats all going on in the source to determine why.
# radosgw-admin user create --uid=dashboard --display-name=dashboard --system couldn't init storage provider # radosgw-admin user create --uid=sysuser-asdf --display-name=sysuser-asdf --access-key=xxxxxxxxxxxxx --secret-key=yyyyyyyyyyyyyyy --system --rgw-realm=asdf Unable to initialize site config.
Any thoughts? It seems like I may have messed something up in the ceph config database from multiple tear downs and restarts of the multisite setup but I'm not seeing what it could be.
ceph 19.2.1 using cephadm on alma 9
Thanks! Kevin _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Not sure if it can help in your case but the rgw mgr module provides some automation around multisite configuration: https://docs.ceph.com/en/quincy/mgr/rgw/ On Mon, Sep 15, 2025 at 4:45 PM Kevin Hrpcek <khrpcek@ucar.edu> wrote:
I went through multiple deletions and rebuilds but the steps I usually took were: - ceph orch rm the rgw daemons - remove secondary realm - commit period - Remove primary realm - Remove zongroup on both - Remove realm on both - Delete realm related pools
On the last delete when I wanted to ensure I removed everything and attempt a completely fresh start I also deleted the .rgw.root and any default realm pools (that I never used) on both clusters. Though I believe I skipped a couple steps after the realm removal because I thought deleting the .rgw.root pools would give it a fresh start.
Kevin
On Sun, Sep 14, 2025 at 2:48 AM Eugen Block <eblock@nde.ag> wrote:
How exactly did you clean up the RGW config?
Zitat von Kevin Hrpcek <khrpcek@ucar.edu>:
Hey all,
I've been having some problems with a relatively basic rgw multisite set up. I initially had it working a few weeks ago with a simple config but rebuilt it with ssl enabled for the endpoints and then started having some problems to work through. I eventually had that working but then tore it all down so I could verify the recreation steps but now I'm having problems with the multisite. There are a few different errors....
I set up the master zone without problems, but if I use the rgw module with a token and spec it fails to sync the sync user and the primary zone can't sync back from the secondary zone. A permission denied error for data syncing primary->secondary but secondary-> primary works because it has the credentials.
So i used to be able to manually create the user on the secondary zone and get data syncing but the metadata wouldn't sync with the master showing an error message but when I check the mdlog on the primary it looks okay.
2025-09-11T15:15:18.771+0000 7fc29e96c640 1 req 14061042141473454829 0.001000005s failed to read mdlog history: (2) No such file or directory
I ended up trying a full clean reset of rgw on both clusters so I deleted the daemons and all related pools. The primary starts up but I still have the problem of the sync user not being created on the secondary and now when I try to create a user on the secondary for syncing or just for the dashboard to work I get errors that I've tracked down to
https://github.com/ceph/ceph/blob/62bcf65e8c0995783bb3e368909716346874ad62/s...
but am not quite following whats all going on in the source to determine why.
# radosgw-admin user create --uid=dashboard --display-name=dashboard --system couldn't init storage provider # radosgw-admin user create --uid=sysuser-asdf --display-name=sysuser-asdf --access-key=xxxxxxxxxxxxx --secret-key=yyyyyyyyyyyyyyy --system --rgw-realm=asdf Unable to initialize site config.
Any thoughts? It seems like I may have messed something up in the ceph config database from multiple tear downs and restarts of the multisite setup but I'm not seeing what it could be.
ceph 19.2.1 using cephadm on alma 9
Thanks! Kevin _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi, For the secondary site sync user you can edit the zone configuration on both sites and add the credentials for the synchronisation system user , as follow: radosgw-admin zone modify --rgw-zone=<zone-name> --access-key=<access-key> --secret=<secret-key> radosgw-admin period update --commit Regards On 12.09.2025 01:03, Kevin Hrpcek wrote:
Hey all,
I've been having some problems with a relatively basic rgw multisite set up. I initially had it working a few weeks ago with a simple config but rebuilt it with ssl enabled for the endpoints and then started having some problems to work through. I eventually had that working but then tore it all down so I could verify the recreation steps but now I'm having problems with the multisite. There are a few different errors....
I set up the master zone without problems, but if I use the rgw module with a token and spec it fails to sync the sync user and the primary zone can't sync back from the secondary zone. A permission denied error for data syncing primary->secondary but secondary-> primary works because it has the credentials.
So i used to be able to manually create the user on the secondary zone and get data syncing but the metadata wouldn't sync with the master showing an error message but when I check the mdlog on the primary it looks okay.
2025-09-11T15:15:18.771+0000 7fc29e96c640 1 req 14061042141473454829 0.001000005s failed to read mdlog history: (2) No such file or directory
I ended up trying a full clean reset of rgw on both clusters so I deleted the daemons and all related pools. The primary starts up but I still have the problem of the sync user not being created on the secondary and now when I try to create a user on the secondary for syncing or just for the dashboard to work I get errors that I've tracked down to https://github.com/ceph/ceph/blob/62bcf65e8c0995783bb3e368909716346874ad62/s... but am not quite following whats all going on in the source to determine why.
# radosgw-admin user create --uid=dashboard --display-name=dashboard --system couldn't init storage provider # radosgw-admin user create --uid=sysuser-asdf --display-name=sysuser-asdf --access-key=xxxxxxxxxxxxx --secret-key=yyyyyyyyyyyyyyy --system --rgw-realm=asdf Unable to initialize site config.
Any thoughts? It seems like I may have messed something up in the ceph config database from multiple tear downs and restarts of the multisite setup but I'm not seeing what it could be.
ceph 19.2.1 using cephadm on alma 9
Thanks! Kevin _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
@Redouane Kachach I've been largely using the rgw module and it was originally working for a non ssl setup but now that is what is actually failing. A very simple multisite following the instructions fails to correctly initialize because the secondary cluster isn't correctly creating it's synchronization user on the secondary cluster. @wissem mimouna The secondary sync user has needed to be manually created as its not syncing from the primary zone when the secondary realm/zone is created with the rgw module. Once I create it then I can get data syncing to run but the metadata syncing is failing with the mdlog error. Which is what I'm assuming is stopping the secondary sync user from being created in the first place. Much appreciated, Kevin On Mon, Sep 15, 2025 at 9:11 AM wissem mimouna <ceph-users@ik.me> wrote:
Hi,
For the secondary site sync user you can edit the zone configuration on both sites and add the credentials for the synchronisation system user , as follow:
radosgw-admin zone modify --rgw-zone=<zone-name> --access-key=<access-key> --secret=<secret-key>
radosgw-admin period update --commit
Regards
Hey all,
I've been having some problems with a relatively basic rgw multisite set up. I initially had it working a few weeks ago with a simple config but rebuilt it with ssl enabled for the endpoints and then started having some problems to work through. I eventually had that working but then tore it all down so I could verify the recreation steps but now I'm having
with the multisite. There are a few different errors....
I set up the master zone without problems, but if I use the rgw module with a token and spec it fails to sync the sync user and the primary zone can't sync back from the secondary zone. A permission denied error for data syncing primary->secondary but secondary-> primary works because it has
On 12.09.2025 01:03, Kevin Hrpcek wrote: problems the
credentials.
So i used to be able to manually create the user on the secondary zone and get data syncing but the metadata wouldn't sync with the master showing an error message but when I check the mdlog on the primary it looks okay.
2025-09-11T15:15:18.771+0000 7fc29e96c640 1 req 14061042141473454829 0.001000005s failed to read mdlog history: (2) No such file or directory
I ended up trying a full clean reset of rgw on both clusters so I deleted the daemons and all related pools. The primary starts up but I still have the problem of the sync user not being created on the secondary and now when I try to create a user on the secondary for syncing or just for the dashboard to work I get errors that I've tracked down to
https://github.com/ceph/ceph/blob/62bcf65e8c0995783bb3e368909716346874ad62/s...
but am not quite following whats all going on in the source to determine why.
# radosgw-admin user create --uid=dashboard --display-name=dashboard --system couldn't init storage provider # radosgw-admin user create --uid=sysuser-asdf --display-name=sysuser-asdf --access-key=xxxxxxxxxxxxx --secret-key=yyyyyyyyyyyyyyy --system --rgw-realm=asdf Unable to initialize site config.
Any thoughts? It seems like I may have messed something up in the ceph config database from multiple tear downs and restarts of the multisite setup but I'm not seeing what it could be.
ceph 19.2.1 using cephadm on alma 9
Thanks! Kevin _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ohhh I see! Initial support was meant for http only deployment, so not sure how it would behave in case of RGW + SSL The command "rgw realm bootstrap" also supports passing the spec by using the "-i" flag so you might try passing an RGW spec which has SSL enabled and see if that works... nothing that I have tested but maybe it can help. On Mon, Sep 15, 2025 at 5:19 PM Kevin Hrpcek <khrpcek@ucar.edu> wrote:
@Redouane Kachach I've been largely using the rgw module and it was originally working for a non ssl setup but now that is what is actually failing. A very simple multisite following the instructions fails to correctly initialize because the secondary cluster isn't correctly creating it's synchronization user on the secondary cluster.
@wissem mimouna The secondary sync user has needed to be manually created as its not syncing from the primary zone when the secondary realm/zone is created with the rgw module. Once I create it then I can get data syncing to run but the metadata syncing is failing with the mdlog error. Which is what I'm assuming is stopping the secondary sync user from being created in the first place. Much appreciated, Kevin
On Mon, Sep 15, 2025 at 9:11 AM wissem mimouna <ceph-users@ik.me> wrote:
Hi,
For the secondary site sync user you can edit the zone configuration on both sites and add the credentials for the synchronisation system user , as follow:
radosgw-admin zone modify --rgw-zone=<zone-name> --access-key=<access-key> --secret=<secret-key>
radosgw-admin period update --commit
Regards
Hey all,
I've been having some problems with a relatively basic rgw multisite set up. I initially had it working a few weeks ago with a simple config but rebuilt it with ssl enabled for the endpoints and then started having some problems to work through. I eventually had that working but then tore it all down so I could verify the recreation steps but now I'm having
with the multisite. There are a few different errors....
I set up the master zone without problems, but if I use the rgw module with a token and spec it fails to sync the sync user and the primary zone can't sync back from the secondary zone. A permission denied error for data syncing primary->secondary but secondary-> primary works because it has
credentials.
So i used to be able to manually create the user on the secondary zone and get data syncing but the metadata wouldn't sync with the master showing an error message but when I check the mdlog on the primary it looks okay.
2025-09-11T15:15:18.771+0000 7fc29e96c640 1 req 14061042141473454829 0.001000005s failed to read mdlog history: (2) No such file or
On 12.09.2025 01:03, Kevin Hrpcek wrote: problems the directory
I ended up trying a full clean reset of rgw on both clusters so I
the daemons and all related pools. The primary starts up but I still have the problem of the sync user not being created on the secondary and now when I try to create a user on the secondary for syncing or just for
deleted the
dashboard to work I get errors that I've tracked down to
https://github.com/ceph/ceph/blob/62bcf65e8c0995783bb3e368909716346874ad62/s...
but am not quite following whats all going on in the source to determine why.
# radosgw-admin user create --uid=dashboard --display-name=dashboard --system couldn't init storage provider # radosgw-admin user create --uid=sysuser-asdf --display-name=sysuser-asdf --access-key=xxxxxxxxxxxxx --secret-key=yyyyyyyyyyyyyyy --system --rgw-realm=asdf Unable to initialize site config.
Any thoughts? It seems like I may have messed something up in the ceph config database from multiple tear downs and restarts of the multisite setup but I'm not seeing what it could be.
ceph 19.2.1 using cephadm on alma 9
Thanks! Kevin _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
I was trying to get the rgw module + ssl working with a spec file but never seemed to get it right. My current attempts have been to use the rgw module to set up the multisite then deploy a new set of rgws using ceph orch apply -i spec.yaml that is set up for ssl. Unfortunately whatever problem I'm currently running into with the radosgw-admin command and basic multisite setup is keeping me from making any other progress. Right now I can't as much as list users one the secondary cluster: # rg user list Unable to initialize site config. Kevin On Mon, Sep 15, 2025 at 9:30 AM Redouane Kachach <rkachach@redhat.com> wrote:
ohhh I see! Initial support was meant for http only deployment, so not sure how it would behave in case of RGW + SSL
The command "rgw realm bootstrap" also supports passing the spec by using the "-i" flag so you might try passing an RGW spec which has SSL enabled and see if that works... nothing that I have tested but maybe it can help.
On Mon, Sep 15, 2025 at 5:19 PM Kevin Hrpcek <khrpcek@ucar.edu> wrote:
@Redouane Kachach I've been largely using the rgw module and it was originally working for a non ssl setup but now that is what is actually failing. A very simple multisite following the instructions fails to correctly initialize because the secondary cluster isn't correctly creating it's synchronization user on the secondary cluster.
@wissem mimouna The secondary sync user has needed to be manually created as its not syncing from the primary zone when the secondary realm/zone is created with the rgw module. Once I create it then I can get data syncing to run but the metadata syncing is failing with the mdlog error. Which is what I'm assuming is stopping the secondary sync user from being created in the first place. Much appreciated, Kevin
On Mon, Sep 15, 2025 at 9:11 AM wissem mimouna <ceph-users@ik.me> wrote:
Hi,
For the secondary site sync user you can edit the zone configuration on both sites and add the credentials for the synchronisation system user , as follow:
radosgw-admin zone modify --rgw-zone=<zone-name> --access-key=<access-key> --secret=<secret-key>
radosgw-admin period update --commit
Regards
Hey all,
I've been having some problems with a relatively basic rgw multisite set up. I initially had it working a few weeks ago with a simple config but rebuilt it with ssl enabled for the endpoints and then started having some problems to work through. I eventually had that working but then tore it all down so I could verify the recreation steps but now I'm having
with the multisite. There are a few different errors....
I set up the master zone without problems, but if I use the rgw module with a token and spec it fails to sync the sync user and the primary zone can't sync back from the secondary zone. A permission denied error for data syncing primary->secondary but secondary-> primary works because it has
credentials.
So i used to be able to manually create the user on the secondary zone and get data syncing but the metadata wouldn't sync with the master showing an error message but when I check the mdlog on the primary it looks okay.
2025-09-11T15:15:18.771+0000 7fc29e96c640 1 req 14061042141473454829 0.001000005s failed to read mdlog history: (2) No such file or
On 12.09.2025 01:03, Kevin Hrpcek wrote: problems the directory
I ended up trying a full clean reset of rgw on both clusters so I
the daemons and all related pools. The primary starts up but I still have the problem of the sync user not being created on the secondary and now when I try to create a user on the secondary for syncing or just for
deleted the
dashboard to work I get errors that I've tracked down to
https://github.com/ceph/ceph/blob/62bcf65e8c0995783bb3e368909716346874ad62/s...
but am not quite following whats all going on in the source to determine why.
# radosgw-admin user create --uid=dashboard --display-name=dashboard --system couldn't init storage provider # radosgw-admin user create --uid=sysuser-asdf --display-name=sysuser-asdf --access-key=xxxxxxxxxxxxx --secret-key=yyyyyyyyyyyyyyy --system --rgw-realm=asdf Unable to initialize site config.
Any thoughts? It seems like I may have messed something up in the ceph config database from multiple tear downs and restarts of the multisite setup but I'm not seeing what it could be.
ceph 19.2.1 using cephadm on alma 9
Thanks! Kevin _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
When using the rgw module you can pass the -i with your spec directly (internally it will call cephadm to pass the spec). You can use either custom certificates embedded in the spec or set *ssl: true *and *generate_cert: true* if you want to use cephadm self-signed certificates. In that case you would need this fix https://github.com/ceph/ceph/pull/63162 which wasn't not ported to Tentacle yet. On Mon, Sep 15, 2025 at 8:28 PM Kevin Hrpcek <khrpcek@ucar.edu> wrote:
I was trying to get the rgw module + ssl working with a spec file but never seemed to get it right. My current attempts have been to use the rgw module to set up the multisite then deploy a new set of rgws using ceph orch apply -i spec.yaml that is set up for ssl. Unfortunately whatever problem I'm currently running into with the radosgw-admin command and basic multisite setup is keeping me from making any other progress. Right now I can't as much as list users one the secondary cluster: # rg user list Unable to initialize site config.
Kevin
On Mon, Sep 15, 2025 at 9:30 AM Redouane Kachach <rkachach@redhat.com> wrote:
ohhh I see! Initial support was meant for http only deployment, so not sure how it would behave in case of RGW + SSL
The command "rgw realm bootstrap" also supports passing the spec by using the "-i" flag so you might try passing an RGW spec which has SSL enabled and see if that works... nothing that I have tested but maybe it can help.
On Mon, Sep 15, 2025 at 5:19 PM Kevin Hrpcek <khrpcek@ucar.edu> wrote:
@Redouane Kachach I've been largely using the rgw module and it was originally working for a non ssl setup but now that is what is actually failing. A very simple multisite following the instructions fails to correctly initialize because the secondary cluster isn't correctly creating it's synchronization user on the secondary cluster.
@wissem mimouna The secondary sync user has needed to be manually created as its not syncing from the primary zone when the secondary realm/zone is created with the rgw module. Once I create it then I can get data syncing to run but the metadata syncing is failing with the mdlog error. Which is what I'm assuming is stopping the secondary sync user from being created in the first place. Much appreciated, Kevin
On Mon, Sep 15, 2025 at 9:11 AM wissem mimouna <ceph-users@ik.me> wrote:
Hi,
For the secondary site sync user you can edit the zone configuration on both sites and add the credentials for the synchronisation system user , as follow:
radosgw-admin zone modify --rgw-zone=<zone-name> --access-key=<access-key> --secret=<secret-key>
radosgw-admin period update --commit
Regards
Hey all,
I've been having some problems with a relatively basic rgw multisite set up. I initially had it working a few weeks ago with a simple config but rebuilt it with ssl enabled for the endpoints and then started having some problems to work through. I eventually had that working but then tore it all down so I could verify the recreation steps but now I'm having
with the multisite. There are a few different errors....
I set up the master zone without problems, but if I use the rgw module with a token and spec it fails to sync the sync user and the primary zone can't sync back from the secondary zone. A permission denied error for data syncing primary->secondary but secondary-> primary works because it has
credentials.
So i used to be able to manually create the user on the secondary zone and get data syncing but the metadata wouldn't sync with the master showing an error message but when I check the mdlog on the primary it looks okay.
2025-09-11T15:15:18.771+0000 7fc29e96c640 1 req 14061042141473454829 0.001000005s failed to read mdlog history: (2) No such file or
On 12.09.2025 01:03, Kevin Hrpcek wrote: problems the directory
I ended up trying a full clean reset of rgw on both clusters so I
the daemons and all related pools. The primary starts up but I still have the problem of the sync user not being created on the secondary and now when I try to create a user on the secondary for syncing or just for
deleted the
dashboard to work I get errors that I've tracked down to
https://github.com/ceph/ceph/blob/62bcf65e8c0995783bb3e368909716346874ad62/s...
but am not quite following whats all going on in the source to determine why.
# radosgw-admin user create --uid=dashboard --display-name=dashboard --system couldn't init storage provider # radosgw-admin user create --uid=sysuser-asdf --display-name=sysuser-asdf --access-key=xxxxxxxxxxxxx --secret-key=yyyyyyyyyyyyyyy --system --rgw-realm=asdf Unable to initialize site config.
Any thoughts? It seems like I may have messed something up in the ceph config database from multiple tear downs and restarts of the multisite setup but I'm not seeing what it could be.
ceph 19.2.1 using cephadm on alma 9
Thanks! Kevin _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (4)
-
Eugen Block
-
Kevin Hrpcek
-
Redouane Kachach
-
wissem mimouna