On Wed, Sep 2, 2020 at 12:41 AM Szabo, Istvan (Agoda)
<Istvan.Szabo(a)agoda.com> wrote:
Hi,
Could someone help me what is the issue with our dployment steps please?
Initial RGW Cluster_1
===================================================================================ADD_RGW_TO_CLUSTERCreate
Default Realm
- sudo radosgw-admin realm create --rgw-realm=default --defaultCreate Default Zone Group
- sudo radosgw-admin zonegroup create --rgw-zonegroup=default --master --defaultCreate
Default Zone
- sudo radosgw-admin zone create --rgw-zonegroup=default --rgw-zone=default --master
--defaultUpdate Period
- sudo radosgw-admin period update --rgw-realm=default --commitCreate RGW on 3 OSD_Node
- sudo ceph orch apply rgw default default --placement=“3 bk-otpsmon-1001 bk-otpsmon-1002
bk-otpsmon-1003”
===================================================================================Initial
RGW Cluster_2
- sudo ceph orch apply rgw default default --placement=“3 hy-otpsmon-2001 hy-otpsmon-2002
bk-otpsmon-2003"
HOW_TO_CONFIGURE_CEPH_OCTOPUS_MULTISITE
===================================================================================================
configure multisite
radosgw-admin realm create \
--rgw-realm=agoda \
--default
radosgw-admin zonegroup create \
--rgw-zonegroup=data \
--endpoints=http://bk-otpsmon-1001:80,http://bk-otpsmon-1002:80,http://bk-otpsmon-1003:80
\
--rgw-realm=agoda \
--master \
--default
***** BUG Run Command-line => Can't send shot command-line *****
radosgw-admin zone create --rgw-zonegroup=data --rgw-zone=bk --master --default
--endpoints=http://bk-otpsmon-1001:80,http://bk-otpsmon-1002:80,http://bk-otpsmon-1003:80
***** BUG Can't Delete Pool *****
***** ceph config set mon mon_allow_pool_delete true
***** ceph tell mon.\* injectargs '--mon-allow-pool-delete=true'
***** (optional) systemctl |grep -i ceph
***** (optional) systemctl restart
ceph-85b0c358-e2a2-11ea-8864-000c29fa922a(a)mon.bk-otpsmon-1001.service
# radosgw-admin zonegroup remove --rgw-zonegroup=default --rgw-zone=default
# radosgw-admin period update --commit
# radosgw-admin zone delete --rgw-zone=default
# radosgw-admin period update --commit
# radosgw-admin zonegroup delete --rgw-zonegroup=default
# radosgw-admin period update --commit
# ceph osd pool rm default.rgw.meta default.rgw.meta --yes-i-really-really-mean-it
# ceph osd pool rm default.rgw.control default.rgw.control --yes-i-really-really-mean-it
# ceph osd pool rm default.rgw.log default.rgw.log --yes-i-really-really-mean-it
radosgw-admin user create --uid="ceph-sync"
--display-name="ceph-sync" --system
"user": "ceph-sync",
"access_key": "VWV6566957QGVDV6ITJM",
"secret_key": "qAEGXslUHBeWv7O6VMCmdo0z2AgMyBZlcKqg38H7"
radosgw-admin zone modify \
--rgw-zone=bk \
--access-key=VWV6566957QGVDV6ITJM \
--secret=qAEGXslUHBeWv7O6VMCmdo0z2AgMyBZlcKqg38H7
radosgw-admin period update --rgw-realm=agoda --commit
Configure Rados Gateway Client at RGW Nodes
[client.rgw.bk-otpsmon-1001]
host = bk-otpsmon-1001
rgw frontends = "civetweb port=80"
rgw_zone=bk
[client.rgw.bk-otpsmon-1002]
host = bk-otpsmon-1002
rgw frontends = "civetweb port=80"
rgw_zone=bk
[client.rgw.bk-otpsmon-1003]
host = bk-otpsmon-1003
rgw frontends = "civetweb port=80"
rgw_zone=bk
Restart All RGW Container Node
rgw_1: systemctl restart
ceph-89c72a6c-eb95-11ea-b88b-000c29147836(a)rgw.default.default.bk-otpsmon-1001.atdfmv.service
rgw_2: systemctl restart
ceph-89c72a6c-eb95-11ea-b88b-000c29147836(a)rgw.default.default.bk-otpsmon-1002.wkieqj.service
rgw_3: systemctl restart
ceph-89c72a6c-eb95-11ea-b88b-000c29147836(a)rgw.default.default.bk-otpsmon-1003.jpyzdq.service
Enable All RGW Container Node
rgw_1: systemctl enable
ceph-89c72a6c-eb95-11ea-b88b-000c29147836(a)rgw.default.default.bk-otpsmon-1001.atdfmv.service
rgw_2: systemctl enable
ceph-89c72a6c-eb95-11ea-b88b-000c29147836(a)rgw.default.default.bk-otpsmon-1002.wkieqj.service
rgw_3: systemctl enable
ceph-89c72a6c-eb95-11ea-b88b-000c29147836(a)rgw.default.default.bk-otpsmon-1003.jpyzdq.service
=====================================
Secondary Zones
radosgw-admin realm pull --url={url-to-master-zone-gateway} --access-key={access-key}
--secret={secret}
radosgw-admin realm pull --url=http://bk-otpsmon-1001:80
--access-key=CL7NF0DLYL7D2YYVR9HA --secret=C49mBXDNgHl9fNwibdaQamvffB9QM2RNj5snUq03
radosgw-admin realm pull --url=http://bk-otpsmon-1001:80
--access-key=CL7NF0DLYL7D2YYVR9HA --secret=C49mBXDNgHl9fNwibdaQamvffB9QM2RNj5snUq03
--rgw-realm=agoda
I'm not sure where these keys came from. They should be the ones
associated with the ceph-sync user created above.
And here is the errors:
Error From Cluster_2 After Realm Pull[root@hy-otpsmon-2001 ~]# radosgw-admin realm pull
--url=http://bk-otpsmon-1001:80<http://bk-otpsmon-1001/>
--access-key=CL7NF0DLYL7D2YYVR9HA --secret=C49mBXDNgHl9fNwibdaQamvffB9QM2RNj5snUq03
--rgw-realm=agoda
request failed: (13) Permission denied
If the realm has been changed on the master zone, the master zone's gateway may need
to be restarted to recognize this user.
[root@hy-otpsmon-2001 ~]#
Error From Cluster_1 RGW Container LogSep 02 11:20:52 bk-otpsmon-1001 bash[1246]: debug
2020-09-02T04:20:52.963+0000 7f3b0747f700 1 ====== starting new request
req=0x7f3b455118a0 =====
Sep 02 11:20:52 bk-otpsmon-1001 bash[1246]: debug 2020-09-02T04:20:52.967+0000
7f3b0747f700 1 op->ERRORHANDLER: err_no=-2028 new_err_no=-2028
Sep 02 11:20:52 bk-otpsmon-1001 bash[1246]: debug 2020-09-02T04:20:52.967+0000
7f3b0747f700 1 ====== req done req=0x7f3b455118a0 op status=0 http_status=403
latency=0.002999559s ======
All help appreciated.
________________________________
This message is confidential and is for the sole use of the intended recipient(s). It may
also be privileged or otherwise protected by copyright or other legal rules. If you have
received it by mistake please let us know by reply email and delete it from your system.
It is prohibited to copy this message or disclose its content to anyone. Any
confidentiality or privilege is not waived or lost by any mistaken delivery or
unauthorized disclosure of the message. All messages sent to and from Agoda may be
monitored to ensure compliance with company policies, to protect the company's
interests and to remove potential malware. Electronic messages may be intercepted,
amended, lost or deleted, or contain viruses.
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io