manually configure radosgw
I am trying to manually create a radosgw instance for a small development installation. I was able to muddle through and get a working mon, mgr, and osd (x2), but the docs for radosgw are based on ceph-deploy which is not part of the octopus release. The host systems are all lxc/lxd containers with centos 7 and the http://download.ceph.com/rpm-octopus/el7/$basearch repos. The radosgw setip that I did was essentially creating a keyring following the steps in https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/1.2.3/htm... as far as I could, but I have to admit that even after getting mon, mgr, and a couple of osds working I still don't grok the keyring usage... is there a more conceptual doc that explains how they are used and what has to be on each host? ** start rgw0 (systemctl start ceph-radosgw@rgw0) ** The failure on rgw0 host (journalctl -f -u ceph-radosgw@rgw0.service) Apr 27 17:00:20 rgw0 systemd[1]: Started Ceph rados gateway. Apr 27 17:00:20 rgw0 radosgw[8668]: 2020-04-27T17:00:20.791+0000 7f330173d700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2] Apr 27 17:00:20 rgw0 radosgw[8668]: failed to fetch mon config (--no-mon-config to skip) Apr 27 17:00:20 rgw0 systemd[1]: ceph-radosgw@rgw0.service: main process exited, code=exited, status=1/FAILURE [root@cephadm ~]# ceph mon dump dumped monmap epoch 2 epoch 2 fsid f286f47e-7043-4dca-aeb0-03fda1910378 last_changed 2020-04-20T16:23:49.311252+0000 created 2020-04-20T16:18:21.389527+0000 min_mon_release 15 (octopus) 0: [v2:10.121.149.102:3300/0,v1:10.121.149.102:6789/0] mon.mon0 ** on mon0 I also see a lot of this in the logs: 2020-04-22T00:29:41.293+0000 7f35cb1da700 0 cephx server client.rgw0: handle_request failed to decode CephXAuthenticate: buffer::end_of_buffer Any help would be appreciated. -- Patrick Dowler Canadian Astronomy Data Centre Victoria, BC, Canada
CAn you please display your keyring you use in the radosgw containers and also the ceph config? Seems like authentication issue or your containers don't pick up your ceph config?
This is out dated but will get you throuh it (especially the pools and civetweb) yum install ceph-radosgw ceph osd pool create default.rgw 8 ceph osd pool create default.rgw.meta 8 ceph osd pool create default.rgw.control 8 ceph osd pool create default.rgw.log 8 ceph osd pool create .rgw.root 8 ceph osd pool create .rgw.gc 8 ceph osd pool create .rgw.buckets 16 ceph osd pool create .rgw.buckets.index 8 ceph osd pool create .rgw.buckets.extra 8 ceph osd pool create .intent-log 8 ceph osd pool create .usage 8 ceph osd pool create .users 8 ceph osd pool create .users.email 8 ceph osd pool create .users.swift 8 ceph osd pool create .users.uid 8 ceph auth get-or-create client.rgw1 ceph auth caps client.rgw1 osd 'allow rwx' mon 'allow rwx' (umask 027 ; sudo -u ceph mkdir -p /var/lib/ceph/radosgw/ceph-rgw1) (umask 077; ceph auth export client.rgw1 2>/dev/null | head -2 | sudo -u ceph tee /var/lib/ceph/radosgw/ceph-rgw1/keyring ) service ceph-radosgw@rgw1 start systemctl enable ceph-radosgw@rgw1 [client.rgw1] rgw_frontends = civetweb port=80+7480s ssl_certificate=/etc/ceph/cert.pem
On Mon, Apr 27, 2020 at 11:21 AM Patrick Dowler <pdowler.cadc@gmail.com> wrote:
I am trying to manually create a radosgw instance for a small development installation. I was able to muddle through and get a working mon, mgr, and osd (x2), but the docs for radosgw are based on ceph-deploy which is not part of the octopus release.
Here are the steps I did two weeks ago on CentOS 7 for Nautilus. I have not tried Octopus manually yet. # Install RGW: yum -y install ceph-radosgw mkdir -p /var/lib/ceph/radosgw/ceph-rgw.ceph1 # My /etc/ceph/ceph.conf file: ------- [global] fsid = cbc2b0e2-ed16-4f09-8578-f8c962b1a0ef mon initial members = ceph1 mon host = 192.168.2.101 public network = 192.168.2.0/24 cluster network = 192.168.2.0/24 [client.rgw.ceph1] host = ceph1 rgw frontends = "civetweb port=80" # Uncomment this to write RGW logs: # log file = /var/log/ceph/ceph-rgw.log # Note: set up wildcard DNS for "*.ceph.example.com" this to work: rgw resolve cname = true rgw dns name = ceph.example.com --------- # Create RGW keyring: ceph auth get-or-create client.rgw.ceph1 osd 'allow rwx' mon 'allow rw' -o /var/lib/ceph/radosgw/ceph-rgw.ceph1/keyring # Ensure permissions are correct: chown -R ceph:ceph /var/lib/ceph/radosgw/ # Configure RGW as "done": sudo -u ceph touch /var/lib/ceph/radosgw/ceph-rgw.ceph1/done # Start the service: systemctl enable --now ceph-radosgw@rgw.ceph1 # Create a RGW user: radosgw-admin user create --uid=kdreyer --display-name="Ken Dreyer" --email=kdreyer@example.com ------ Then, on the s3 client: # Write our config file: s3cmd --no-ssl --access_key=ASDF123456 --secret_key=FOOBAR123 --host=ceph.example.com --host-bucket="%(bucket)s.ceph.example.com" --dump-config > $HOME/.s3cfg # Make a "kdreyer" bucket s3cmd mb s3://kdreyer # Make the whole bucket public: s3cmd setacl s3://kdreyer/ --acl-public # Upload a test file: echo "Hello World" > /tmp/hello.txt s3cmd put /tmp/hello.txt s3://kdreyer # Fetch file: curl http://kdreyer.ceph.example.com/hello.txt
Marc and Ken, Thanks for the tips! I will try to work through this asap. -- Patrick Dowler Canadian Astronomy Data Centre Victoria, BC, Canada On Tue, 28 Apr 2020 at 16:04, Ken Dreyer <kdreyer@redhat.com> wrote:
On Mon, Apr 27, 2020 at 11:21 AM Patrick Dowler <pdowler.cadc@gmail.com> wrote:
I am trying to manually create a radosgw instance for a small development installation. I was able to muddle through and get a working mon, mgr,
and
osd (x2), but the docs for radosgw are based on ceph-deploy which is not part of the octopus release.
Here are the steps I did two weeks ago on CentOS 7 for Nautilus. I have not tried Octopus manually yet.
# Install RGW: yum -y install ceph-radosgw mkdir -p /var/lib/ceph/radosgw/ceph-rgw.ceph1
# My /etc/ceph/ceph.conf file: ------- [global] fsid = cbc2b0e2-ed16-4f09-8578-f8c962b1a0ef mon initial members = ceph1 mon host = 192.168.2.101 public network = 192.168.2.0/24 cluster network = 192.168.2.0/24
[client.rgw.ceph1] host = ceph1 rgw frontends = "civetweb port=80" # Uncomment this to write RGW logs: # log file = /var/log/ceph/ceph-rgw.log
# Note: set up wildcard DNS for "*.ceph.example.com" this to work: rgw resolve cname = true rgw dns name = ceph.example.com
---------
# Create RGW keyring: ceph auth get-or-create client.rgw.ceph1 osd 'allow rwx' mon 'allow rw' -o /var/lib/ceph/radosgw/ceph-rgw.ceph1/keyring
# Ensure permissions are correct: chown -R ceph:ceph /var/lib/ceph/radosgw/
# Configure RGW as "done": sudo -u ceph touch /var/lib/ceph/radosgw/ceph-rgw.ceph1/done
# Start the service: systemctl enable --now ceph-radosgw@rgw.ceph1
# Create a RGW user: radosgw-admin user create --uid=kdreyer --display-name="Ken Dreyer" --email=kdreyer@example.com
------
Then, on the s3 client:
# Write our config file: s3cmd --no-ssl --access_key=ASDF123456 --secret_key=FOOBAR123 --host=ceph.example.com --host-bucket="%(bucket)s.ceph.example.com" --dump-config > $HOME/.s3cfg
# Make a "kdreyer" bucket s3cmd mb s3://kdreyer
# Make the whole bucket public: s3cmd setacl s3://kdreyer/ --acl-public
# Upload a test file: echo "Hello World" > /tmp/hello.txt s3cmd put /tmp/hello.txt s3://kdreyer
# Fetch file: curl http://kdreyer.ceph.example.com/hello.txt
participants (4)
-
Ken Dreyer
-
Marc Roos
-
Patrick Dowler
-
tdados＠hotmail.com