Hi folks, My ceph cluster with Quincy and Rocky9 is up and running. But I'm having issues with swift authenticating with keystone. Was wondering if I'm missed anything in the configuration. From the debug logs below, it appears that radosgw is still trying to authenticate with Swift instead of Keystone. Any pointers will be appreciated. thanks, Here is my configuration. # ceph config dump | grep rgw client advanced debug_rgw 20/20 client advanced rgw_keystone_accepted_roles admin,user * client advanced rgw_keystone_admin_domain Default * client advanced rgw_keystone_admin_password <secret> * client advanced rgw_keystone_admin_project service * client advanced rgw_keystone_admin_user ceph-ks-svc * client advanced rgw_keystone_api_version 3 client advanced rgw_keystone_implicit_tenants false * client advanced rgw_keystone_token_cache_size 0 client basic rgw_keystone_url <Identity URL> * client advanced rgw_s3_auth_use_keystone true client advanced rgw_swift_account_in_url true client basic rgw_thread_pool_size 512 client.rgw.s_rgw.dev-ipp1-u1-control01.ojmddc basic rgw_frontends beast port=7480 * client.rgw.s_rgw.dev-ipp1-u1-control02.adnjrx basic rgw_frontends beast port=7480 Here's the debug log. If I interpret it correctly, it is trying to do a swift authentication and failing. Am I missing any configuration for Keystone based authentication ? Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: beast: 0x7fddeb8e7710: 10.117.53.10 - - [03/Jun/2023:18:47:03.060 +0000] "GET /swift/v1/AUTH_c668ed224e434c88a9e0fce125056112?format=json HTTP/1.1" 401 119 - "openstacksdk/0.52.0 keystoneauth1/4.0.0 python-requests/2.22.0 CPython/3.8.10" - latency=0.000000000s Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: HTTP_ACCEPT=*/* Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: HTTP_ACCEPT_ENCODING=gzip, deflate Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: HTTP_CONNECTION=close Jun 03 11:47:03 dev-ipp1-u1-control02.radosgw[2802861]: HTTP_HOST=dev-ipp1-u1-object-store Jun 03 11:47:03 dev-ipp1-u1-control02radosgw[2802861]: HTTP_USER_AGENT=openstacksdk/0.52.0 keystoneauth1/4.0.0 python-requests/2.22.0 CPython/3.8.10 Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: HTTP_VERSION=1.1 Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: HTTP_X_AUTH_TOKEN=gAAAAABke4qn779UQ_XMz0EDL3P3TgjBQsGG6p-MNhviJxLZTuMTnTDmpT5Yfi9UpgO_T3LOOsPjQAw6zoMUIaC22wPeryp5x-UumB3XwXOWp-qSXLbuN3b9oj_Qg5kCZWA0waWNRHzQ1mwtlEmmpTgvTXbU5V1ym6hEBOn6Q3RWhn34Hj3cF9o Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: HTTP_X_FORWARDED_FOR=10.117.148.3 Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: QUERY_STRING=format=json Jun 03 11:47:03 dev-ipp1-u1-control02.radosgw[2802861]: REMOTE_ADDR=10.117.53.10 Jun 03 11:47:03 dev-ipp1-u1-control02.radosgw[2802861]: REQUEST_METHOD=GET Jun 03 11:47:03 dev-ipp1-u1-control02.radosgw[2802861]: REQUEST_URI=/swift/v1/AUTH_c668ed224e434c88a9e0fce125056112?format=json Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: SCRIPT_URI=/swift/v1/AUTH_c668ed224e434c88a9e0fce125056112 Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: SERVER_PORT=7480 Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: ====== starting new request req=0x7fddeb8e7710 ===== Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s initializing for trans_id = tx000003991cfc5c1791f95-00647b8aa7-30c56-default Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s rgw api priority: s3=8 s3website=7 Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s host=dev-ipp1-u1-object-store Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s subdomain= domain= in_hosted_domain=0 in_hosted_domain_s3website=0 Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s final domain/bucket subdomain= domain= in_hosted_domain=0 in_hosted_domain_s3website=0 s->info.domain= s->info.request_uri=/swift/v1/AUTH_c668ed224e434c88a9e0fce125056112 Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s name: format val: json Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s ver=v1 first= req= Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s handler=29RGWHandler_REST_Service_SWIFT Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s getting op 0 Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s get_system_obj_state: rctx=0x7fddeb8e6790 obj=default.rgw.log:script.prerequest. state=0x55f743b97720 s->prefetch_data=0 Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s cache get: name=default.rgw.log++script.prerequest. : hit (negative entry) Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s swift:list_buckets scheduling with throttler client=3 cost=1 Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s swift:list_buckets op=29RGWListBuckets_ObjStore_SWIFT Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s swift:list_buckets verifying requester Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s swift:list_buckets rgw::auth::swift::DefaultStrategy: trying rgw::auth::swift::TempURLEngine Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s swift:list_buckets rgw::auth::swift::TempURLEngine denied with reason=-13 Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s swift:list_buckets rgw::auth::swift::DefaultStrategy: trying rgw::auth::swift::SignedTokenEngine Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s swift:list_buckets rgw::auth::swift::SignedTokenEngine denied with reason=-1 Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s swift:list_buckets rgw::auth::swift::DefaultStrategy: trying rgw::auth::swift::SwiftAnonymousEngine Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s swift:list_buckets rgw::auth::swift::SwiftAnonymousEngine denied with reason=-1 Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s swift:list_buckets Failed the auth strategy, reason=-1 Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: failed to authorize request Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s op->ERRORHANDLER: err_no=-1 new_err_no=-1 Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s get_system_obj_state: rctx=0x7fddeb8e6790 obj=default.rgw.log:script.postrequest. state=0x55f743b97960 s->prefetch_data=0 Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s cache get: name=default.rgw.log++script.postrequest. : hit (negative entry) Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s swift:list_buckets op status=0 Jun 03 11:47:03 dev-ipp1-u1-control02 radosgw[2802861]: req 4148325180046385045 0.000000000s swift:list_buckets http status=401