5 Oct
2020
5 Oct
'20
3:07 p.m.
Hello, we have integrated Ceph's RGW with LDAP and have authenticated users using the
mail attribute successfully. We would like to shift to SSO and are evaluating the new OIDC
feature in Ceph together with dexIdP with an LDAP connector as an upstream IdP.
We are trying to understand the flow of the user authentication and how it will effect my
current LDAP users buckets which are already created in Ceph as LDAP users.
Will the Ceph RGW be able to pass the token to be verified to the IdP and what type of
user will then be created in Ceph? Is this the intended way of OIDC integration?
Thanks for any assistance
9 Oct
9 Oct
7:20 a.m.
Hello,
If it is possible for the uid that has been used for LDAP users to be the
same for OIDC users (which is based off the 'sub' field of the OpenID
connect token), then there are no extra migration steps needed.
Which version of Ceph are you using? In octopus, offline token validation
has been introduced, where an incoming web token is validated using the
certificate of the IDP. Uptil Octopus, there were no shadow users for OIDC
users, but we have introduced shadow user creation in the 'master' branch,
and that is done automatically when an AssumeRoleWithWebIdentity call is
made. So the metadata to look at right now would be <tenant>$<uid>$buckets
which stores the user stats and make sure that the same uid is being used
across both LDAP and OIDC (if that is possible), else there is a
radosgw-admin user rename command that will rename the user and update all
other metadata.
Also, please note that currently AssumeRoleWithWebIdentity has been tested
only with Keycloak. The documentation for STS in Octopus is here:
https://docs.ceph.com/en/octopus/radosgw/STS/
Thanks,
Pritha
On Mon, Oct 5, 2020 at 9:56 PM <technical(a)zylacomputing.com> wrote:
Hello, we have integrated Ceph's RGW with LDAP and
have authenticated
users using the mail attribute successfully. We would like to shift to SSO
and are evaluating the new OIDC feature in Ceph together with dexIdP with
an LDAP connector as an upstream IdP.
We are trying to understand the flow of the user authentication and how it
will effect my current LDAP users buckets which are already created in Ceph
as LDAP users.
Will the Ceph RGW be able to pass the token to be verified to the IdP and
what type of user will then be created in Ceph? Is this the intended way of
OIDC integration?
Thanks for any assistance
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
13 Oct
13 Oct
7:21 p.m.
Hi Pritha and thanks for your reply. We are using Ceph Octopus and we have switched to
Keycloak from dexIdP.
Having said that we have followed the guide from
https://docs.ceph.com/en/octopus/radosgw/STS/ but we are constantly having an issue with
the AssumeRoleWithWebIdentity example.
We are using 2 different accounts for role creation and policy creation and those 2 parts
of the example script are working fine but when we move over to the
assume_role_with_web_identity part we have forbidden error from Ceph.
We have used cephadm to install Ceph which is at:
# ceph --version
ceph version 15.2.5 (2c93eff00150f0cc5f106a559557a58d3d7b6f1f) octopus (stable)
We used the following command to add the role capabilities for both users:
radosgw-admin caps add --uid="TESTER" --caps="roles=*"
radosgw-admin caps add --uid="TESTER1" --caps="roles=*"
We have set the capabilities for the 2 users mentioned above as shown here:
buckets (*)
metadata (*)
roles (*)
usage (*)
user-policy (*)
users (*)
zone (*)
---
Can you please confirm that the key values have actually spaces in them or are they
missing an underscore?
[client.radosgw.gateway]
rgw sts key = {sts key for encrypting the session token}
rgw s3 auth use sts = true
---
We are also getting "NameError: name 'client' is not defined" error from
AssumeRoleWithWebIdentity example in this part shown below. Shouldn't it be
"sts_client.assume_role_with_web_identity" from
"client.assume_role_with_web_identity" as it is being defined as sts_client in
the code above it?
sts_client = boto3.client('sts',
aws_access_key_id=<access_key of TESTER1>,
aws_secret_access_key=<secret_key of TESTER1>,
endpoint_url=<STS URL>,
region_name='',
)
response = client.assume_role_with_web_identity(
RoleArn=role_response['Role']['Arn'],
RoleSessionName='Bob',
DurationSeconds=3600,
WebIdentityToken=<Web Token>
)
Can you or anyone give us some pointers to this issue please?
14 Oct
14 Oct
9:14 a.m.
Hello,
rgw sts key should be a key of length 16 since we use AES 128 for
encryption (e.g. rgw sts key = abcdefghijklmnop)
Yes it should be 'sts_client' and not 'client'. The errors in
documentation
have been noted and will be corrected.
Also please note that the backport to octopus of the new changes is
underway (https://github.com/ceph/ceph/pull/37640), and this should be
available in the next Octopus release.
Thanks,
Pritha
On Tue, Oct 13, 2020 at 9:22 PM <technical(a)zylacomputing.com> wrote:
Hi Pritha and thanks for your reply. We are using Ceph
Octopus and we have
switched to Keycloak from dexIdP.
Having said that we have followed the guide from
https://docs.ceph.com/en/octopus/radosgw/STS/ but we are constantly
having an issue with the AssumeRoleWithWebIdentity example.
We are using 2 different accounts for role creation and policy creation
and those 2 parts of the example script are working fine but when we move
over to the assume_role_with_web_identity part we have forbidden error from
Ceph.
We have used cephadm to install Ceph which is at:
# ceph --version
ceph version 15.2.5 (2c93eff00150f0cc5f106a559557a58d3d7b6f1f) octopus
(stable)
We used the following command to add the role capabilities for both users:
radosgw-admin caps add --uid="TESTER" --caps="roles=*"
radosgw-admin caps add --uid="TESTER1" --caps="roles=*"
We have set the capabilities for the 2 users mentioned above as shown here:
buckets (*)
metadata (*)
roles (*)
usage (*)
user-policy (*)
users (*)
zone (*)
---
Can you please confirm that the key values have actually spaces in them or
are they missing an underscore?
[client.radosgw.gateway]
rgw sts key = {sts key for encrypting the session token}
rgw s3 auth use sts = true
---
We are also getting "NameError: name 'client' is not defined" error
from
AssumeRoleWithWebIdentity example in this part shown below. Shouldn't it be
"sts_client.assume_role_with_web_identity" from
"client.assume_role_with_web_identity" as it is being defined as sts_client
in the code above it?
sts_client = boto3.client('sts',
aws_access_key_id=<access_key of TESTER1>,
aws_secret_access_key=<secret_key of TESTER1>,
endpoint_url=<STS URL>,
region_name='',
)
response = client.assume_role_with_web_identity(
RoleArn=role_response['Role']['Arn'],
RoleSessionName='Bob',
DurationSeconds=3600,
WebIdentityToken=<Web Token>
)
Can you or anyone give us some pointers to this issue please?
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
9:47 p.m.
Hi Pritha and thanks again for your reply. Unfortunately we are still stuck at the
AssumeRoleWithWebIdentity API call as shown below:
2020-10-14T08:24:26.314+0000 7ff6600ff700 1 ====== starting new request
req=0x7ff6b69496b0 =====
2020-10-14T08:24:26.314+0000 7ff6600ff700 2 req 7 0s initializing for trans_id =
tx000000000000000000007-005f86b5ba-1e4f8-ZylaS3
2020-10-14T08:24:26.314+0000 7ff6600ff700 10 rgw api priority: s3=8 s3website=7
2020-10-14T08:24:26.314+0000 7ff6600ff700 10 host=192.168.35.28
2020-10-14T08:24:26.314+0000 7ff6600ff700 20 subdomain= domain= in_hosted_domain=0
in_hosted_domain_s3website=0
2020-10-14T08:24:26.314+0000 7ff6600ff700 20 final domain/bucket subdomain= domain=
in_hosted_domain=0 in_hosted_domain_s3website=0 s->info.domain=
s->info.request_uri=/
2020-10-14T08:24:26.314+0000 7ff6600ff700 20 req 7 0s get_handler
handler=26RGWHandler_REST_Service_S3
2020-10-14T08:24:26.314+0000 7ff6600ff700 10 handler=26RGWHandler_REST_Service_S3
2020-10-14T08:24:26.314+0000 7ff6600ff700 2 req 7 0s getting op 4
2020-10-14T08:24:26.314+0000 7ff6600ff700 10 Content of POST:
Action=AssumeRoleWithWebIdentity&Version=2011-06-15&RoleArn=arn%3Aaws%3Aiam%3A%3A%3Arole%2FS3Access&RoleSessionName=KC-Client&DurationSeconds=3600&WebIdentityToken=<token_here>
2020-10-14T08:24:26.315+0000 7ff6600ff700 10 req 7 0.001000005s
sts:assume_role_web_identity scheduling with dmclock client=0 cost=1
2020-10-14T08:24:26.315+0000 7ff6600ff700 10 op=31RGWSTSAssumeRoleWithWebIdentity
2020-10-14T08:24:26.315+0000 7ff6600ff700 2 req 7 0.001000005s
sts:assume_role_web_identity verifying requester
2020-10-14T08:24:26.315+0000 7ff6600ff700 20 req 7 0.001000005s
sts:assume_role_web_identity rgw::auth::sts::DefaultStrategy: trying
rgw::auth::sts::WebTokenEngine
2020-10-14T08:24:26.315+0000 7ff6600ff700 20 req 7 0.001000005s
sts:assume_role_web_identity rgw::auth::sts::WebTokenEngine denied with reason=-13
2020-10-14T08:24:26.315+0000 7ff6600ff700 5 req 7 0.001000005s
sts:assume_role_web_identity Failed the auth strategy, reason=-13
2020-10-14T08:24:26.315+0000 7ff6600ff700 10 failed to authorize request
2020-10-14T08:24:26.315+0000 7ff6600ff700 1 op->ERRORHANDLER: err_no=-13
new_err_no=-13
2020-10-14T08:24:26.315+0000 7ff6600ff700 2 req 7 0.001000005s
sts:assume_role_web_identity op status=0
2020-10-14T08:24:26.315+0000 7ff6600ff700 2 req 7 0.001000005s
sts:assume_role_web_identity http status=403
2020-10-14T08:24:26.315+0000 7ff6600ff700 1 ====== req done req=0x7ff6b69496b0 op
status=0 http_status=403 latency=0.001000005s ======
2020-10-14T08:24:26.315+0000 7ff6600ff700 1 beast: 0x7ff6b69496b0: 192.168.35.116 - -
[2020-10-14T08:24:26.315479+0000] "POST / HTTP/1.1" 403 185 -
"Boto3/1.15.16 Python/3.8.5 Linux/5.4.0-48-generic Botocore/1.18.16" -
2020-10-14T08:24:26.383+0000 7ff66690c700 20 failed to read header: end of stream
to get the web token we are using the below script and pasting the output in the
AssumeRoleWithWebIdentity API call:
#!/bin/bash
KC_REALM=myrealm
KC_CLIENT=rgw-app
KC_CLIENT_SECRET=<string from credentials tab in KC of the client>
KC_SERVER=<IP>:<port>
KC_CONTEXT=auth
# Request Tokens for credentials
KC_RESPONSE=$( \
curl -k --no-progress-meter -X POST \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "scope=openid" \
-d "grant_type=client_credentials" \
-d "client_id=$KC_CLIENT" \
-d "client_secret=$KC_CLIENT_SECRET" \
"https://$KC_SERVER/$KC_CONTEXT/realms/$KC_REALM/protocol/openid-connect/token"
\
| jq .
)
KC_ACCESS_TOKEN=$(echo $KC_RESPONSE| jq -r .access_token)
echo $KC_ACCESS_TOKEN
For the above token request to work we had to turn "ON" the option "Service
Accounts Enabled" in the settings tab of the client
I suspect that the issue is more in relation to the capabilities of the user when calling
the STS engine for the AssumeRoleWithWebIdentity API call.
If anyone can help... You are more than welcome :-)
15 Oct
15 Oct
7:40 a.m.
Hello again,
The issues that you are seeing are because as I mentioned in my previous
email, I missed backporting some commits to Octopus (apologies for the
same), and I have opened a backport PR ((
https://github.com/ceph/ceph/pull/37640) and this should be available in
the next Octopus release.
Thanks,
Pritha
On Wed, Oct 14, 2020 at 11:48 PM <technical(a)zylacomputing.com> wrote:
Hi Pritha and thanks again for your reply.
Unfortunately we are still
stuck at the AssumeRoleWithWebIdentity API call as shown below:
2020-10-14T08:24:26.314+0000 7ff6600ff700 1 ====== starting new request
req=0x7ff6b69496b0 =====
2020-10-14T08:24:26.314+0000 7ff6600ff700 2 req 7 0s initializing for
trans_id = tx000000000000000000007-005f86b5ba-1e4f8-ZylaS3
2020-10-14T08:24:26.314+0000 7ff6600ff700 10 rgw api priority: s3=8
s3website=7
2020-10-14T08:24:26.314+0000 7ff6600ff700 10 host=192.168.35.28
2020-10-14T08:24:26.314+0000 7ff6600ff700 20 subdomain= domain=
in_hosted_domain=0 in_hosted_domain_s3website=0
2020-10-14T08:24:26.314+0000 7ff6600ff700 20 final domain/bucket
subdomain= domain= in_hosted_domain=0 in_hosted_domain_s3website=0
s->info.domain= s->info.request_uri=/
2020-10-14T08:24:26.314+0000 7ff6600ff700 20 req 7 0s get_handler
handler=26RGWHandler_REST_Service_S3
2020-10-14T08:24:26.314+0000 7ff6600ff700 10
handler=26RGWHandler_REST_Service_S3
2020-10-14T08:24:26.314+0000 7ff6600ff700 2 req 7 0s getting op 4
2020-10-14T08:24:26.314+0000 7ff6600ff700 10 Content of POST:
Action=AssumeRoleWithWebIdentity&Version=2011-06-15&RoleArn=arn%3Aaws%3Aiam%3A%3A%3Arole%2FS3Access&RoleSessionName=KC-Client&DurationSeconds=3600&WebIdentityToken=<token_here>
2020-10-14T08:24:26.315+0000 7ff6600ff700 10 req 7 0.001000005s
sts:assume_role_web_identity scheduling with dmclock client=0 cost=1
2020-10-14T08:24:26.315+0000 7ff6600ff700 10
op=31RGWSTSAssumeRoleWithWebIdentity
2020-10-14T08:24:26.315+0000 7ff6600ff700 2 req 7 0.001000005s
sts:assume_role_web_identity verifying requester
2020-10-14T08:24:26.315+0000 7ff6600ff700 20 req 7 0.001000005s
sts:assume_role_web_identity rgw::auth::sts::DefaultStrategy: trying
rgw::auth::sts::WebTokenEngine
2020-10-14T08:24:26.315+0000 7ff6600ff700 20 req 7 0.001000005s
sts:assume_role_web_identity rgw::auth::sts::WebTokenEngine denied with
reason=-13
2020-10-14T08:24:26.315+0000 7ff6600ff700 5 req 7 0.001000005s
sts:assume_role_web_identity Failed the auth strategy, reason=-13
2020-10-14T08:24:26.315+0000 7ff6600ff700 10 failed to authorize request
2020-10-14T08:24:26.315+0000 7ff6600ff700 1 op->ERRORHANDLER: err_no=-13
new_err_no=-13
2020-10-14T08:24:26.315+0000 7ff6600ff700 2 req 7 0.001000005s
sts:assume_role_web_identity op status=0
2020-10-14T08:24:26.315+0000 7ff6600ff700 2 req 7 0.001000005s
sts:assume_role_web_identity http status=403
2020-10-14T08:24:26.315+0000 7ff6600ff700 1 ====== req done
req=0x7ff6b69496b0 op status=0 http_status=403 latency=0.001000005s ======
2020-10-14T08:24:26.315+0000 7ff6600ff700 1 beast: 0x7ff6b69496b0:
192.168.35.116 - - [2020-10-14T08:24:26.315479+0000] "POST / HTTP/1.1" 403
185 - "Boto3/1.15.16 Python/3.8.5 Linux/5.4.0-48-generic Botocore/1.18.16" -
2020-10-14T08:24:26.383+0000 7ff66690c700 20 failed to read header: end of
stream
to get the web token we are using the below script and pasting the output
in the AssumeRoleWithWebIdentity API call:
#!/bin/bash
KC_REALM=myrealm
KC_CLIENT=rgw-app
KC_CLIENT_SECRET=<string from credentials tab in KC of the client>
KC_SERVER=<IP>:<port>
KC_CONTEXT=auth
# Request Tokens for credentials
KC_RESPONSE=$( \
curl -k --no-progress-meter -X POST \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "scope=openid" \
-d "grant_type=client_credentials" \
-d "client_id=$KC_CLIENT" \
-d "client_secret=$KC_CLIENT_SECRET" \
"https://$KC_SERVER/$KC_CONTEXT/realms/$KC_REALM/protocol/openid-connect/token"
\
| jq .
)
KC_ACCESS_TOKEN=$(echo $KC_RESPONSE| jq -r .access_token)
echo $KC_ACCESS_TOKEN
For the above token request to work we had to turn "ON" the option
"Service Accounts Enabled" in the settings tab of the client
I suspect that the issue is more in relation to the capabilities of the
user when calling the STS engine for the AssumeRoleWithWebIdentity API call.
If anyone can help... You are more than welcome :-)
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
19 Oct
19 Oct
6:29 p.m.
Dear Pritha, thanks a lot for your feedback and apologies for missing your comment about
the backporting. Would you have a rough estimate on the next Octopus release by any
chance?
On another note on the same subject, would you be able to give us some feedback on how the
users will be created in Ceph? (for example when we used ldap, an ldap user used to be
created in Ceph for "mapping", will it be the same in this case)
If we have multiple tenants (unique usernames "emails" in KeyCloak) how will the
introspect url's be defined for different tenants?
Thanks in advance
21 Oct
21 Oct
7:50 a.m.
Hello,
The next Octopus release should be there in 3-4 weeks.
In Octopus, shadow users aren't created ((for federated oidc users). But we
later realised that shadow users are needed to maintain user stats, hence
the code for the same is under the process of being added as of now and
should be available in Pacific release.
We have also done away with the token introspection url in the latest code
and have switched over to offline token validation using the IDP's certs
since using token introspection url wouldn't scale well for multiple
clients.
There is a related Ceph Tech Talk here, that you can watch:
https://www.youtube.com/watch?v=Lc32meILfNI&t=410s
Thanks,
Pritha
On Mon, Oct 19, 2020 at 8:30 PM <technical(a)zylacomputing.com> wrote:
Dear Pritha, thanks a lot for your feedback and
apologies for missing your
comment about the backporting. Would you have a rough estimate on the next
Octopus release by any chance?
On another note on the same subject, would you be able to give us some
feedback on how the users will be created in Ceph? (for example when we
used ldap, an ldap user used to be created in Ceph for "mapping", will it
be the same in this case)
If we have multiple tenants (unique usernames "emails" in KeyCloak) how
will the introspect url's be defined for different tenants?
Thanks in advance
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
1302
days inactive
1318
days old
7 comments
2 participants
participants (2)
-
Pritha Srivastava -
technical＠zylacomputing.com