Re: Authentication failure at radosgw for presigned urls
++ceph-users On Fri, Sep 20, 2019 at 4:48 PM Biswajeet Patra < biswajeet.patra@flipkart.com> wrote:
Hi, We recently faced an issue with radosgw authentication of presigned urls. The presigned url generated by client to download object fails at radosgw with http error 403 i.e SignatureDoesNotMatch.
The radosgw computes the signature (*v2 signature in this case*) using the s3 specification https://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html. In this process, a string_to_sign is created by concatenating selected elements from requests which is then authenticated using hmac to create the final signature. If this signature matches with the client signature, the authentication is successful or else it fails with SignatureDoesNotMatch error. As part of the StringToSign, the CanonicalizedAmzHeaders should only include headers that start with "x-amz-" and ignore other headers. But in the radosgw code, there are other meta prefixes that are checked against the http request headers and if matched are included in the CanonicalizedAmzHeaders to compute the final signature. For e.g, if a request header contains "HTTP_X_ACCOUNT" its selected by radosgw to include in amz_headers but the same will be ignored by AWS SDK as it does not start with "x-amz-". This will result in different signature computed by client and radosgw.
Code Snippet: rgw_common.cc struct str_len meta_prefixes[] = { STR_LEN_ENTRY("HTTP_X_AMZ"), STR_LEN_ENTRY("HTTP_X_GOOG"), STR_LEN_ENTRY("HTTP_X_DHO"), STR_LEN_ENTRY("HTTP_X_RGW"), STR_LEN_ENTRY("HTTP_X_OBJECT"), STR_LEN_ENTRY("HTTP_X_CONTAINER"), STR_LEN_ENTRY("HTTP_X_ACCOUNT"), {NULL, 0} };
The method init_meta_info() which matches the above prefixes is called from RGWREST::preprocess() which is invoked for all s3 requests. It will be helpful to know as to why these prefixes that are not specified by AWS S3 comes in the path of s3 authentication. Was it added for swift use-cases only? If so, then why its included in rgw_common.cc? As a proposed fix, we can remove the highlighted meta prefixes that are not specified by aws from the s3 authentication path signature calculation. Let me know if you have any queries or solutions.
Regards, Biswajeet
-- *-----------------------------------------------------------------------------------------* *This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee, you should not disseminate, distribute or copy this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.***** **** *Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the organization. Any information on shares, debentures or similar instruments, recommended product pricing, valuations and the like are for information purposes only. It is not meant to be an instruction or recommendation, as the case may be, to buy or to sell securities, products, services nor an offer to buy or sell securities, products or services unless specifically stated to be so on behalf of the Flipkart group. Employees of the Flipkart group of companies are expressly required not to make defamatory statements and not to infringe or authorise any infringement of copyright or any other legal right by email communications. Any such communication is contrary to organizational policy and outside the scope of the employment of the individual concerned. The organization will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising.***** **** *Our organization accepts no liability for the content of this email, or for the consequences of any actions taken on the basis of the information *provided,* unless that information is subsequently confirmed in writing. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.* _-----------------------------------------------------------------------------------------_
<<bumping this up>> Can anybody take a look into this ? why are other prefixes apart from 'x-amz-' in headers are checked and used to sign s3 requests? This maybe a use case for swift requests but the same code flow is also used for s3 requests and that results in signature mismatch at client and server end. Regards, Biswajeet On Sun, Sep 22, 2019 at 11:57 AM Biswajeet Patra < biswajeet.patra@flipkart.com> wrote:
++ceph-users
On Fri, Sep 20, 2019 at 4:48 PM Biswajeet Patra < biswajeet.patra@flipkart.com> wrote:
Hi, We recently faced an issue with radosgw authentication of presigned urls. The presigned url generated by client to download object fails at radosgw with http error 403 i.e SignatureDoesNotMatch.
The radosgw computes the signature (*v2 signature in this case*) using the s3 specification https://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html. In this process, a string_to_sign is created by concatenating selected elements from requests which is then authenticated using hmac to create the final signature. If this signature matches with the client signature, the authentication is successful or else it fails with SignatureDoesNotMatch error. As part of the StringToSign, the CanonicalizedAmzHeaders should only include headers that start with "x-amz-" and ignore other headers. But in the radosgw code, there are other meta prefixes that are checked against the http request headers and if matched are included in the CanonicalizedAmzHeaders to compute the final signature. For e.g, if a request header contains "HTTP_X_ACCOUNT" its selected by radosgw to include in amz_headers but the same will be ignored by AWS SDK as it does not start with "x-amz-". This will result in different signature computed by client and radosgw.
Code Snippet: rgw_common.cc struct str_len meta_prefixes[] = { STR_LEN_ENTRY("HTTP_X_AMZ"), STR_LEN_ENTRY("HTTP_X_GOOG"), STR_LEN_ENTRY("HTTP_X_DHO"), STR_LEN_ENTRY("HTTP_X_RGW"), STR_LEN_ENTRY("HTTP_X_OBJECT"), STR_LEN_ENTRY("HTTP_X_CONTAINER"), STR_LEN_ENTRY("HTTP_X_ACCOUNT"), {NULL, 0} };
The method init_meta_info() which matches the above prefixes is called from RGWREST::preprocess() which is invoked for all s3 requests. It will be helpful to know as to why these prefixes that are not specified by AWS S3 comes in the path of s3 authentication. Was it added for swift use-cases only? If so, then why its included in rgw_common.cc? As a proposed fix, we can remove the highlighted meta prefixes that are not specified by aws from the s3 authentication path signature calculation. Let me know if you have any queries or solutions.
Regards, Biswajeet
-- *-----------------------------------------------------------------------------------------* *This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee, you should not disseminate, distribute or copy this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.***** **** *Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the organization. Any information on shares, debentures or similar instruments, recommended product pricing, valuations and the like are for information purposes only. It is not meant to be an instruction or recommendation, as the case may be, to buy or to sell securities, products, services nor an offer to buy or sell securities, products or services unless specifically stated to be so on behalf of the Flipkart group. Employees of the Flipkart group of companies are expressly required not to make defamatory statements and not to infringe or authorise any infringement of copyright or any other legal right by email communications. Any such communication is contrary to organizational policy and outside the scope of the employment of the individual concerned. The organization will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising.***** **** *Our organization accepts no liability for the content of this email, or for the consequences of any actions taken on the basis of the information *provided,* unless that information is subsequently confirmed in writing. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.* _-----------------------------------------------------------------------------------------_
participants (1)
-
Biswajeet Patra