when calling the CreateTopic operation: Unknown
hi, i want to use ceph bucket notification . i try to created topic with below command but get error when used kafka with user/password how can i solved this problem ? my syntax have any problem? https://www.ibm.com/docs/en/storage-ceph/7?topic=management-creating-bucket-... https://docs.ceph.com/en/latest/radosgw/notifications/ # aws sns create-topic --name storage --profile=test-notifications --endpoint-url=http://192.168.115.59 --attributes=file://topic.json --debug ##### topic.json: {"push-endpoint": "kafka://usr-test:p@ssw#0rd@kafka.test:9092","verify-ssl": "False", "kafka-ack-level": "broker", "persistent":"true"} ##### error is :---------------- content-type:application/x-www-form-urlencoded; charset=utf-8 host:192.168.115.59 x-amz-date:20240712T120700Z content-type;host;x-amz-date 6f328bc5b0736f23ae2cdf68ccffe1a45c705dd1636f61b999350ae18f8d5ad1 2024-07-12 12:07:00,439 - MainThread - botocore.auth - DEBUG - StringToSign: AWS4-HMAC-SHA256 20240712T120700Z 20240712/podspace/sns/aws4_request 38b7d8721abdd98c214ea763d9dcc324fcbc5982990353140f6b734455e9c01e 2024-07-12 12:07:00,439 - MainThread - botocore.auth - DEBUG - Signature: 99a0f7d40bc011c09dfe7c6aed4bae7f739df52278e23bd4f16fdb1888c001b7 2024-07-12 12:07:00,440 - MainThread - botocore.endpoint - DEBUG - Sending http request: <AWSPreparedRequest stream_output=False, method=POST, url= http://192.168.115.59/, headers={'Content-Type': b'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': b'aws-cli/1.18.156 Python/3.6.8 Linux/5.4.17-2136.326.6.el8uek.x86_64 botocore/1.18.15', 'X-Amz-Date': b'20240712T120700Z', 'Authorization': b'AWS4-HMAC-SHA256 Credential=2MRG0TXJFISQAIVE47F1/20240712/podspace/sns/aws4_request, SignedHeaders=content-type;host;x-amz-date, Signature=99a0f7d40bc011c09dfe7c6aed4bae7f739df52278e23bd4f16fdb1888c001b7', 'Content-Length': '371'}> 2024-07-12 12:07:00,441 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTP connection (1): 192.168.115.59:80 2024-07-12 12:07:00,446 - MainThread - urllib3.connectionpool - DEBUG - http://192.168.115.59:80 "POST / HTTP/1.1" 400 206 2024-07-12 12:07:00,447 - MainThread - botocore.parsers - DEBUG - Response headers: {'content-length': '206', 'x-amz-request-id': 'tx00000c7a58038e67798c0-0066911c64-19910-site1', 'accept-ranges': 'bytes', 'content-type': 'application/xml', 'date': 'Fri, 12 Jul 2024 12:07:00 GMT'} 2024-07-12 12:07:00,447 - MainThread - botocore.parsers - DEBUG - Response body: b'<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidArgument</Code><Message></Message><RequestId>tx00000c7a58038e67798c0-0066911c64-19910-site1</RequestId><HostId>19910-site1-podspace</HostId></Error>' 2024-07-12 12:07:00,448 - MainThread - botocore.hooks - DEBUG - Event needs-retry.sns.CreateTopic: calling handler <botocore.retryhandler.RetryHandler object at 0x7f35e0a7f860> 2024-07-12 12:07:00,448 - MainThread - botocore.retryhandler - DEBUG - No retry needed. 2024-07-12 12:07:00,449 - MainThread - awscli.clidriver - DEBUG - Exception caught in main() Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/awscli/clidriver.py", line 217, in main return command_table[parsed_args.command](remaining, parsed_args) File "/usr/lib/python3.6/site-packages/awscli/clidriver.py", line 358, in __call__ return command_table[parsed_args.operation](remaining, parsed_globals) File "/usr/lib/python3.6/site-packages/awscli/clidriver.py", line 530, in __call__ call_parameters, parsed_globals) File "/usr/lib/python3.6/site-packages/awscli/clidriver.py", line 650, in invoke client, operation_name, parameters, parsed_globals) File "/usr/lib/python3.6/site-packages/awscli/clidriver.py", line 662, in _make_client_call **parameters) File "/usr/lib/python3.6/site-packages/botocore/client.py", line 357, in _api_call return self._make_api_call(operation_name, kwargs) File "/usr/lib/python3.6/site-packages/botocore/client.py", line 676, in _make_api_call raise error_class(parsed_response, operation_name) botocore.exceptions.ClientError: An error occurred (Unknown) when calling the CreateTopic operation: Unknown 2024-07-12 12:07:00,449 - MainThread - awscli.clidriver - DEBUG - Exiting with rc 255 An error occurred (Unknown) when calling the CreateTopic operation: Unknown
Hi, sending user/password in the topic, must be done over secure transport. this means that the connection between the boto client and the RGW has to be HTTPS and the connection to the kafka server has to be secure as well. please check the section dealing with kafka in the "CreateTopic" documentation: https://docs.ceph.com/en/latest/radosgw/notifications/#create-a-topic On Fri, Jul 12, 2024 at 3:18 PM farhad kh <farhad.khedriyan@gmail.com> wrote:
hi, i want to use ceph bucket notification . i try to created topic with below command but get error when used kafka with user/password how can i solved this problem ? my syntax have any problem?
https://www.ibm.com/docs/en/storage-ceph/7?topic=management-creating-bucket-... https://docs.ceph.com/en/latest/radosgw/notifications/
# aws sns create-topic --name storage --profile=test-notifications --endpoint-url=http://192.168.115.59 --attributes=file://topic.json --debug
##### topic.json: {"push-endpoint": "kafka://usr-test:p@ssw#0rd@kafka.test :9092","verify-ssl": "False", "kafka-ack-level": "broker", "persistent":"true"} #####
error is :---------------- content-type:application/x-www-form-urlencoded; charset=utf-8 host:192.168.115.59 x-amz-date:20240712T120700Z
content-type;host;x-amz-date 6f328bc5b0736f23ae2cdf68ccffe1a45c705dd1636f61b999350ae18f8d5ad1 2024-07-12 12:07:00,439 - MainThread - botocore.auth - DEBUG - StringToSign: AWS4-HMAC-SHA256 20240712T120700Z 20240712/podspace/sns/aws4_request 38b7d8721abdd98c214ea763d9dcc324fcbc5982990353140f6b734455e9c01e 2024-07-12 12:07:00,439 - MainThread - botocore.auth - DEBUG - Signature: 99a0f7d40bc011c09dfe7c6aed4bae7f739df52278e23bd4f16fdb1888c001b7 2024-07-12 12:07:00,440 - MainThread - botocore.endpoint - DEBUG - Sending http request: <AWSPreparedRequest stream_output=False, method=POST, url= http://192.168.115.59/, headers={'Content-Type': b'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': b'aws-cli/1.18.156 Python/3.6.8 Linux/5.4.17-2136.326.6.el8uek.x86_64 botocore/1.18.15', 'X-Amz-Date': b'20240712T120700Z', 'Authorization': b'AWS4-HMAC-SHA256 Credential=2MRG0TXJFISQAIVE47F1/20240712/podspace/sns/aws4_request, SignedHeaders=content-type;host;x-amz-date,
Signature=99a0f7d40bc011c09dfe7c6aed4bae7f739df52278e23bd4f16fdb1888c001b7', 'Content-Length': '371'}> 2024-07-12 12:07:00,441 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTP connection (1): 192.168.115.59:80 2024-07-12 12:07:00,446 - MainThread - urllib3.connectionpool - DEBUG - http://192.168.115.59:80 "POST / HTTP/1.1" 400 206 2024-07-12 12:07:00,447 - MainThread - botocore.parsers - DEBUG - Response headers: {'content-length': '206', 'x-amz-request-id': 'tx00000c7a58038e67798c0-0066911c64-19910-site1', 'accept-ranges': 'bytes', 'content-type': 'application/xml', 'date': 'Fri, 12 Jul 2024 12:07:00 GMT'} 2024-07-12 12:07:00,447 - MainThread - botocore.parsers - DEBUG - Response body: b'<?xml version="1.0"
encoding="UTF-8"?><Error><Code>InvalidArgument</Code><Message></Message><RequestId>tx00000c7a58038e67798c0-0066911c64-19910-site1</RequestId><HostId>19910-site1-podspace</HostId></Error>' 2024-07-12 12:07:00,448 - MainThread - botocore.hooks - DEBUG - Event needs-retry.sns.CreateTopic: calling handler <botocore.retryhandler.RetryHandler object at 0x7f35e0a7f860> 2024-07-12 12:07:00,448 - MainThread - botocore.retryhandler - DEBUG - No retry needed. 2024-07-12 12:07:00,449 - MainThread - awscli.clidriver - DEBUG - Exception caught in main() Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/awscli/clidriver.py", line 217, in main return command_table[parsed_args.command](remaining, parsed_args) File "/usr/lib/python3.6/site-packages/awscli/clidriver.py", line 358, in __call__ return command_table[parsed_args.operation](remaining, parsed_globals) File "/usr/lib/python3.6/site-packages/awscli/clidriver.py", line 530, in __call__ call_parameters, parsed_globals) File "/usr/lib/python3.6/site-packages/awscli/clidriver.py", line 650, in invoke client, operation_name, parameters, parsed_globals) File "/usr/lib/python3.6/site-packages/awscli/clidriver.py", line 662, in _make_client_call **parameters) File "/usr/lib/python3.6/site-packages/botocore/client.py", line 357, in _api_call return self._make_api_call(operation_name, kwargs) File "/usr/lib/python3.6/site-packages/botocore/client.py", line 676, in _make_api_call raise error_class(parsed_response, operation_name) botocore.exceptions.ClientError: An error occurred (Unknown) when calling the CreateTopic operation: Unknown 2024-07-12 12:07:00,449 - MainThread - awscli.clidriver - DEBUG - Exiting with rc 255
An error occurred (Unknown) when calling the CreateTopic operation: Unknown _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (2)
-
farhad kh
-
Yuval Lifshitz