[ceph-users] Bucket notification with kafka error

11 Mar 2020

      HI,
I'm sorry to bother you again

I want to use kafka to queue the notifications , I add a topic named kafka,and put the notification config xml

The topic info:

<ListTopicsResponse xmlns="https://sns.amazonaws.com/doc/2010-03-31/">
    <ListTopicsResult>
        <Topics>
            <member>
                <User>sr</User>
                <Name>kafka</Name>
                <EndPoint>
                    <EndpointAddress>kafka://192.168.3.250:9092</EndpointAddress>
                    <EndpointArgs>kafka-ack-level=broker&push-endpoint=kafka://192.168.3.250:9092</EndpointArgs>
                    <EndpointTopic>kafka</EndpointTopic>
                </EndPoint>
                <TopicArn>arn:aws:sns:default::kafka</TopicArn>
            </member>
            <member>
                <User>sr</User>
                <Name>kafka_kafka</Name>
                <EndPoint>
                    <EndpointAddress>kafka://192.168.3.250:9092</EndpointAddress>
                    <EndpointArgs>kafka-ack-level=broker&push-endpoint=kafka://192.168.3.250:9092</EndpointArgs>
                    <EndpointTopic>kafka</EndpointTopic>
                </EndPoint>
                <TopicArn>arn:aws:sns:default::kafka</TopicArn>
            </member>
            <member>
                <User>sr</User>
                <Name>webno</Name>
                <EndPoint>
                    <EndpointAddress>http://192.168.1.114:8080/s3/sn</EndpointAddress>
                    <EndpointArgs>push-endpoint=http://192.168.1.114:8080/s3/sn</EndpointArgs>
                    <EndpointTopic>webno</EndpointTopic>
                </EndPoint>
                <TopicArn>arn:aws:sns:default::webno</TopicArn>
            </member>
        </Topics>
    </ListTopicsResult>
    <ResponseMetadata>
        <RequestId>c4b84c5b-1e88-4f16-9863-7f68872d91a4.744394.135</RequestId>
    </ResponseMetadata>
</ListTopicsResponse>

and the put notification body :
<NotificationConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"> <TopicConfiguration> <Id>kafka</Id> <Topic>arn:aws:sns:default::kafka</Topic> </TopicConfiguration> </NotificationConfiguration>

The web notification works fine ,but when I use the kafka(version 1.0 jdk 1.8)
I got the debug info :

020-03-11 12:46:38.612 7fd81eeb1700 20 get_system_obj_state: s->obj_tag was set empty
2020-03-11 12:46:38.612 7fd81eeb1700 10 cache get: name=default.rgw.log++pubsub.user.sr.bucket.osstest/c4b84c5b-1e88-4f16-9863-7f68872d91a4.14175.1 : hit (requested=0x1, cached=0x17)
2020-03-11 12:46:38.612 7fd81eeb1700 20 notification: 'kafka' on topic: 'kafka' and bucket: 'osstest' (unique topic: 'kafka_kafka') apply to event of type: 's3:ObjectCreated:Put'
2020-03-11 12:46:38.612 7fd81eeb1700  1 ERROR: failed to create push endpoint: kafka://192.168.3.250:9092 due to: pubsub endpoint configuration error: unknown schema in: kafka://192.168.3.250:9092
2020-03-11 12:46:38.612 7fd81eeb1700  5 req 126 0.186s s3:put_obj WARNING: publishing notification failed, with error: -22
2020-03-11 12:46:38.612 7fd81eeb1700  2

[ceph-users] Bucket notification with kafka error

曹 海旺

曹海旺