Hi I am trying to setup the “High availability service for RGW” using SSL both to the HAProxy and from the HAProxy to the RGW backend. The SSL certificate gets applied to both HAProxy and the RGW. If I use the RGW instances directly they work as expected. The RGW config is as follows: service_type: rgw service_id: rgw service_name: rgw.rgw placement: label: rgw count_per_host: 2 spec: ssl: true rgw_frontend_port: 6443 rgw_frontend_ssl_certificate: | -----BEGIN CERTIFICATE---- -----END PRIVATE KEY----- Ingress as follows: service_type: ingress service_id: rgw.rgw placement: hosts: - cephrgw01 - cephrgw02 - cephrgw03 spec: backend_service: rgw.rgw virtual_ip: 172.16.1.130/16 frontend_port: 443 monitor_port: 1967 ssl_cert: | -----BEGIN CERTIFICATE----- -----END CERTIFICATE----- The issue is that the haproxy.cfg gets generated like this, without SSL enabled on the backends: # This file is generated by cephadm. global log127.0.0.1 local2 chroot/var/lib/haproxy pidfile/var/lib/haproxy/haproxy.pid maxconn8000 daemon stats socket /var/lib/haproxy/stats defaults modehttp logglobal optionhttplog optiondontlognull option http-server-close option forwardforexcept 127.0.0.0/8 optionredispatch retries3 timeout queue20s timeout connect5s timeout http-request1s timeout http-keep-alive 5s timeout client1s timeout server1s timeout check5s maxconn8000 frontend stats mode http bind 172.16.1.130:1967 bind localhost:1967 stats enable stats uri /stats stats refresh 10s stats auth admin:abcdefg http-request use-service prometheus-exporter if { path /metrics } monitor-uri /health frontend frontend bind 172.16.1.130:443 ssl crt /var/lib/haproxy/haproxy.pem default_backend backend backend backend option forwardfor balance static-rr option httpchk HEAD / HTTP/1.0 server rgw.rgw.cephrgw01.euvqmd 172.16.1.131:6443 check weight 100 server rgw.rgw.cephrgw01.aphsnx 172.16.1.131:6444 check weight 100 server rgw.rgw.cephrgw02.ovckaw 172.16.1.132:6443 check weight 100 server rgw.rgw.cephrgw02.jevtrb 172.16.1.132:6444 check weight 100 server rgw.rgw.cephrgw03.gzdame 172.16.1.133:6443 check weight 100 server rgw.rgw.cephrgw03.bchspq 172.16.1.133:6444 check weight 100 This of course does not work as the backend use SSL. Is there some configuration that I have missed or should I file a bug report? /Jimmy
I forget to add that the Ceph version is 17.2.5 managed with cephadm. /Jimmy
# This file is generated by cephadm. global log127.0.0.1 local2 chroot/var/lib/haproxy pidfile/var/lib/haproxy/haproxy.pid maxconn8000 daemon stats socket /var/lib/haproxy/stats
defaults modehttp logglobal optionhttplog optiondontlognull option http-server-close option forwardforexcept 127.0.0.0/8 optionredispatch retries3 timeout queue20s timeout connect5s timeout http-request1s timeout http-keep-alive 5s timeout client1s timeout server1s timeout check5s maxconn8000
frontend stats mode http bind 172.16.1.130:1967 bind localhost:1967 stats enable stats uri /stats stats refresh 10s stats auth admin:abcdefg http-request use-service prometheus-exporter if { path /metrics } monitor-uri /health
frontend frontend bind 172.16.1.130:443 ssl crt /var/lib/haproxy/haproxy.pem default_backend backend
backend backend option forwardfor balance static-rr option httpchk HEAD / HTTP/1.0 server rgw.rgw.cephrgw01.euvqmd 172.16.1.131:6443 check weight 100 server rgw.rgw.cephrgw01.aphsnx 172.16.1.131:6444 check weight 100 server rgw.rgw.cephrgw02.ovckaw 172.16.1.132:6443 check weight 100 server rgw.rgw.cephrgw02.jevtrb 172.16.1.132:6444 check weight 100 server rgw.rgw.cephrgw03.gzdame 172.16.1.133:6443 check weight 100 server rgw.rgw.cephrgw03.bchspq 172.16.1.133:6444 check weight 100
This of course does not work as the backend use SSL.
Is there some configuration that I have missed or should I file a bug report?
Can this be because of your http check on https? Maybe you have to add ssl at the server as well? I have this option httpchk GET /swift/healthcheck .. server-template rgw2 1 _https._rgw2.prod.xxx ssl server-template rgw1 1 _https._rgw1.prod.xxx ssl
The config file for HAProxy is generated by Ceph and I think it should include "ssl verify none" on each backed line as the config use plain ip:port notation. What I wonder is if my yaml config for the RGW and Ingress miss something or if it is a bug in the HAProxy config file generator.
participants (2)
-
Jimmy Spets
-
Marc