Re: Performance issue after tentacle upgrade
Hey Moti, Zitat von Moti via ceph-users <ceph-users@ceph.io>:
I recently upgraded from 18.2.4 to 20.2.1
I did notice this issue: https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/message/C5BXQ26KD3M... but that's not much similar to our case since we don't see any jump in CPU or memory usage.
Although the symptoms don't look the same, I'd recommend upgrading to v20.2.3 to get the fix for this issue and the other fixes that went in: https://docs.ceph.com/en/latest/releases/tentacle/#v20-2-3-tentacle After v20.2.3 upgrade, if you still see the same issue, then I'd suggest collecting "perf" output per process per component (rgw, osd mainly) to further analyze the bottleneck. You can also quickly check rgw logs to see what RGW reports as the latency for those PUT/GET/HEAD requests whether it matches the latency you see on your end (your wkld generator); especially for HEAD where no data streaming happens. RGW logs "start" and "end" of a request. This is to rule out whether bottleneck is outside of ceph (e.g., load-balancers). Cheers, Oguzhan
participants (1)
-
Oguzhan Ozmen (BLOOMBERG/ 120 PARK)