Folks, on two completely unrelated NVMe-only clusters, I am seeing detrimental performance for benchmarks with QD=256 and BS=4K: Where a comparable SAS-SSD cluster has 95k IOPS, with NVMes, I see only 7000 (one cluster) and 14000 (for the other one). I must be overseeing something very obvious here. What I have found out that „ceph tell osd bench“ for all OSDs is fine and within the expectations (50k+ IOPS) and network is fine (24,7Gbit/s). Benchmark is performed using „rados bench“ with „-t 256“ and -b 4K“. Total number of OSDs is 12. Dell Enterprise NVMes, 15.6TB per piece. During benchmarks, it looks as if only one OSD is even active on each of the nodes according to htop and iostat. The other ones appear to be inactive. I’m going nuts over this … What is going on here? :( Thank you very much in advance. M. -- Martin Gerhard Loschwitz Geschäftsführer / CEO, True West IT Services GmbH Phone: +49 2433 5253130 Mobile: +49 176 61832178 Address: Schmiedegasse 24a, 41836 Hückelhoven, Germany Legal: HRB 21985, Amtsgericht Mönchengladbach VAT: DE363893844 True West IT Services GmbH is compliant with the GDPR regulation on data protection and privacy in the European Union and the European Economic Area. You can request the information on how we collect and process your private data according to the law by contacting the email sender.