[ceph-users] mons excessive writes to local disk and SSD wearout

Hi, on our large ceph cluster with 60 servers, 1600 OSDs, we have observed that small system nvmes are wearing out rapidly. Our monitoring shows mon writes on average about 10MB/s to store.db. For small system nvmes of 250GB and DWPD of ~1, this turns out to be too much, 0.8TB/day or 1.5PB in 5 years, too much even for 3DWPD of the same capacity. Apart from replacing the drives with larger ones, more durable, preferably  both, do you have any suggestions if these writes can be reduced? Actually, the mon writes match 0.15Hz  rate of .sst file creation of 64MB.... Best regards, Andrej -- _____________________________________________________________ prof. dr. Andrej Filipcic, E-mail: Andrej.Filipcic(a)ijs.si Department of Experimental High Energy Physics - F9 Jozef Stefan Institute, Jamova 39, P.o.Box 3000 SI-1001 Ljubljana, Slovenia Tel.: +386-1-477-3674 Fax: +386-1-425-7074 -------------------------------------------------------------