Hi, I would appreciate any help/hints to solve this issue iscis (gwcli) cannot see the images anymore This configuration worked fine for many months What changed was that ceph is "nearly full" I am in the process of cleaning it up ( by deleting objects from one of the pools) and I do see reads and writes on the cluster as well as images info so not sure what gwcli does not like ( targetcli ls not working either - just froze ) Below some info ceph version ceph version 13.2.2 (02899bfda814146b021136e9d8e80eba494e1126) mimic (stable) gwcli --version gwcli - 2.7 ceph osd dump | grep ratio full_ratio 0.96 backfillfull_ratio 0.92 nearfull_ratio 0.9 [root@osd02 ~]# rbd -p rbd info rep01 rbd image 'rep01': size 7 TiB in 1835008 objects order 22 (4 MiB objects) id: 15b366b8b4567 block_name_prefix: rbd_data.15b366b8b4567 format: 2 features: layering, exclusive-lock, object-map, fast-diff, deep-flatten op_features: flags: create_timestamp: Thu Nov 1 15:57:52 2018 [root@osd02 ~]# rbd -p rbd info vmware01 rbd image 'vmware01': size 6 TiB in 1572864 objects order 22 (4 MiB objects) id: 16d3f6b8b4567 block_name_prefix: rbd_data.16d3f6b8b4567 format: 2 features: layering, exclusive-lock, object-map, fast-diff, deep-flatten op_features: flags: create_timestamp: Thu Nov 29 13:56:28 2018 [root@osd02 ~]# ceph df GLOBAL: SIZE AVAIL RAW USED %RAW USED 33 TiB 7.5 TiB 25 TiB 77.16 POOLS: NAME ID USED %USED MAX AVAIL OBJECTS cephfs_metadata 22 173 MiB 0.01 1.4 TiB 469 cephfs_data 23 1.7 TiB 69.78 775 GiB 486232 rbd 24 11 TiB 93.74 775 GiB 2974077 [root@osd02 ~]# ceph health detail HEALTH_ERR 2 nearfull osd(s); 2 pool(s) nearfull; Module 'prometheus' has failed: IOError("Port 9283 not free on '10.10.35.20'",) OSD_NEARFULL 2 nearfull osd(s) osd.12 is near full osd.17 is near full POOL_NEARFULL 2 pool(s) nearfull pool 'cephfs_data' is nearfull pool 'rbd' is nearfull gwcli /iscsi-target...nner-21faa413> info Client Iqn .. iqn.1998-01.com.vmware:banner-21faa413 Ip Address .. Alias .. Logged In .. Auth - chap .. cephuser/PASSWORD Group Name .. Luns - rbd.rep01 .. lun_id=0 - rbd.vmware01 .. lun_id=1 osd02 journal: client update failed on iqn.1998-01.com.vmware:banner-21faa413 : Non-existent images ['rbd.vmware01'] requested for iqn.1998-01.com.vmware:banner-21faa413 Aug 7 14:15:39 osd02 journal: 127.0.0.1 - - [07/Aug/2020 14:15:39] "PUT /api/_clientlun/iqn.1998-01.com.vmware:banner-21faa413 HTTP/1.1" 500 - Aug 7 14:15:39 osd02 journal: _clientlun change on 127.0.0.1 failed with 500 Aug 7 14:15:39 osd02 journal: 127.0.0.1 - - [07/Aug/2020 14:15:39] "DELETE /api/clientlun/iqn.1998-01.com.vmware:banner-21faa413 HTTP/1.1" 500 -