Hello, is it possible to restart the rbd-target-api without restarting the entire container? Ceph version: ceph version 17.2.6 (d7ff0d10654d2280e08f1ab989c7cdf3064446a5) root@vxxx-sx-xx-iscsi0:~# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 949dabe059eb quay.io/ceph/ceph "/usr/bin/rbd-target…" 2 months ago Up 2 months ceph-c404fafe-767c-11ee-bc37-0509d00921ba-iscsi-sx-xx-vxxxgw-pool0-vxxx-sx-xx-iscsi0-kwjqrn 43a298fe835b quay.io/ceph/ceph "/usr/bin/tcmu-runner" 2 months ago Up 2 months ceph-c404fafe-767c-11ee-bc37-0509d00921ba-iscsi-sx-xx-vxxxgw-pool0-vxxx-sx-xx-iscsi0-kwjqrn-tcmu 1eef0e6084a2 quay.io/prometheus/node-exporter:v1.5.0 "/bin/node_exporter …" 6 months ago Up 6 months ceph-c404fafe-767c-11ee-bc37-0509d00921ba-node-exporter-vxxx-sx-xx-iscsi0 4d94c24cebec quay.io/ceph/ceph "/usr/bin/ceph-crash…" 6 months ago Up 6 months ceph-c404fafe-767c-11ee-bc37-0509d00921ba-crash-vxxx-sx-xx-iscsi0 root@vxxx-sx-xx-iscsi0:~# docker exec -it 949dabe059eb /bin/bash [root@vxxx-sx-xx-iscsi0/]# ps auxf USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 134572 0.2 0.0 14152 3284 pts/4 Ss 13:42 0:00 /bin/bash root 134590 0.0 0.0 46800 3460 pts/4 R+ 13:42 0:00 \_ ps auxf root 1 0.0 0.0 1020 676 ? Ss Jul14 6:09 /sbin/docker-init -- /usr/bin/rbd-target-api root 8 0.1 0.3 3785056 253040 ? Sl Jul14 152:47 /usr/bin/python3.6 -s /usr/bin/rbd-target-api Best Regards, Laszlo Kardos -----Original Message----- From: Anthony D'Atri <aad@dreamsnake.net> Sent: Tuesday, September 30, 2025 6:05 PM To: Laszlo Budai <laszlo@componentsoft.eu> Cc: Kardos László <laszlo.kardos@acetelecom.hu>; ceph-users@ceph.io Subject: [ceph-users] Re: Ceph GWCLI issue
The PG numbers are still very low in my opinion. you have 42 OSDs and only 614 PGs that makes roughly 15 PG / OSD. That's quite far from the rule of thumb of 100 PG/OSD.
I've been trying to clear up this nuance in the docs when I can. The PGS field in `ceph osd df` and the target value (aka "PG ratio") are for PG replicas, not PGs, so one has to factor in replication. For EC, the replication factor is k+m. For a cluster with one pool: pg_num = (#OSDs * ratio) / replication ratio = (pg_num * replication) / #OSDs Round to the nearest power of 2, if in doubt round up. When you have multiple pools it gets more complicated. One can use the pgcalc, or leverage the PG autoscaler. That said the default target of 100 is way too low, especially since it's a max not a target as such. global advanced mon_max_pg_per_osd 600 global advanced mon_target_pg_per_osd 300
But maybe your problem is located in a different place. You may want to check whether all your `rbd-target-api` services are up and running. gwcli relies on them.
Kind regards, Laszlo Budai
On 9/30/25 10:31, Kardos László wrote:
Hello, I apologize for sending the wrong pool details earlier. We store the data in the following data pool: xxxx0-data
pool 15 'xxxx0-data' erasure profile laurel_ec size 4 min_size 3 crush_rule
If this is a 3+1 pool, note that a value of m=1 is ... fraught If this is a 2+2 pool, note that with current releases, EC for RBD is usually a significant latency liability. Tentacle's fast EC improves that dynamic.
8 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 30830 lfor 0/0/30825 flags hashpspool,ec_overwrites,selfmanaged_snaps stripe_width 12288 application rbd,rgw
cluster: id: c404fafe-767c-11ee-bc37-0509d00921ba health: HEALTH_OK
services: mon: 5 daemons, quorum v188-ceph-mgr0,v188-ceph-mgr1,v188-ceph-iscsigw2,v188-ceph6,v188-ceph 5 (age 5d) mgr: v188-ceph-mgr0.rxcecw(active, since 11w), standbys: v188-ceph-mgr1.hmbuma mds: 1/1 daemons up, 1 standby osd: 42 osds: 42 up (since 2M), 42 in (since 3M) tcmu-runner: 10 portals active (4 hosts)
data: volumes: 1/1 healthy pools: 11 pools, 614 pgs objects: 13.63M objects, 51 TiB usage: 75 TiB used, 71 TiB / 147 TiB avail pgs: 613 active+clean 1 active+clean+scrubbing+deep
io: client: 8.1 MiB/s rd, 105 MiB/s wr, 320 op/s rd, 2.31k op/s wr
Best Regards, Laszlo Kardos
-----Original Message----- From: Eugen Block<eblock@nde.ag> Sent: Tuesday, September 30, 2025 9:03 AM To:ceph-users@ceph.io Subject: [ceph-users] Re: Ceph GWCLI issue
Hi,
I don't have an answer why the image is in unknown state, but I'd be concerned about the pool's pg_num. You have Terabytes in a pool with a single PG? That's awful and should be increased to a more suitable value. I can't say if that would fix anything regarding the unknown issue, but that's definitely not good at all.
What is the overall Ceph status (ceph -s)?
Regards, Eugen
Zitat von Kardos László<laszlo.kardos@acetelecom.hu>:
Hello,
We have encountered the following issue in our production environment:
A new RBD Image was created within an existing pool, and its status is reported as "unknown" in GWCLI. Based on our tests, this does not appear to cause operational issues, but we would like to investigate the root cause. No relevant information regarding this issue was found in the logs.
GWCLI output:
o- / .......................................................................... ............................................... [...]
o- cluster .......................................................................... ............................... [Clusters: 1]
| o- ceph .......................................................................... .................................. [HEALTH_OK]
| o- pools .......................................................................... ............................... [Pools: 11]
| | o- .mgr ................................................................ [(x3), Commit: 0.00Y/15591725M (0%), Used: 194124K]
| | o- .nfs ................................................................. [(x3), Commit: 0.00Y/15591725M (0%), Used: 16924b]
| | o- xxxx-test ............................................................. [(2+1), Commit: 0.00Y/23727198M (0%), Used: 0.00Y]
| | o- xxxxx-erasure-0 ............................................ [(2+1), Commit: 0.00Y/23727198M (0%), Used: 61519257668K]
| | o- xxxxxx-repl ...................................................... [(x3), Commit: 0.00Y/15591725M (0%), Used: 130084b]
| | o- cephfs.cephfs-test.data ............................................ [(x3), Commit: 0.00Y/15591725M (0%), Used: 9090444K]
| | o- cephfs.cephfs-test.meta .......................................... [(x3), Commit: 0.00Y/15591725M (0%), Used: 516415713b]
| | o- xxxxx-data ..................................................... [(3+1), Commit: 0.00Y/9604386M (0%), Used: 7547753556K]
| | o- xxxxx-rpl .......................................................... [(x3), Commit: 12.0T/4268616M (294%), Used: 85265b]
| | o- xxxxx-data ................................................... [(3+1), Commit: 0.00Y/5011626M (0%), Used: 10955179612K]
| | o- replicated_xxxx ............................................... [(x3), Commit: 25.0T/2280846592K (1176%), Used: 46912b]
| o- topology .......................................................................... ..................... [OSDs: 42,MONs: 5]
o- disks .......................................................................... ............................. [37.0T, Disks: 3]
| o- xxxx-rpl .......................................................................... ................... [xxxx-rpl (12.0T)]
| | o- xxxxx_lun0 ........................................................................ [xxxx-rpl/xxxxx_lun0 (Online, 12.0T)]
| o- replicated_xxxx .......................................................................... ..... [replicated_xxxx (25.0T)]
| o- xxxx_lun0 ............................................................... [replicated_xxxx/xxxx_lun0 (Online, 12.0T)]
| o- xxxx_lun_new ........................................................ [replicated_xxxx/xxxx_lun_new (Unknown, 13.0T)]
The image (xxxx_lun_new) is provisioned to multiple ESXi hosts, mounted, and formatted with VMFS6. The datastore is writable and readable by the hosts.
There is a change in the block size of the RBD Image: the older RBD Images use a 4 MiB block size, while the new RBD Image uses a 512 KiB block size.
RBD Image Parameters:
For replicated_xxxx / xxxx_lun0 (Online status in GWCLI):
rbd image 'xxxx_lun0':
size 12 TiB in 3145728 objects
order 22 (4 MiB objects)
snapshot_count: 0
id: 5c1b5ecfdfa46
data_pool: xxxx0-data
block_name_prefix: rbd_data.14.5c1b5ecfdfa46
format: 2
features: exclusive-lock, data-pool
op_features:
flags:
create_timestamp: Tue Jul 8 13:02:11 2025
access_timestamp: Thu Sep 25 13:49:47 2025
modify_timestamp: Thu Sep 25 13:50:05 2025
For replicated_xxxx / xxxx_lun_new (Unknown status in GWCLI):
rbd image 'xxxx_lun_new': size 13 TiB in 27262976 objects order 19 (512 KiB objects) snapshot_count: 0 id: 1945d9cf9f41ab data_pool: xxxx0-data block_name_prefix: rbd_data.14.1945d9cf9f41ab format: 2 features: exclusive-lock, data-pool op_features: flags: create_timestamp: Wed Sep 24 11:21:21 2025 access_timestamp: Thu Sep 25 13:50:42 2025 modify_timestamp: Thu Sep 25 13:49:48 2025
Pool Parameters:
pool 14 'replicated_xxxx' replicated size 3 min_size 2 crush_rule 7 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode on last_change 30743 flags hashpspool stripe_width 0 application rbd,rgw
Ceph version:
ceph --version
ceph version 17.2.6 (d7ff0d10654d2280e08f1ab989c7cdf3064446a5) quincy (stable)
Question:
What could be causing the RBD Image (xxxx_lun_new) to appear in an "unknown" state in GWCLI?
_______________________________________________ ceph-users mailing list --ceph-users@ceph.io To unsubscribe send an email toceph-users-leave@ceph.io _______________________________________________ ceph-users mailing list --ceph-users@ceph.io To unsubscribe send an email toceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io