yes, I used the same ecpool_hdd also for cephfs file systems. The new pool
ecpool_test I've created for a test, I've also created it with application
profile 'cephfs', but there aren't any cephfs filesystem attached to it.
root@zephir:~# ceph fs status
backups - 2 clients
=======
RANK STATE MDS ACTIVITY DNS INOS DIRS
CAPS
0 active backups.debian.runngh Reqs: 0 /s 253k 253k 21.3k
899
POOL TYPE USED AVAIL
cephfs.backups.meta metadata 1366M 2115G
cephfs.backups.data data 16.7T 16.4T
ecpool_hdd data 29.3T 29.6T
rgysi - 5 clients
=====
RANK STATE MDS ACTIVITY DNS INOS DIRS
CAPS
0 active rgysi.debian.uhgqen Reqs: 0 /s 409k 408k 40.8k 24.5k
POOL TYPE USED AVAIL
cephfs.rgysi.meta metadata 1453M 2115G
cephfs.rgysi.data data 4898G 17.6T
ecpool_hdd data 29.3T 29.6T
jellyfin - 1 clients
========
RANK STATE MDS ACTIVITY DNS INOS DIRS
CAPS
0 active jellyfin.debian.dcsocv Reqs: 0 /s 11.2k 10.9k 1935
1922
POOL TYPE USED AVAIL
cephfs.jellyfin.meta metadata 1076M 2115G
cephfs.jellyfin.data data 0 17.6T
ecpool_hdd data 29.3T 29.6T
STANDBY MDS
jellyfin.zephir.iqywsn
backups.zephir.ygigch
rgysi.zephir.diylss
MDS version: ceph version 17.2.6 (d7ff0d10654d2280e08f1ab989c7cdf3064446a5)
quincy (stable)
root@zephir:~#
I think I remember that I've read once something in documentation that
using the same pool for <x> could lead to ?naming? conflicts or something.
But later on I couldn't find it anymore and couldn't remember what <x> was,
and then I forgot about it.
So my understanding of the pull request is that I should migrate the cephfs
data from ecpool_hdd to a separate erasure code pool for cephfs and then
remove the 'cephfs' application tag from the ecpool_hdd pool, correct?
Am Mi., 19. Apr. 2023 um 09:37 Uhr schrieb Ilya Dryomov <idryomov(a)gmail.com
On Tue, Apr 18, 2023 at 11:34 PM Reto Gysi
<rlgysi(a)gmail.com> wrote:
Ah, yes indeed I had disabled log-to-stderr in cluster wide config.
root@zephir:~# rbd -p rbd snap create ceph-dev@backup --id admin
--debug-ms 1
--debug-rbd 20 --log-to-stderr=true >/home/rgysi/log.txt 2>&1
Hi Reto,
So "rbd snap create" is failing to allocate a snap ID:
2023-04-18T23:25:42.779+0200 7f4a8963a700 5
librbd::SnapshotCreateRequest: 0x7f4a68013ec0 send_allocate_snap_id
2023-04-18T23:25:42.779+0200 7f4a8963a700 1 --
192.168.1.1:0/1547580829 -->
[v2:192.168.1.10:3300/0,v1:192.168.1.10:6789/0] -- pool_op(create
unmanaged snap pool 37 tid 22 name v0) v4 -- 0x7f4a68017430 con
0x55637d589a60
2023-04-18T23:25:42.779+0200 7f4a7bfff700 1 --
192.168.1.1:0/1547580829 <== mon.1 v2:192.168.1.10:3300/0 6 ====
pool_op_reply(tid 22 (95) Operation not supported v72776) v1 ====
43+0+0 (secure 0 0 0) 0x7f4a80087080 con 0x55637d589a60
2023-04-18T23:25:42.779+0200 7f4a89e3b700 5
librbd::SnapshotCreateRequest: 0x7f4a68013ec0 handle_allocate_snap_id:
r=-95, snap_id=18446744073709551614
It's most likely coming from
https://github.com/ceph/ceph/pull/47753
(which was backported to 17.2.6, this explains why it showed up after
the upgrade). The fact that both the old and the new EC pools have
a cephfs application tag instead of just rbd is suspicious:
pool 37 'ecpool_hdd' erasure profile 3-2-jerasure size 5 min_size 4
crush_rule 5 object_hash rjenkins pg_num 128 pgp_num 128
autoscale_mode on last_change 72385 lfor 0/0/65311 flags
hashpspool,ec_overwrites,selfmanaged_snaps stripe_width 12288
compression_algorithm lz4 compression_mode aggressive
compression_required_ratio 0.875 application cephfs,rbd
pool 87 'ecpool_test' erasure profile 3-2-jerasure size 5 min_size 4
crush_rule 9 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode on
last_change 72720 flags hashpspool,ec_overwrites,selfmanaged_snaps
stripe_width 12288 compression_algorithm lz4 compression_mode
aggressive compression_required_ratio 0.825 application cephfs,rbd
Do you recall attaching either of these to a filesystem?
Thanks,
Ilya