On Mon, Jan 11, 2021 at 10:09 AM Gaƫl THEROND <gael.therond(a)bitswalk.com> wrote:
Hi Ilya,
Here is additional information:
My cluster is a three OSD Nodes cluster with each node having 24 4TB SSD disks.
The mkfs.xfs command fail with the following error:
https://pastebin.com/yTmMUtQs
I'm using the following command to format the image: mkfs.xfs
/dev/rbd/<pool_name>/<image_name>
I'm facing the same problem (and same sectors) if I'm directly targeting the
device with mkfs.xfs /dev/rbb<devMapID>
The client authentication caps are as follows:
https://pastebin.com/UuAHRycF
Regarding your questions, yes, it is a persistent issue as soon as I try to create a
large image from a newly created pool.
Yes, after the first attempt, all new attempts fail too.
Yes, it is always the same set of sectors that fails.
Have you tried writing to sector 0, just to take mkfs.xfs out of the
picture? E.g. "dd if=/dev/zero of=/dev/rbd17 bs=512 count=1 oflag=direct"?
Strange thing is, if I use an already existing pool, and create this 80Tb image within
this pool, it formats it correctly.
What do you mean by a newly created pool? A metadata pool, a data pool
or both?
Are you deleting and re-creating pools (whether metadata or data) with
the same name? It would help if you paste all commands, starting with
how you create pools all the way to a failing write.
Have you tried mapping using the admin user ("rbd map --id admin ...")?
Thanks,
Ilya