1 Oct
2020
1 Oct
'20
12:37 p.m.
Hi,
on octopus 15.2.4 I have an issue with cephfs tag auth. The following
works fine:
client.f9desktop
key: ....
caps: [mds] allow rw
caps: [mon] allow r
caps: [osd] allow rw pool=cephfs_data, allow rw pool=ssd_data,
allow rw pool=fast_data, allow rw pool=arich_data, allow rw
pool=ecfast_data
but this one does not.
client.f9desktopnew
key: ....
caps: [mds] allow rw
caps: [mon] allow r
caps: [osd] allow rw tag cephfs data=cephfs
For the 2nd, mds works, files can be created or removed, but client
read/write (native client, kernel version 5.7.4) fails with I/O error,
so osd part does not seem to be working properly.
Any clues what can be wrong? the cephfs was created in jewel...
Another issue is: if osd caps are updated (adding data pool), then some
clients refresh the caps, but most of them do not, and the only way to
refresh it is to remount the filesystem. working tag would solve it.
Best regards,
Andrej
--
_____________________________________________________________
prof. dr. Andrej Filipcic, E-mail: Andrej.Filipcic(a)ijs.si
Department of Experimental High Energy Physics - F9
Jozef Stefan Institute, Jamova 39, P.o.Box 3000
SI-1001 Ljubljana, Slovenia
Tel.: +386-1-477-3674 Fax: +386-1-477-3166
-------------------------------------------------------------
1 Oct
1 Oct
1:33 p.m.
Hi,
I have a one-node-cluster (also 15.2.4) for testing purposes and just
created a cephfs with the tag, it works for me. But my node is also
its own client, so there's that. And it was installed with 15.2.4, no
upgrade.
For the 2nd, mds works, files can be created or
removed, but client
read/write (native client, kernel version 5.7.4) fails with I/O
error, so osd part does not seem to be working properly.
You mean it works if you mount it from a different host (within the
cluster maybe) with the new client's key but it doesn't work with the
designated clients? I'm not sure about the OSD part since the other
syntax seems to work, you say.
Can you share more details about the error? The mount on the clients
works but they can't read/write?
Regards,
Eugen
Zitat von Andrej Filipcic <andrej.filipcic(a)ijs.si>si>:
> Hi,
>
> on octopus 15.2.4 I have an issue with cephfs tag auth. The
> following works fine:
>
> client.f9desktop
> key: ....
> caps: [mds] allow rw
> caps: [mon] allow r
> caps: [osd] allow rw pool=cephfs_data, allow rw
> pool=ssd_data, allow rw pool=fast_data, allow rw pool=arich_data,
> allow rw pool=ecfast_data
>
> but this one does not.
>
> client.f9desktopnew
> key: ....
> caps: [mds] allow rw
> caps: [mon] allow r
> caps: [osd] allow rw tag cephfs data=cephfs
>
For the 2nd, mds works, files can be created or
removed, but client
read/write (native client, kernel version 5.7.4) fails with I/O
error, so osd part does not seem to be working properly.
>
> Any clues what can be wrong? the cephfs was created in jewel...
>
> Another issue is: if osd caps are updated (adding data pool), then
> some clients refresh the caps, but most of them do not, and the only
> way to refresh it is to remount the filesystem. working tag would
> solve it.
>
> Best regards,
> Andrej
>
> --
> _____________________________________________________________
> prof. dr. Andrej Filipcic, E-mail: Andrej.Filipcic(a)ijs.si
> Department of Experimental High Energy Physics - F9
> Jozef Stefan Institute, Jamova 39, P.o.Box 3000
> SI-1001 Ljubljana, Slovenia
> Tel.: +386-1-477-3674 Fax: +386-1-477-3166
> -------------------------------------------------------------
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
1:56 p.m.
There used to be / is a bug in ceph fs commands when using data pools. If you enable the
application cephfs on a pool explicitly before running cephfs add datapool, the fs-tag is
not applied. Maybe its that? There is an older thread on the topic in the users-list and
also a fix/workaround.
Best regards,
=================
Frank Schilder
AIT Risø Campus
Bygning 109, rum S14
________________________________________
From: Eugen Block <eblock(a)nde.ag>
Sent: 01 October 2020 15:33:53
To: ceph-users(a)ceph.io
Subject: [ceph-users] Re: cephfs tag not working
Hi,
I have a one-node-cluster (also 15.2.4) for testing purposes and just
created a cephfs with the tag, it works for me. But my node is also
its own client, so there's that. And it was installed with 15.2.4, no
upgrade.
For the 2nd, mds works, files can be created or
removed, but client
read/write (native client, kernel version 5.7.4) fails with I/O
error, so osd part does not seem to be working properly.
You mean it works if you mount it from a different host (within the
cluster maybe) with the new client's key but it doesn't work with the
designated clients? I'm not sure about the OSD part since the other
syntax seems to work, you say.
Can you share more details about the error? The mount on the clients
works but they can't read/write?
Regards,
Eugen
Zitat von Andrej Filipcic <andrej.filipcic(a)ijs.si>si>:
> Hi,
>
> on octopus 15.2.4 I have an issue with cephfs tag auth. The
> following works fine:
>
> client.f9desktop
> key: ....
> caps: [mds] allow rw
> caps: [mon] allow r
> caps: [osd] allow rw pool=cephfs_data, allow rw
> pool=ssd_data, allow rw pool=fast_data, allow rw pool=arich_data,
> allow rw pool=ecfast_data
>
> but this one does not.
>
> client.f9desktopnew
> key: ....
> caps: [mds] allow rw
> caps: [mon] allow r
> caps: [osd] allow rw tag cephfs data=cephfs
>
For the 2nd, mds works, files can be created or
removed, but client
read/write (native client, kernel version 5.7.4) fails with I/O
error, so osd part does not seem to be working properly.
>
> Any clues what can be wrong? the cephfs was created in jewel...
>
> Another issue is: if osd caps are updated (adding data pool), then
> some clients refresh the caps, but most of them do not, and the only
> way to refresh it is to remount the filesystem. working tag would
> solve it.
>
> Best regards,
> Andrej
>
> --
> _____________________________________________________________
> prof. dr. Andrej Filipcic, E-mail: Andrej.Filipcic(a)ijs.si
> Department of Experimental High Energy Physics - F9
> Jozef Stefan Institute, Jamova 39, P.o.Box 3000
> SI-1001 Ljubljana, Slovenia
> Tel.: +386-1-477-3674 Fax: +386-1-477-3166
> -------------------------------------------------------------
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
5:58 p.m.
On Thu, Oct 1, 2020 at 6:57 AM Frank Schilder <frans(a)dtu.dk> wrote:
There used to be / is a bug in ceph fs commands when using data pools. If you enable the
application cephfs on a pool explicitly before running cephfs add datapool, the fs-tag is
not applied. Maybe its that? There is an older thread on the topic in the users-list and
also a fix/workaround.
This is likely to be the problem. Please add the application tag to
your CephFS data pools:
https://docs.ceph.com/en/latest/rados/operations/pools/#associate-pool-to-a…
--
Patrick Donnelly, Ph.D.
He / Him / His
Principal Software Engineer
Red Hat Sunnyvale, CA
GPG: 19F28A586F808C2402351B93C3301A3E258DD79D
6:15 p.m.
On 2020-10-01 15:56, Frank Schilder wrote:
There used to be / is a bug in ceph fs commands when
using data pools. If you enable the application cephfs on a pool explicitly before running
cephfs add datapool, the fs-tag is not applied. Maybe its that? There is an older thread
on the topic in the users-list and also a fix/workaround.
Thanks, found it, that was
it. I enabled application before adding the
pool. Only the latest pool had the cephfs key/value in. The fix worked.
Best regards,
Andrej
Best regards,
=================
Frank Schilder
AIT Risø Campus
Bygning 109, rum S14
________________________________________
From: Eugen Block <eblock(a)nde.ag>
Sent: 01 October 2020 15:33:53
To: ceph-users(a)ceph.io
Subject: [ceph-users] Re: cephfs tag not working
Hi,
I have a one-node-cluster (also 15.2.4) for testing purposes and just
created a cephfs with the tag, it works for me. But my node is also
its own client, so there's that. And it was installed with 15.2.4, no
upgrade.
--
_____________________________________________________________
prof. dr. Andrej Filipcic, E-mail: Andrej.Filipcic(a)ijs.si
Department of Experimental High Energy Physics - F9
Jozef Stefan Institute, Jamova 39, P.o.Box 3000
SI-1001 Ljubljana, Slovenia
Tel.: +386-1-477-3674 Fax: +386-1-425-7074
-------------------------------------------------------------
For the 2nd, mds works, files can be created or
removed, but client
read/write (native client, kernel version 5.7.4) fails with I/O
error, so osd part does not seem to be working properly.
You mean it works if you
mount it from a different host (within the
cluster maybe) with the new client's key but it doesn't work with the
designated clients? I'm not sure about the OSD part since the other
syntax seems to work, you say.
Can you share more details about the error? The mount on the clients
works but they can't read/write?
Regards,
Eugen
Zitat von Andrej Filipcic <andrej.filipcic(a)ijs.si>si>:
> Hi,
>
> on octopus 15.2.4 I have an issue with cephfs tag auth. The
> following works fine:
>
> client.f9desktop
> key: ....
> caps: [mds] allow rw
> caps: [mon] allow r
> caps: [osd] allow rw pool=cephfs_data, allow rw
> pool=ssd_data, allow rw pool=fast_data, allow rw pool=arich_data,
> allow rw pool=ecfast_data
>
> but this one does not.
>
> client.f9desktopnew
> key: ....
> caps: [mds] allow rw
> caps: [mon] allow r
> caps: [osd] allow rw tag cephfs data=cephfs
>
For the 2nd, mds works, files can be created or
removed, but client
read/write (native client, kernel version 5.7.4) fails with I/O
error, so osd part does not seem to be working properly.
>
> Any clues what can be wrong? the cephfs was created in jewel...
>
> Another issue is: if osd caps are updated (adding data pool), then
> some clients refresh the caps, but most of them do not, and the only
> way to refresh it is to remount the filesystem. working tag would
> solve it.
>
> Best regards,
> Andrej
>
> --
> _____________________________________________________________
> prof. dr. Andrej Filipcic, E-mail: Andrej.Filipcic(a)ijs.si
> Department of Experimental High Energy Physics - F9
> Jozef Stefan Institute, Jamova 39, P.o.Box 3000
> SI-1001 Ljubljana, Slovenia
> Tel.: +386-1-477-3674 Fax: +386-1-477-3166
> -------------------------------------------------------------
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
1302
days inactive
1302
days old
4 comments
4 participants
participants (4)
-
Andrej Filipcic -
Eugen Block -
Frank Schilder -
Patrick Donnelly