16 Dec
2020
16 Dec
'20
7:54 a.m.
Oops.. sent this to the "wrong" list previously. (lists.ceph.com)
Lets try the proper one this time :-/
not sure if this an actual bug or I'm doing something else wrong. but in Octopus, I
have on the master node
# ceph --version
ceph version 15.2.7 (88e41c6c49beb18add4fdb6b4326ca466d931db8) octopus (stable)
# tail -3 /etc/ceph/ceph.conf
[client]
rbd cache = false
rbd cache writethrough until flush = false
but I have restarted ALL nodes.. and yet rbd cache is still on.
# ceph --admin-daemon `find /var/run/ceph -name 'ceph-mon*'` config show |grep
rbd_cache
"rbd_cache": "true",
What else am I supposed to do???
--
Philip Brown| Sr. Linux System Administrator | Medata, Inc.
5 Peters Canyon Rd Suite 250
Irvine CA 92606
Office 714.918.1310| Fax 714.918.1325
pbrown(a)medata.com| www.medata.com
17 Dec
17 Dec
4:21 a.m.
Hi,
[client]
rbd cache = false
rbd cache writethrough until flush = false
this is the rbd client's config, not the global MON config you're
reading here:
# ceph --admin-daemon `find /var/run/ceph -name
'ceph-mon*'` config
show |grep rbd_cache
"rbd_cache": "true",
If you want to change the global "rbd_cache" setting you have to
change that in the [global] (or maybe [mon]) section.
But I'm not sure which effect this will have.
Regards,
Eugen
Zitat von Philip Brown <pbrown(a)medata.com>om>:
> Oops.. sent this to the "wrong" list previously. (lists.ceph.com)
> Lets try the proper one this time :-/
>
> not sure if this an actual bug or I'm doing something else wrong.
> but in Octopus, I have on the master node
>
>
> # ceph --version
> ceph version 15.2.7 (88e41c6c49beb18add4fdb6b4326ca466d931db8)
> octopus (stable)
> # tail -3 /etc/ceph/ceph.conf
[client]
rbd cache = false
rbd cache writethrough until flush = false
>
>
> but I have restarted ALL nodes.. and yet rbd cache is still on.
>
# ceph --admin-daemon `find /var/run/ceph -name
'ceph-mon*'` config
show |grep rbd_cache
"rbd_cache": "true",
>
>
> What else am I supposed to do???
>
>
>
>
> --
> Philip Brown| Sr. Linux System Administrator | Medata, Inc.
> 5 Peters Canyon Rd Suite 250
> Irvine CA 92606
> Office 714.918.1310| Fax 714.918.1325
> pbrown(a)medata.com| www.medata.com
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
5:09 a.m.
On Thu, Dec 17, 2020 at 7:22 AM Eugen Block <eblock(a)nde.ag> wrote:
Hi,
If you want to change global (librbd) settings, please use "rbd config
global set ..." (which is just a wrapper around "ceph config set
global ..."). The conf files only apply to the single host where they
exist.
[client]
rbd cache = false
rbd cache writethrough until flush = false
this is the rbd client's config, not the global MON config you're
reading here:
# ceph --admin-daemon `find /var/run/ceph -name
'ceph-mon*'` config
show |grep rbd_cache
"rbd_cache": "true",
If you want to change the global "rbd_cache" setting you have to
change that in the [global] (or maybe [mon]) section.
But I'm not sure which effect this will have. Regards,
Eugen
Zitat von Philip Brown <pbrown(a)medata.com>om>:
--
Jason
Oops.. sent this to the "wrong" list
previously. (lists.ceph.com)
Lets try the proper one this time :-/
not sure if this an actual bug or I'm doing something else wrong.
but in Octopus, I have on the master node
# ceph --version
ceph version 15.2.7 (88e41c6c49beb18add4fdb6b4326ca466d931db8)
octopus (stable)
# tail -3 /etc/ceph/ceph.conf
[client]
rbd cache = false
rbd cache writethrough until flush = false
but I have restarted ALL nodes.. and yet rbd cache is still on.
# ceph --admin-daemon `find /var/run/ceph -name 'ceph-mon*'` config
show |grep rbd_cache
"rbd_cache": "true",
What else am I supposed to do???
--
Philip Brown| Sr. Linux System Administrator | Medata, Inc.
5 Peters Canyon Rd Suite 250
Irvine CA 92606
Office 714.918.1310| Fax 714.918.1325
pbrown(a)medata.com| www.medata.com
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
7:40 a.m.
Huhhh...
Its unfortunate that every google search i did for turning off rbd cache, specified
"put it in the [client] section".
Doh.
Maybe this would make a good candidate to update the ceph rbd docs?
Speaking of which.. what is the *exact* syntax for that command please?
None of the below work:
rbd config global set rbd_cache false
rbd config global set rbd cache false
rbd config global set conf_rbd_cache false
rbd config global set rbd cache false
rbd config global set global cache false
and the error messages are not helpful. for example,
# rbd config global set cache false
rbd: invalid config entity: cache (must be global, client or client.<id>)
??? I already specified global??
but then
# rbd config global set global cache false
rbd: not rbd option: cache
Very frustrating.
----- Original Message -----
From: "Jason Dillaman" <jdillama(a)redhat.com>
To: "Eugen Block" <eblock(a)nde.ag>
Cc: "ceph-users" <ceph-users(a)ceph.io>
Sent: Thursday, December 17, 2020 5:09:17 AM
Subject: [ceph-users] Re: bug? cant turn off rbd cache?
If you want to change global (librbd) settings, please use "rbd config
global set ..." (which is just a wrapper around "ceph config set
global ..."). The conf files only apply to the single host where they
exist.
7:48 a.m.
On Thu, Dec 17, 2020 at 10:41 AM Philip Brown <pbrown(a)medata.com> wrote:
Huhhh...
Its unfortunate that every google search i did for turning off rbd cache, specified
"put it in the [client] section".
Doh.
Maybe this would make a good candidate to update the ceph rbd docs?
As an open source project, I highly encourage you to give back in any
way you can [1].
Speaking of which.. what is the *exact* syntax for
that command please?
None of the below work:
rbd config global set rbd_cache false
rbd config global set rbd cache false
rbd config global set conf_rbd_cache false
rbd config global set rbd cache false
rbd config global set global cache false
and the error messages are not helpful. for example,
# rbd config global set cache false
rbd: invalid config entity: cache (must be global, client or client.<id>)
??? I already specified global??
... because a global configuration override can apply to all entities,
to all clients, or to specific clients (hence the examples for the
entity parameter).
but then
# rbd config global set global cache false
rbd: not rbd option: cache
... the configuration option is "rbd_cache" as documented here [2].
Very frustrating.
----- Original Message -----
From: "Jason Dillaman" <jdillama(a)redhat.com>
To: "Eugen Block" <eblock(a)nde.ag>
Cc: "ceph-users" <ceph-users(a)ceph.io>
Sent: Thursday, December 17, 2020 5:09:17 AM
Subject: [ceph-users] Re: bug? cant turn off rbd cache?
If you want to change global (librbd) settings, please use "rbd config
global set ..." (which is just a wrapper around "ceph config set
global ..."). The conf files only apply to the single host where they
exist.
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
[1] https://docs.ceph.com/en/latest/start/documenting-ceph/
[2] https://docs.ceph.com/en/latest/rbd/rbd-config-ref/#cache-settings
--
Jason
8:21 a.m.
I guess I left out in my examples, where I tried rbd_cache as well, and failed
# rbd config global set rbd_cache false
rbd: invalid config entity: rbd_cache (must be global, client or client.<id>)
So, while i am happy to file a documentation pull request.. I still need to find the
specific command line that actually *works*, for the "rbd config" variant, etc.
----- Original Message -----
From: "Jason Dillaman" <jdillama(a)redhat.com>
To: "Philip Brown" <pbrown(a)medata.com>
Cc: "ceph-users" <ceph-users(a)ceph.io>
Sent: Thursday, December 17, 2020 7:48:22 AM
Subject: Re: [ceph-users] Re: bug? cant turn off rbd cache?
On Thu, Dec 17, 2020 at 10:41 AM Philip Brown <pbrown(a)medata.com> wrote:
Huhhh...
Its unfortunate that every google search i did for turning off rbd cache, specified
"put it in the [client] section".
Doh.
Maybe this would make a good candidate to update the ceph rbd docs?
As an open source project, I highly encourage you to give back in any
way you can [1].
Speaking of which.. what is the *exact* syntax for
that command please?
None of the below work:
rbd config global set rbd_cache false
rbd config global set rbd cache false
rbd config global set conf_rbd_cache false
rbd config global set rbd cache false
rbd config global set global cache false
and the error messages are not helpful. for example,
# rbd config global set cache false
rbd: invalid config entity: cache (must be global, client or client.<id>)
??? I already specified global??
... because a global configuration override can apply to all entities,
to all clients, or to specific clients (hence the examples for the
entity parameter).
but then
# rbd config global set global cache false
rbd: not rbd option: cache
... the configuration option is "rbd_cache" as documented here [2].
>
>
> Very frustrating.
>
>
>
8:24 a.m.
On Thu, Dec 17, 2020 at 11:21 AM Philip Brown <pbrown(a)medata.com> wrote:
I guess I left out in my examples, where I tried rbd_cache as well, and failed
# rbd config global set rbd_cache false
rbd: invalid config entity: rbd_cache (must be global, client or client.<id>)
But that's not a valid command -- you attempted to provide the
configuration key name for the entity (as the error message is telling
you).
$ rbd config global set global rbd_cache false
... or ...
$ rbd config global set client rbd_cache false
... or ...
$ rbd config global set client.host_a rbd_cache false
So, while i am happy to file a documentation pull request.. I still need to find the
specific command line that actually *works*, for the "rbd config" variant, etc.
----- Original Message -----
From: "Jason Dillaman" <jdillama(a)redhat.com>
To: "Philip Brown" <pbrown(a)medata.com>
Cc: "ceph-users" <ceph-users(a)ceph.io>
Sent: Thursday, December 17, 2020 7:48:22 AM
Subject: Re: [ceph-users] Re: bug? cant turn off rbd cache?
On Thu, Dec 17, 2020 at 10:41 AM Philip Brown <pbrown(a)medata.com> wrote:
--
Jason
Huhhh...
Its unfortunate that every google search i did for turning off rbd cache, specified
"put it in the [client] section".
Doh.
Maybe this would make a good candidate to update the ceph rbd docs?
As an open source project, I highly encourage you to give back in any
way you can [1].
Speaking of which.. what is the *exact* syntax
for that command please?
None of the below work:
rbd config global set rbd_cache false
rbd config global set rbd cache false
rbd config global set conf_rbd_cache false
rbd config global set rbd cache false
rbd config global set global cache false
and the error messages are not helpful. for example,
# rbd config global set cache false
rbd: invalid config entity: cache (must be global, client or client.<id>)
??? I already specified global??
... because a global configuration override can apply to all entities,
to all clients, or to specific clients (hence the examples for the
entity parameter).
but then
# rbd config global set global cache false
rbd: not rbd option: cache
... the configuration option is "rbd_cache" as documented here [2].
Very frustrating.
9:08 a.m.
Huhhh..
It seems worthwhile to point out two inconsistencies, then.
1. the "old way", of ceph config set global rbd_cache false
doesnt require this odd redundant "global set global" syntax. It is confusing to
users to have to specify "global" twice.
may I suggest that the syntax for rbd config be adjusted, so that people can simply use
rbd config global ...
or
rbd config client ...
not
rbd config global set (global/client)
2. The fact that "ceph config set global rbd_cache false" works, seems to imply
that putting
[global]
rbd cache = false
in /etc/ceph/ceph.conf should work also.
Except it doesnt.
Even after fully shutting down every node in the ceph cluster and doing a cold startup.
is that a bug?
----- Original Message -----
From: "Jason Dillaman" <jdillama(a)redhat.com>
To: "Philip Brown" <pbrown(a)medata.com>
Cc: "dillaman" <dillaman(a)redhat.com>om>, "ceph-users"
<ceph-users(a)ceph.io>
Sent: Thursday, December 17, 2020 8:24:59 AM
Subject: Re: [ceph-users] Re: bug? cant turn off rbd cache?
On Thu, Dec 17, 2020 at 11:21 AM Philip Brown <pbrown(a)medata.com> wrote:
I guess I left out in my examples, where I tried rbd_cache as well, and failed
# rbd config global set rbd_cache false
rbd: invalid config entity: rbd_cache (must be global, client or client.<id>)
But that's not a valid command -- you attempted to provide the
configuration key name for the entity (as the error message is telling
you).
$ rbd config global set global rbd_cache false
... or ...
$ rbd config global set client rbd_cache false
... or ...
$ rbd config global set client.host_a rbd_cache false
>
>
9:37 a.m.
On Thu, Dec 17, 2020 at 12:09 PM Philip Brown <pbrown(a)medata.com> wrote:
Huhhh..
It seems worthwhile to point out two inconsistencies, then.
1. the "old way", of ceph config set global rbd_cache false
doesnt require this odd redundant "global set global" syntax. It is confusing
to users to have to specify "global" twice.
may I suggest that the syntax for rbd config be adjusted, so that people can simply use
rbd config global ...
or
rbd config client ...
not
rbd config global set (global/client)
Everyone has different opinions. "rbd help config global set" clearly
defines all the expected parameters. We can't be all things to all
people.
2. The fact that "ceph config set global
rbd_cache false" works, seems to imply that putting
[global]
rbd cache = false
in /etc/ceph/ceph.conf should work also.
Except it doesnt.
Even after fully shutting down every node in the ceph cluster and doing a cold startup.
is that a bug?
Nope [1]. How would changing a random configuration file on a random
node affect the configuration on another node?
----- Original Message -----
From: "Jason Dillaman" <jdillama(a)redhat.com>
To: "Philip Brown" <pbrown(a)medata.com>
Cc: "dillaman" <dillaman(a)redhat.com>om>, "ceph-users"
<ceph-users(a)ceph.io>
Sent: Thursday, December 17, 2020 8:24:59 AM
Subject: Re: [ceph-users] Re: bug? cant turn off rbd cache?
On Thu, Dec 17, 2020 at 11:21 AM Philip Brown <pbrown(a)medata.com> wrote:
[1] https://docs.ceph.com/en/latest/rados/configuration/ceph-conf/#config-sourc…
--
Jason
I guess I left out in my examples, where I tried rbd_cache as well, and failed
# rbd config global set rbd_cache false
rbd: invalid config entity: rbd_cache (must be global, client or client.<id>)
But that's not a valid command -- you attempted to provide the
configuration key name for the entity (as the error message is telling
you).
$ rbd config global set global rbd_cache false
... or ...
$ rbd config global set client rbd_cache false
... or ...
$ rbd config global set client.host_a rbd_cache false
1239
days inactive
1240
days old
8 comments
3 participants
participants (3)
-
Eugen Block -
Jason Dillaman -
Philip Brown