Hello everybody, I'm trying to figure out how often the ceph client is contacting the monitors for updating its own information about the cluster map. Can anyone point me to a document describing this client <-> monitor communication? Thank you, Laszlo
Hi Laszlo, Which client are you talking about? On 7/15/20, 11:54, "Budai Laszlo" <laszlo.budai@gmail.com> wrote: Hello everybody, I'm trying to figure out how often the ceph client is contacting the monitors for updating its own information about the cluster map. Can anyone point me to a document describing this client <-> monitor communication? Thank you, Laszlo _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi Nghia, in the docs (https://docs.ceph.com/docs/master/architecture/#about-pools) there is the statement "Ceph Clients retrieve a Cluster Map from a Ceph Monitor, and write objects to pools." My question is how often the client is retrieving the Cluster Map? How does the client get the knowledge about a change in the cluster? Thank you, Laszlo On 7/15/20 7:57 AM, Nghia Viet Tran wrote:
Hi Laszlo,
Which client are you talking about?
On 7/15/20, 11:54, "Budai Laszlo" <laszlo.budai@gmail.com> wrote:
Hello everybody,
I'm trying to figure out how often the ceph client is contacting the monitors for updating its own information about the cluster map. Can anyone point me to a document describing this client <-> monitor communication?
Thank you, Laszlo _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
to be more specific: if we have an RBD volume used by a client (a hypervisor, or or mapped with rbd), we assume continuous activity on the volume. How often will the RBD client contact the monitor to get the current map? Are you aware of any documentation page that describes this interaction? Thank you, Laszlo On 7/15/20 8:12 AM, Budai Laszlo wrote:
Hi Nghia,
in the docs (https://docs.ceph.com/docs/master/architecture/#about-pools) there is the statement "Ceph Clients retrieve a Cluster Map from a Ceph Monitor, and write objects to pools." My question is how often the client is retrieving the Cluster Map? How does the client get the knowledge about a change in the cluster?
Thank you, Laszlo
On 7/15/20 7:57 AM, Nghia Viet Tran wrote:
Hi Laszlo,
Which client are you talking about?
On 7/15/20, 11:54, "Budai Laszlo" <laszlo.budai@gmail.com> wrote:
Hello everybody,
I'm trying to figure out how often the ceph client is contacting the monitors for updating its own information about the cluster map. Can anyone point me to a document describing this client <-> monitor communication?
Thank you, Laszlo _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi Budai, When you ask "*how often the client is retrieving the Cluster Map?*" . The obvious answer to that is there is nothing 'often' in it. Whenever there is a change in the map, the monitor will inform the client. I think you need to read about the CRUSH algorithm in Ceph. Because that will explain you the map changes and data movement. While going through CRUSH, forget there is a monitor node. Just suppose there is a client machine and this client machine READ/WRITE to a cluster ( number of OSDs). Because theoretically a Ceph client can also be a monitor (not at all recommended for practical purposes). Once you have understood CRUSH, I am quite sure that will answer many of your questions. And feel free to ask about CRUSH. I would be glad to answer. BR On Wed, Jul 15, 2020 at 8:54 AM Budai Laszlo <laszlo.budai@gmail.com> wrote:
to be more specific: if we have an RBD volume used by a client (a hypervisor, or or mapped with rbd), we assume continuous activity on the volume. How often will the RBD client contact the monitor to get the current map? Are you aware of any documentation page that describes this interaction?
Thank you, Laszlo
Hi Nghia,
in the docs (https://docs.ceph.com/docs/master/architecture/#about-pools)
On 7/15/20 8:12 AM, Budai Laszlo wrote: there is the statement "Ceph Clients retrieve a Cluster Map from a Ceph Monitor, and write objects to pools." My question is how often the client is retrieving the Cluster Map? How does the client get the knowledge about a change in the cluster?
Thank you, Laszlo
On 7/15/20 7:57 AM, Nghia Viet Tran wrote:
Hi Laszlo,
Which client are you talking about?
On 7/15/20, 11:54, "Budai Laszlo" <laszlo.budai@gmail.com> wrote:
Hello everybody,
I'm trying to figure out how often the ceph client is contacting
the monitors for updating its own information about the cluster map.
Can anyone point me to a document describing this client <->
monitor communication?
Thank you, Laszlo _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi Bobby, Thank you for your answer. You are saying "Whenever there is a change in the map, the monitor will inform the client." Can you please give me some ceph documentation link where I could read these details? For me it is logical to have the monitors update the clients about changes in the cluster map. I'm looking for the details of this process. Like how does the monitor knows which clients to update? Kind regards, Laszlo On 7/15/20 1:57 PM, Bobby wrote:
Hi Budai,
When you ask "/how often the client is retrieving the Cluster Map?/" . The obvious answer to that is there is nothing 'often' in it. Whenever there is a change in the map, the monitor will inform the client.
I think you need to read about the CRUSH algorithm in Ceph. Because that will explain you the map changes and data movement. While going through CRUSH, forget there is a monitor node. Just suppose there is a client machine and this client machine READ/WRITE to a cluster ( number of OSDs). Because theoretically a Ceph client can also be a monitor (not at all recommended for practical purposes). Once you have understood CRUSH, I am quite sure that will answer many of your questions.
And feel free to ask about CRUSH. I would be glad to answer.
BR
On Wed, Jul 15, 2020 at 8:54 AM Budai Laszlo <laszlo.budai@gmail.com <mailto:laszlo.budai@gmail.com>> wrote:
to be more specific: if we have an RBD volume used by a client (a hypervisor, or or mapped with rbd), we assume continuous activity on the volume. How often will the RBD client contact the monitor to get the current map? Are you aware of any documentation page that describes this interaction?
Thank you, Laszlo
On 7/15/20 8:12 AM, Budai Laszlo wrote: > Hi Nghia, > > in the docs (https://docs.ceph.com/docs/master/architecture/#about-pools) there is the statement "Ceph Clients retrieve a Cluster Map from a Ceph Monitor, and write objects to pools." My question is how often the client is retrieving the Cluster Map? How does the client get the knowledge about a change in the cluster? > > Thank you, > Laszlo > > On 7/15/20 7:57 AM, Nghia Viet Tran wrote: >> Hi Laszlo, >> >> Which client are you talking about? >> >> On 7/15/20, 11:54, "Budai Laszlo" <laszlo.budai@gmail.com <mailto:laszlo.budai@gmail.com>> wrote: >> >> Hello everybody, >> >> I'm trying to figure out how often the ceph client is contacting the monitors for updating its own information about the cluster map. >> Can anyone point me to a document describing this client <-> monitor communication? >> >> Thank you, >> Laszlo >> _______________________________________________ >> ceph-users mailing list -- ceph-users@ceph.io <mailto:ceph-users@ceph.io> >> To unsubscribe send an email to ceph-users-leave@ceph.io <mailto:ceph-users-leave@ceph.io> >> > _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io <mailto:ceph-users@ceph.io> To unsubscribe send an email to ceph-users-leave@ceph.io <mailto:ceph-users-leave@ceph.io>
On Wed, Jul 15, 2020 at 1:41 PM Budai Laszlo <laszlo.budai@gmail.com> wrote:
Hi Bobby,
Thank you for your answer. You are saying "Whenever there is a change in the map, the monitor will inform the client." Can you please give me some ceph documentation link where I could read these details? For me it is logical to have the monitors update the clients about changes in the cluster map. I'm looking for the details of this process. Like how does the monitor knows which clients to update?
Clients subscribe to map updates, either continuous or one-time. The monitor keeps track of clients that asked for continuous updates and whenever a new map of the respective type is published, sends it to those clients (for osdmaps, what gets sent is an incremental on top of the previous osdmap as the full osdmap can be quite large). Most clients don't subscribe to continuous map updates and instead rely on lazy map propagation. When an OSD receives a request from a client with an old osdmap, it will send an incremental along with the reply (or instead of the reply, if, according to the current osdmap, the request should have been directed to a different OSD). The client processes the incremental and resends the request if needed. Thanks, Ilya
participants (4)
-
Bobby
-
Budai Laszlo
-
Ilya Dryomov
-
Nghia Viet Tran