Servicing multiple OpenStack clusters from the same Ceph cluster
Hello, We have a medium-sized Ceph Luminous cluster that, up til now, has been the RBD image backend solely for an OpenStack Newton cluster that's marked for upgrade to Stein later this year. Recently we deployed a brand new Stein cluster however, and I'm curious whether the idea of pointing the new OpenStack cluster at the same RBD pools for Cinder/Glance/Nova as the Luminous cluster would be considered bad practice, or even potentially dangerous. One argument for doing it may be that multiple CInder/Glance/Nova pools serving disparate groups of clients would come at a PG cost to the cluster, though the separation of multiple, distinct pools also has its advantages. The UUIDs generated for RBD images in the pools by OpenStack services *should* be unique and collision-less between the 2 OpenStack clusters, in theory. One other point I was curious about was RBD image feature sets; Stein Ceph clients will be running later versions of Ceph libraries than Newton clients. If the 2 sets of clients were to share pools, would that itself cause problems (in the case that neither set needed to share RBD images within pools, only the pool itself) with some images in the pool having different feature lists? -- ******************* Paul Browne Research Computing Platforms University Information Services Roger Needham Building JJ Thompson Avenue University of Cambridge Cambridge United Kingdom E-Mail: pfb29@cam.ac.uk Tel: 0044-1223-746548 *******************
Hi, On 29/01/2020 16:40, Paul Browne wrote:
Recently we deployed a brand new Stein cluster however, and I'm curious whether the idea of pointing the new OpenStack cluster at the same RBD pools for Cinder/Glance/Nova as the Luminous cluster would be considered bad practice, or even potentially dangerous.
I think that would be pretty risky - here we have a Ceph cluster that provides backing for our OpenStacks, and each OpenStack has its own set of pools -metrics,-images,-volumes,-vms (and its own credential). Regards, Matthew
On Wed, 29 Jan 2020 at 16:52, Matthew Vernon <mv3@sanger.ac.uk> wrote:
Hi,
On 29/01/2020 16:40, Paul Browne wrote:
Recently we deployed a brand new Stein cluster however, and I'm curious whether the idea of pointing the new OpenStack cluster at the same RBD pools for Cinder/Glance/Nova as the Luminous cluster would be considered bad practice, or even potentially dangerous.
I think that would be pretty risky - here we have a Ceph cluster that provides backing for our OpenStacks, and each OpenStack has its own set of pools -metrics,-images,-volumes,-vms (and its own credential).
Hi Matthew, I think I've come around to that thinking now too. Despite using different keys, the 2 sets of clients in different OpenStack clusters would require the same capabilities on the shared pools, which widens the blast radius a bit too far for me, I think (unless there were also a capability to restrict the sets of clients' keys to specific namespaces within the shared pools similar to the caps given out to CephFS clients) Thanmks, Paul
Regards,
Matthew
-- ******************* Paul Browne Research Computing Platforms University Information Services Roger Needham Building JJ Thompson Avenue University of Cambridge Cambridge United Kingdom E-Mail: pfb29@cam.ac.uk Tel: 0044-1223-746548 *******************
Hi, Quoting Paul Browne (pfb29@cam.ac.uk):
On Wed, 29 Jan 2020 at 16:52, Matthew Vernon <mv3@sanger.ac.uk> wrote:
Hi,
On 29/01/2020 16:40, Paul Browne wrote:
Recently we deployed a brand new Stein cluster however, and I'm curious whether the idea of pointing the new OpenStack cluster at the same RBD pools for Cinder/Glance/Nova as the Luminous cluster would be considered bad practice, or even potentially dangerous.
I think that would be pretty risky - here we have a Ceph cluster that provides backing for our OpenStacks, and each OpenStack has its own set of pools -metrics,-images,-volumes,-vms (and its own credential).
Hi Matthew,
I think I've come around to that thinking now too.
Despite using different keys, the 2 sets of clients in different OpenStack clusters would require the same capabilities on the shared pools, which widens the blast radius a bit too far for me, I think (unless there were also a capability to restrict the sets of clients' keys to specific namespaces within the shared pools similar to the caps given out to CephFS clients)
This is supported since Nautilus: namespace support for librbd. I do not now however if there is already support for this in qemu/libvirt/openstack. OpenNebula support is pending [1]. Gr. Stefan [1]: https://github.com/OpenNebula/one/issues/3141 -- | BIT BV https://www.bit.nl/ Kamer van Koophandel 09090351 | GPG: 0xD14839C6 +31 318 648 688 / info@bit.nl
Hello, We have recently deployed that and it's working fine. We have deployed different keys for the different openstack clusters ofcourse and they are using the same cinder/nova/glance pools. The only risk is if a client from one openstack cluster creates a volume and the id that will be generated ends up being the same on an existing volume from the other openstack cluster. But that's like possibility of 1 in 5 billion or something. We took the risk. Regards
You should have used separate pool name scemes for each OpenStack cluster.. ________________________________ From: tdados@hotmail.com <tdados@hotmail.com> Sent: Wednesday, January 29, 2020 12:29 PM To: ceph-users@ceph.io <ceph-users@ceph.io> Subject: [ceph-users] Re: Servicing multiple OpenStack clusters from the same Ceph cluster Hello, We have recently deployed that and it's working fine. We have deployed different keys for the different openstack clusters ofcourse and they are using the same cinder/nova/glance pools. The only risk is if a client from one openstack cluster creates a volume and the id that will be generated ends up being the same on an existing volume from the other openstack cluster. But that's like possibility of 1 in 5 billion or something. We took the risk. Regards _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (5)
-
Matthew H
-
Matthew Vernon
-
Paul Browne
-
Stefan Kooman
-
tdados＠hotmail.com