2 Mar
2021
2 Mar
'21
3:34 a.m.
hello!
I'm trying to understand how Bluestore cooperates with RBD image clones, so
my test is simple
1. create an image (2G) and fill with data
2. create a snapshot
3. protect it
4. create a clone of the image
5. write a small portion of data (4K) to clone
6. check how it changed and if just 4K are used to prove CoW allocated new
extent instead of copying out snapped data.
Unfortunately it occurs that at least rbd du reports that 4M was changed
and the clone consumes 4M of data instead of expected 4K...
'''
rbd du rbd/clone1
NAME PROVISIONED USED
clone1 2 GiB 4 MiB
'''
How can I trace/prove Bluestore CoW really works in this case, and prevent
copying the rest of the 4M stripe like Filestore did ?
p.s tested on Luminous/Octopus, ssd devices, min_alloc_size: 16k,
block_size: 4k
best regards!
--
Pawel S.
2 Mar
2 Mar
3:47 a.m.
On Mon, Mar 1, 2021 at 1:35 PM Pawel S <pejotes(a)gmail.com> wrote:
hello!
I'm trying to understand how Bluestore cooperates with RBD image clones, so
my test is simple
1. create an image (2G) and fill with data
2. create a snapshot
3. protect it
4. create a clone of the image
5. write a small portion of data (4K) to clone
6. check how it changed and if just 4K are used to prove CoW allocated new
extent instead of copying out snapped data.
Unfortunately it occurs that at least rbd du reports that 4M was changed
and the clone consumes 4M of data instead of expected 4K...
'''
rbd du rbd/clone1
NAME PROVISIONED USED
clone1 2 GiB 4 MiB
'''
How can I trace/prove Bluestore CoW really works in this case, and prevent
copying the rest of the 4M stripe like Filestore did ?
RBD clones are not the same as RBD snapshots. Writing to an
unallocated extent within an RBD clone image will always copy up to
the "object-size" amount of data from the parent image. In that
respect, there is no difference between Bluestore and Filestore since
this logic is agnostic to the underlying OSD object store.
RBD snapshots, however, do result in copy-on-write like semantics
within Bluestore, although it's technically a redirect-on-write since
older data is not moved to preserve the snapshot history and instead
the new write request is redirected to a newly allocated block within
Bluestore and metadata is updated to reference the new block.
p.s tested on Luminous/Octopus, ssd devices,
min_alloc_size: 16k,
block_size: 4k
best regards!
--
Pawel S.
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
--
Jason
5:06 a.m.
Hello Jason!
On Mon, Mar 1, 2021, 19:48 Jason Dillaman <jdillama(a)redhat.com> wrote:
On Mon, Mar 1, 2021 at 1:35 PM Pawel S
<pejotes(a)gmail.com> wrote:
Thanks for clarification, so the only way to reduce impact here is to
decrease size of objects ?
hello!
I'm trying to understand how Bluestore cooperates with RBD image clones,
so
my test is simple
1. create an image (2G) and fill with data
2. create a snapshot
3. protect it
4. create a clone of the image
5. write a small portion of data (4K) to clone
6. check how it changed and if just 4K are used to prove CoW allocated
new
extent instead of copying out snapped data.
Unfortunately it occurs that at least rbd du reports that 4M was changed
and the clone consumes 4M of data instead of expected 4K...
'''
rbd du rbd/clone1
NAME PROVISIONED USED
clone1 2 GiB 4 MiB
'''
How can I trace/prove Bluestore CoW really works in this case, and
prevent
copying the rest of the 4M stripe like Filestore
did ?
RBD clones are not the same as RBD snapshots. Writing to an
unallocated extent within an RBD clone image will always copy up to
the "object-size" amount of data from the parent image. In that
respect, there is no difference between Bluestore and Filestore since
this logic is agnostic to the underlying OSD object store.
RBD snapshots, however, do result in copy-on-write
like semantics
within Bluestore, although it's technically a redirect-on-write since
older data is not moved to preserve the snapshot history and instead
the new write request is redirected to a newly allocated block within
Bluestore and metadata is updated to reference the new block.
Thanks!!! So I've wrongly adopted the same for clones.
--
Pawel S.
5:16 a.m.
On Mon, Mar 1, 2021 at 3:07 PM Pawel S <pejotes(a)gmail.com> wrote:
--
Jason
Hello Jason!
On Mon, Mar 1, 2021, 19:48 Jason Dillaman <jdillama(a)redhat.com> wrote:
Yes, although since the Octopus release this copy-up from parent to
child operation is now a sparse-read from parent/sparse-write to child
(i.e. if the overlapping parent object extent was empty, no data would
need to be copied to the child). However, in your test case where
there is a full 4MiB of data in the overlapping parent extent, it
would all need to be copied to the child.
On Mon, Mar 1, 2021 at 1:35 PM Pawel S
<pejotes(a)gmail.com> wrote:
Thanks for clarification, so the only way to reduce impact here is to
decrease size of objects ?
hello!
I'm trying to understand how Bluestore cooperates with RBD image clones,
so
my test is simple
1. create an image (2G) and fill with data
2. create a snapshot
3. protect it
4. create a clone of the image
5. write a small portion of data (4K) to clone
6. check how it changed and if just 4K are used to prove CoW allocated
new
extent instead of copying out snapped data.
Unfortunately it occurs that at least rbd du reports that 4M was changed
and the clone consumes 4M of data instead of expected 4K...
'''
rbd du rbd/clone1
NAME PROVISIONED USED
clone1 2 GiB 4 MiB
'''
How can I trace/prove Bluestore CoW really works in this case, and
prevent
copying the rest of the 4M stripe like Filestore
did ?
RBD clones are not the same as RBD snapshots. Writing to an
unallocated extent within an RBD clone image will always copy up to
the "object-size" amount of data from the parent image. In that
respect, there is no difference between Bluestore and Filestore since
this logic is agnostic to the underlying OSD object store.
RBD snapshots, however, do result in
copy-on-write like semantics
within Bluestore, although it's technically a redirect-on-write since
older data is not moved to preserve the snapshot history and instead
the new write request is redirected to a newly allocated block within
Bluestore and metadata is updated to reference the new block.
Thanks!!! So I've wrongly adopted the same for clones.
--
Pawel S.
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
12:24 p.m.
New subject: [Suspicious newsletter] RBD clone on Bluestore
I might be wrong but the object size is 4M in ceph so this might be the case also. But
yeah, on the lower level of data storing I'm not familiar.
Istvan Szabo
Senior Infrastructure Engineer
---------------------------------------------------
Agoda Services Co., Ltd.
e: istvan.szabo(a)agoda.com
---------------------------------------------------
-----Original Message-----
From: Pawel S <pejotes(a)gmail.com>
Sent: Tuesday, March 2, 2021 1:34 AM
To: ceph-users(a)ceph.io
Subject: [Suspicious newsletter] [ceph-users] RBD clone on Bluestore
Email received from outside the company. If in doubt don't click links nor open
attachments!
________________________________
hello!
I'm trying to understand how Bluestore cooperates with RBD image clones, so my test is
simple
1. create an image (2G) and fill with data 2. create a snapshot 3. protect it 4. create a
clone of the image 5. write a small portion of data (4K) to clone 6. check how it changed
and if just 4K are used to prove CoW allocated new extent instead of copying out snapped
data.
Unfortunately it occurs that at least rbd du reports that 4M was changed and the clone
consumes 4M of data instead of expected 4K...
'''
rbd du rbd/clone1
NAME PROVISIONED USED
clone1 2 GiB 4 MiB
'''
How can I trace/prove Bluestore CoW really works in this case, and prevent copying the
rest of the 4M stripe like Filestore did ?
p.s tested on Luminous/Octopus, ssd devices, min_alloc_size: 16k,
block_size: 4k
best regards!
--
Pawel S.
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an email to
ceph-users-leave(a)ceph.io
________________________________
This message is confidential and is for the sole use of the intended recipient(s). It may
also be privileged or otherwise protected by copyright or other legal rules. If you have
received it by mistake please let us know by reply email and delete it from your system.
It is prohibited to copy this message or disclose its content to anyone. Any
confidentiality or privilege is not waived or lost by any mistaken delivery or
unauthorized disclosure of the message. All messages sent to and from Agoda may be
monitored to ensure compliance with company policies, to protect the company's
interests and to remove potential malware. Electronic messages may be intercepted,
amended, lost or deleted, or contain viruses.
1161
days inactive
1162
days old
4 comments
3 participants
participants (3)
-
Jason Dillaman -
Pawel S -
Szabo, Istvan (Agoda)