2 Oct
2019
2 Oct
'19
11:32 p.m.
From my initial testing it looks like 14.2.4 fully supports the
deduplication mentioned here:
https://docs.ceph.com/docs/master/dev/deduplication/
However, I'm not sure where the struct object_manifest script goes in
relation to foo and foo-chunk, and I'm not aware of what the
offsets/caspool should be.
If this still isn't fully implemented how does the dedup tool work? If I
remove a file but it exists elsewhere on the volume, will it be purged or
would the tool need to run again to clear the data?
9 Oct
9 Oct
5:55 p.m.
So since nobody who's actually working on it has chimed in:
While there is some deduplication functionality built into the system,
AFAIK it's not something considered for users at this point. It's
under ongoing development, doesn't have performance data, and isn't
plumbed through into a lot of the system. Last I heard about it was a
discussion about how to handle the reference counting used to delete
data when it's no longer needed. (See the thread "refcounting chunks
vs snapshots" and https://github.com/ceph/ceph/pull/29283)
-Greg
On Wed, Oct 2, 2019 at 4:48 PM The Zombie Hunter
<thezombiehunter(a)gmail.com> wrote:
From my initial testing it looks like 14.2.4 fully supports the deduplication mentioned
here:
https://docs.ceph.com/docs/master/dev/deduplication/
However, I'm not sure where the struct object_manifest script goes in relation to foo
and foo-chunk, and I'm not aware of what the offsets/caspool should be.
If this still isn't fully implemented how does the dedup tool work? If I remove a
file but it exists elsewhere on the volume, will it be purged or would the tool need to
run again to clear the data?
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
6:15 p.m.
On Wed, Oct 9, 2019 at 12:56 PM Gregory Farnum <gfarnum(a)redhat.com> wrote:
So since nobody who's actually working on it has
chimed in:
While there is some deduplication functionality built into the system,
AFAIK it's not something considered for users at this point. It's
under ongoing development, doesn't have performance data, and isn't
plumbed through into a lot of the system. Last I heard about it was a
discussion about how to handle the reference counting used to delete
data when it's no longer needed. (See the thread "refcounting chunks
vs snapshots" and https://github.com/ceph/ceph/pull/29283)
-Greg
I was holding off, as this is really not a part of Ceph, but we have done
some preliminary testing of VDO as a client on top of RBD, and it looked
fine. We also run borg on top of RBD, which is also stable and able to
handle large volumes of data. Hope this helps.
--
Alex Gorbachev
Intelligent Systems Services Inc.
On Wed, Oct 2, 2019 at 4:48 PM The Zombie Hunter
<thezombiehunter(a)gmail.com> wrote:
From my initial testing it looks like 14.2.4 fully supports the
deduplication
mentioned here:
https://docs.ceph.com/docs/master/dev/deduplication/
However, I'm not sure where the struct object_manifest script goes in
relation
to foo and foo-chunk, and I'm not aware of what the
offsets/caspool should be.
If this still isn't fully implemented how does the dedup tool work? If I
remove a file but it exists elsewhere on the volume, will it be purged or
would the tool need to run again to clear the data?
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
1683
days inactive
1690
days old
2 comments
3 participants
participants (3)
-
Alex Gorbachev -
Gregory Farnum -
The Zombie Hunter