On Sun, Jul 30, 2023 at 7:09 PM Ilya Dryomov <idryomov(a)gmail.com> wrote:
On Sat, Jul 29, 2023 at 2:11 PM Mykola Golub <to.my.trociny(a)gmail.com> wrote:
2) How can we
fix the current situation in the cluster? Would it be safe
enough to just run `ceph-kvstore-tool rocksdb store.db rm-prefix osd_snap`
to remove all osd_snap keys (including purged_epoch keys)? Due to
large db size I don't think it would be possible to selectively remove
keys with `ceph-kvstore-tool rocksdb store.db rm {prefix} {key}`
command and we may use only the `rm-prefix` command. Looking at the
code and actually trying it in a test environment it seems like it could
work, but I may be missing something dangerous here?
Adding Radek.
Trying to resurrect this thread. Actually this is the question we are
most interested
in, i.e how the osd_snap prefixed keys can be properly trimmed.
Note, it is easy to write a script that would list all existing
snapshots in the pool
with `rbd snap ls --all {image}` command and then issue
`remove_self_managed_snap`
for all IDs that are not in this list. One thing is that it would run
for an unrealistically
long time. But another thing is that removing a snapshot may merge purged_snap
ranges (reducing the number of purged_snap keys) but it will add purged_epoch
key (which contains a list of snapshots removed for this epoch). And I
don't see how
the purged_epoch keys are supposed to be normally trimmed. It looks
like we never
trim these keys.
--
Mykola Golub