- ceph-users - lists.ceph.io

OSD Metadata Imbalance
by Paul Kramme 01 Dec '20

01 Dec '20

Hello, my cluster is currently showing a metadata imbalance. Normally, all OSDs have around 23GB metadata (META column), but 4 OSDs out of 56 have 34 GB metadata. Compacting reduces the data for some OSDs, but not for others. OSDs where the compaction worked quickly grow to back to 34GB. Our cluster configuration: * 8 nodes, each with 6 HDDs OSDs and 1 SSD used for blockdb and WAL * k=4 m=2 EC * v14.2.14 Normal OSD: ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META AVAIL %USE VAR PGS STATUS 40 hdd 11.09470 1.00000 11 TiB 8.6 TiB 8.4 TiB 1.3 GiB 23 GiB 2.5 TiB 77.15 1.01 130 up Big OSD: ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META AVAIL %USE VAR PGS STATUS 0 hdd 11.09499 1.00000 11 TiB 8.6 TiB 8.4 TiB 1.8 GiB 30 GiB 2.5 TiB 77.59 1.02 130 up There are 56 OSDs in the cluster, 4 OSDs of which are bigger. These OSDs are all in different hosts. Why is that? Is that dangerous or could lead to problems such as performance degrades? Thanks, Paul

1 0

osd out cant' bring it back online
by Oliver Weinmann 01 Dec '20

01 Dec '20

Hi, I'm still evaluating ceph 15.2.5 in a lab so the problem is not really hurting me, but I want to understand it and hopefully fix it. It is a good practice. To test the resilience of the cluster I try to break it by doing all kinds of things. Today I powered off (clean shutdown) one osd node and powered it back on. Last time I tried this there was no problem getting it back online. After a few minutes the cluster health was back to ok. This time it stayed degraded forever. I checked and noticed that the service osd.0 on the osd node was failing. So i used google and there people recommended to simply delete the osd and re-create it. I tried it and still can't get the osd back in service. First I removed the osd: [root@gedasvl02 ~]# ceph osd out 0 INFO:cephadm:Inferring fsid d0920c36-2368-11eb-a5de-005056b703af INFO:cephadm:Inferring config /var/lib/ceph/d0920c36-2368-11eb-a5de-005056b703af/mon.gedasvl02/config INFO:cephadm:Using recent ceph image docker.io/ceph/ceph:v15 osd.0 is already out. [root@gedasvl02 ~]# ceph auth del 0 INFO:cephadm:Inferring fsid d0920c36-2368-11eb-a5de-005056b703af INFO:cephadm:Inferring config /var/lib/ceph/d0920c36-2368-11eb-a5de-005056b703af/mon.gedasvl02/config INFO:cephadm:Using recent ceph image docker.io/ceph/ceph:v15 Error EINVAL: bad entity name [root@gedasvl02 ~]# ceph auth del osd.0 INFO:cephadm:Inferring fsid d0920c36-2368-11eb-a5de-005056b703af INFO:cephadm:Inferring config /var/lib/ceph/d0920c36-2368-11eb-a5de-005056b703af/mon.gedasvl02/config INFO:cephadm:Using recent ceph image docker.io/ceph/ceph:v15 updated [root@gedasvl02 ~]# ceph osd rm 0 INFO:cephadm:Inferring fsid d0920c36-2368-11eb-a5de-005056b703af INFO:cephadm:Inferring config /var/lib/ceph/d0920c36-2368-11eb-a5de-005056b703af/mon.gedasvl02/config INFO:cephadm:Using recent ceph image docker.io/ceph/ceph:v15 removed osd.0 [root@gedasvl02 ~]# ceph osd tree INFO:cephadm:Inferring fsid d0920c36-2368-11eb-a5de-005056b703af INFO:cephadm:Inferring config /var/lib/ceph/d0920c36-2368-11eb-a5de-005056b703af/mon.gedasvl02/config INFO:cephadm:Using recent ceph image docker.io/ceph/ceph:v15 ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF -1 0.43658 root default -7 0.21829 host gedaopl01 2 ssd 0.21829 osd.2 up 1.00000 1.00000 -3 0 host gedaopl02 -5 0.21829 host gedaopl03 3 ssd 0.21829 osd.3 up 1.00000 1.00000 Looks ok it's gone... Then i zapped it: [root@gedasvl02 ~]# ceph orch device zap gedaopl02 /dev/sdb --force INFO:cephadm:Inferring fsid d0920c36-2368-11eb-a5de-005056b703af INFO:cephadm:Inferring config /var/lib/ceph/d0920c36-2368-11eb-a5de-005056b703af/mon.gedasvl02/config INFO:cephadm:Using recent ceph image docker.io/ceph/ceph:v15 INFO:cephadm:/usr/bin/podman:stderr WARNING: The same type, major and minor should not be used for multiple devices. INFO:cephadm:/usr/bin/podman:stderr --> Zapping: /dev/sdb INFO:cephadm:/usr/bin/podman:stderr --> Zapping lvm member /dev/sdb. lv_path is /dev/ceph-3bf1bb28-0858-4464-a848-d7f56319b40a/osd-block-3a79800d-2a19-45d8-a850-82c6a8113323 INFO:cephadm:/usr/bin/podman:stderr Running command: /usr/bin/dd if=/dev/zero of=/dev/ceph-3bf1bb28-0858-4464-a848-d7f56319b40a/osd-block-3a79800d-2a19-45d8-a850-82c6a8113323 bs=1M count=10 conv=fsync INFO:cephadm:/usr/bin/podman:stderr stderr: 10+0 records in INFO:cephadm:/usr/bin/podman:stderr 10+0 records out INFO:cephadm:/usr/bin/podman:stderr 10485760 bytes (10 MB, 10 MiB) copied, 0.0314447 s, 333 MB/s INFO:cephadm:/usr/bin/podman:stderr stderr: INFO:cephadm:/usr/bin/podman:stderr --> Only 1 LV left in VG, will proceed to destroy volume group ceph-3bf1bb28-0858-4464-a848-d7f56319b40a INFO:cephadm:/usr/bin/podman:stderr Running command: /usr/sbin/vgremove -v -f ceph-3bf1bb28-0858-4464-a848-d7f56319b40a INFO:cephadm:/usr/bin/podman:stderr stderr: Removing ceph--3bf1bb28--0858--4464--a848--d7f56319b40a-osd--block--3a79800d--2a19--45d8--a850--82c6a8113323 (253:0) INFO:cephadm:/usr/bin/podman:stderr stderr: Archiving volume group "ceph-3bf1bb28-0858-4464-a848-d7f56319b40a" metadata (seqno 5). INFO:cephadm:/usr/bin/podman:stderr stderr: Releasing logical volume "osd-block-3a79800d-2a19-45d8-a850-82c6a8113323" INFO:cephadm:/usr/bin/podman:stderr stderr: Creating volume group backup "/etc/lvm/backup/ceph-3bf1bb28-0858-4464-a848-d7f56319b40a" (seqno 6). INFO:cephadm:/usr/bin/podman:stderr stdout: Logical volume "osd-block-3a79800d-2a19-45d8-a850-82c6a8113323" successfully removed INFO:cephadm:/usr/bin/podman:stderr stderr: Removing physical volume "/dev/sdb" from volume group "ceph-3bf1bb28-0858-4464-a848-d7f56319b40a" INFO:cephadm:/usr/bin/podman:stderr stdout: Volume group "ceph-3bf1bb28-0858-4464-a848-d7f56319b40a" successfully removed INFO:cephadm:/usr/bin/podman:stderr Running command: /usr/bin/dd if=/dev/zero of=/dev/sdb bs=1M count=10 conv=fsync INFO:cephadm:/usr/bin/podman:stderr stderr: 10+0 records in INFO:cephadm:/usr/bin/podman:stderr 10+0 records out INFO:cephadm:/usr/bin/podman:stderr stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0355641 s, 295 MB/s INFO:cephadm:/usr/bin/podman:stderr --> Zapping successful for: <Raw Device: /dev/sdb> And re-added it: [root@gedasvl02 ~]# ceph orch daemon add osd gedaopl02:/dev/sdb INFO:cephadm:Inferring fsid d0920c36-2368-11eb-a5de-005056b703af INFO:cephadm:Inferring config /var/lib/ceph/d0920c36-2368-11eb-a5de-005056b703af/mon.gedasvl02/config INFO:cephadm:Using recent ceph image docker.io/ceph/ceph:v15 Created osd(s) 0 on host 'gedaopl02' But the osd is still out... [root@gedasvl02 ~]# ceph osd tree INFO:cephadm:Inferring fsid d0920c36-2368-11eb-a5de-005056b703af INFO:cephadm:Inferring config /var/lib/ceph/d0920c36-2368-11eb-a5de-005056b703af/mon.gedasvl02/config INFO:cephadm:Using recent ceph image docker.io/ceph/ceph:v15 ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF -1 0.43658 root default -7 0.21829 host gedaopl01 2 ssd 0.21829 osd.2 up 1.00000 1.00000 -3 0 host gedaopl02 -5 0.21829 host gedaopl03 3 ssd 0.21829 osd.3 up 1.00000 1.00000 0 0 osd.0 down 0 1.00000 Looking at the cluster log in the webui i see the following error: Failed to apply osd.dashboard-admin-1606745745154 spec DriveGroupSpec(name=dashboard-admin-1606745745154->placement=PlacementSpec(host_pattern='*'), service_id='dashboard-admin-1606745745154', service_type='osd', data_devices=DeviceSelection(size='223.6GB', rotational=False, all=False), osd_id_claims={}, unmanaged=False, filter_logic='AND', preview_only=False): No filters applied Traceback (most recent call last): File "/usr/share/ceph/mgr/cephadm/module.py", line 2108, in _apply_all_services if self._apply_service(spec): File "/usr/share/ceph/mgr/cephadm/module.py", line 2005, in _apply_service self.osd_service.create_from_spec(cast(DriveGroupSpec, spec)) File "/usr/share/ceph/mgr/cephadm/services/osd.py", line 43, in create_from_spec ret = create_from_spec_one(self.prepare_drivegroup(drive_group)) File "/usr/share/ceph/mgr/cephadm/services/osd.py", line 127, in prepare_drivegroup drive_selection = DriveSelection(drive_group, inventory_for_host) File "/lib/python3.6/site-packages/ceph/deployment/drive_selection/selector.py", line 32, in __init__ self._data = self.assign_devices(self.spec.data_devices) File "/lib/python3.6/site-packages/ceph/deployment/drive_selection/selector.py", line 138, in assign_devices if not all(m.compare(disk) for m in FilterGenerator(device_filter)): File "/lib/python3.6/site-packages/ceph/deployment/drive_selection/selector.py", line 138, in <genexpr> if not all(m.compare(disk) for m in FilterGenerator(device_filter)): File "/lib/python3.6/site-packages/ceph/deployment/drive_selection/matchers.py", line 410, in compare raise Exception("No filters applied") Exception: No filters applied I have another error "pgs undersized", maybe this is also causing trouble? [root@gedasvl02 ~]# ceph -s INFO:cephadm:Inferring fsid d0920c36-2368-11eb-a5de-005056b703af INFO:cephadm:Inferring config /var/lib/ceph/d0920c36-2368-11eb-a5de-005056b703af/mon.gedasvl02/config INFO:cephadm:Using recent ceph image docker.io/ceph/ceph:v15 cluster: id: d0920c36-2368-11eb-a5de-005056b703af health: HEALTH_WARN Degraded data redundancy: 13142/39426 objects degraded (33.333%), 176 pgs degraded, 225 pgs undersized services: mon: 1 daemons, quorum gedasvl02 (age 2w) mgr: gedasvl02.vqswxg(active, since 2w), standbys: gedaopl02.yrwzqh mds: cephfs:1 {0=cephfs.gedaopl01.zjuhem=up:active} 1 up:standby osd: 3 osds: 2 up (since 4d), 2 in (since 94m) task status: scrub status: mds.cephfs.gedaopl01.zjuhem: idle data: pools: 7 pools, 225 pgs objects: 13.14k objects, 77 GiB usage: 148 GiB used, 299 GiB / 447 GiB avail pgs: 13142/39426 objects degraded (33.333%) 176 active+undersized+degraded 49 active+undersized io: client: 0 B/s rd, 6.1 KiB/s wr, 0 op/s rd, 0 op/s wr Best Regards, Oliver

2 5

v15.2.7 Octopus released
by David Galloway 30 Nov '20

30 Nov '20

This is the 7th backport release in the Octopus series. This release fixes a serious bug in RGW that has been shown to cause data loss when a read of a large RGW object (i.e., one with at least one tail segment) takes longer than one half the time specified in the configuration option `rgw_gc_obj_min_wait`. The bug causes the tail segments of that read object to be added to the RGW garbage collection queue, which will in turn cause them to be deleted after a period of time. Changelog --------- * rgw: during GC defer, prevent new GC enqueue Getting Ceph ------------ * Git at git://github.com/ceph/ceph.git * Tarball at http://download.ceph.com/tarballs/ceph-15.2.7.tar.gz * For packages, see http://docs.ceph.com/docs/master/install/get-packages/ * Release git sha1: 88e41c6c49beb18add4fdb6b4326ca466d931db8

1 0

librdbpy examples
by Vladimir Prokofev 30 Nov '20

30 Nov '20

Hi. Just want to notice that if you google for ceph python lib examples it leads to 404 https://www.google.ru/search?hl=ru&q=ceph+python+rbd https://docs.ceph.com/en/latest/rbd/api/librbdpy/ Some 3rd party sites and chinese version works fine though http://docs.ceph.org.cn/rbd/librbdpy/ https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/3/html/b…

1 0

high memory usage in osd_pglog
by Robert Brooks 30 Nov '20

30 Nov '20

We are seeing very high osd_pglog usage in mempools for ceph osds. For example... "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 41857200, "bluestore_alloc_items": 523215, "bluestore_cache_data_bytes": 50876416, "bluestore_cache_data_items": 1326, "bluestore_cache_onode_bytes": 6814080, "bluestore_cache_onode_items": 13104, "bluestore_cache_other_bytes": 57793850, "bluestore_cache_other_items": 2599669, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 29904, "bluestore_txc_items": 42, "bluestore_writing_deferred_bytes": 733191, "bluestore_writing_deferred_items": 96, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 101400, "bluefs_items": 1885, "buffer_anon_bytes": 21505818, "buffer_anon_items": 14949, "buffer_meta_bytes": 1161512, "buffer_meta_items": 13199, "osd_bytes": 1962920, "osd_items": 167, "osd_mapbl_bytes": 825079, "osd_mapbl_items": 17, "osd_pglog_bytes": 14099381936, "osd_pglog_items": 134285429, "osdmap_bytes": 734616, "osdmap_items": 26508, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, Where roughly 14g is required for pg_logs. Cluster has 106 OSD and 2432 placement groups. The pg log count for placement groups is much less than 134285429 logs. Top counts are... 1486 1.41c 883 7.3 834 7.f 683 7.13 669 7.a 623 7.5 565 7.8 560 7.1c 546 7.16 544 7.19 Summing these gives 21594 pg logs. Overall the performance of the cluster is poor, OSD memory usage is high (20-30G resident), and with a moderate workload we are seeing iowait on OSD hosts. The memory allocated to caches appears to be low, I believe because osd_pglog is taking most of the available memory. Regards, Rob -- ******************************************************************* This message was sent from RiskIQ, and is intended only for the designated recipient(s). It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not a designated recipient, you may not review, copy or distribute this message. If you receive this in error, please notify the sender by reply e-mail and delete this message. Thank you. *******************************************************************

2 2

CEPH-ISCSI fails when restarting rbd-target-api and won't work anymore
by Hamidreza Hosseini 30 Nov '20

30 Nov '20

I have an issue on ceph-iscsi ( ubuntu 20 LTS and Ceph 15.2.6) after I restart rbd-target-api, it fails and not starting again: ``` sudo systemctl status rbd-target-api.service ● rbd-target-api.service - Ceph iscsi target configuration API Loaded: loaded (/lib/systemd/system/rbd-target-api.service; enabled; vendor preset: enabled) Active: deactivating (stop-sigterm) since Sat 2020-11-28 17:01:40 +0330; 20s ago Main PID: 37651 (rbd-target-api) Tasks: 55 (limit: 9451) Memory: 141.4M CGroup: /system.slice/rbd-target-api.service ├─15289 /usr/bin/python3 /usr/bin/rbd-target-api └─37651 /usr/bin/python3 /usr/bin/rbd-target-api Nov 28 14:36:53 dev11 systemd[1]: Started Ceph iscsi target configuration API. Nov 28 14:36:54 dev11 rbd-target-api[37651]: Started the configuration object watcher Nov 28 14:36:54 dev11 rbd-target-api[37651]: Processing osd blacklist entries for this node Nov 28 14:36:54 dev11 rbd-target-api[37651]: Checking for config object changes every 1s Nov 28 14:36:55 dev11 rbd-target-api[37651]: Reading the configuration object to update local LIO configuration Nov 28 14:36:55 dev11 rbd-target-api[37651]: Processing Gateway configuration Nov 28 14:36:55 dev11 rbd-target-api[37651]: Setting up iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw Nov 28 14:36:55 dev11 rbd-target-api[37651]: (Gateway.load_config) successfully loaded existing target definition Nov 28 17:01:40 dev11 systemd[1]: Stopping Ceph iscsi target configuration API... ``` journalctl: ``` Nov 28 17:00:01 dev11 kernel: Unable to locate Target Portal Group on iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw Nov 28 17:00:01 dev11 kernel: iSCSI Login negotiation failed. Nov 28 17:00:04 dev11 kernel: Unable to locate Target Portal Group on iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw Nov 28 17:00:04 dev11 kernel: iSCSI Login negotiation failed. Nov 28 17:00:06 dev11 ceph-mgr[3184]: [172.16.1.3:57002] [GET] [500] [45.074s] [admin] [513.0B] /api/health/minimal Nov 28 17:00:06 dev11 ceph-mgr[3184]: [b'{"status": "500 Internal Server Error", "detail": "The server encountered an unexpected condition which prevented it from fulfilling the request.", "request_id": "68eed46b-3ece-4e60-bc17-a172358f2d76"} '] Nov 28 17:00:06 dev11 ceph-mgr[3184]: [172.16.1.3:60128] [GET] [500] [45.070s] [admin] [513.0B] /api/health/minimal Nov 28 17:00:06 dev11 ceph-mgr[3184]: [b'{"status": "500 Internal Server Error", "detail": "The server encountered an unexpected condition which prevented it from fulfilling the request.", "request_id": "5b6fdaa2-dc70-48a7-b01f-ca554ecfec41"} '] Nov 28 17:00:07 dev11 kernel: Unable to locate Target Portal Group on iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw Nov 28 17:00:07 dev11 kernel: iSCSI Login negotiation failed. Nov 28 17:00:11 dev11 kernel: Unable to locate Target Portal Group on iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw Nov 28 17:00:11 dev11 kernel: iSCSI Login negotiation failed. Nov 28 17:00:11 dev11 ceph-mgr[3184]: ::ffff:127.0.0.1 - - [28/Nov/2020:17:00:11] "GET /metrics HTTP/1.1" 200 151419 "" "Prometheus/2.7.2" Nov 28 17:00:14 dev11 kernel: Unable to locate Target Portal Group on iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw Nov 28 17:00:14 dev11 kernel: iSCSI Login negotiation failed. Nov 28 17:00:17 dev11 kernel: Unable to locate Target Portal Group on iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw Nov 28 17:00:17 dev11 kernel: iSCSI Login negotiation failed. Nov 28 17:00:20 dev11 kernel: Unable to locate Target Portal Group on iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw Nov 28 17:00:20 dev11 kernel: iSCSI Login negotiation failed. Nov 28 17:00:22 dev11 ceph-mgr[3184]: [172.16.1.3:59834] [GET] [500] [45.062s] [admin] [513.0B] /api/health/minimal Nov 28 17:00:22 dev11 ceph-mgr[3184]: [b'{"status": "500 Internal Server Error", "detail": "The server encountered an unexpected condition which prevented it from fulfilling the request.", "request_id": "1ba61331-1dfd-43e7-8ced-9f28aeb8a39c"} '] Nov 28 17:00:23 dev11 kernel: Unable to locate Target Portal Group on iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw Nov 28 17:00:23 dev11 kernel: iSCSI Login negotiation failed. Nov 28 17:00:26 dev11 kernel: Unable to locate Target Portal Group on iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw Nov 28 17:00:26 dev11 kernel: iSCSI Login negotiation failed. Nov 28 17:00:26 dev11 ceph-mgr[3184]: ::ffff:127.0.0.1 - - [28/Nov/2020:17:00:26] "GET /metrics HTTP/1.1" 200 151420 "" "Prometheus/2.7.2" Nov 28 17:00:27 dev11 ceph-mgr[3184]: [172.16.1.3:60132] [GET] [500] [45.081s] [admin] [513.0B] /api/health/minimal Nov 28 17:00:27 dev11 ceph-mgr[3184]: [b'{"status": "500 Internal Server Error", "detail": "The server encountered an unexpected condition which prevented it from fulfilling the request.", "request_id": "9c1dd49b-07fb-4c49-a033-f5d8d82d9cbe"} '] Nov 28 17:00:29 dev11 kernel: Unable to locate Target Portal Group on iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw Nov 28 17:00:29 dev11 kernel: iSCSI Login negotiation failed. Nov 28 17:00:32 dev11 kernel: Unable to locate Target Portal Group on iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw Nov 28 17:00:32 dev11 kernel: iSCSI Login negotiation failed. Nov 28 17:00:35 dev11 kernel: Unable to locate Target Portal Group on iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw Nov 28 17:00:35 dev11 kernel: iSCSI Login negotiation failed. ``` What should I do for this problem?

2 1

RESTful manager module deprecation
by Ernesto Puerta 30 Nov '20

30 Nov '20

Hi all, Since Ceph-Dashboard RESTful API <https://docs.ceph.com/en/latest/mgr/ceph_api/> is becoming the official RESTful API for Ceph (starting Pacific release), the proposal is to mark RESTful Module <https://docs.ceph.com/en/latest/mgr/restful/> for deprecation by Pacific and for removal on Q-release. You may find a detailed feature-gap analysis in this tracker issue <https://tracker.ceph.com/issues/47066>. We'd like to know about existing users of the RESTful module and their concerns and suggestions for this proposal. Thank you! Kind regards, Ernesto

1 0

Re: rbd image backup best practice
by Janne Johansson 30 Nov '20

30 Nov '20

Den fre 27 nov. 2020 kl 23:21 skrev Marc Roos <M.Roos(a)f1-outsourcing.eu>: > Is there a best practice or guide for backuping rbd images? > One would think that most things that apply to an iscsi mounted device would be equally valid for an RBD mount, so you might look into that for hints and tips on how to backup remote network data devices if you are seeing this from the mounting client point of view. If now, it probably matters what you are aiming for since "backup" is quite a wide concept apart from "copy of my data". Is it best practice for storing sparse images for conserving backup space, quick restores, valid images, legal archiving demands, or "want to try this weird update and be able to move backwards 30 minutes" ? The solution will be quite different depending on what the problem is, more than perhaps "what the mountpoint type is". -- May the most significant bit of your life be positive.

2 1

Re: Manual bucket resharding problem
by Mateusz Skała 30 Nov '20

30 Nov '20

Thank You for response, how I can upload this to metadata? Is this operation safe? Regards Mateusz Skała W dniu sob., 21.11.2020 o 18:01 Amit Ghadge <amitg.b14(a)gmail.com> napisał(a): > I go through this and you need to update bucket metadata, radosgw-admin > metadata get bucket.instance:bucket:xxx > bucket.json, update two parameter > I don't remember but it's look reshard: false and next_marker set empty. > > -AmitG > On Sat, 21 Nov, 2020, 2:04 PM Mateusz Skała, <mateusz.skala(a)gmail.com> > wrote: > >> Hello Community. >> I need Your help. Few days ago I started manual resharding of one bucket >> with large objects. Unfortunately I interrupted this by Ctrl+c. At now I >> can’t start this process again. >> There is message: >> # radosgw-admin bucket reshard --bucket objects --num-shards 2 >> ERROR: the bucket is currently undergoing resharding and cannot be added >> to the reshard list at this time >> >> But list of reshard process is empty: >> # radosgw-admin reshard list >> [] >> >> # radosgw-admin reshard status --bucket objects >> [ >> { >> "reshard_status": "not-resharding", >> "new_bucket_instance_id": "", >> "num_shards": -1 >> } >> ] >> >> How can I fix this situation ? How to restore possibility resharding this >> bucket? >> And BTW is resharding process locking writes/reads on bucket? >> Regards >> Mateusz Skała >> _______________________________________________ >> ceph-users mailing list -- ceph-users(a)ceph.io >> To unsubscribe send an email to ceph-users-leave(a)ceph.io >> >

2 3

rbd image backup best practice
by Marc Roos 30 Nov '20

30 Nov '20

Is there a best practice or guide for backuping rbd images?

2 1