Evening,
We are running into issues exporting a disk image from ceph rbd. When we attempt to export
an rbd image in a cache tiered erasure-coded pool on Luminus.
All the other disks are working fine but this one is acting up. We have a bit of important
data on other disks so obviously want to make sure this doesn't happen to those.
[root@ceph-p-mon1 home]# rbd export one/one-177-588-0 one-177-588-0
Exporting image: 8% complete...rbd: error reading from source image at offset 5456789504:
(5) Input/output error
2020-03-23 20:11:29.210718 7f2f3effd700 -1 librbd::io::ObjectRequest: 0x7f2f2c128f90
handle_read_object: failed to read from object: (5) Input/output error
2020-03-23 20:11:29.565184 7f2f3e7fc700 -1 librbd::io::ObjectRequest: 0x7f2f280c84d0
handle_read_cache: failed to read from cache: (5) Input/output error
Exporting image: 8% complete...failed.
rbd: export error: (5) Input/output error
Any thoughts would be appreciated.
Some info:
[root@ceph-p-mon1 home]# rbd info one/one-177-588-0
rbd image 'one-177-588-0':
size 58.6GiB in 15000 objects
order 22 (4MiB objects)
block_name_prefix: rbd_data.84a01279e2a9e3
format: 2
features: layering, exclusive-lock, object-map, fast-diff, deep-flatten
flags:
create_timestamp: Fri Apr 20 17:06:09 2018
parent: one/one-177@snap
overlap: 2.20GiB
[root@ceph-p-mon1 home]# ceph status
cluster:
id: 6a2e8f21-bca2-492b-8869-eecc995216cc
health: HEALTH_OK
services:
mon: 3 daemons, quorum ceph-p-mon2,ceph-p-mon1,ceph-p-mon3
mgr: ceph-p-mon2(active)
mds: cephfsec-1/1/1 up {0=ceph-p-mon2=up:active}, 6 up:standby
osd: 155 osds: 154 up, 154 in
data:
pools: 6 pools, 5904 pgs
objects: 145.53M objects, 192TiB
usage: 253TiB used, 290TiB / 543TiB avail
pgs: 5896 active+clean
8 active+clean+scrubbing+deep
io:
client: 921KiB/s rd, 5.68MiB/s wr, 110op/s rd, 29op/s wr
cache: 5.65MiB/s flush, 0op/s promote
[root@ceph-p-mon1 home]# rpm -qa | grep ceph
ceph-common-12.2.9-0.el7.x86_64
ceph-mds-12.2.9-0.el7.x86_64
ceph-radosgw-12.2.9-0.el7.x86_64
ceph-mgr-12.2.9-0.el7.x86_64
ceph-12.2.9-0.el7.x86_64
collectd-ceph-5.8.1-1.el7.x86_64
ceph-deploy-2.0.1-0.noarch
libcephfs2-12.2.9-0.el7.x86_64
python-cephfs-12.2.9-0.el7.x86_64
ceph-selinux-12.2.9-0.el7.x86_64
ceph-osd-12.2.9-0.el7.x86_64
ceph-base-12.2.9-0.el7.x86_64
ceph-mon-12.2.9-0.el7.x86_64
ceph-release-1-1.el7.noarch
Rhian Resnick
Associate Director Research Computing
Enterprise Systems
Office of Information Technology
Florida Atlantic University
777 Glades Road, CM22, Rm 173B
Boca Raton, FL 33431
Phone 561.297.2647
Fax 561.297.0222
[image] <https://hpc.fau.edu/wp-content/uploads/2015/01/image.jpg>
Show replies by date