- ceph-users - lists.ceph.io

Can kstore be used as OSD objectstore backend when deploying a Ceph Storage Cluster? If can, how to?

by R.R.Yuan

Hi, All, When deploying a development cluster, there are three types of OSD objectstore backend: filestore, bluestore and kstore. But there is no "--kstore" option when using "ceph-deploy osd"command to deploy a real ceph cluster. Can kstore be used as OSD objectstore backend when deploy a real ceph cluster? If can, how to ? Thanks a lot R.R.Yuan

4 years, 9 months

1
0
0 0

Can kstore be used as OSD objectstore backend when deploying a Ceph Storage Cluster? If can, how to?

by R_R_Yuan＠163.com

Hi, All, When deploying a development cluster, there are three types of OSD objectstore backend: filestore, bluestore and kstore. But there is no "--kestore" option when using "ceph-deploy osd" command to deploy a real ceph cluster. Can kstore be used as OSD objectstore backend when deploy a real ceph cluster? If can, how to ? Thanks a lot!

4 years, 9 months

1
0
0 0

Multi zone syncing not working

by Freddy Andersen

Hi, I have tried to do this twice now and can’t get it to work 100%. In my last attempt I have secondary zone correctly syncing: sudo radosgw-admin sync status realm 2cc7226c-2cce-4b75-a719-97ba22756289 (prod) zonegroup 04702b5e-faa4-4110-9e98-6097f5e5e4b9 (us) zone ece9f510-e910-49be-a89f-7b1f47b3faf8 (zone2) metadata sync syncing full sync: 0/64 shards incremental sync: 64/64 shards metadata is caught up with master data sync source: 8f0e7a41-f71c-45e8-869a-9bfbadbcbf2c (zone1) syncing full sync: 0/128 shards incremental sync: 128/128 shards data is caught up with source but master zone can’t sync from secondary: sudo radosgw-admin sync status realm 2cc7226c-2cce-4b75-a719-97ba22756289 (prod) zonegroup 04702b5e-faa4-4110-9e98-6097f5e5e4b9 (us) zone 8f0e7a41-f71c-45e8-869a-9bfbadbcbf2c (zone1) metadata sync no sync (zone is master) 2019-08-06 16:43:36.228 7f059c7ab640 0 data sync zone:ece9f510 ERROR: failed to fetch datalog info data sync source: ece9f510-e910-49be-a89f-7b1f47b3faf8 (zone2) failed to retrieve sync info: (13) Permission denied I can see in our secondary rgw hosts that there is a 403 from master zone 2019-08-06 16:27:58.137 7ff3f3d30700 1 ====== req done req=0x7ff3f3d298d0 op status=0 http_status=403 latency=0s ====== 2019-08-06 16:27:58.137 7ff3f3d30700 1 civetweb: 0x563d52f64000: 10.130.30.210 - - [06/Aug/2019:16:27:58 -0700] "GET /admin/log?type=data&rgwx-zonegroup=04702b5e-faa4-4110-9e98-6097f5e5e4b9 HTTP/1.1" 403 320 - -

4 years, 9 months

1
0
0 0

rados cache-flush-evict-all unable to flush all data into backend storage

by 化氢

hi， I created a cache layer using ceph osd tier. When I remove the cache layer, I use cache-flush-evict-all to synchronize the data, but the cache layer sometimes has several objects left, is this a bug? thanks

4 years, 9 months

1
0
0 0

Migrate RBD header and metadata for EC backed image to new pool

by Jason Lorsung

I'm looking at decommissioning a pool (rbd) that is the header and metadata storage pool for a number of ec backed images (created with --data-pool ecrbd). Pool ecrbd is remaining intact so I don't want to have to migrate all the image data if unnecessary. Is there a way to migrate all the header and metadata off of pool rbd to pool rbdnew, while leaving all the data inplace?

4 years, 9 months

1
0
0 0

OSD crashes on start - while loading pgs

by Jesper Stemann Andersen

Hi, Any ideas for resolving an issue where an OSD crashes on start-up? I have one (large hdd) OSD that will no longer start – it crashes while loading pgs - see attached log file - excerpt below: 2019-08-02 10:08:21.021207 7fea86d7be00 0 osd.1 1844 load_pgs 2019-08-02 10:08:39.370112 7fea86d7be00 -1 *** Caught signal (Aborted) ** in thread 7fea86d7be00 thread_name:ceph-osd ceph version 12.2.12 (39cfebf25a7011204a9876d2950e4b28aba66d11) luminous (stable) 1: (()+0xa59c94) [0x55b835a6dc94] 2: (()+0x110e0) [0x7fea843800e0] 3: (gsignal()+0xcf) [0x7fea83347fff] 4: (abort()+0x16a) [0x7fea8334942a] 5: (__gnu_cxx::__verbose_terminate_handler()+0x15d) [0x7fea83c600ad] 6: (()+0x8f066) [0x7fea83c5e066] 7: (()+0x8f0b1) [0x7fea83c5e0b1] 8: (()+0x8f2c9) [0x7fea83c5e2c9] 9: (pg_log_entry_t::decode_with_checksum(ceph::buffer::list::iterator&)+0x156) [0x55b8356f57c6] 10: (void PGLog::read_log_and_missing<pg_missing_set<true> >(ObjectStore*, coll_t, coll_t, ghobject_t, pg_info_t const&, PGLog::IndexedLog&, pg_missing_set<true>&, bool, std::__cxx11::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >&, bool, bool*, DoutPrefixProvider const*, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >*, bool)+0x1ab4) [0x55b8355a6584] 11: (PG::read_state(ObjectStore*, ceph::buffer::list&)+0x38b) [0x55b83554b7eb] 12: (OSD::load_pgs()+0x8b8) [0x55b835496678] 13: (OSD::init()+0x2237) [0x55b8354b75c7] 14: (main()+0x3092) [0x55b8353bf1c2] 15: (__libc_start_main()+0xf1) [0x7fea833352e1] 16: (_start()+0x2a) [0x55b83544b8ca] I have ensured that kernel.pid_max is set to a high value – sysctl reports kernel.pid_max = 4194304 This issue arose following an expansion of the ceph cluster: https://forum.proxmox.com/threads/unable-to-start-osd-crashes-while-loading… In summary: I added a third node, with extra OSD’s, and increased pg_num and pgp_num for one pool before the cluster had settled. However, by now the cluster has settled – I no longer have the global setting mon_max_pg_per_osd = 1000. Only the issue with the OSD that will not start remains. Best regards, Jesper Stemann Andersen Lead Software Engineer, R&D IHP Systems A/S +45 26 25 23 91

4 years, 9 months

1
0
0 0

Help needed pls (rbd cache)

by Muhammad Junaid

Hi there, Sorry for asking a question, which may be of very basic nature and asked many times before. But much Google search can not satisfy me. The question is about RBD Cache in write-back mode using KVM/libvirt. If we enable this, it uses local KVM Host's RAM as cache for VM's write requests. And KVM Host immediately responds to VM's OS that data has been written to Disk (Actually it is still not on OSD's yet). Then how can be it power failure safe? Is my understanding correct? If not, pls correct. This is very important for me. Thank you very much in advance. Best regards. Muhammad Junaid

4 years, 9 months

2
2
0 0

Ceph packages for Debian buster

by yuval.freund＠cloud.ionos.com

We've been using ceph on Debian for several years now, and would really like seeing official packages for buster. We recently resorted to building our own Luminous packages for buster (based on the Ubuntu packages), due to the fact that: * We've already upgraded our clusters to 12.2.12 , and the debian.org packages offer 12.2.11 . * The official ceph packages for stretch are built against libcurl3, which has been removed in buster in favor of libcurl4 . Sage stated last year that Luminous and Nautilus (and possibly Mimic) packages for buster should be provided at some point (http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-June/027478.html), and we really hope that this decision hasn't been reverted - a lot of Debian users are looking forward to being able to upgrade their clusters to a newer release. We also believe having official Luminous packages will allow most Debian users who've kept their systems up-to-date to have a clean upgrade path. We are aware of croit's Nautilus packages for buster and have played around with them (thanks!), but would rather go with official packages in our production clusters, preferably built against python3 (for bonus points ;) ). We're also undecided regarding whether we should go straight to Nautilus, mainly due to the fact Mimic seems like the more stable choice for the time being. We'd be happy and willing to assist in any way we can. Cheers, Yuval -- Yuval Freund System Engineer 1&1 IONOS Cloud GmbH | Greifswalder Str. 207 | 10405 Berlin | Germany Web: www.ionos.de Head Office: Berlin, Germany District Court Berlin Charlottenburg, Registration number: HRB 125506 B Executive Management: Christoph Steffens, Matthias Steinberg, Achim Weiss Member of United Internet

4 years, 9 months

1
0
0 0

Ceph rbd cannot delete after I reset ceph cluster

by xu xingci

Hi guys: I had setup ceph cluster and mount rbd on one machine. I delete ceph cluster and reinstall follow the manual. but I still have rbd device mount on my machine. I can not access mount point. This is my detail info, I want to delete all old rbd device, what should I do? node1 $> rbd device list id pool namespace image snap device 0 rbd foo - /dev/rbd0 1 kube kubernetes-dynamic-pvc-1cc43c5b-ade1-11e9-9a92-863e3c12afd1 - /dev/rbd1 node1 $> df Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/rootvg-lv_root 10995712 2528388 8467324 23% / devtmpfs 2012656 0 2012656 0% /dev tmpfs 2023588 0 2023588 0% /dev/shm tmpfs 2023588 207340 1816248 11% /run tmpfs 2023588 0 2023588 0% /sys/fs/cgroup /dev/sda1 520868 116936 403932 23% /boot /dev/mapper/rootvg-lv_var 5232640 3226816 2005824 62% /var /dev/mapper/rootvg-lv_tmp 5232640 33060 5199580 1% /tmp /dev/rbd0 3997376 16392 3754888 1% /mnt tmpfs 404720 0 404720 0% /run/user/1001 node1 $> rbd trush list rbd: error opening default pool 'rbd' Ensure that the default pool has been created or specify an alternate pool name. node1 $> rbd info rbd/foo rbd: error opening default pool 'rbd' Ensure that the default pool has been created or specify an alternate pool name. [https://ipmcdn.avast.com/images/icons/icon-envelope-tick-round-orange-anima…]<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> 无病毒。www.avast.com<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…>

4 years, 9 months

1
0
0 0

Help pls (RBD Cache)

by Muhammad Junaid

Hi there, Sorry for asking a question, which may be of very basic nature and asked many times before. But much Google search can not satisfy me. The question is about RBD Cache in write-back mode using KVM/libvirt. If we enable this, it uses local KVM Host's RAM as cache for VM's write requests. And KVM Host immediately responds to VM's OS that data has been written to Disk (Actually it is still not on OSD's yet). Then how can be it power failure safe? Is my understanding correct? If not, pls correct. This is very important for me. Thank you very much in advance. Best regards. Muhammad Junaid

4 years, 9 months

1
0
0 0

2024

2023

2022

2021

2020

2019

ceph-users