December 2019 - Dev - lists.ceph.io

by Yuri Weinstein

Details of this release summarized here: https://tracker.ceph.com/issues/43128#note-1 This is a short release and included limited number of suites to test. smoke - PASSED rados - PASSED rgw - FAILED, Casey pls affirm approval. upgrade/client-upgrade-jewel - PASSED upgrade/client-upgrade-luminous - PASSED upgrade/luminous-x (mimic) - PASSED upgrade/mimic-x (nautilus) - FAILED, Sage, Neha pls review upgrade/mimic-p2p - PASSED Thx YuriW

3 years, 3 months

5
5
0 0

12/12/2019 perf meeting is on at 8AM PST!

by Mark Nelson

Hi Folks, Perf meeting is on in ~35 minutes! Let's continue the conversation from last week about nvdimms. Today we have new test results showing the effect of PGLog omap removal and pglog removal (mostly). Feel free to add your own topic as well! Etherpad: https://pad.ceph.com/p/performance_weekly Bluejeans: https://bluejeans.com/908675367 Thanks, Mark

3 years, 3 months

1
0
0 0

cephadm vs ceph-container

by Sage Weil

ceph-daemon and mgr/ssh are now (or soon to be) cephadm: https://github.com/ceph/ceph/pull/32193 We want to merge this as quickly as possible to unblock other conflicting work. The key change is that ceph-mgr-ssh is now ceph-mgr-cephadm. Note that this package is only needed on master (not nautilus or earlier). And ceph-mgr-ssh isn't needed on nautilus or earlier (tho it was there before). Here's my attempt: https://github.com/ceph/ceph-container/pull/1533 Will that work? Or can you adjust as needed? A bunch of the qa tests mostly on the latest-master-devel image, but that's won't work until the above two PRs merge. So basically as soon as we can merge something into ceph-container that includes the new package (and neither for nautilus), we can merge the ceph PR too. Thanks! sage

3 years, 3 months

2
1
0 0

DocuBetter APAC -- 2019 Dec 11

by John Zachary Dover

Hi everyone. The next DocuBetter meeting is scheduled for: 11 Dec 2019 0930 PST 11 Dec 2019 1730 UTC 12 Dec 2019 0330 AEST Etherpad: https://pad.ceph.com/p/Ceph_Documentation Meeting: https://bluejeans.com/908675367 Agenda: 1. Resolution of outstanding Sphinx issues 2. Documentation Bug Progress Report 3. Old Business -- CephFS manual 4. Old Business -- RADOS Documentation 5. Old Business -- docs bug reporting and improvement-request procedure Thanks, everyone. Zac Dover

3 years, 3 months

1
0
0 0

rgw: decoupling the bucket replication log from the bucket index

by Casey Bodley

The bucket index logs used for multisite replication are currently stored in omap on the bucket index shards, along with the rest of the bucket index entries. Storing them in the index was a natural choice, because cls_rgw can write these log entries atomically when completing a bucket index transaction. To replicate a bucket, other zones process each of its bucket index shard logs independently, and store sync status markers with their position in each shard. This tight coupling between the replication strategy and the bucket's sharding scheme is the main challenge to supporting bucket resharding in multisite, because shuffling these log entries to a new set of shards would invalidate the sync status markers stored in other zones. My proposal, then, is to move the replication logs out of bucket index shards into a single log per bucket, and extend the consistency model to make up for the lack of atomic writes that we get from cls_rgw. The existing consistency model for object writes involves a) calling cls_rgw to prepare a bucket index transaction, b) writing the object's head to the data pool, then c) calling cls_rgw to complete the transaction. Since the write in b) is what makes the object visible to GET requests, we can reply to the client without waiting for c) to finish. If either b) or c) fails, the next bucket listing will find an entry that was prepared but not completed, and we'll check whether the head object exists and use the 'dir suggest' call to update the bucket index accordingly. If we move the replication log to a separate object, we'll need to write to that as well before completing the transaction. And when dir suggest finds head objects for uncompleted transactions, it can (re)write their replication log entries before updating the bucket index. This recovery means that we can still reply to the client before writing to the replication log, so the client won't see any extra latency. This change also gives us the opportunity to move away from omap and the challenges associated with trimming. Yehuda wrote cls_fifo in https://github.com/ceph/ceph/pull/30797 with the datalog in mind, and that could be a good fit for these bucket replication logs as well.

3 years, 3 months

4
7
0 0

Cephalocon 2020

by Sage Weil

Hi everyone, The next Cephalocon is coming up on March 3-5 in Seoul! The CFP is open until Friday (get your talks in!). We expect to have the program ready for the first week of January. Registration (early bird) will be available soon. We're also looking for sponsors for the conference. The prospectus is available here: https://ceph.io/wp-content/uploads/2019/12/sponsor-Cephalocon20-112719.pdf Thanks! sage

3 years, 3 months

1
1
0 0

using ceph-el8 copr repo today

by Alfredo Deza

Hi Ken There is a lot of interest in getting the Python3/CentOS8 pr in master merged [0], but we can't do that at the moment since the dependencies are missing. As you know, we've been building everything we need in the ceph-el8 copr repo [1], and I would like your take on merging the PR in master while we wait to get packages in EPEL8. This would unblock development in master that can't move forward unless the CentOS8 support is there. I value your guidance here, and I'd like to know if you are OK with this approach, or if you have any other ideas to achieve progress. Thanks! [0] https://github.com/ceph/ceph/pull/25969 [1] https://copr.fedorainfracloud.org/coprs/ktdreyer/ceph-el8/

3 years, 3 months

4
4
0 0

renaming ceph-daemon

by Sage Weil

During CDM on Wednesday I suggested that ceph-daemon should probably be renamed before octopus (and before the name sticks). In speech "ceph daemon" is confusing, and the name doesn't really reflect what it is: a user-facing tool - to bootstrap a new cluster, - launch a (containerized) shell, - enter an existing daemon container, - tail a daemon's log, or - adopt a daemon deployed with a legacy tool (ceph-deploy, ceph-ansible, etc) into a ceph-daemon style container - remove all trace of a cluster from the localhost, and an internal tool used by ssh-orch to - deploy or remove a container running a ceph daemon - start, stop, or update an existing container - run ceph-volume (to gather device inventory, create osds, zap, etc.) - gather a host inventory of services (containers) The original tool was more like "ceph daemon tool" but it was shortened to ceph-daemon at the start. We voted on some alternatives here: https://pad.ceph.com/p/ceph-daemon-tool but the winner is currently 'cephctl' with is IMO a non-starteer. Usually 'ctl' utilities are for interacting with running systems/daemons via some runtime API--that's basically what the 'ceph' CLI utility is. ceph-daemon is more akin to kubeadm or ceph-deploy or something like that. So... I shortened/pruned the list again. Please weigh in. And if you have a bright idea for a better name, feel free to add it. Thanks! sage

3 years, 3 months

11
15
0 0

_get_ceph_option Function not find when running tox-run-mgr

by Willem Jan Withagen

Since a few days my jenkins tests fail in running tox-run-mgr. And in my places, but all look like: What am I missing here? Thanx, --WjW ssh/module.py:243: in __init__ super(SSHOrchestrator, self).__init__(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <ssh.module.SSHOrchestrator object at 0x803113ba8>, module_name = 'ssh' py_modules_ptr = 0, this_ptr = 0 def __init__(self, module_name, py_modules_ptr, this_ptr): self.module_name = module_name super(MgrModule, self).__init__(py_modules_ptr, this_ptr) > mgr_level = self._ceph_get_option("debug_mgr") E AttributeError: 'SSHOrchestrator' object has no attribute '_ceph_get_option' mgr_module.py:644: AttributeError

3 years, 3 months

2
2
0 0

Multi-site sync failed when buckets uses not default placement in source zone

by liuchang0812

Hi, @Yehuda Sadeh-Weinraub <yehuda(a)redhat.com> @Casey Bodley <cbodley(a)redhat.com> @Matt Benjamin <mbenjamin(a)redhat.com> and Cephers We met a problem with the Elastic Search sync module: bucket with custom placement could not be synced to the target zone. The target zone tries to create a bucket instance based on the placement name, but the target zone does have the placement. logs as following: meta sync: ERROR: can't store key: bucket.instance: * data sync: ERROR: failed to fetch bucket instance info for * ERROR: select_bucket_placement() returned -22 we are going to fix this issue. here are some questions: 1. should we sync those buckets to the ES zone? it seems not necessary 2. should we sync placements? if we sync placements to the target zone, we also need to create rados pools in the target zone thanks Chang Liu

3 years, 3 months

2
3
0 0

2023

2022

2021

2020

2019

Dev December 2019