Hello everyone,
every rgw request comes down core functions ObjectReadOperation and
ObjectWriteOperation, defined librados.hpp file, the and rgw code calls
this functions via rgw_tools.cc rgw_rados_operate(), is it necessary to
trace these functions because due to async nature of these functions I have
not able to trace these functions.
So is it okay if I dont trace these functions?
Around 18JUN2020 0700 UTC, an errant `sudo rm -rf ceph` from the root
directory on a senta unfortunately wiped out almost all data on the Ceph
cluster in our upstream Sepia lab (AKA Long Running Cluster or LRC).
Only teuthology job logs were preserved.
I would guess because teuthology workers were actively writing jobs logs
and files, the /teuthology-archive directory didn't get entirely wiped out.
Here is a list of directories we lost:
bz
cephdrop (drop.ceph.com)
cephfs-perf
chacra (chacra.ceph.com)
containers (quay.ceph.io)
dgalloway
diskprediction_config.txt
doug-is-great
el8
filedump.ceph.com
firmware
home.backup01
home.gitbuilder-archive
job1.0.0
jspray.senta02.home.tar.gz
old.repos
post (files submitted using ceph-post-file)
sftp (drop.ceph.com/qa)
shaman
signer (signed upstream release packages)
tmp
traces
While I /did/ have backups of chacra.ceph.com binaries, the amount of
data (> 1TB) backed up was too much to keep snapshots of. My daily
backup script performs an `rsync --delete-delay` so if files are gone on
the source, they get deleted from the backup. This is fine (and
preferred) for backups we have snapshots of. However, the backup script
ran *after* the errant `rm -rf` so unfortunately everything on
chacra.ceph.com is gone. I have patched the backup script to *not*
--delete-delay backups that we don't keep snapshots of.
I restored the vagrant and valgrind chacra.ceph.com repos because I saw
teuthology jobs failing because of those missing repos. Kefu also
rebuilt and pushed ceph-libboost 1.72. (THANK YOU, KEFU!)
We started using the quay.ceph.io registry (instead of quay.io) on June
17. Containers pushed to that registry were stored on the LRC as well
so I had to delete the repo and start over this morning. Anything you
see in the web UI should pull without issue:
https://quay.ceph.io/repository/ceph-ci/ceph?tab=tags
To prevent data loss in the future, Patrick graciously set up new
filesystems and client credentials on the LRC. Because senta{02..04}
are considered developer playgrounds, all users have sudo access. The
sentas now mount /teuthology-archive read-only at /teuthology. If you
need to unzip and inspect log files on a senta, you can do so in
/scratch (another new filesystem on the LRC).
It will likely take weeks of "where did X go" e-mails to mailing lists,
job and build failures, bugs filed, IRC pings, etc. for me to find and
restore everything that was used on a regular basis. I appreciate your
patience and understanding in the meantime.
Take care & be well,
--
David Galloway
Systems Administrator, RDU
Ceph Engineering
IRC: dgalloway
Dev leads and all
We are getting ready for the next point release 15.2.4.
Please make effort to prep backports for it and tag PRs with
`octopus-batch-1`+`needs-qa` ASAP.
Thx
YuriW
(I sent this to a smaller list of folks yesterday, but I think it
probably warrants wider discussion).
Recently Varsha added the necessary infrastructure to bring up
nfs-ganesha via vstart.sh. The current implementation requires that
ganesha already be installed on the box (usually via distro packaging),
but that poses a bit of a problem.
A distro ganesha package will have likely been built vs. a completely
different version of libcephfs and librados. Even if you build right off
of ceph master branch, you won't get the benefit of any recent client
bugfixes when you want to test ganesha. You'd have to build new ganesha
packages, install them, etc.
I think we ought to consider making a nfs-ganesha build an optional part
of a ceph build (maybe enable it with cmake -DNFS_GANESHA=ON or
something).
It doesn't take very long to build it (typically only a minute or two on
my box), and we could disable the parts that ceph doesn't care about
(other FSALs primarily). We could also have vstart just error out when
you run it with NFS=X on a build that didn't have ganesha enabled.
OTOH, the potential downside here is that it'll likely add other build-
time dependencies, and would require some extra cmake or scripting
wizardry. Nothing insurmountable, but it might represent a maintenance
burden going forward, particularly for something that's basically only
going to be used for vstart.
I'm also not sure how we'd do this in practice. I don't think you can do
optional submodules, so we might have to look at other methods of
pulling in the ganesha tree, or just live with it as a submodule that
only gets used when ganesha is enabled.
Thoughts?
--
Jeff Layton <jlayton(a)redhat.com>
+ceph dev
On Sat, 13 Jun 2020, 20:34 Amit Ghadge, <amitg.b14(a)gmail.com> wrote:
> Hello All,
>
> We saw all nodes mon services are restarted at the same time after
> enabling msgr2, So this make an impact on production running cluster? We
> are upgrading from Luminous to Nautilus.
>
>
>
> Thanks,
> AmitG
>
Hi Justin,
The EPEL 8 repository has all the Ceph packages that we need now, right?
Can we delete all the existing builds from
https://copr.fedorainfracloud.org/coprs/ktdreyer/ceph-el8/ ?
I think that would make it clear to everyone that we don't need this
Copr repository anymore.
- Ken
Hello everyone, I found that s3 has the api for rgw request like post
object,bucket notification,multipart upload but I cant find the
implementation of those. Can someone show me or give me a link about those
implementations,
Thank You
Hi,
I have a question regarding Ceph CRUSH. I have been going through Crush.h
file. It says that *struct crush_bucket **buckets * (below) is an array of
pointers. My understanding is that this particular array of pointers is a
collection of addresses of six scalar values namely __s32 id; __u16
type; __u8 alg, __u8 hash, __u32 weight, __u32 size and the reason it
has double pointer **buckets because it also points to another pointer
namely __s32 *items? Please correct me if I am wrong.
/** @ingroup API
*
* A crush map define a hierarchy of crush_bucket that end with leaves
* (buckets and leaves are called items) and a set of crush_rule to
* map an integer to items with the crush_do_rule() function.
*
*/
struct crush_map {
/*! An array of crush_bucket pointers of size __max_buckets__.
* An element of the array may be NULL if the bucket was removed
with
* crush_remove_bucket(). The buckets must be added with
crush_add_bucket().
* The bucket found at __buckets[i]__ must have a crush_bucket.id
== -1-i.
*/
struct crush_bucket **buckets;
/*! An array of crush_rule pointers of size __max_rules__.
* An element of the array may be NULL if the rule was removed
(there is
* no API to do so but there may be one in the future). The rules
must be added
* with crush_add_rule().
*/
struct crush_rule **rules;
__s32 max_buckets; /*!< the size of __buckets__ */
__u32 max_rules; /*!< the size of __rules__ */
/*! The value of the highest item stored in the crush_map + 1
*/
__s32 max_devices;
/*! Backward compatibility tunable. It implements a bad solution
* and must always be set to 0 except for backward compatibility
* purposes
*/
__u32 choose_local_tries;
/*! Backward compatibility tunable. It implements a bad solution
* and must always be set to 0 except for backward compatibility
* purposes
*/
__u32 choose_local_fallback_tries;
/*! Tunable. The default value when the CHOOSE_TRIES or
* CHOOSELEAF_TRIES steps are omitted in a rule. See the
* documentation for crush_rule_set_step() for more
* information
*/
__u32 choose_total_tries;
/*! Backward compatibility tunable. It should always be set
* to 1 except for backward compatibility. Implemented in 2012
* it was generalized late 2013 and is mostly unused except
* in one border case, reason why it must be set to 1.
*
* Attempt chooseleaf inner descent once for firstn mode; on
* reject retry outer descent. Note that this does *not*
* apply to a collision: in that case we will retry as we
* used to.
*/
__u32 chooseleaf_descend_once;
/*! Backward compatibility tunable. It is a fix for bad
* mappings implemented in 2014 at
* https://github.com/ceph/ceph/pull/1185. It should always
* be set to 1 except for backward compatibility.
*
* If non-zero, feed r into chooseleaf, bit-shifted right by
* (r-1) bits. a value of 1 is best for new clusters. for
* legacy clusters that want to limit reshuffling, a value of
* 3 or 4 will make the mappings line up a bit better with
* previous mappings.
*/
__u8 chooseleaf_vary_r;
/*! Backward compatibility tunable. It is an improvement that
* avoids unnecessary mapping changes, implemented at
* https://github.com/ceph/ceph/pull/6572 and explained in
* this post: "chooseleaf may cause some unnecessary pg
* migrations" in October 2015
*
https://www.mail-archive.com/ceph-devel@vger.kernel.org/msg26075.html
* It should always be set to 1 except for backward compatibility.
*/
__u8 chooseleaf_stable;
/*! @cond INTERNAL */
/* This value is calculated after decode or construction by
the builder. It is exposed here (rather than having a
'build CRUSH working space' function) so that callers can
reserve a static buffer, allocate space on the stack, or
otherwise avoid calling into the heap allocator if they
want to. The size of the working space depends on the map,
while the size of the scratch vector passed to the mapper
depends on the size of the desired result set.
Nothing stops the caller from allocating both in one swell
foop and passing in two points, though. */
size_t working_size;
#ifndef __KERNEL__
/*! @endcond */
/*! Backward compatibility tunable. It is a fix for the straw
* scaler values for the straw algorithm which is deprecated
* (straw2 replaces it) implemented at
* https://github.com/ceph/ceph/pull/3057. It should always
* be set to 1 except for backward compatibility.
*
*/
__u8 straw_calc_version;
/*! @cond INTERNAL */
/*
* allowed bucket algs is a bitmask, here the bit positions
* are CRUSH_BUCKET_*. note that these are *bits* and
* CRUSH_BUCKET_* values are not, so we need to or together (1
* << CRUSH_BUCKET_WHATEVER). The 0th bit is not used to
* minimize confusion (bucket type values start at 1).
*/
__u32 allowed_bucket_algs;
__u32 *choose_tries;
#endif
/*! @endcond */
};
BR
Hi all,
FYI, I created a new github team "Doc writers" and added some inaugural
members that I could think of:
https://github.com/orgs/ceph/teams/doc-writers/members
The purpose of this team is to improve the handling of pull requests
that contain documentation changes, by getting notified on PRs that
change documentation files.
I also updated the CODEOWNERS file accordingly:
CODEOWNERS: added doc-writers team — https://github.com/ceph/ceph/pull/35604
Please let me know if you would like to be added to the team and please
review/comment on the pull request - did I capture the relevant files?
Thanks,
Lenz
--
SUSE Software Solutions Germany GmbH - Maxfeldstr. 5 - 90409 Nuernberg
GF: Felix Imendörffer, HRB 36809 (AG Nürnberg)
I'm happy to announce the another release of the go-ceph API
bindings. This is a regular release following our every-two-months release
cadence.
https://github.com/ceph/go-ceph/releases/tag/v0.4.0
The bindings aim to play a similar role to the "pybind" python bindings in the
ceph tree but for the Go language. These API bindings require the use of cgo.
There are already a few consumers of this library in the wild and the
ceph-csi project is starting to make use of this library.
Specific questions, comments, bugs etc are best directed at our github issues
tracker.
---
John Mulligan
phlogistonjohn(a)asynchrono.us
jmulligan(a)redhat.com
