September 2019 - Dev - lists.ceph.io

by Patrick Donnelly

For developers submitting jobs using teuthology, we now have recommendations on what priority level to use: https://docs.ceph.com/docs/master/dev/developer_guide/#testing-priority -- Patrick Donnelly, Ph.D. He / Him / His Senior Software Engineer Red Hat Sunnyvale, CA GPG: 19F28A586F808C2402351B93C3301A3E258DD79D

6 months, 1 week

5
7
0 0

build on RHEL8

by kefu chai

hi folks, just want to share my findings regarding to building ceph on RHEL8 here. today, i was trying to build ceph on RHEL8. but it seems we are missing some build dependencies on this distro, because quite a few packages were removed from RHEL8 [0], and these packages are still missing EPEL8: No matching package to install: 'gperftools-devel >= 2.6.1' No matching package to install: 'leveldb-devel > 1.2' No matching package to install: 'libbabeltrace-devel' No matching package to install: 'liboath-devel' No matching package to install: 'python3-cherrypy' No matching package to install: 'python3-coverage' No matching package to install: 'python3-pecan' No matching package to install: 'python3-routes' No matching package to install: 'python3-tox' No matching package to install: 'xmlstarlet' i've added EPEL8 repo by following [1]. probably packages being added to EPEL8 in future will help to ease the pain. but at this moment, some unnecessary dependencies are just missing. cheers, --- [0] https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/ht… [1] https://fedoraproject.org/wiki/EPEL -- Regards Kefu Chai

3 years, 10 months

2
3
0 0

Authentication failure at radosgw for presigned urls

by Biswajeet Patra

Hi, We recently faced an issue with radosgw authentication of presigned urls. The presigned url generated by client to download object fails at radosgw with http error 403 i.e SignatureDoesNotMatch. The radosgw computes the signature (*v2 signature in this case*) using the s3 specification https://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html. In this process, a string_to_sign is created by concatenating selected elements from requests which is then authenticated using hmac to create the final signature. If this signature matches with the client signature, the authentication is successful or else it fails with SignatureDoesNotMatch error. As part of the StringToSign, the CanonicalizedAmzHeaders should only include headers that start with "x-amz-" and ignore other headers. But in the radosgw code, there are other meta prefixes that are checked against the http request headers and if matched are included in the CanonicalizedAmzHeaders to compute the final signature. For e.g, if a request header contains "HTTP_X_ACCOUNT" its selected by radosgw to include in amz_headers but the same will be ignored by AWS SDK as it does not start with "x-amz-". This will result in different signature computed by client and radosgw. Code Snippet: rgw_common.cc struct str_len meta_prefixes[] = { STR_LEN_ENTRY("HTTP_X_AMZ"), STR_LEN_ENTRY("HTTP_X_GOOG"), STR_LEN_ENTRY("HTTP_X_DHO"), STR_LEN_ENTRY("HTTP_X_RGW"), STR_LEN_ENTRY("HTTP_X_OBJECT"), STR_LEN_ENTRY("HTTP_X_CONTAINER"), STR_LEN_ENTRY("HTTP_X_ACCOUNT"), {NULL, 0} }; The method init_meta_info() which matches the above prefixes is called from RGWREST::preprocess() which is invoked for all s3 requests. It will be helpful to know as to why these prefixes that are not specified by AWS S3 comes in the path of s3 authentication. Was it added for swift use-cases only? If so, then why its included in rgw_common.cc? As a proposed fix, we can remove the highlighted meta prefixes that are not specified by aws from the s3 authentication path signature calculation. Let me know if you have any queries or solutions. Regards, Biswajeet -- *-----------------------------------------------------------------------------------------* *This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee, you should not disseminate, distribute or copy this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.***** **** *Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the organization. Any information on shares, debentures or similar instruments, recommended product pricing, valuations and the like are for information purposes only. It is not meant to be an instruction or recommendation, as the case may be, to buy or to sell securities, products, services nor an offer to buy or sell securities, products or services unless specifically stated to be so on behalf of the Flipkart group. Employees of the Flipkart group of companies are expressly required not to make defamatory statements and not to infringe or authorise any infringement of copyright or any other legal right by email communications. Any such communication is contrary to organizational policy and outside the scope of the employment of the individual concerned. The organization will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising.***** **** *Our organization accepts no liability for the content of this email, or for the consequences of any actions taken on the basis of the information *provided,* unless that information is subsequently confirmed in writing. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.* _-----------------------------------------------------------------------------------------_

3 years, 11 months

1
2
0 0

Test Work coordination/revitalizing the Testing Weekly

by Gregory Farnum

Hi all, About a year ago I started a weekly meeting focused on Ceph testing: https://calendar.google.com/calendar/event?eid=MmxkMWZubzg2Zzg0a3VjaDJocGU4… It meets every Wednesday at 8AM Pacific Time and has mostly focused on teuthology and the qa suites. We've gotten some nice stuff done through the meeting around teuthology and getting the Suse fork merged into upstream, but things have mostly stalled out since then. I know lots of other things are happening outside of that meeting, but I haven't advertised it in a while and several people were or would be involved have time conflicts. So I'd like to see if there are other times that are good for people to meet to try and coordinate some of those efforts more broadly, or if there are other meetings we should piggy-back off of and expand to try and get a broader and more stable audience. Things I know about: * downgrade testing (Josh and Yuri?) * several efforts around Kubernetes * getting openSUSE into the upstream test matrix * Building and testing containers (Dan+) I've added some people I know are working on things or whose input would be appreciated by others on the list, so please discuss! :) -Greg

3 years, 11 months

2
3
0 0

slow op warnings from OSDs due to recent master changes

by Patrick Donnelly

https://tracker.ceph.com/issues/41834 This is broadly affecting Ceph QA. Hoping this mail will get the notice of the person whose changes maybe broke it. -- Patrick Donnelly, Ph.D. He / Him / His Senior Software Engineer Red Hat Sunnyvale, CA GPG: 19F28A586F808C2402351B93C3301A3E258DD79D

3 years, 11 months

5
7
0 0

Error with make on raspberry pi nautilus tags/v14.2.2

by Sharath Gururaj

On raspberry pi 4 (arm v7 architecture), I am able to do ./install_deps as well as ./do_cmake but when I make, I get the following compile error /home/sharath/os/ceph/src/include/buffer.h:740:12: error: 'void ceph::buffer::v14_2_0::list::iterator_impl<is_const>::advance(size_t)' cannot be overloaded with 'void ceph::buffer::v14_2_0::list::iterator_impl<is_const>::advance(unsigned int)' void advance(size_t o) { advance(static_cast<unsigned>(o)); } ^~~~~~~ my gcc version is sharath@raspberrypi:~/os/ceph/build $ gcc --version gcc (Raspbian 8.3.0-6+rpi1) 8.3.0 excerpt from ./do_cmake + cmake -DCMAKE_BUILD_TYPE=Debug -DCMAKE_BUILD_TYPE=RelWithDebInfo .. -- The CXX compiler identification is GNU 8.3.0 -- The C compiler identification is GNU 8.3.0 -- The ASM compiler identification is GNU -- Found assembler: /usr/bin/cc -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works Please help

3 years, 11 months

2
7
0 0

Why BlueRocksDirectory::Fsync only sync metadata?

by Xuehan Xu

Hi, everyone. I'm trying to read the source code of BlueStore. My question is why it is sufficient to only flush the log in BlueRocksDirectory::Fsync? Shouldn't it flush the file data first? Is it because rocksdb always flush file data before doing fsync? Thanks:-)

3 years, 11 months

2
4
0 0

Memstore information/documentation

by Mason-Williams, Gabryel (DLSLtd,RAL,LSCI)

I was wondering if there is anywhere I can find thorough documentation on memstore and if there are any documented use cases? As the only documentation, I can find is https://ceph.io/geen-categorie/ceph-real-men-use-the-memstore-backend/ Thanks Gabryel Mason-Williams, Placement Student Address: Diamond Light Source Ltd., Diamond House, Harwell Science & Innovation Campus, Didcot, Oxfordshire OX11 0DE Email: gabryel.mason-williams(a)diamond.ac.uk -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 years, 11 months

2
1
0 0

teuthology-suite

by Jos Collin

Hi Everyone, How can I make teuthology-suite command to fetch the code from my local machine (laptop) and create a few jobs for me to test the code changes in qa/tasks/? This would save a lot of time when I work on qa/tasks/, as shaman would normally take 4-5 hrs to build. I tried changing --suite-repo and --suite-branch, but it always points to shaman. Thanks, Jos Collin

3 years, 11 months

4
7
0 0

Re: High-end and low-cost photographic equipment

by Joyce

3 years, 11 months

1
0
0 0

2023

2022

2021

2020

2019

Dev September 2019