Hi Folks,
The weekly performance meeting is starting in 20 minutes! Today I will
share some results from testing the new on-disk format changes for
Pacific based on some earlier tests that Gabi performed. Please feel
free to add your own topic as well.
Hope to see you there!
Etherpad:
https://pad.ceph.com/p/performance_weekly
Bluejeans:
https://bluejeans.com/908675367
Thanks,
Mark
Dear Ceph developers,
Presently Ceph has a single config option named min_size which decides the
minimum number of copies that must be available before any client I/O
operation (read or write) can be performed on a given RADOS pool.
Would it make sense to split it into two i.e. read_min_size and
write_min_size to allow better data availability?
For instance, in a pool with replication size of 3 (where 3 copies are
stored), if two OSDs go down, we would want to avoid client write
operations (to reduce risk of data loss) but allow client read operations
from the single copy that is available. This can be done by setting
read_min_size to 1, but retaining write_min_size to 2.
Are there any technical reasons why this cannot work? Any pitfalls that I
don't foresee?
Thanks,
K.Prasad
--
*-----------------------------------------------------------------------------------------*
*This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they are
addressed. If you have received this email in error, please notify the
system manager. This message contains confidential information and is
intended only for the individual named. If you are not the named addressee,
you should not disseminate, distribute or copy this email. Please notify
the sender immediately by email if you have received this email by mistake
and delete this email from your system. If you are not the intended
recipient, you are notified that disclosing, copying, distributing or
taking any action in reliance on the contents of this information is
strictly prohibited.*****
****
*Any views or opinions presented in this
email are solely those of the author and do not necessarily represent those
of the organization. Any information on shares, debentures or similar
instruments, recommended product pricing, valuations and the like are for
information purposes only. It is not meant to be an instruction or
recommendation, as the case may be, to buy or to sell securities, products,
services nor an offer to buy or sell securities, products or services
unless specifically stated to be so on behalf of the Flipkart group.
Employees of the Flipkart group of companies are expressly required not to
make defamatory statements and not to infringe or authorise any
infringement of copyright or any other legal right by email communications.
Any such communication is contrary to organizational policy and outside the
scope of the employment of the individual concerned. The organization will
not accept any liability in respect of such communication, and the employee
responsible will be personally liable for any damages or other liability
arising.*****
****
*Our organization accepts no liability for the
content of this email, or for the consequences of any actions taken on the
basis of the information *provided,* unless that information is
subsequently confirmed in writing. If you are not the intended recipient,
you are notified that disclosing, copying, distributing or taking any
action in reliance on the contents of this information is strictly
prohibited.*
_-----------------------------------------------------------------------------------------_
Thanks to David Galloway, there's a fresh version of sentry setup at
https://sentry.ceph.com. This should help us track teuthology failures
and make it easier to fix issues and keep the tests passing.
Sentry is a general tool for tracking errors, and it's hooked up to
teuthology. Whenever a job fails, teuthology reports the stacktrace and
a bunch of metadata to sentry (the metadata is customizable, so we can
add more fields easily [0]).
Sentry groups these failures ('events' in sentry terms) together into
a single 'issue', which you can search and filter by
suite/branch/os/etc.
You can login with your github account, add yourself to the ceph org,
and look around the sepia project to see all the failures from the
sepia lab. Pulpito also links to the sentry event directly for each
failure.
Each issue has a timeline of occurrences so it's easy to see if it's
recently introduced or a long-present intermittent failure. There's
also a chart of the distribution of the metadata, so you can see if
there's a common element like OS, branch, etc. This should help us
detect when an issue was introduced, and whether it is new or not.
The way sentry groups failures is also customizable - the default is
based on the stacktrace from teuthology, which is too coarse in some
cases [1]. There are likely more tweaks here that will make it easier
to use. A few ideas:
* report ceph crashes rather than the teuthology traceback when
there's a coredump
* parse test framework output and report the failed tests rather than
the command that ran them
* differentiate between different kinds of timeouts
* some automation with tracker tickets (you can link to redmine from
comments on sentry events, and vice versa today)
Check it out, and let us know if there are ways to improve it!
Josh
[0]
https://github.com/ceph/teuthology/blob/master/teuthology/run_tasks.py#L107…
[1] https://github.com/ceph/teuthology/pull/1593
Hey folks, we're getting towards the end of the pacific cycle.
The component leads discussed and decided this time we're going to do a
hard feature freeze on January 15th, 2021.
After that, we'll branch off pacific from master, and use the usual
backport process to merge bug fixes - first going to master,
then backporting to the pacific branch. Since this is the same
procedure as the rest of the development cycle, it should cause
less confusion.
This will let us focus on stabilizing pacific and improving the test
suites, and make sure only changes consciously targeted for pacific are
merged there. As usual, we're targeting the release for March, so we'll
have ~2 months of stabilization.
Josh
Hi Folks,
The weekly performance meeting is starting now. :) Topics today include
trying to get crimson working with CBT (not using vstart) and rocksdb
on-disk format changes for pacific. Please feel free to add your own
topic as well.
Hope to see you there!
Etherpad:
https://pad.ceph.com/p/performance_weekly
Bluejeans:
https://bluejeans.com/908675367
Thanks,
Mark
On Wed, Jan 6, 2021 at 1:06 AM Matt Wilder <matt.wilder(a)bitmex.com> wrote:
> I took a look at your scan results.
>
> All of the CRITICAL vulnerabilities listed are for samba related libraries
> and reference the same CVE (https://avd.aquasec.com/nvd/cve-2020-1472),
> which is about the domain controller part of samba, and therefore does not
> affect Ceph.
>
> Of the HIGH vulnerabilities listed:
> sqlite-libs - https://avd.aquasec.com/nvd/cve-2019-5827 is specifically
> about chromium-browser, so Ceph is not affected.
> lodash - https://avd.aquasec.com/nvd/cve-2020-8203 is about distributed
> tracing, so Ceph MIGHT be affected if you are using this feature, or it
> might not. Someone with more context on this would need to weigh in.
> https://www.npmjs.com/advisories/1523 seems to reference the same thing.
>
>
The moral of the story here is that these security scanners do a very
> simple check for CVEs against software versions installed inside a
> package. The affected libraries should definitely be upgraded, but just
> because a scanner like this registers a vulnerability is not an absolute
> indicator that a given service is actually vulnerable.
>
agree with your point, I have similar point of view, cc'ing Mona if s/he
has any further query.
Thanks Matt!
> On Tue, Jan 5, 2021 at 7:56 AM Deepika Upadhyay <dupadhya(a)redhat.com>
> wrote:
>
>>
>>
>> ---------- Forwarded message ---------
>> From: Minor, Mona <Mona.Minor(a)unisys.com>
>> Date: Tue, Dec 29, 2020 at 8:15 PM
>> Subject: how to fix ceph vulnerability
>> To: dupadhya(a)redhat.com <dupadhya(a)redhat.com>
>>
>>
>> Hi Deepika,
>>
>> I am working on a project where I need storage for my kubernetes pods.
>> I am looking to get the storage from ceph cluster.
>> ceph is very nice tool for completing most of the storage requirements.
>>
>> but, I am in doubt to proceed ahead as I found that ceph is “vulnerable”.
>> I tried to setup cluster with cephadm tool as well as ceph-ansible tool
>> as well. After then that I also tried ceph with rook as well.
>> the image that’s available on docker hub (ceph/ceph) that doesn’t having
>> any Dockerfile.
>> I scanned the ceph:v15.xx image with “trivy”, and its generated report
>> with some vulnerability (with HIGH , CRITICAL ).
>>
>> I am interested to get any ceph image that is not vulnerable.
>> please let me know if any image is available or any process that I have
>> to follow for getting ceph image that is not vulnerable.
>>
>> For your reference I have attached generated trivy report for ceph.
>> Kindly have a look on them
>>
>> Thank You and Regards,
>>
>> Mona Minor
>>
>>
>> _______________________________________________
>> Dev mailing list -- dev(a)ceph.io
>> To unsubscribe send an email to dev-leave(a)ceph.io
>>
>
> This e-mail and all information in, attached to, or linked via this e-mail
> (together the ‘e-mail’) is confidential and may be legally privileged. It
> is intended solely for the intended addressee(s). Access to, or any onward
> transmission, of this e-mail by any other person is not authorised. If you
> are not the intended recipient, you are requested to immediately alert the
> sender of this e-mail and to immediately delete this e-mail. Any disclosure
> in any form of all or part of this e-mail, or of any the parties to it,
> including any copying, distribution or any action taken or omitted to be
> taken in reliance on it, is prohibited and may be unlawful.
>
>
> This e-mail is not, and is not intended to be, and should not be construed
> as being, (a) any offer, solicitation, or promotion of any kind; (b) the
> basis of any investment or other decision(s); (c) any recommendation to
> buy, sell or transact in any manner any good(s), product(s) or service(s),
> nor engage in any investment(s) or other transaction(s) or activities; or
> (d) the provision of, or related to, any advisory service(s) or activities,
> including regarding any investment, tax, legal, financial, accounting,
> consulting or any other related service(s).
>
Hey folks, happy new year!
With people coming back from holidays and busy with the last couple
weeks of Pacific feature development, we'll skip January and reconvene
February 3.
Josh
---------- Forwarded message ---------
From: Minor, Mona <Mona.Minor(a)unisys.com>
Date: Tue, Dec 29, 2020 at 8:15 PM
Subject: how to fix ceph vulnerability
To: dupadhya(a)redhat.com <dupadhya(a)redhat.com>
Hi Deepika,
I am working on a project where I need storage for my kubernetes pods.
I am looking to get the storage from ceph cluster.
ceph is very nice tool for completing most of the storage requirements.
but, I am in doubt to proceed ahead as I found that ceph is “vulnerable”.
I tried to setup cluster with cephadm tool as well as ceph-ansible tool as
well. After then that I also tried ceph with rook as well.
the image that’s available on docker hub (ceph/ceph) that doesn’t having
any Dockerfile.
I scanned the ceph:v15.xx image with “trivy”, and its generated report with
some vulnerability (with HIGH , CRITICAL ).
I am interested to get any ceph image that is not vulnerable.
please let me know if any image is available or any process that I have to
follow for getting ceph image that is not vulnerable.
For your reference I have attached generated trivy report for ceph. Kindly
have a look on them
Thank You and Regards,
Mona Minor
