25 Jun
2020
25 Jun
'20
4:21 p.m.
hello
i want to unsubscribe this mail list . help me please.
在 2020/6/25 22:42, ceph-users-request(a)ceph.io 写道:
> Send ceph-users mailing list submissions to
> ceph-users(a)ceph.io
>
> To subscribe or unsubscribe via email, send a message with subject or
> body 'help' to
> ceph-users-request(a)ceph.io
>
> You can reach the person managing the list at
> ceph-users-owner(a)ceph.io
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of ceph-users digest..."
>
> Today's Topics:
>
> 1. Re: Removing pool in nautilus is incredibly slow
> (Francois Legrand)
> 2. Re: Bench on specific OSD (Marc Roos)
> 3. Re: Removing pool in nautilus is incredibly slow
> (Wout van Heeswijk)
> 4. Lifecycle message on logs (Marcelo Miziara)
> 5. Re: Feedback of the used configuration (Simon Sutter)
> 6. Re: Removing pool in nautilus is incredibly slow
> (Francois Legrand)
> 7. Re: Removing pool in nautilus is incredibly slow (Eugen Block)
>
>
> ----------------------------------------------------------------------
>
> Date: Thu, 25 Jun 2020 07:57:48 -0000
> From: "Francois Legrand" <fleg(a)lpnhe.in2p3.fr>
> Subject: [ceph-users] Re: Removing pool in nautilus is incredibly slow
> To: ceph-users(a)ceph.io
> Message-ID: <159307186843.20.7354239610800797635@mailman-web>
> Content-Type: text/plain; charset="utf-8"
>
> Does someone have an idea ?
> F.
>
> ------------------------------
>
> Date: Thu, 25 Jun 2020 11:36:24 +0200
> From: "Marc Roos" <M.Roos(a)f1-outsourcing.eu>
> Subject: [ceph-users] Re: Bench on specific OSD
> To: ceph-users <ceph-users(a)ceph.io>io>, seenafallah
> <seenafallah(a)gmail.com>
> Message-ID:
<"H0000071001729d1.1593077784.sx.f1-outsourcing.eu*"@MHS>
> Content-Type: text/plain; charset="US-ASCII"
>
>
>
> What is wrong with just doing multiple tests and group in your charts
> osd's by host?
>
>
> -----Original Message-----
> To: ceph-users
> Subject: [ceph-users] Bench on specific OSD
>
> Hi all.
>
> Is there anyway to completely health check one OSD host or instance?
> For example rados bech just on that OSD or do some checks for disk and
> front and back netowrk?
>
> Thanks.
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an
> email to ceph-users-leave(a)ceph.io
>
>
> ------------------------------
>
> Date: Thu, 25 Jun 2020 14:26:59 +0200
> From: Wout van Heeswijk <wout(a)42on.com>
> Subject: [ceph-users] Re: Removing pool in nautilus is incredibly slow
> To: fleg(a)lpnhe.in2p3.fr
> Cc: ceph-users(a)ceph.io
> Message-ID: <38a5d557-7797-a68a-1b67-4c8f0e1ecf4c(a)42on.com>
> Content-Type: text/plain; charset=utf-8; format=flowed
>
> Hi Francois,
>
> Have you already looked at the option "osd_delete_sleep"? It will not
> speed up the process but I will give you some control over your cluster
> performance.
>
> Something like:
>
> ceph tell osd.\* injectargs '--osd_delete_sleep1'
>
> kind regards,
>
> Wout
> 42on
>
> On 25-06-2020 09:57, Francois Legrand wrote:
>> Does someone have an idea ?
>> F.
>> _______________________________________________
>> ceph-users mailing list -- ceph-users(a)ceph.io
>> To unsubscribe send an email to ceph-users-leave(a)ceph.io
>
> ------------------------------
>
> Date: Thu, 25 Jun 2020 09:53:09 -0300
> From: Marcelo Miziara <raxidex(a)gmail.com>
> Subject: [ceph-users] Lifecycle message on logs
> To: ceph-users(a)ceph.io
> Message-ID:
> <CAGxsqB296eczKux19OKObkimXg3PZ_UDEWVTTqEnrAf=Agm0+A(a)mail.gmail.com>
> Content-Type: text/plain; charset="UTF-8"
>
> Hello...it's the first time I need to use the lifecycle, and I created a
> bucket and set it to expire in one day with s3cmd:
> s3cmd expire --expiry-days=1 s3://bucket
>
> The rgw_lifecycle_work_time is set to the default values(00:00-06:00). But
> I noticed in the rgw logs a lot of messages like:
> 2020-06-16 00:00:00.311369 7fe2cac87700 0 RGWLC::process() failed to get
> obj entry lc.8
> 2020-06-16 00:00:00.311623 7fe2c8c83700 0 RGWLC::process() failed to get
> obj entry lc.16
> 2020-06-16 00:00:00.311862 7fe2c6c7f700 0 RGWLC::process() failed to get
> obj entry lc.4
> 2020-06-16 00:00:00.319424 7fe2cac87700 0 RGWLC::process() failed to get
> obj entry lc.10
> 2020-06-16 00:00:00.319647 7fe2c8c83700 0 RGWLC::process() failed to get
> obj entry lc.18
> 2020-06-16 00:00:00.320682 7fe2c6c7f700 0 RGWLC::process() failed to get
> obj entry lc.16
> 2020-06-16 00:00:00.327770 7fe2cac87700 0 RGWLC::process() failed to get
> obj entry lc.6
> 2020-06-16 00:00:00.328941 7fe2c8c83700 0 RGWLC::process() failed to get
> obj entry lc.17
> 2020-06-16 00:00:00.332463 7fe2c6c7f700 0 RGWLC::process() failed to get
> obj entry lc.20
> 2020-06-16 00:00:00.336788 7fe2cac87700 0 RGWLC::process() failed to get
> obj entry lc.1
> 2020-06-16 00:00:00.336924 7fe2c8c83700 0 RGWLC::process() failed to get
> obj entry lc.24
> 2020-06-16 00:00:00.340915 7fe2c6c7f700 0 RGWLC::process() failed to get
> obj entry lc.2
>
> The object was deleted, but these messages keep appearing.
> Is it safe to ignore them?
>
> For the records, i'm using redhat luminous 12.2.12
>
> Thanks, Marcelo.
>
> ------------------------------
>
> Date: Thu, 25 Jun 2020 13:19:38 +0000
> From: Simon Sutter <ssutter(a)hosttech.ch>
> Subject: [ceph-users] Re: Feedback of the used configuration
> To: Paul Emmerich <paul.emmerich(a)croit.io>
> Cc: "ceph-users(a)ceph.io" <ceph-users(a)ceph.io>
> Message-ID: <f6e11ee189a74c35815c495d64cbe4cb(a)hosttech.ch>
> Content-Type: text/plain; charset="utf-8"
>
> Hello Paul,
>
> Thanks for the Answer.
> I took a look at the subvolumes, but they are a bit odd in my opinion.
> If I create one with a subvolume-group, the folder structure will look like this:
> /cephfs/volumes/group-name/subvolume-name/random-uuid/
> And I have to issue two commands, first set the group and then set the volume name,
but why so complicated?
>
> Wouldn't it be easier to just make subvolumes anywhere inside the cephfs?
> I can see the intended use for groups, but if I want to publish a pool in some
different directory that's not possible (except for setfattr).
> Without first creating subvolume-groups, the orchestrator creates subvolumes in the
/cephfs/volumes/_nogroup/subvolume-name/randmon-uuid/ folder.
>
> And the more important question is, why is there a new folder with a random uuid
inside the subvolume?
> I try to understand the points the devs had, when they developed this, but for me,
this is something I have to explain to some devs in our team and at the moment I
can't.
>
> It is indeed easier to deploy but comes with much less flexibility.
> Maybe something to write in the tracker about?
>
> Thanks in advance,
> Simon
>
> Von: Paul Emmerich [mailto:paul.emmerich@croit.io]
> Gesendet: Mittwoch, 24. Juni 2020 17:35
> An: Simon Sutter <ssutter(a)hosttech.ch>
> Cc: ceph-users(a)ceph.io
> Betreff: Re: [ceph-users] Feedback of the used configuration
>
> Have a look at cephfs subvolumes:
https://docs.ceph.com/docs/master/cephfs/fs-volumes/#fs-subvolumes
>
> They are internally just directories with quota/pool placement layout/namespace with
some mgr magic to make it easier than doing that all by hand
>
> Paul
>
> --
> Paul Emmerich
>
> Looking for help with your Ceph cluster? Contact us at https://croit.io
>
> croit GmbH
> Freseniusstr. 31h
> 81247 München
> www.croit.io<http://www.croit.io>
> Tel: +49 89 1896585 90
>
>
> On Wed, Jun 24, 2020 at 4:38 PM Simon Sutter
<ssutter@hosttech.ch<mailto:ssutter@hosttech.ch>> wrote:
> Hello,
>
> After two months of the "ceph try and error game", I finally managed to get
an Octopuss cluster up and running.
> The unconventional thing about it is, it's just for hot backups, no virtual
machines on there.
> All the nodes are without any caching ssd's, just plain hdd's.
> At the moment there are eight of them with a total of 50TB. We are planning to go up
to 25 and bigger disks so we end on 300TB-400TB
>
> I decided to go with cephfs, because I don't have any experience in things like
S3 and I need to read the same file system from more than one client.
>
> I made one cephfs with a replicated pool.
> On there I added erasure-coded pools to save some Storage.
> To add those pools, I did it with the setfattr command like this:
> setfattr -n ceph.dir.layout.pool -v ec_data_server1 /cephfs/nfs/server1
>
> Some of our servers cannot use cephfs (old kernels, special OS's) so I have to
use nfs.
> This is set up with the included ganesha-nfs.
> Exported is the /cephfs/nfs folder and clients can mount folders below this.
>
> There are two final questions:
>
> - Was it right to go with the way of "mounting" pools with
setfattr, or should I have used multiple cephfs?
>
> First I was thinking about using multiple cephfs but there are warnings everywhere.
The deeper I got in, the more it seems I would have been fine with multiple cephfs.
>
> - Is there a way I don't know, but it would be easier?
>
> I still don't know much about Rest, S3, RBD etc... so there may be a better way
>
> Other remarks are desired.
>
> Thanks in advance,
> Simon
> _______________________________________________
> ceph-users mailing list -- ceph-users@ceph.io<mailto:ceph-users@ceph.io>
> To unsubscribe send an email to
ceph-users-leave@ceph.io<mailto:ceph-users-leave@ceph.io>
>
> ------------------------------
>
> Date: Thu, 25 Jun 2020 16:22:12 +0200
> From: Francois Legrand <fleg(a)lpnhe.in2p3.fr>
> Subject: [ceph-users] Re: Removing pool in nautilus is incredibly slow
> To: Wout van Heeswijk <wout(a)42on.com>
> Cc: ceph-users(a)ceph.io
> Message-ID: <db34e791-1172-8b94-09e7-4ab5790d3162(a)lpnhe.in2p3.fr>
> Content-Type: text/plain; charset=utf-8; format=flowed
>
> Thanks for the hint.
> I tryed but it doesn't seems to change anything...
> Moreover, as the osds seems quite loaded I had regularly some osd marked
> down which triggered some new peering and thus more load !!!
> I set the osd no down flag, but I still have some osd reported (wrongly)
> as down (and back up in the minute) which generate peering and
> remapping. I don't really understand the action of no down parameter !
> Is there a way to tell ceph not to peer immediately after an osd is
> reported down (let say wait for 60s) ?
> I am thinking about restarting all osd (or maybe the whole cluster) to
> get osd_op_queue_cut_off changed to high and osd_op_thread_timeout to
> something higher than 15 (but I don't think it will really improve the
> situation).
> F.
>
>
> Le 25/06/2020 à 14:26, Wout van Heeswijk a écrit :
>> Hi Francois,
>>
>> Have you already looked at the option "osd_delete_sleep"? It will not
>> speed up the process but I will give you some control over your
>> cluster performance.
>>
>> Something like:
>>
>> ceph tell osd.\* injectargs '--osd_delete_sleep1'
>> kind regards,
>>
>> Wout
>> 42on
>> On 25-06-2020 09:57, Francois Legrand wrote:
>>> Does someone have an idea ?
>>> F.
>>> _______________________________________________
>>> ceph-users mailing list --ceph-users(a)ceph.io
>>> To unsubscribe send an email toceph-users-leave(a)ceph.io
>
> ------------------------------
>
> Date: Thu, 25 Jun 2020 14:42:57 +0000
> From: Eugen Block <eblock(a)nde.ag>
> Subject: [ceph-users] Re: Removing pool in nautilus is incredibly slow
> To: ceph-users(a)ceph.io
> Message-ID:
> <20200625144257.Horde.I8JPcddeor47WdOehQKQPNY(a)webmail.nde.ag>
> Content-Type: text/plain; charset=utf-8; format=flowed; DelSp=Yes
>
> I'm not sure if your OSDs have their rocksDB on faster devices, if not
> it sounds a lot like rocksdb fragmentation [1] leading to a very high
> load on the OSDs and occasionally crashing OSDs. If you don't plan to
> delete so much data at once on a regular basis you could sit this one
> out, but one solution is to re-create the OSDs with rocksDB/WAL on
> faster devices.
>
>
> [1] https://www.mail-archive.com/ceph-users@ceph.io/msg03160.html
>
>
> Zitat von Francois Legrand <fleg(a)lpnhe.in2p3.fr>fr>:
>
>> Thanks for the hint.
>> I tryed but it doesn't seems to change anything...
>> Moreover, as the osds seems quite loaded I had regularly some osd
>> marked down which triggered some new peering and thus more load !!!
>> I set the osd no down flag, but I still have some osd reported
>> (wrongly) as down (and back up in the minute) which generate peering
>> and remapping. I don't really understand the action of no down
>> parameter !
>> Is there a way to tell ceph not to peer immediately after an osd is
>> reported down (let say wait for 60s) ?
>> I am thinking about restarting all osd (or maybe the whole cluster)
>> to get osd_op_queue_cut_off changed to high and
>> osd_op_thread_timeout to something higher than 15 (but I don't think
>> it will really improve the situation).
>> F.
>>
>>
>> Le 25/06/2020 à 14:26, Wout van Heeswijk a écrit :
>>> Hi Francois,
>>>
>>> Have you already looked at the option "osd_delete_sleep"? It will
>>> not speed up the process but I will give you some control over your
>>> cluster performance.
>>>
>>> Something like:
>>>
>>> ceph tell osd.\* injectargs '--osd_delete_sleep1'
>>> kind regards,
>>>
>>> Wout
>>> 42on
>>> On 25-06-2020 09:57, Francois Legrand wrote:
>>>> Does someone have an idea ?
>>>> F.
>>>> _______________________________________________
>>>> ceph-users mailing list --ceph-users(a)ceph.io
>>>> To unsubscribe send an email toceph-users-leave(a)ceph.io
>> _______________________________________________
>> ceph-users mailing list -- ceph-users(a)ceph.io
>> To unsubscribe send an email to ceph-users-leave(a)ceph.io
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
> %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
>
>
> ------------------------------
>
> End of ceph-users Digest, Vol 89, Issue 112
> *******************************************