- ceph-users - lists.ceph.io

Nfs-ganesha 2.6 upgrade to 2.7
by Marc Roos 26 Sep '19

26 Sep '19

Is it really necessary to have these dependencies in nfs-ganesha 2.7 Dep-Install samba-client-libs-4.8.3-4.el7.x86_64 @CentOS7 Dep-Install samba-common-4.8.3-4.el7.noarch @CentOS7 Dep-Install samba-common-libs-4.8.3-4.el7.x86_64 @CentOS7

1 0

Cephfs corruption(?) causing nfs-ganesha to "clients failing to respond to capability release" / "MDSs report slow requests"
by Marc Roos 26 Sep '19

26 Sep '19

Can it be that there is something wrong with the cephfs, that causes the nfs-ganesha to fail? I have the impression that when I do an rsync on only a specific nfs mount/cephfs tree. I am getting this error. All nfs mounts are inaccessible on that client. Other clients (with other mounts) are fine.

1 0

Re: Balancer active plan
by Massimo Sgaravatto 26 Sep '19

26 Sep '19

I think you should see this info in the mgr log (with debug mgr = 4/5) Cheers, Massimo On Thu, Sep 26, 2019 at 3:43 PM Marc Roos <M.Roos(a)f1-outsourcing.eu> wrote: > > Suddenly I have objects misplaced, I assume this is because of the > balancer being active. But > > 1. how can I see the currently/last executed plan of the balancer? > 2. when it was activated? > > > 7 active+remapped+backfill_wait > 7 active+remapped+backfilling > > > > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -. > F1 Outsourcing Development Sp. z o.o. > Poland > > t: +48 (0)124466845 > f: +48 (0)124466843 > e: marc(a)f1-outsourcing.eu > > _______________________________________________ > ceph-users mailing list -- ceph-users(a)ceph.io > To unsubscribe send an email to ceph-users-leave(a)ceph.io >

1 0

Balancer active plan
by Marc Roos 26 Sep '19

26 Sep '19

Suddenly I have objects misplaced, I assume this is because of the balancer being active. But 1. how can I see the currently/last executed plan of the balancer? 2. when it was activated? 7 active+remapped+backfill_wait 7 active+remapped+backfilling - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -. F1 Outsourcing Development Sp. z o.o. Poland t: +48 (0)124466845 f: +48 (0)124466843 e: marc(a)f1-outsourcing.eu

1 0

CephFS deleted files' space not reclaimed
by Josh Haft 26 Sep '19

26 Sep '19

Hi, I've been migrating data from one EC pool to another EC pool: two directories are mounted with ceph.dir.layout.pool file attribute set appropriately, then rsync from old to new and finally, delete the old files. I'm using the kernel client to do this. While the removed files are no longer present on the filesystem, they still appear to be accounted for via "ceph df". When I tally up the sizes reported by "ls -lh" on all subdirectories under the root CephFS using a FUSE client mount (except for those on the new EC pool), it totals just under 2PiB. However, "ceph df" shows the original EC pool as 2.5PiB used. I've copied + deleted approximately 545TiB so far, so it seems like the unlinked files aren't being fully released/purged. I've only observed the num_strays counter from "ceph daemon mds.$name perf dump" for a few days now since I first suspected an issue, but I've never seen it drop below roughly 310k. From other ML postings I've gathered that stat has something to do with files pending deletion, but I'm not positive. So far all I've done is restart the mds and mon daemons, which hasn't helped. What are the next steps for troubleshooting? I can turn up mds debug logging, but am not sure what to look for. Thanks for your help! Josh

2 3

Luminous 12.2.12 "clients failing to respond to capability release"
by Marc Roos 26 Sep '19

26 Sep '19

I am getting this error I have in two sessions[0] num_caps high ( I assume the error is about num_caps ). I am using a default luminous and a default centos7 with default kernel 3.10. Do I really still need to change to a not stock kernel to resolve this? I read this in posts of 2016 and 2019, yet I also saw some post on tuning the mds. [0] ceph daemon mds.c session ls { "id": 3634861, "num_leases": 0, "num_caps": 2183, "state": "open", "request_load_avg": 181, "uptime": 2545057.775869, "replay_requests": 0, "completed_requests": 0, "reconnecting": false, { "id": 3644756, "num_leases": 0, "num_caps": 2080, "state": "open", "request_load_avg": 32, "uptime": 2545057.775869, "replay_requests": 0, "completed_requests": 0, "reconnecting": false,

1 4

Cephfs + docker
by Alex L 26 Sep '19

26 Sep '19

Hi, I am trying to figure out why my portainer and pi-hole in docker keeps getting broken databases. All other docker applications are working flawlessly but not these. I am running Ubuntu 18.04 + kernel ceph mount for the data directory. Have looked at how others do it, and they seem to all use the docker plugin for getting an RBD image but looking at those plugins many seem to not have been updated for years.. Thanks in advance, A

3 2

Nautilus dashboard: MDS performance graph doesn't refresh
by Eugen Block 26 Sep '19

26 Sep '19

Hi all, I regularly check the MDS performance graphs in the dashboard, especially the requests per second is interesting in my case. Since our upgrade to Nautilus the values in the activity column are still refreshed every 5 seconds (I believe), but the graphs are not refreshed since that upgrade anymore. I couldn't find anything in the tracker or the mailing list, can anyone comment on this? Thank you & best regards, Eugen

2 3

Re: Nautilus : ceph dashboard ssl not working
by Lenz Grimmer 26 Sep '19

26 Sep '19

Hi Miha, interesting observation, I don't think we've noticed this before. Would you mind submitting a bug report about this on our tracker, including these logs? https://tracker.ceph.com/projects/mgr/issues/new Thanks in advance! Lenz On 9/26/19 10:01 AM, Miha Verlic wrote: > On 24. 09. 19 14:53, Lenz Grimmer wrote: >> On 9/24/19 1:37 PM, Miha Verlic wrote: >> >>> I've got slightly different problem. After a few days of running fine, >>> dashboard stops working because it is apparently seeking for wrong >>> certificate file in /tmp. If I restart ceph-mgr it starts to work again. >> >> Does the restart trigger the creation of a similar-looking file in /tmp? >> I wonder if there's some kind of cron job that cleans up the /tmp >> directory every now and then... > > There is systemd-tmpfiles-clean.timer, but it ignores PrivateTmp folders. > > But as said, crt and key files do exist in tmp and according to > timestamps they were created when ceph-mgr daemon was started; it seems > to me like ceph-mgr simply starts looking for wrong filenames after a while. > > Going through ceph-mgr logs I'd say the problem is internal webserver is > restarted, but ceph-mgr is not notified about new crt/key files: > > Sep 19 12:54:16 cephtest01 ceph-mgr[2247]: [19/Sep/2019:12:54:16] ENGINE > Bus STOPPING > > Sep 19 12:54:16 cephtest01 ceph-mgr[2247]: [19/Sep/2019:12:54:16] ENGINE > HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 8443)) shut down > > Sep 19 12:54:16 cephtest01 ceph-mgr[2247]: [19/Sep/2019:12:54:16] ENGINE > Stopped thread '_TimeoutMonitor'. > > Sep 19 12:54:16 cephtest01 ceph-mgr[2247]: [19/Sep/2019:12:54:16] ENGINE > Bus STOPPED > > Sep 19 12:54:16 cephtest01 ceph-mgr[2247]: [19/Sep/2019:12:54:16] ENGINE > Bus STARTING > > Sep 19 12:54:16 cephtest01 ceph-mgr[2247]: [19/Sep/2019:12:54:16] ENGINE > Started monitor thread '_TimeoutMonitor'. > > Sep 19 12:54:16 cephtest01 ceph-mgr[2247]: 2019-09-19 12:54:16.618 > 7f3a796e4700 -1 client.0 error registering admin socket command: (17) > File exists > > Sep 19 12:54:16 cephtest01 ceph-mgr[2247]: 2019-09-19 12:54:16.618 > 7f3a796e4700 -1 client.0 error registering admin socket command: (17) > File exists > > Sep 19 12:54:16 cephtest01 ceph-mgr[2247]: 2019-09-19 12:54:16.618 > 7f3a796e4700 -1 client.0 error registering admin socket command: (17) > File exists > > Sep 19 12:54:16 cephtest01 ceph-mgr[2247]: 2019-09-19 12:54:16.618 > 7f3a796e4700 -1 client.0 error registering admin socket command: (17) > File exists > > Sep 19 12:54:16 cephtest01 ceph-mgr[2247]: 2019-09-19 12:54:16.618 > 7f3a796e4700 -1 client.0 error registering admin socket command: (17) > File exists > > Sep 19 12:54:16 cephtest01 ceph-mgr[2247]: [19/Sep/2019:12:54:16] ENGINE > Serving on :::8443 > > Sep 19 12:54:16 cephtest01 ceph-mgr[2247]: [19/Sep/2019:12:54:16] ENGINE > Bus STARTED -- SUSE Software Solutions Germany GmbH - Maxfeldstr. 5 - 90409 Nuernberg GF: Felix Imendörffer, HRB 247165 (AG Nürnberg)

1 0

how many monitor should to deploy in a 1000+ osd cluster
by 展荣臻（信泰） 26 Sep '19

26 Sep '19

hi all: I have a production cluster, and it had 24 hosts (528 osds,3mons) at a former. Now we want to add 36 hosts so the osd increase to 1320 . does the monitor need to increase?how many numbers of monitor node is recommended? Another question is which monitor does monclient commnuicate with? And how it decide? Any suggestions are welcome!

7 7