January 2020 - Dev - lists.ceph.io

Re: ceph issue discuss: read pg info incorrect cause unfound objects in EC scenario

by Sage Weil

Hi Song, On Mon, 13 Jan 2020, song wrote: > Hi Sage, > > happy new year! > > I am a software engineer from China. Recently I found a issue for fastinfo in Ceph and want to consult you about it. > > In the scenario of EC deployment, suppose we done a peering process for a pg and changed one shard's last_update from lu1(e1'3) to lu2(e1'2) .lu1 was written as fastinfo and lu2 was written as info. After that we restarted this osd and loaded pgs again. when we read pg info from disk, we will find the pg info is lu1 applied to lu2, which becomes incorrect. the true value should be lu2. That may cause the coming peering execute incorrectly and result in unfound objects. > I currently considered below two options: > 1. delete fastinfo when we need to change info; > 2. add extra sequence number to fastinfo and info structure to make it keep them in the right order. > > I am looking forward to hearing your suggestions about this issue and preferred solution. > if you need any more info, please let me know. Ah, that does look like a bug. I've opened a tracker ticket for this, https://tracker.ceph.com/issues/43580 Does that look right? I think the fix is pretty simple: https://github.com/ceph/ceph/pull/32615 Thanks! sage > > > thanks, > Song > >

4 years, 3 months

1
0
0 0

Re: New Ceph Plugin - assistance

by Paul Cuzner

Hi, Couple of things that may help; First doulbe check that your module is where it's supposed to be. For example, if you're seeing ENOENT, that seems to imply that the module is not in the /usr/share/ceph/mgr/ directory e.g. [root@ceph ~]# ceph mgr module enable thinair Error ENOENT: all mgr daemons do not support module 'thinair', pass --force to force enablement However, IIRC ENOENT is also returned if the module has been marked bad due to a prior failure. In this case disable and enable the module with --force - as the message states. On Mon, Jan 13, 2020 at 3:08 AM Khoury, Rima (Nokia - IL/Kfar Sava) < rima.khoury(a)nokia.com> wrote: > Thank you! The logs did help but I had to add log_to_file (as true) in the > ceph.conf in order to see it. > The plugin was successfully installed and did run as expected and could > produce output as desired and implemented in the code. However, I tried to > write a new function, but unfortunately it produced an error, but when I > restored the working code I could not enable the plugin again. Now it says: > " Error ENOENT: all mgr daemons do not support module 'newplugin', pass > --force to force enablement". > I tried restarting the ceph-mgr, and made sure to do all the steps I did > before, but seems that it's still stuck on this error. > Do you have any ideas of what might help? Could this be a bug in ceph? > BTW I am using nautilus > > > -----Original Message----- > From: Willem Jan Withagen <wjw(a)digiware.nl> > Sent: Thursday, January 9, 2020 1:41 PM > To: Juan Miguel Olmo Martinez <jolmomar(a)redhat.com>; Khoury, Rima (Nokia > - IL/Kfar Sava) <rima.khoury(a)nokia.com> > Cc: dev(a)ceph.io > Subject: Re: New Ceph Plugin - assistance > > On 9-1-2020 09:01, Juan Miguel Olmo Martinez wrote: > > Hi, > > > > Depending of the Ceph version you can have the files in different > > places, but in any case is easy to search the file "module.py" in any > > of your manager hosts, the results will show you the location of the > > manager plugins. > > In Nautilus they are in "/usr/share/ceph/mgr/" > > > > You can see the active manager using the command "ceph -s" > > > > A good source of information to see what is happening is the manager > > log file (in var/log/ceph) in the manager host, > > > > You can change the log level to debug using: > > ceph daemon <your-mgr> config set debug_mgr 20/5 restore the log > > level: > > ceph daemon <your-mgr> config set debug_mgr 1/5 > > I had a lot of trouble getting plugins to work, since they needed py-libs > that I did not (yet) have. > Python tracebacks are logged in the ceph-mgr log file. > And there is was usually quite fast clear what the problem was. > > Big chance that ceph-mgr is not loading you plugin due to loading errors > but the traceback in the log should give you more info. > > --WjW > > _______________________________________________ > Dev mailing list -- dev(a)ceph.io > To unsubscribe send an email to dev-leave(a)ceph.io >

4 years, 3 months

1
0
0 0

Re: New Ceph Plugin - assistance

by Juan Miguel Olmo Martinez

Hi, Depending of the Ceph version you can have the files in different places, but in any case is easy to search the file "module.py" in any of your manager hosts, the results will show you the location of the manager plugins. In Nautilus they are in "/usr/share/ceph/mgr/" You can see the active manager using the command "ceph -s" A good source of information to see what is happening is the manager log file (in var/log/ceph) in the manager host, You can change the log level to debug using: ceph daemon <your-mgr> config set debug_mgr 20/5 restore the log level: ceph daemon <your-mgr> config set debug_mgr 1/5 BR On Wed, Jan 8, 2020 at 4:01 PM Khoury, Rima (Nokia - IL/Kfar Sava) < rima.khoury(a)nokia.com> wrote: > Hi, > > > > I did the steps as suggested in the documentation, but the plugin still > can not run. > > I have tried to get the mgr module path but there is no configuration > option for that. Do you know how can I know what is the path? > > And is there any command to get details on the active manager? Just to > make sure it is running on my host. > > > > Thanks, > > > > *From:* Juan Miguel Olmo Martinez <jolmomar(a)redhat.com> > *Sent:* Wednesday, January 8, 2020 4:21 PM > *To:* Khoury, Rima (Nokia - IL/Kfar Sava) <rima.khoury(a)nokia.com> > *Cc:* dev(a)ceph.io > *Subject:* Re: New Ceph Plugin - assistance > > > > Hi Rima, > > > > - Review documentation: > > https://docs.ceph.com/docs/master/mgr/modules/#creating-a-module > > > > - Take a look at one example, in this folder you can find several of them: > > https://github.com/jmolmo/ceph/tree/master/src/pybind/mgr > > > > Hello: is the most simple manager module > > ssh: the most updated > > > > And ... be sure that your manager module files are located in the same > host where the active manager is running. > > > > BR > > > > On Wed, Jan 8, 2020 at 2:55 PM Khoury, Rima (Nokia - IL/Kfar Sava) < > rima.khoury(a)nokia.com> wrote: > > Hi there, > > > > I have been trying to add a new plugin to the ceph mgr but can't seem to > figure out why it is not working. > I have created a directory under /ceph/mgr/ and added the module.py file > there. Afterwards I added the plugin's name to the file > /usr/share/ceph-ansible/roles/ceph-mgr/defaults/main.yml under > ceph_mgr_modules field. But when I run "ceph mgr module enable > <plugin-name>" I get this error: Error ENOENT: module 'newplugin' reports > that it cannot run on the active manager daemon: (pass --force to force > enablement). > > Did anyone try this before and can share some insights. > > This matter is urgent to me so I would really appreciate any help I can > get. > > > > Thank you > > > > _______________________________________________ > Dev mailing list -- dev(a)ceph.io > To unsubscribe send an email to dev-leave(a)ceph.io > > > > > -- > > *Juan Miguel Olmo Martínez* > > Senior Software Engineer > > Red Hat <https://www.redhat.com/> > > jolmomar(a)redhat.com > > [image: Image removed by sender.] <https://www.redhat.com/> > > > -- Juan Miguel Olmo Martínez Senior Software Engineer Red Hat <https://www.redhat.com/> jolmomar(a)redhat.com <https://www.redhat.com/>

4 years, 3 months

3
2
0 0

Seastore storage structure design consideration based on HLC and R-tree

by Xuehan Xu

Hi, everyone. The following link is a thorough description of what we think may be an option for implementing the on-disk structure of seastore. Please take a look, thanks:-) https://docs.google.com/document/d/1hs5jJ1F8rVz7m2PZbBONlsiJsjRAh_DjVTrHVY3…

4 years, 3 months

5
11
0 0

v14.2.6 Nautilus released

by Abhishek Lekshmanan

We're happy to announce the availability of the sixth update to the Ceph Nautilus release series. This is a hotfix release primarily fixing a regression introduced in v14.2.5, all nautilus users are advised to upgrade to this release. Notable Changes --------------- * This release fixes a `ceph-mgr` bug that caused mgr becoming unresponsive on larger clusters issue#43364 https://tracker.ceph.com/issues/43364, pr#32466 (https://github.com/ceph/ceph/pull/32466 David Zafman, Neha Ojha) The release notes can be also found at https://ceph.io/releases/v14-2-6-nautilus-released/ Getting Ceph ------------ * Git at git://github.com/ceph/ceph.git * Tarball at http://download.ceph.com/tarballs/ceph-14.2.6.tar.gz * For packages, see http://docs.ceph.com/docs/master/install/get-packages/ * Release git sha1: f0aa067ac7a02ee46ea48aa26c6e298b5ea272e9 -- Abhishek Lekshmanan SUSE Software Solutions Germany GmbH

4 years, 3 months

1
0
0 0

14.2.6 hot-fix QE Validation

by Yuri Weinstein

nautilus branch had been locked for the ho-fix testing and release, see https://tracker.ceph.com/issues/43491#note-1 Please don't unlock and/or merge on this branch. Thx YuriW

4 years, 3 months

2
3
0 0

14.2.6 is done

by David Galloway

A couple FYIs: - arm64 builds failed. This is because all the arm64 builders got updated to Bionic and CentOS8 and we only target Xenial and CentOS7 for arm64 builds. I'll have to look at this further. - The formatting of the Release Notes on ceph.io might be off. I did my best. -- David Galloway Systems Administrator, RDU Ceph Engineering IRC: dgalloway

4 years, 3 months

2
1
0 0

Re: New Ceph Plugin - assistance

by Juan Miguel Olmo Martinez

Hi Rima, - Review documentation: https://docs.ceph.com/docs/master/mgr/modules/#creating-a-module - Take a look at one example, in this folder you can find several of them: https://github.com/jmolmo/ceph/tree/master/src/pybind/mgr Hello: is the most simple manager module ssh: the most updated And ... be sure that your manager module files are located in the same host where the active manager is running. BR On Wed, Jan 8, 2020 at 2:55 PM Khoury, Rima (Nokia - IL/Kfar Sava) < rima.khoury(a)nokia.com> wrote: > Hi there, > > > > I have been trying to add a new plugin to the ceph mgr but can't seem to > figure out why it is not working. > I have created a directory under /ceph/mgr/ and added the module.py file > there. Afterwards I added the plugin's name to the file > /usr/share/ceph-ansible/roles/ceph-mgr/defaults/main.yml under > ceph_mgr_modules field. But when I run "ceph mgr module enable > <plugin-name>" I get this error: Error ENOENT: module 'newplugin' reports > that it cannot run on the active manager daemon: (pass --force to force > enablement). > > Did anyone try this before and can share some insights. > > This matter is urgent to me so I would really appreciate any help I can > get. > > > > Thank you > > > _______________________________________________ > Dev mailing list -- dev(a)ceph.io > To unsubscribe send an email to dev-leave(a)ceph.io > -- Juan Miguel Olmo Martínez Senior Software Engineer Red Hat <https://www.redhat.com/> jolmomar(a)redhat.com <https://www.redhat.com/>

4 years, 3 months

2
1
0 0

New Ceph Plugin - assistance

by Khoury, Rima (Nokia - IL/Kfar Sava)

Hi there, I have been trying to add a new plugin to the ceph mgr but can't seem to figure out why it is not working. I have created a directory under /ceph/mgr/ and added the module.py file there. Afterwards I added the plugin's name to the file /usr/share/ceph-ansible/roles/ceph-mgr/defaults/main.yml under ceph_mgr_modules field. But when I run "ceph mgr module enable <plugin-name>" I get this error: Error ENOENT: module 'newplugin' reports that it cannot run on the active manager daemon: (pass --force to force enablement). Did anyone try this before and can share some insights. This matter is urgent to me so I would really appreciate any help I can get. Thank you

4 years, 3 months

1
0
0 0

DocuBetter APAC -- 2020 Jan 08

by John Zachary Dover

Hi everyone. The next DocuBetter meeting is scheduled for: 08 Jan 2020 0930 PST 08 Jan 2020 1730 UTC 09 Jan 2020 0330 AEST Etherpad: https://pad.ceph.com/p/Ceph_Documentation Meeting: https://bluejeans.com/908675367 Agenda: 1. Crisp Getting Started Guide PR 2. PG Repair Bug 3. Other bugs -- removing very stale bugs Thanks, everyone. Zac Dover

4 years, 3 months

1
0
0 0

2024

2023

2022

2021

2020

2019

Dev January 2020