Hi Ashley, Thanks for your response. Nothing that I can think of would have happened. We are using max_mds =1. We do have 4 so used to have 3 standby. Within minutes they all crash. On Fri, May 1, 2020 at 2:21 PM Ashley Merrick <singapore@amerrick.co.uk> wrote:
Quickly checking the code that calls that assert
if (version > omap_version) {
omap_version = version;
omap_num_objs = num_objs;
omap_num_items.resize(omap_num_objs);
journal_state = jstate;
} else if (version == omap_version) {
ceph_assert(omap_num_objs == num_objs);
if (jstate > journal_state)
journal_state = jstate;
}
}
Im not a dev, but not sure if this will help, seems could mean that MDS thinks its behind on omaps/too far ahead.
Anything happened recently? Just running a single MDS?
Hopefully someone else may see this and shine some light on what could be causing it.
---- On Sat, 02 May 2020 02:10:58 +0800 marcopizzolo@gmail.com wrote ----
Hello,
Hoping you can help me.
Ceph had been largely problem free for us for the better part of a year. We have a high file count in a single CephFS filesystem, and are seeing this error in the logs:
/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/gigantic/release/14.2.9/rpm/el7/BUILD/ceph-14.2.9/src/mds/OpenFileTable.cc: 777: FAILED ceph_assert(omap_num_objs == num_objs)
The issued seemed to occur this morning, and restarting the MDS as well as rebooting the servers doesn't correct the problem.
Not really sure where to look next as the MDS daemons crash.
Appreciate any help you can provide
Marco _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io