August 2019 - Dev - lists.ceph.io

"message too long"

by Tapas Jana

tapas(a)wolk.com

4 years, 7 months

1
0
0 0

EC Data pools with RGW - recovery issues

by Wyllys Ingersoll

We are running some tests under Luminous using an erasure code data pool for RGW where we push large amounts of data to the gateway using COSbench and then monitor performance while we remove devices and watch the subsequent recovery. The goal is to see how the RGW handles failures and recovery when using EC data pools. Our EC data pool is consists of 13 NVME bluestore devices using jerasure with K=7, M=3 and min_size=7. Under this configuration, I would expect that we could lose 3 devices without risk of data loss. But what we see is that once a single OSD is removed and recovery begins, the RGW stops working (its still running, just not doing anything) and will not begin processing requests (read or write) until the cluster has no more degraded PGs, usually this means re-adding the device and allowing the rebalance to backfill the degraded PGs. We did not appear to lose any data, but the fact that RGW stops processing requests while a recovery is happening is troubling. We expected the data to continue to be processed since we still had enough OSDs in the pool to satisfy the EC parameters (7/3) Is this expected behavior? thanks, Wyllys Ingersoll

4 years, 7 months

1
0
0 0

14.2.3 QE Nautilus validation status

by Yuri Weinstein

Details of this release summarized here: https://tracker.ceph.com/issues/41249#note-2 smoke - PASSED rados - FAILED - Neha pls review/approve/comment/ticket # rgw - FAILED - Casey pls review/approve/comment/ticket # rbd - FAILED - Jason pls review/approve/comment/ticket # krbd - FAILED - Ilya pls review/approve/comment/ticket # fs - FAILED - Patrick pls review/approve/comment/ticket # kcephfs - FAILED - Patrick pls review/approve/comment/ticket # multimds - FAILED - Patrick pls review/approve/comment/ticket # ceph-deploy - FAILED Sage, Alfredo FYI and what's next ? ceph-disk - N/A upgrade/client-upgrade-hammer (nautilus) - N/A upgrade/client-upgrade-jewel (nautilus) - PASSED upgrade/client-upgrade-mimic (nautilus) - FAILED needs more looking upgrade/client-upgrade-luminous-nautilus (nautilus) - FAILED needs more looking upgrade/luminous-p2p - FAILED needs more looking powercycle - PASSED ceph-ansible - FAILED Brad is looking upgrade/luminous-x (nautilus) - FAILED needs more looking ceph-volume - PASSED (please speak up if something is missing) PS: I am on vacation next week, that's why sending this status now and some runs require more reviewing/troubleshooting Thx YuriW

4 years, 7 months

11
16
0 0

08/29/2019 perf meeting is on!

by Mark Nelson

Hi Folks, Perf meeting is on in ~1 hour! Today we'll talk about a lot of rapid work happening on RGW and the OSD to improve the performance of put throughput and bucket listing. Please feel free to add your own agenda items to the etherpad! Etherpad: https://pad.ceph.com/p/performance_weekly Bluejeans: https://bluejeans.com/908675367 Thanks, Mark

4 years, 7 months

1
0
0 0

***** SPAM 5.8 ***** Re: camera&phone tripod from experienced Factory

by Joyce

4 years, 8 months

1
0
0 0

DocuBetter APAC - RADOS

by Neha Ojha

Hi Folks, The next DocuBetter meeting is on August 28, 2019 at 6 pm PT. Etherpad: https://pad.ceph.com/p/Ceph_Documentation Meeting: https://bluejeans.com/908675367 Agenda: We'll be discussing documentation gaps in RADOS and brainstorming ideas to make the existing documentation more easy to understand. (and any other topics that folks want to bring up) Hope to see you there! Thanks, Neha

4 years, 8 months

1
0
0 0

***** SPAM 5.8 ***** Re: customized women's knitted clothing service

by rongsha66178

4 years, 8 months

1
0
0 0

ceph-dencoder output has differences...

by Willem Jan Withagen

Hi Kefu Since a few days there is a difference in the output of bluestore_bdev_t in ceph-dencoder. And it looks like the buffer is eihter too long, or not properly zero-ed out. the encode/decode stuff has not been touched for a while. But if it uses the pre-initialized test-label, this work out oke. Dump_json actually also reports identical results. So I guess it is somewhere in other code that manages the buffers?? Suggestions as how to debug it? --WjW [~/master36] wjw(a)cephdev.digiware.nl> build/bin/ceph-dencoder type bluestore_bdev_label_t select_test 1 encode export /tmp/typ-yFISvjvgj [~/master36] wjw(a)cephdev.digiware.nl> hexdump -C !$ hexdump -C /tmp/typ-yFISvjvgj 00000000 62 6c 75 65 73 74 6f 72 65 20 62 6c 6f 63 6b 20 |bluestore block | 00000010 64 65 76 69 63 65 0a 30 30 30 30 30 30 30 30 2d |device.00000000-| 00000020 30 30 30 30 2d 30 30 30 30 2d 30 30 30 30 2d 30 |0000-0000-0000-0| 00000030 30 30 30 30 30 30 30 30 30 30 30 0a 02 01 28 00 |00000000000...(.| 00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000050 00 00 70 74 03 04 00 00 00 00 00 00 00 00 00 00 |..pt............| 00000060 00 00 00 00 00 00 00 00 00 00 |..........| 0000006a [~/master36] wjw(a)cephdev.digiware.nl> build/bin/ceph-dencoder type bluestore_bdev_label_t select_test 1 encode decode encode export /tmp/typ-MjWXdCpzJ [~/master36] wjw(a)cephdev.digiware.nl> hexdump -C !$ hexdump -C /tmp/typ-MjWXdCpzJ 00000000 62 6c 75 65 73 74 6f 72 65 20 62 6c 6f 63 6b 20 |bluestore block | 00000010 64 65 76 69 63 65 0a 30 30 30 30 30 30 30 30 2d |device.00000000-| 00000020 30 30 30 30 2d 30 30 30 30 2d 30 30 30 30 2d 30 |0000-0000-0000-0| 00000030 30 30 30 30 30 30 30 30 30 30 30 0a 02 01 28 00 |00000000000...(.| 00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000050 00 00 73 64 00 00 00 00 00 00 00 00 00 00 00 00 |..sd............| 00000060 00 00 00 00 00 00 00 00 00 00 |..........| 0000006a [~/master36] wjw(a)cephdev.digiware.nl> build/bin/ceph-dencoder type bluestore_bdev_label_t select_test 2 encode decode encode export /tmp/test [~/master36] wjw(a)cephdev.digiware.nl> hexdump -C /tmp/test 00000000 62 6c 75 65 73 74 6f 72 65 20 62 6c 6f 63 6b 20 |bluestore block | 00000010 64 65 76 69 63 65 0a 30 30 30 30 30 30 30 30 2d |device.00000000-| 00000020 30 30 30 30 2d 30 30 30 30 2d 30 30 30 30 2d 30 |0000-0000-0000-0| 00000030 30 30 30 30 30 30 30 30 30 30 30 0a 02 01 3b 00 |00000000000...;.| 00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000050 00 00 7b 00 00 00 00 00 00 00 04 00 00 00 05 00 |..{.............| 00000060 00 00 05 00 00 00 66 61 6b 65 79 01 00 00 00 03 |......fakey.....| 00000070 00 00 00 66 6f 6f 03 00 00 00 62 61 72 |...foo....bar| 0000007d [~/master36] wjw(a)cephdev.digiware.nl> build/bin/ceph-dencoder type bluestore_bdev_label_t select_test 2 encode export /tmp/test2 [~/master36] wjw(a)cephdev.digiware.nl> hexdump -C /tmp/test2 00000000 62 6c 75 65 73 74 6f 72 65 20 62 6c 6f 63 6b 20 |bluestore block | 00000010 64 65 76 69 63 65 0a 30 30 30 30 30 30 30 30 2d |device.00000000-| 00000020 30 30 30 30 2d 30 30 30 30 2d 30 30 30 30 2d 30 |0000-0000-0000-0| 00000030 30 30 30 30 30 30 30 30 30 30 30 0a 02 01 3b 00 |00000000000...;.| 00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000050 00 00 7b 00 00 00 00 00 00 00 04 00 00 00 05 00 |..{.............| 00000060 00 00 05 00 00 00 66 61 6b 65 79 01 00 00 00 03 |......fakey.....| 00000070 00 00 00 66 6f 6f 03 00 00 00 62 61 72 |...foo....bar| 0000007d

4 years, 8 months

1
0
0 0

Re: fix for hidden corei7 requirement in binary packages

by Brad Hubbard

+dev@ceph On Sun, Aug 18, 2019 at 9:50 AM Alexandre Oliva <oliva(a)gnu.org> wrote: > > Hi, > > After upgrading some old Phenom servers from Fedora/Freed-ora 29 to > 30's, to ceph-14.2.2, I was surprised by very early crashes of both > ceph-mon and ceph-osd. After ruling out disk and memory corruption, I > investigated a bit noticed all of them crashed during pre-main() init > section processing, at an instruction not available on the Phenom X6 > processors that support sse4a, but not e.g. sse4.1. > > It turns out that much of librte is built with -march=corei7. That's a > little excessive, considering that x86/rte_memcpy.h would be happy > enough with -msse4.1, but not with earlier sse versions that Fedora is > supposed to target. > > I understand rte_memcpy.h is meant for better performance, inlining > fixed-size and known-alignment implementations of memcpy into users. > Alas, that requires setting a baseline target processor, and you'll only > get as efficient an implementation as what's built in. > > I noticed an attempt for dynamic selection, but GCC rightfully won't > inline across different target flags, so we'd have to give up inlining > to get better dynamic behavior. The good news is that glibc already > offers dynamic selection of memcpy implementations, so hopefully the > impact of this change won't be much worse than that of enabling dynamic > selection, without the complications. > > If that's not good enough, compiling ceph with flags that enable SSE4.1, > AVX2 or AVX512, or with a -march flag that implicitly enables them, > would restore current performance, but without that, you will (with the > patch below) get a package that runs on a broader range of processors, > that the base distro (through the compiler's baseline flags) chooses to > support. It's not nice when you install a package on a processor that's > supposed to be supported and suddenly you're no longer sure it is ;-) > > Perhaps building a shared librte, so that one could build and install > builds targeting different ISA versions, without having to rebuild all > of ceph, would be a reasonable way to address the better tuning of these > performance-critical bits. > > > > src/spdk/dpdk: > > diff --git a/lib/librte_eal/common/include/arch/x86/rte_memcpy.h b/lib/librte_eal/common/include/arch/x86/rte_memcpy.h > index 7b758094d..ce714bf02 100644 > --- a/lib/librte_eal/common/include/arch/x86/rte_memcpy.h > +++ b/lib/librte_eal/common/include/arch/x86/rte_memcpy.h > @@ -40,6 +40,16 @@ extern "C" { > static __rte_always_inline void * > rte_memcpy(void *dst, const void *src, size_t n); > > +#ifndef RTE_MACHINE_CPUFLAG_SSE4_1 > + > +static __rte_always_inline void * > +rte_memcpy(void *dst, const void *src, size_t n) > +{ > + return memcpy(dst, src, n); > +} > + > +#else /* RTE_MACHINE_CPUFLAG_SSE4_1 */ > + > #ifdef RTE_MACHINE_CPUFLAG_AVX512F > > #define ALIGNMENT_MASK 0x3F > @@ -869,6 +879,8 @@ rte_memcpy(void *dst, const void *src, size_t n) > return rte_memcpy_generic(dst, src, n); > } > > +#endif /* RTE_MACHINE_CPUFLAG_SSE4_1 */ > + > #ifdef __cplusplus > } > #endif > diff --git a/mk/machine/default/rte.vars.mk b/mk/machine/default/rte.vars.mk > index df08d3b03..6bf695849 100644 > --- a/mk/machine/default/rte.vars.mk > +++ b/mk/machine/default/rte.vars.mk > @@ -27,4 +27,4 @@ > # CPU_LDFLAGS = > # CPU_ASFLAGS = > > -MACHINE_CFLAGS += -march=corei7 > +# MACHINE_CFLAGS += -march=corei7 > > > -- > Alexandre Oliva, freedom fighter he/him https://FSFLA.org/blogs/lxo > Be the change, be Free! FSF Latin America board member > GNU Toolchain Engineer Free Software Evangelist > Hay que enGNUrecerse, pero sin perder la terGNUra jamás - Che GNUevara -- Cheers, Brad

4 years, 8 months

3
7
0 0

***** SPAM 6.3 ***** China Supplier Full Color Transparent LED Display

by ts1＠a.ambf.cc

4 years, 8 months

1
0
0 0

2024

2023

2022

2021

2020

2019

Dev August 2019