Hi Istvan:
Any update for directional sync?
I am trying to upgrade Ceph to 15.2.10. Also I have some issues that RGW
may crash after I ran data sync init. :(
Thanks
I was having difficulty doing this myself, and I came across this semi-recent thread:
https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/T4R76XJN2NE…
" I've tried adding OSDs with ceph orch daemon add ... but it's pretty limited. ...you can't [have] a separate db device. "
Has this been fixed yet?
Is it GOING to be fixed?
--
Philip Brown| Sr. Linux System Administrator | Medata, Inc.
5 Peters Canyon Rd Suite 250
Irvine CA 92606
Office 714.918.1310| Fax 714.918.1325
pbrown(a)medata.com| www.medata.com
Hi,
I am testing 15.2.10 on a large cluster (RH8). cephfs pool (size=1) with
122 nvme OSDs works fine till the number of clients is relatively low.
Writing from 400 kernel clients (ior benchmark), 8 streams each, causes
issues. Writes are initially fast at 100GB/s but then they drop to
<1GB/s after few minutes, while the OSDs use 300% cpu each.
My guess is that the OSDs are overloaded with requests from too many
clients, since it does not happen till there are ~3-4 streams OSD. The
OSDs log do not show anything problematic.
tried to increase osd_op_num_threads_per_shard_ssd, did not help.
Restarting OSDs recovers the situation for few minutes.
Writing to HDD pool with 1500 HDDs does not have any issues at all under
same conditions.
Any hints, settings to improve this?
Cheers,
Andrej
--
_____________________________________________________________
prof. dr. Andrej Filipcic, E-mail: Andrej.Filipcic(a)ijs.si
Department of Experimental High Energy Physics - F9
Jozef Stefan Institute, Jamova 39, P.o.Box 3000
SI-1001 Ljubljana, Slovenia
Tel.: +386-1-477-3674 Fax: +386-1-425-7074
-------------------------------------------------------------
Hey all,
We will be having a Ceph science/research/big cluster call on Wednesday
March 24th. If anyone wants to discuss something specific they can add
it to the pad linked below. If you have questions or comments you can
contact me.
This is an informal open call of community members mostly from
hpc/htc/research environments where we discuss whatever is on our minds
regarding ceph. Updates, outages, features, maintenance, etc...there is
no set presenter but I do attempt to keep the conversation lively.
https://pad.ceph.com/p/Ceph_Science_User_Group_20210324
<https://pad.ceph.com/p/Ceph_Science_User_Group_20200923>
We try to keep it to an hour or less.
Ceph calendar event details:
March 24th, 2021
15:00 UTC
4pm Central European
10am Central US
Description: Main pad for discussions:
https://pad.ceph.com/p/Ceph_Science_User_Group_Index
Meetings will be recorded and posted to the Ceph Youtube channel.
To join the meeting on a computer or mobile phone:
https://bluejeans.com/908675367?src=calendarLink
To join from a Red Hat Deskphone or Softphone, dial: 84336.
Connecting directly from a room system?
1.) Dial: 199.48.152.152 or bjn.vc
2.) Enter Meeting ID: 908675367
Just want to dial in on your phone?
1.) Dial one of the following numbers: 408-915-6466 (US)
See all numbers: https://www.redhat.com/en/conference-numbers
2.) Enter Meeting ID: 908675367
3.) Press #
Want to test your video connection? https://bluejeans.com/111
Kevin
--
Kevin Hrpcek
NASA VIIRS Atmosphere SIPS
Space Science & Engineering Center
University of Wisconsin-Madison
When looking over the changelog for 15.2.10 I noticed some bluefs
changes. One in particular caught my eye, and it was called out as a
notable change:
os/bluestore: fix huge reads/writes at BlueFS (pr#39701
<https://github.com/ceph/ceph/pull/39701>, Jianpeng Ma, Igor Fedotov)
It wasn't obvious to me how important that one was for data integrity.
We'll roll 15.2.10 onto our clusters shortly, but would appreciate an
opinion on whether any delay in installing this would increase the risk
of anything nasty (e.g. silent data corruption). If that was the case we
will push the release through rather more urgently!
Thanks, Chris
Terribly sorry for the mistake. There was a bug in the script I use to
sync packages to download.ceph.com that wasn't listing directories in
the desired order. That meant the download.ceph.com/{rpm,deb}-octopus
symlinks still pointed to 15.2.9. This is fixed.
I'm re-running the container jobs to get those pushed too.
On 3/18/21 10:45 AM, David Orman wrote:
> Hi David,
>
> The "For Packages" link in your email/the blog posts do not appear to
> work. Additionally, we browsed the repo, and it doesn't appear the
> packages are uploaded, at least for debian-octopus:
> http://download.ceph.com/debian-octopus/pool/main/c/ceph/. We only use
> the release packages for cephadm bootstrapping, so it's not a
> deal-breaker for us, just wanted to give you a head's up.
>
> Cheers,
> David Orman
>
> On Thu, Mar 18, 2021 at 9:11 AM David Galloway <dgallowa(a)redhat.com> wrote:
>>
>> We're happy to announce the 10th backport release in the Octopus series.
>> We recommend users to update to this release. For a detailed release
>> notes with links & changelog please refer to the official blog entry at
>> https://ceph.io/releases/v15-2-10-octopus-released
>>
>> Notable Changes
>> ---------------
>>
>> * The containers include an updated tcmalloc that avoids crashes seen on
>> 15.2.9. See `issue#49618 <https://tracker.ceph.com/issues/49618>`_ for
>> details.
>>
>> * RADOS: BlueStore handling of huge(>4GB) writes from RocksDB to BlueFS
>> has been fixed.
>>
>> * When upgrading from a previous cephadm release, systemctl may hang
>> when trying to start or restart the monitoring containers. (This is
>> caused by a change in the systemd unit to use `type=forking`.) After the
>> upgrade, please run::
>>
>> ceph orch redeploy nfs
>> ceph orch redeploy iscsi
>> ceph orch redeploy node-exporter
>> ceph orch redeploy prometheus
>> ceph orch redeploy grafana
>> ceph orch redeploy alertmanager
>>
>>
>> Getting Ceph
>> ------------
>> * Git at git://github.com/ceph/ceph.git
>> * Tarball at http://download.ceph.com/tarballs/ceph-15.2.10.tar.gz
>> * For packages, see http://docs.ceph.com/docs/master/install/get-packages/
>> * Release git sha1: 27917a557cca91e4da407489bbaa64ad4352cc02
>> _______________________________________________
>> Dev mailing list -- dev(a)ceph.io
>> To unsubscribe send an email to dev-leave(a)ceph.io
>
Hi all,
How have folks implemented getting email or snmp alerts out of Ceph? Getting things like osd/pool nearly full or osd/daemon failures etc.
Kind regards
Andrew
Sent from my iPhone
We're happy to announce the 10th backport release in the Octopus series.
We recommend users to update to this release. For a detailed release
notes with links & changelog please refer to the official blog entry at
https://ceph.io/releases/v15-2-10-octopus-released
Notable Changes
---------------
* The containers include an updated tcmalloc that avoids crashes seen on
15.2.9. See `issue#49618 <https://tracker.ceph.com/issues/49618>`_ for
details.
* RADOS: BlueStore handling of huge(>4GB) writes from RocksDB to BlueFS
has been fixed.
* When upgrading from a previous cephadm release, systemctl may hang
when trying to start or restart the monitoring containers. (This is
caused by a change in the systemd unit to use `type=forking`.) After the
upgrade, please run::
ceph orch redeploy nfs
ceph orch redeploy iscsi
ceph orch redeploy node-exporter
ceph orch redeploy prometheus
ceph orch redeploy grafana
ceph orch redeploy alertmanager
Getting Ceph
------------
* Git at git://github.com/ceph/ceph.git
* Tarball at http://download.ceph.com/tarballs/ceph-15.2.10.tar.gz
* For packages, see http://docs.ceph.com/docs/master/install/get-packages/
* Release git sha1: 27917a557cca91e4da407489bbaa64ad4352cc02
