Hello everyone,
I've been trying to use CephFS and mix it together with fscache, however i was never able to have multiple mounts with fscache enabled.
Is this a known intentional limitation or a bug?
It would be possible to work around it by mounting the root of the filesystem and using bind mounts but i have separate volumes that need to be separately mounted.
How to replicate:
mount -t ceph -o fsc admin@.filesystem1=/path1 /tmp/one # Succeeds
mount -t ceph -o fsc admin@.filesystem1=/path2 /tmp/two # Fails complaining about no mds being available
The alternative of using no fscache works just fine:
mount -t ceph admin@.filesystem1=/path1 /tmp/one # Succeeds
mount -t ceph admin@.filesystem1=/path2 /tmp/two # Succeeds
Versions:
- ceph quincy 17.2.6
- linux 6.4.6
- cachefilesd 0.10.10
--
Alex D.
RedXen System & Infrastructure Administration
https://redxen.eu/
Hi,
We are in a transition where I'd like to ask my user who stores 2B objects in 1 bucket to split it some way.
Thinking for the future we identified to make it future proof and don't store huge amount of objects in 1 bucket, we would need to create 65xxx buckets.
Is there anybody aware of any issue with this amount of buckets please?
I guess better to split to multiple buckets rather than have gigantic bucket.
Thank you the advises
________________________________
This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses.
Hello. It's been a while. I have a Nautilus cluster with 72 x 12GB HDD
OSDs (BlueStore) and mostly of EC 8+2 pools/PGs. It's been working great -
some nodes went nearly 900 days without a reboot.
As of yesterday I found that I have 3 OSDs with a Smart status of 'Pending
Failure'. New drives are ordered and will be here next week. There is a
procedure in the documentation for replacing an OSD, but I can't do that
directly until I receive the drives.
My inclination is to mark these 3 OSDs 'OUT' before they crash completely,
but I want to confirm my understanding of Ceph's response to this. Mainly,
given my EC pools (or replicated pools for that matter), if I mark all 3
OSD out all at once will I risk data loss?
If I have it right, marking an OSD out will simply cause Ceph to move all
of the PG shards from that OSD to other OSDs, so no major risk of data
loss. However, if it would be better to do them one per day or something,
I'd rather be safe.
I also assume that I should wait for the rebalance to complete before I
initiate the replacement procedure.
Your thoughts?
Thanks.
-Dave
--
Dave Hall
Binghamton University
kdhall(a)binghamton.edu
Hi,
We know snapshot is on a point of time. Is this point of time tracked internally by
some sort of sequence number, or the timestamp showed by "snap ls", or something else?
I noticed that when "deep cp", the timestamps of all snapshot are changed to copy-time.
Say I create a snapshot at 1PM and make a copy at 3PM, the timestamp of snapshot in
the copy is 3PM. If I rollback the copy to this snapshot, I'd assume it will actually bring me
back to the state of 1PM. Is that correct?
If the above is true, I won't be able to rely on timestamp to track snapshots.
Say I create a snapshot every hour and make a backup by copy at the end of the day.
Then the original image is damaged and backup is used to restore the work. On this
backup image, how do I know which snapshot was on 1PM, which was on 2PM, etc.?
Any advices to track snapshots properly in such case?
I can definitely build something else to help on this, but I'd like to know how much
Ceph can support it.
Thanks!
Tony
Hi,
There is a snap ID for each snapshot. How is this ID allocated, sequentially?
Did some tests, it seems this ID is per pool, starting from 4 and always going up.
Is that correct?
What's the max of this ID?
What's going to happen when ID reaches the max, going back to start from 4 again?
Thanks!
Tony
Hi,
During the upgrade from centos7/nautilus to ubuntu 18/nautilus (still updating the MONs) I got a cephfs client who refuses or is refused to mount the ceph fs again.
The clients says: mount error 13 = Permission denied
The cephmds log: lacks required features 0x0000000000001000 client supports 0x00000000000000ff
The mds/mon is still centos7/nautilus. The Clients centos7 as well.
Any ideas ? Thx for suggestions and hints . Best Götz
Hi,
We have a cluster running for a while. From grafana ceph dashboard, I saw
OSD onode hits ratio 92% when cluster was just up and running. After couple
month, it says now 70%. This is not a good trend I think. Just wondering
what should be done to stop this trend.
Many thank,
Ben
I've been digging and I can't see that this has come up anywhere.
I'm trying to update a client from Pacific 17.2.3-2 to 17.2.6-4 and I'm getting the error
Error:
Problem: cannot install the best update candidate for package ceph-base-2:17.2.3-2.el9s.x86_64
- nothing provides liburing.so.2()(64bit) needed by ceph-base-2:17.2.6-4.el9s.x86_64
- nothing provides liburing.so.2(LIBURING_2.0)(64bit) needed by ceph-base-2:17.2.6-4.el9s.x86_64
(try to add '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
Did Ceph Pacific switch to requiring liburing 2? Rocky 9 only provides 0.7-7. CentOS stream seems to have 1.0.7-3 (at least back to when I set up that repo on Foreman; I don't remember if I'm keeping it up-to-date).
Can I/should I just do --nobest when updating? I could probably build it from a source RPM from another RH-based distro, but I'd rather keep it clean with the same distro.
Hi,
We're having sporadic problems with a CephFS filesystem where MDSs end up
on the OSD blocklist. We're still digging around looking for a cause
(Ceph related or other infrastructure cause).
The cluster isn't massive (68 OSDs spread over 34 hosts), each host is a
VM, with MGR/MON/MDS on non-OSD hosts.
Running Ceph 16.2.10
Any suggestions for debugging this further?