Hi.
I am currently using Ceph for replicated storage to store many objects across 5 nodes with 3x replication.
When I generate ~1000 read requests to a single object, they all get serviced by the same primary OSD. I would like to balance the reads across the replicas.
So I use the following:
auto read_op = rados_create_read_op();
rados_read_op_read(read_op, offset, outSize, buffer, &bytes_read, &prval);
err = rados_read_op_operate(read_op, pool->ioctx, keyName.c_str(), LIBRADOS_OPERATION_BALANCE_READS);
However, this does not seem to balance the reads across replicas. I do not see what I am doing wrong in the above code.
Could you please guide me on this?
ceph-mon and ceph-osd are run on Ubuntu 22.04 installed via apt-get update ceph ceph-mds ceph-volume
If I should ask this question somewhere else, please point me in the right direction.
Thanks and regards,
Alan.
The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. Is e buidheann carthannais a th' ann an Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.
Dear all,
We are experiencing with Ceph after deploying it by PVE with the network backed by a 10G Cisco switch with VPC feature on. We are encountering a slow OSD heartbeat and have not been able to identify any network traffic issues.
Upon checking, we found that the ping is around 0.1ms, and there is occasional 2% packet loss when using flood ping, but not consistently. We also noticed a large number of UDP port 5405 packets and the 'corosync' process utilizing a significant amount of CPU.
When running the 'ceph -s' command, we observed a slow OSD heartbeat on the back and front, with the longest latency being 2250.54ms. We suspect that this may be a network issue, but we are unsure of how Ceph detects such long latency. Additionally, we are wondering if a 2% packet loss can significantly affect Ceph's performance and even cause the OSD process to fail sometimes.
We have heard about potential issues with rockdb 6 causing OSD process failures, and we are curious about how to check the rockdb version. Furthermore, we are wondering how severe traffic package loss and latency must be to cause OSD process crashes, and how the monitoring system determines that an OSD is offline.
We would greatly appreciate any assistance or insights you could provide on these matters.
Thanks,
Hi Venky,
> Also, at one point the kclient wasn't able to handle more than 400 snapshots (per file system), but we have come a long way from that and that is not a constraint right now.
Does it mean that there is no more limit to the number of snapshots per filesystem? And, if not, do you know what is the max number of snapshots now (per filesystem) ??
Cheers,
Arnaud
Hi folks,
Armed with bucket-specific sync policy feature, I found that we could move objects of a bucket between zones. It is migration via sync followed by object removal at the source. This allows us to better utilize available capacities in different clusters/zones. However, to achieve this, we need a way to reset an empty bucket so that it can serve as a destination for a migration after it serves as a source before. ceph/rgw currently doesn't seem to be able to do that. So I create a feature request for it https://tracker.ceph.com/issues/59593
My own prototype shows that this feature is fairly simply to implement and works well for bucket migration.
Cheers,
Yixin
Hi,
When creating a ceph cluster, a failover domain is created, and by default
it uses host as a minimal domain, that domain can be modified to chassis,
or rack, ...etc.
My question is :
Suppose I have three osd nodes, my replication is 3 and my failover domain
is host, which means that each copy of data is stored on a different node.
What happens when one node crashes, does Ceph use the remaining free space
on the other two to create the third copy, or the ceph cluster will run in
degraded mode, like a RAID5
which lost a disk.
Regards.
Hello,
Lately I’ve been playing with Lua scripting on top of RGW.
I would like to implement a request blocking based on bucket name -> when there is a dot in a bucket name return error code and a message that this name is invalid.
Here is the code I was able to came up with.
if string.find(Request.HTTP.URI, '%.') then
Request.Response.HTTPStatusCode = 400
Request.Response.HTTPStatus = “InvalidBucketName"
Request.Response.Message = “Dots in bucket name are not allowed."
end
This works fine, but the request for creating a bucket would be processed and the bucket will be created. I thought about a dirty workaround with setting the Request.Bucket.Name to a bucket that already exists but it seems that this field is not writable in Quincy.
Is there a way to block the request from processing?
Any help is much appreciated.
Kind regards,
Ondrej
