Hi,
yesterday I added a new zonegroup and it looks like it seems to cycle over
the same requests over and over again.
In the log of the main zone I see these requests:
2023-06-20T09:48:37.979+0000 7f8941fb3700 1 beast: 0x7f8a602f3700:
fd00:2380:0:24::136 - - [2023-06-20T09:48:37.979941+0000] "GET
/admin/log?type=metadata&id=62&period=e8fc96f1-ae86-4dc1-b432-470b0772fded&max-entries=100&&rgwx-zonegroup=b39392eb-75f8-47f0-b4f3-7d3882930b26
HTTP/1.1" 200 44 - - -
Only thing that changes is the &id.
We have two other zonegroups that are configured identical (ceph.conf and
period) and these don;t seem to spam the main rgw.
root@host:~# radosgw-admin sync status
realm 5d6f2ea4-b84a-459b-bce2-bccac338b3ef (main)
zonegroup b39392eb-75f8-47f0-b4f3-7d3882930b26 (dc3)
zone 96f5eca9-425b-4194-a152-86e310e91ddb (dc3)
metadata sync syncing
full sync: 0/64 shards
incremental sync: 64/64 shards
metadata is caught up with master
root@host:~# radosgw-admin period get
{
"id": "e8fc96f1-ae86-4dc1-b432-470b0772fded",
"epoch": 92,
"predecessor_uuid": "5349ac85-3d6d-4088-993f-7a1d4be3835a",
"sync_status": [
"",
...
""
],
"period_map": {
"id": "e8fc96f1-ae86-4dc1-b432-470b0772fded",
"zonegroups": [
{
"id": "b39392eb-75f8-47f0-b4f3-7d3882930b26",
"name": "dc3",
"api_name": "dc3",
"is_master": "false",
"endpoints": [
],
"hostnames": [
],
"hostnames_s3website": [
],
"master_zone": "96f5eca9-425b-4194-a152-86e310e91ddb",
"zones": [
{
"id": "96f5eca9-425b-4194-a152-86e310e91ddb",
"name": "dc3",
"endpoints": [
],
"log_meta": "false",
"log_data": "false",
"bucket_index_max_shards": 11,
"read_only": "false",
"tier_type": "",
"sync_from_all": "true",
"sync_from": [],
"redirect_zone": ""
}
],
"placement_targets": [
{
"name": "default-placement",
"tags": [],
"storage_classes": [
"STANDARD"
]
}
],
"default_placement": "default-placement",
"realm_id": "5d6f2ea4-b84a-459b-bce2-bccac338b3ef",
"sync_policy": {
"groups": []
}
},
...
--
Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend im
groüen Saal.
Hi,
I had some trouble in the past with my CephFS which I was able to
resolve - mostly with your help.
Now I have about 150GB of data in lost+found in my CephFS. No matter
what I try and how I change permissions, every time when I try to delete
or move something from there I only get the reply: "mv: cannot remove
'lost+found/100002b047c': Read-only file system".
I searched the web and configuration items but I didn't find a way to
get rid of these files. I copyied most of them to another place,
identified them and have them back. So in lost+found there are mostly
useless copies.
Cheers,
Thomas
--
http://www.widhalm.or.at
GnuPG : 6265BAE6 , A84CB603
Threema: H7AV7D33
Telegram, Signal: widhalmt(a)widhalm.or.at
hi , starting upgrade from 15.2.17 i got this error
Module 'cephadm' has failed: Expecting value: line 1 column 1 (char 0)
Cluster was in health ok before starting.
Hello,
we have a Ceph 17.2.5 cluster with a total of 26 nodes, where 15 nodes that have faulty NVMe drives,
where the db/wal resides (one NVMe for the first 6 OSDs and another for the remaining 6).
We replaced them with new drives and pvmoved it to avoid losing the OSDs.
So far, there are no issues, and the OSDs are functioning properly.
ceph see the correct news disks
root@node02:/# ceph daemon osd.26 list_devices
[
{
"device": "/dev/nvme0n1",
"device_id": "INTEL_SSDPEDME016T4S_CVMD516500851P6KGN"
},
{
"device": "/dev/sdc",
"device_id": "SEAGATE_ST18000NM004J_ZR52TT830000C148JFSJ"
}
]
However, the Cephadm GUI still shows the old NVMe drives and hasn't recognized the device change.
How can we make the GUI and Cephadm recognize the new devices?
I tried restarting the managers, thinking that it would rescan the OSDs during startup, but it didn't work.
If you have any ideas, I would appreciate it.
Should I perform something like that: ceph orch daemon reconfig osd.*
Thank you for your help.
Hi all,
Is there any way for clients (without rgw-admin access) to get the
placement target of their S3 buckets? The "GetBucketLocation'' api returns
"default" for all placement targets and I couldn't find any other S3 api
for this purpose.
Can anyone help me with this?
Hi,
might be a dump question, but is there a way to list the rgw instances that
are running in a ceph cluster?
Before pacific it showed up in `ceph status` but now it only tells me how
many daemons are active, now which daemons are active.
ceph orch ls tells me that I need to configure a backend but we are not at
the stage that we are going to implement the orchestrator yet.
Cheers
Boris
I deploy the rgw service and the default pool is created automatically But
I get an error in the dashboard
``
Error connecting to Object Gateway: RGW REST API request failed with
default 404 status code","HostId":"736528-default-default"}')
``
There is a dashboard user but I created the bucket manually
# radosgw-admin user info --uid=dashboard
{
"user_id": "dashboard",
"display_name": "Ceph Dashboard",
"email": "",
"suspended": 0,
"max_buckets": 1000,
"subusers": [],
"keys": [
{
"user": "dashboard",
"access_key": "C8YG708VBA3M3AAJW2U2",
"secret_key": "NpkmIZ5JJVnu3EFa0ytv5vO64NGttK9ks7A3gEQP"
}
],
"swift_keys": [],
"caps": [],
"op_mask": "read, write, delete",
"system": "true",
"default_placement": "",
"default_storage_class": "",
"placement_tags": [],
"bucket_quota": {
"enabled": false,
"check_on_raw": false,
"max_size": -1,
"max_size_kb": 0,
"max_objects": -1
},
"user_quota": {
"enabled": false,
"check_on_raw": false,
"max_size": -1,
"max_size_kb": 0,
"max_objects": -1
},
"temp_url_keys": [],
"type": "rgw",
"mfa_ids": []
}
----------------
# radosgw-admin buckets list
[
"dashboard"
]
How can I solve the problem?
Well in fact it does.
For example in our setup we are parsing the bucket name from the URL. It’s a bit tricky as a client could use both the domain name and path base styles, but that is not an issue for us.
Alternatively you can parse and analyse logs directly from RGWs which have the bucket information without URL parsing needed.
Ondrej
> On 3. 7. 2023, at 9:58, Szabo, Istvan (Agoda) <Istvan.Szabo(a)agoda.com> wrote:
>
> Hi,
>
> Yeah, I'm using haproxy but that doesn't have the bucket information :/
>
> Istvan Szabo
> Staff Infrastructure Engineer
> ---------------------------------------------------
> Agoda Services Co., Ltd.
> e: istvan.szabo(a)agoda.com
> ---------------------------------------------------
>
> -----Original Message-----
> From: Ondřej Kukla <ondrej(a)kuuk.la>
> Sent: Monday, July 3, 2023 2:25 PM
> To: Szabo, Istvan (Agoda) <Istvan.Szabo(a)agoda.com>
> Subject: Re: [ceph-users] Transmit rate metric based per bucket
>
> Email received from the internet. If in doubt, don't click any link nor open any attachment !
> ________________________________
>
> Hello, Istvan,
>
> As far as I’m avare there is no way to do it directly on the RGW. What you can do is using a LB in front of the RGWs something like Nginx or HAProxy.
>
> Then just collect the LB logs using ELK stack, Loki etc. and sum the body bytes sent per request type and bucket.
>
> Ondrej
>
>> On 20. 6. 2023, at 7:01, Szabo, Istvan (Agoda) <Istvan.Szabo(a)agoda.com> wrote:
>>
>> Hello,
>>
>> I'd like to know is there a way to query some metrics/logs in octopus (or if has newer version I'm interested for the future too) about the bandwidth used in the bucket for put/get operations?
>>
>> Thank you
>>
>> ________________________________
>> This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses.
>> _______________________________________________
>> ceph-users mailing list -- ceph-users(a)ceph.io To unsubscribe send an
>> email to ceph-users-leave(a)ceph.io
>
Hi,
I tried to generate a presigned url using SDK PHP, but it doesn't work. (I also tried to use boto3 with the same configures and the url works normally)
Here is my php code:
<?php
require 'aws-autoloader.php';
use Aws\S3\S3Client;
use Aws\Exception\AwsException;
$s3Client = new Aws\S3\S3Client([
'version' => '2006-03-01',
'region' => 'us-east-1',
'signature_version' => 'v4',
'use_path_style_endpoint' => true,
'endpoint' => 'http://hn.ss.bfcplatform.vn',
'credentials' => [
'key' => 'DNMZAFE6G2PP8H9P05UU',
'secret' => 'XXX',
]
]);
$cmd = $s3Client->getCommand('PutObject', [
'Bucket' => 'huynnp-testbucket1',
'Key' => 'testfile.txt',
]);
$request = $s3Client->createPresignedRequest($cmd, '+60 minutes'); // Set the expiration time as desired
$presignedUrl = (string)$request->getUri();
echo "$presignedUrl";
?>
and then:
curl -X PUT -T testfile.txt `php s3.php`
<?xml version="1.0" encoding="UTF-8"?><Error><Code>AccessDenied</Code><RequestId>tx00000b7bb3b2deb6a6ef2-00649d5ebd-d1d50041-hn-1</RequestId><HostId>d1d50041-hn-1-hn</HostId></Error>
I enable the debug_rgw and what I can see is really strange. the domain has been added :8084, so it make "canonical request hash" and "signature" between client and server unmatched. I can't explain why does this happens
2023-06-29T17:10:46.880+0700 7f26014b0700 10 v4 credential format = DNMZAFE6G2PP8H9P05UU/20230629/us-east-1/s3/aws4_request
2023-06-29T17:10:46.880+0700 7f26014b0700 10 access key id = DNMZAFE6G2PP8H9P05UU
2023-06-29T17:10:46.880+0700 7f26014b0700 10 credential scope = 20230629/us-east-1/s3/aws4_request
2023-06-29T17:10:46.880+0700 7f26014b0700 10 req 15647562574720867919 1000005ns canonical headers format = host:hn.ss.bfcplatform.vn:8084
2023-06-29T17:10:46.880+0700 7f26014b0700 10 payload request hash = UNSIGNED-PAYLOAD
2023-06-29T17:10:46.880+0700 7f26014b0700 10 canonical request = PUT
/huynnp-testbucket1/testfile.txt
X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=DNMZAFE6G2PP8H9P05UU%2F20230629%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230629T101046Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host
host:hn.ss.bfcplatform.vn:8084
host
UNSIGNED-PAYLOAD
2023-06-29T17:10:46.880+0700 7f26014b0700 10 canonical request hash = d28e6c3104aff99e9928f902892627d2b284a29d489fbb034ed5c90aa21c566a
2023-06-29T17:10:46.880+0700 7f26014b0700 10 string to sign = AWS4-HMAC-SHA256
20230629T101046Z
20230629/us-east-1/s3/aws4_request
d28e6c3104aff99e9928f902892627d2b284a29d489fbb034ed5c90aa21c566a
