27 Jul
2020
27 Jul
'20
9:02 p.m.
Hi,
I've got a problem on Octopus (15.2.3, debian packages) install, bucket
S3 index shows a file:
s3cmd ls s3://upvid/255/38355 --recursive
2020-07-27 17:48 50584342
s3://upvid/255/38355/juz_nie_zyjesz_sezon_2___oficjalny_zwiastun___netflix_mp4
radosgw-admin bi list also shows it
{
"type": "plain",
"idx":
"255/38355/juz_nie_zyjesz_sezon_2___oficjalny_zwiastun___netflix_mp4",
"entry": { "name":
"255/38355/juz_nie_zyjesz_sezon_2___oficjalny_zwiastun___netflix_mp4",
"instance": "", "ver": {
"pool": 11,
"epoch": 853842
},
"locator": "",
"exists": "true",
"meta": {
"category": 1,
"size": 50584342,
"mtime": "2020-07-27T17:48:27.203008Z",
"etag": "2b31cc8ce8b1fb92a5f65034f2d12581-7",
"storage_class": "",
"owner": "filmweb-app",
"owner_display_name": "filmweb app user",
"content_type": "",
"accounted_size": 50584342,
"user_data": "",
"appendable": "false"
},
"tag": "_3ubjaztglHXfZr05wZCFCPzebQf-ZFP",
"flags": 0,
"pending_map": [],
"versioned_epoch": 0
}
},
but trying to download it via curl (I've set permissions to public0 only gets me
<?xml version="1.0"
encoding="UTF-8"?><Error><Code>NoSuchKey</Code><BucketName>upvid</BucketName><RequestId>tx0000000000000000e716d-005f1f14cb-e478a-pl-war1</RequestId><HostId>e478a-pl-war1-pl</HostId></Error>
(the actually nonexisting files shows access denied in same context)
same with other tools:
$ s3cmd get
s3://upvid/255/38355/juz_nie_zyjesz_sezon_2___oficjalny_zwiastun___netflix_mp4 /tmp
download:
's3://upvid/255/38355/juz_nie_zyjesz_sezon_2___oficjalny_zwiastun___netflix_mp4'
-> '/tmp/juz_nie_zyjesz_sezon_2___oficjalny_zwiastun___netflix_mp4' [1 of 1]
ERROR: S3 error: 404 (NoSuchKey)
cluster health is OK
Any ideas what is happening here ?
--
Mariusz Gronczewski, Administrator
Efigence S. A.
ul. Wołoska 9a, 02-583 Warszawa
T: [+48] 22 380 13 13
NOC: [+48] 22 380 10 20
E: admin(a)efigence.com
28 Jul
28 Jul
12:31 a.m.
On Mon, Jul 27, 2020 at 08:02:23PM +0200, Mariusz Gronczewski wrote:
Hi,
I've got a problem on Octopus (15.2.3, debian packages) install, bucket
S3 index shows a file:
s3cmd ls s3://upvid/255/38355 --recursive
2020-07-27 17:48 50584342
s3://upvid/255/38355/juz_nie_zyjesz_sezon_2___oficjalny_zwiastun___netflix_mp4
radosgw-admin bi list also shows it
{
"type": "plain",
"idx":
"255/38355/juz_nie_zyjesz_sezon_2___oficjalny_zwiastun___netflix_mp4",
"entry": { "name":
"255/38355/juz_nie_zyjesz_sezon_2___oficjalny_zwiastun___netflix_mp4",
"instance": "", "ver": {
"pool": 11,
"epoch": 853842
},
"locator": "",
"exists": "true",
"meta": {
"category": 1,
"size": 50584342,
"mtime": "2020-07-27T17:48:27.203008Z",
"etag": "2b31cc8ce8b1fb92a5f65034f2d12581-7",
"storage_class": "",
"owner": "filmweb-app",
"owner_display_name": "filmweb app user",
"content_type": "",
"accounted_size": 50584342,
"user_data": "",
"appendable": "false"
},
"tag": "_3ubjaztglHXfZr05wZCFCPzebQf-ZFP",
"flags": 0,
"pending_map": [],
"versioned_epoch": 0
}
},
but trying to download it via curl (I've set permissions to public0 only gets me
Does the RADOS object for this still exist?
try:
radosgw-admin object stat --bucket ... --object
'255/38355/juz_nie_zyjesz_sezon_2___oficjalny_zwiastun___netflix_mp4'
If that doesn't return, then the backing object is gone, and you have a
stale index entry that can be cleaned up in most cases with check
bucket.
For cases where that doesn't fix it, my recommended way to fix it is
write a new 0-byte object to the same name, then delete it.
--
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
E-Mail : robbat2(a)gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136
2:28 p.m.
Dnia 2020-07-27, o godz. 21:31:33
"Robin H. Johnson" <robbat2(a)gentoo.org> napisał(a):
On Mon, Jul 27, 2020 at 08:02:23PM +0200, Mariusz
Gronczewski wrote:
it does exist:
{
"name":
"255/38355/juz_nie_zyjesz_sezon_2___oficjalny_zwiastun___netflix_mp4",
"size": 50584342, "policy": {
"acl": {...},
"owner": {...}
},
"etag": "2b31cc8ce8b1fb92a5f65034f2d12581-7",
"tag": "_3ubjaztglHXfZr05wZCFCPzebQf-ZFP",
"manifest": {
"objs": [],
"obj_size": 50584342,
"explicit_objs": "false",
"head_size": 0,
"max_head_size": 0,
"prefix":
"255/38355/juz_nie_zyjesz_sezon_2___oficjalny_zwiastun___netflix_mp4.2~NTy88SkDkXR9ifSrrRcw5WPDxqN3PO2",
"rules": [ {
"key": 0,
"val": {
"start_part_num": 1,
"start_ofs": 0,
"part_size": 8388608,
"stripe_max_size": 4194304,
"override_prefix": ""
}
},
{
"key": 50331648,
"val": {
"start_part_num": 7,
"start_ofs": 50331648,
"part_size": 252694,
"stripe_max_size": 4194304,
"override_prefix": ""
}
}
],
"tail_instance": "",
"tail_placement": {
"bucket": {
"name": "upvid",
"marker":
"88d4f221-0da5-444d-81a8-517771278350.665933.2", "bucket_id":
"88d4f221-0da5-444d-81a8-517771278350.665933.2", "tenant":
"",
"explicit_placement": {
"data_pool": "",
"data_extra_pool": "",
"index_pool": ""
}
},
"placement_rule": "default-placement"
},
"begin_iter": {
"part_ofs": 0,
"stripe_ofs": 0,
"ofs": 0,
"stripe_size": 4194304,
"cur_part_id": 1,
"cur_stripe": 0,
"cur_override_prefix": "",
"location": {
"placement_rule": "default-placement",
"obj": {
"bucket": {
"name": "upvid",
"marker":
"88d4f221-0da5-444d-81a8-517771278350.665933.2", "bucket_id":
"88d4f221-0da5-444d-81a8-517771278350.665933.2", "tenant":
"",
"explicit_placement": {
"data_pool": "",
"data_extra_pool": "",
"index_pool": ""
}
},
"key": {
"name":
"255/38355/juz_nie_zyjesz_sezon_2___oficjalny_zwiastun___netflix_mp4.2~NTy88SkDkXR9ifSrrRcw5WPDxqN3PO2.1",
"instance": "", "ns": "multipart"
}
},
"raw_obj": {
"pool": "",
"oid": "",
"loc": ""
},
"is_raw": false
}
},
"end_iter": {
"part_ofs": 50584342,
"stripe_ofs": 50584342,
"ofs": 50584342,
"stripe_size": 252694,
"cur_part_id": 8,
"cur_stripe": 0,
"cur_override_prefix": "",
"location": {
"placement_rule": "default-placement",
"obj": {
"bucket": {
"name": "upvid",
"marker":
"88d4f221-0da5-444d-81a8-517771278350.665933.2", "bucket_id":
"88d4f221-0da5-444d-81a8-517771278350.665933.2", "tenant":
"",
"explicit_placement": {
"data_pool": "",
"data_extra_pool": "",
"index_pool": ""
}
},
"key": {
"name":
"255/38355/juz_nie_zyjesz_sezon_2___oficjalny_zwiastun___netflix_mp4.2~NTy88SkDkXR9ifSrrRcw5WPDxqN3PO2.8",
"instance": "", "ns": "multipart"
}
},
"raw_obj": {
"pool": "",
"oid": "",
"loc": ""
},
"is_raw": false
}
}
},
"attrs": {
"user.rgw.pg_ver": "\u0005u",
"user.rgw.source_zone": "�I}�",
"user.rgw.tail_tag":
"88d4f221-0da5-444d-81a8-517771278350.658638.5538809",
"user.rgw.x-amz-acl": "private",
"user.rgw.x-amz-content-sha256":
"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"user.rgw.x-amz-date": "20200619T194517Z" }
}
--
Mariusz Gronczewski, Administrator
Efigence S. A.
ul. Wołoska 9a, 02-583 Warszawa
T: [+48] 22 380 13 13
NOC: [+48] 22 380 10 20
E: admin(a)efigence.com
Hi,
I've got a problem on Octopus (15.2.3, debian packages) install,
bucket S3 index shows a file:
s3cmd ls s3://upvid/255/38355 --recursive
2020-07-27 17:48 50584342
s3://upvid/255/38355/juz_nie_zyjesz_sezon_2___oficjalny_zwiastun___netflix_mp4
radosgw-admin bi list also shows it
{
"type": "plain",
"idx":
"255/38355/juz_nie_zyjesz_sezon_2___oficjalny_zwiastun___netflix_mp4",
"entry": { "name":
"255/38355/juz_nie_zyjesz_sezon_2___oficjalny_zwiastun___netflix_mp4",
"instance": "", "ver": {
"pool": 11,
"epoch": 853842
},
"locator": "",
"exists": "true",
"meta": {
"category": 1,
"size": 50584342,
"mtime": "2020-07-27T17:48:27.203008Z",
"etag": "2b31cc8ce8b1fb92a5f65034f2d12581-7",
"storage_class": "",
"owner": "filmweb-app",
"owner_display_name": "filmweb app user",
"content_type": "",
"accounted_size": 50584342,
"user_data": "",
"appendable": "false"
},
"tag": "_3ubjaztglHXfZr05wZCFCPzebQf-ZFP",
"flags": 0,
"pending_map": [],
"versioned_epoch": 0
}
},
but trying to download it via curl (I've set permissions to public0
only gets me
Does the RADOS object for this still exist?
try:
radosgw-admin object stat --bucket ... --object
'255/38355/juz_nie_zyjesz_sezon_2___oficjalny_zwiastun___netflix_mp4'
If that doesn't return, then the backing object is gone, and you have
a stale index entry that can be cleaned up in most cases with check
bucket.
For cases where that doesn't fix it, my recommended way to fix it is
write a new 0-byte object to the same name, then delete it.
6 Nov
6 Nov
1:03 p.m.
We are having the exact same problem (also Octopus). The object is listed by s3cmd, but
trying to download it results in a 404 error. radosgw-admin object stat shows that the
object still exists. Any further ideas how I can restore access to this object?
1276
days inactive
1378
days old
3 comments
3 participants
participants (3)
-
Janek Bevendorff -
Mariusz Gronczewski -
Robin H. Johnson