13 Mar
2021
13 Mar
'21
4:07 a.m.
Hi all,
I was wondering if anyone knew of documentation regarding the s3-test suite
(https://github.com/ceph/s3-tests <https://github.com/ceph/s3-tests>). We’re trying
to implement some multi-cluster functionality with s3 and part of it is validation testing
to make sure everything works as it should. I’m assuming that a lot of what we’re trying
to do is already covered by the s3-test suite. I’m just looking for more documentation on
what exactly is covered by the tests, and what functionality is tested.
I’ve been able to find docs relating to running the tests on a deployed cluster
(https://docs.ceph.com/en/latest/dev/developer_guide/running-tests-locally/
<https://docs.ceph.com/en/latest/dev/developer_guide/running-tests-locally/>). But
no documentation beyond that about the tests themselves. Just from looking at the code
there’s organization by feature using the attribution feature in nose, so I’m wondering if
there’s existing knowledge that’s recorded but that I’m missing. Something that breaks
down coverage of each attribute, or a list of all the tests being run along with a quick
description.
- Gavin
14 Mar
14 Mar
2:55 a.m.
On Fri, Mar 12, 2021 at 04:07:22PM -0500, Gavin Chen wrote:
I’ve been able to find docs relating to running the
tests on a
deployed cluster
(https://docs.ceph.com/en/latest/dev/developer_guide/running-tests-locally/
<https://docs.ceph.com/en/latest/dev/developer_guide/running-tests-locally/>).
But no documentation beyond that about the tests themselves. Just from
looking at the code there’s organization by feature using the
attribution feature in nose, so I’m wondering if there’s existing
knowledge that’s recorded but that I’m missing. Something that breaks
down coverage of each attribute, or a list of all the tests being run
along with a quick description.
Hi!
I answered you on IRC, but I don't know if you saw it.
There isn't a good list of what's covered, and that's one of the goals I
have in mind when I included a project in the list of potential GSOC
projects for this year:
"RGW: S3 SDK COMPATABILITY" on
https://ceph.io/contribute/google-summer-of-code/
I was hoping to instrument coverage of Boto, per each test within
s3-tests, and that would enable us to improve tags on each test within
s3-tests with the S3 API calls that it does consume, and see if that
matches what we want to have tested. This would then also see what parts
of the S3 API are NOT presently tested by s3-tests. The recent bugs of
s3website error handling w/ conditional HTTP requests clearly show there
are cases there not properly covered in tests today.
--
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
E-Mail : robbat2(a)gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136
15 Mar
15 Mar
12:22 p.m.
Hi Robin.
I’m the Ceph Upstream Docs guy.
More detailed Teuthology documentation is likely to be developed later this
year.
If you have specific requests, let me know and I’ll try to shape the docs
to accommodate them.
Zac Dover
Upstream Docs
Ceph
On Sun, 14 Mar 2021 at 5:55 am, Robin H. Johnson <robbat2(a)gentoo.org> wrote:
On Fri, Mar 12, 2021 at 04:07:22PM -0500, Gavin Chen
wrote:
I’ve been able to find docs relating to running
the tests on a
deployed cluster
(
https://docs.ceph.com/en/latest/dev/developer_guide/running-tests-locally/
<
https://docs.ceph.com/en/latest/dev/developer_guide/running-tests-locally/
).
But no documentation beyond that about the tests themselves. Just from
looking at the code there’s organization by feature using the
attribution feature in nose, so I’m wondering if there’s existing
knowledge that’s recorded but that I’m missing. Something that breaks
down coverage of each attribute, or a list of all the tests being run
along with a quick description.
Hi!
I answered you on IRC, but I don't know if you saw it.
There isn't a good list of what's covered, and that's one of the goals I
have in mind when I included a project in the list of potential GSOC
projects for this year:
"RGW: S3 SDK COMPATABILITY" on
https://ceph.io/contribute/google-summer-of-code/
I was hoping to instrument coverage of Boto, per each test within
s3-tests, and that would enable us to improve tags on each test within
s3-tests with the S3 API calls that it does consume, and see if that
matches what we want to have tested. This would then also see what parts
of the S3 API are NOT presently tested by s3-tests. The recent bugs of
s3website error handling w/ conditional HTTP requests clearly show there
are cases there not properly covered in tests today.
--
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
E-Mail : robbat2(a)gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136
_______________________________________________
Dev mailing list -- dev(a)ceph.io
To unsubscribe send an email to dev-leave(a)ceph.io
1:34 p.m.
On Mon, Mar 15, 2021 at 03:22:14PM +1000, John Zachary Dover wrote:
Hi Robin.
I’m the Ceph Upstream Docs guy.
We've chatted ;-)
More detailed Teuthology documentation is likely to be
developed later this
year.
If you have specific requests, let me know and I’ll try to shape the docs
to accommodate them.
I feel the following tries to hammer Gavin's ask into
specific requests.
1. Document individual s3-tests suites/tests outside of the code somehow?
2. Correlate individual s3-tests with what parts of the S3 API they touch.
3. Better cover how Teuthology pieces kick off parts of s3-tests or use s3-tests
as a dependency of a further test!
qa/tasks/rgw_logsocket.py
qa/tasks/s3readwrite.py
qa/tasks/s3roundtrip.py
qa/tasks/s3tests.py
qa/workunits/rgw/run-s3tests.sh
#2 in that list ties to the GSOC project I intend to mentor if we get a good
student fit.
--
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
E-Mail : robbat2(a)gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136
8:54 p.m.
Hi Robin and John,
Thanks for responding on IRC Robin, sorry I seemed to have missed your response!
It’s good to hear there are already some other eyes on this and a plan to begin addressing
it. I second your list of requests as someone currently going through the tests and trying
to catalog them from a high level view. An external source of info documenting individual
s3 tests would make the tests much more understandable over the current method of having
to read through over all the code itself.
Our current plan is to take a catalog of the existing tests to see if it covers some of
the tests we’ll need to verify our ceph implementation. This will likely involve at first
cataloging the existing tests in some way.
Excited to hear about the GSOC project idea! Look forward to it hopefully working out. If
there’s anything I can help to contribute towards I’d be happy to help.
John, as it stands I would be glad to help contribute towards a summative catalog of the
tests if such a place exists in the documentation. Just let me know if there’s a process
in place to add in what we come up with.
Thanks,
- Gavin
(831) 515-6028
gchen(a)linode.com
On Mar 15, 2021, at 2:34 AM, Robin H. Johnson
<robbat2(a)gentoo.org> wrote:
On Mon, Mar 15, 2021 at 03:22:14PM +1000, John Zachary Dover wrote:
Hi Robin.
I’m the Ceph Upstream Docs guy.
We've chatted ;-)
More detailed Teuthology documentation is likely
to be developed later this
year.
If you have specific requests, let me know and I’ll try to shape the docs
to accommodate them.
I feel the following tries to hammer Gavin's ask into
specific requests.
1. Document individual s3-tests suites/tests outside of the code somehow?
2. Correlate individual s3-tests with what parts of the S3 API they touch.
3. Better cover how Teuthology pieces kick off parts of s3-tests or use s3-tests
as a dependency of a further test!
qa/tasks/rgw_logsocket.py
qa/tasks/s3readwrite.py
qa/tasks/s3roundtrip.py
qa/tasks/s3tests.py
qa/workunits/rgw/run-s3tests.sh
#2 in that list ties to the GSOC project I intend to mentor if we get a good
student fit.
--
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
E-Mail : robbat2(a)gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136
_______________________________________________
Dev mailing list -- dev(a)ceph.io
To unsubscribe send an email to dev-leave(a)ceph.io
1145
days inactive
1148
days old
4 comments
3 participants
participants (3)
-
Gavin Chen -
John Zachary Dover -
Robin H. Johnson