21 Mar
2019
21 Mar
'19
9:49 p.m.
For the benefit of our new folks and for posterity:
Many of our QA tests for CephFS are located in qa/tasks/cephfs/*.
These get run in teuthology with various cluster configurations. What
everyone will need to be able to do is develop these tests locally
without waiting for teuthology so you can rapidly find errors in your
test cases and development builds.
To do this, you need to use the qa/tasks/vstart_runner.py script. This
allows you to use a vstart cluster to execute your tests by providing
the necessary frameworks the tests expect.
On a development box*, build ceph. If you're just testing CephFS, you
can usually get away with a smaller build without rbd/rgw:
/do_cmake.sh -DWITH_PYTHON3:BOOL=ON -DWITH_BABELTRACE=OFF
-DWITH_MANPAGE=OFF -DWITH_RBD=OFF -DWITH_RADOSGW=OFF && time (cd build
&& make -j24 CMAKE_BUILD_TYPE=Debug -k)
Next, build teuthology:
git clone https://github.com/ceph/teuthology.git && cd teuthology &&
virtualenv ./venv && source venv/bin/activate && pip install --upgrade
pip && pip install -r requirements.txt && python setup.py develop
Next, start a vstart cluster:
cd ceph/build && env MDS=3 ../src/vstart.sh -d -b -l -n --without-dashboard
Finally, run vstart_runner:
python2 ../qa/tasks/vstart_runner.py --interactive
tasks.cephfs.test_snapshots.TestSnapshots
^ That's an example test. The format is based on the directory
structure of qa/tasks/cephfs/test_snapshots. The final part is the
class we're testing, TestSnapshots. This invocation of
vstart_runner.py will run every test in TestSnapshots, methods
beginning with "test_". If you want to run a specific test, then we
could do:
python2 ../qa/tasks/vstart_runner.py --interactive
tasks.cephfs.test_snapshots.TestSnapshots.test_snapclient_cache
Please give the above a try sometime soon so you know how to do it and
we can resolve any problems. This is an important skill to have for
developing CephFS.
* Hopefully you're using one of the beefy development boxes that make
compiling Ceph fast. I recommend one of the senta boxes like
senta03.front.sepia.ceph.com.
--
Patrick Donnelly
15 Nov
15 Nov
1:15 p.m.
On 21/03/19 11:49 pm, Patrick Donnelly wrote:
For the benefit of our new folks and for posterity:
Many of our QA tests for CephFS are located in qa/tasks/cephfs/*.
These get run in teuthology with various cluster configurations. What
everyone will need to be able to do is develop these tests locally
without waiting for teuthology so you can rapidly find errors in your
test cases and development builds.
To do this, you need to use the qa/tasks/vstart_runner.py script. This
allows you to use a vstart cluster to execute your tests by providing
the necessary frameworks the tests expect.
On a development box*, build ceph. If you're just testing CephFS, you
can usually get away with a smaller build without rbd/rgw:
/do_cmake.sh -DWITH_PYTHON3:BOOL=ON -DWITH_BABELTRACE=OFF
-DWITH_MANPAGE=OFF -DWITH_RBD=OFF -DWITH_RADOSGW=OFF && time (cd build
&& make -j24 CMAKE_BUILD_TYPE=Debug -k)
Next, build teuthology:
git clone https://github.com/ceph/teuthology.git && cd teuthology &&
virtualenv ./venv && source venv/bin/activate && pip install --upgrade
pip && pip install -r requirements.txt && python setup.py develop
pip install -r requirements.txt fails on Fedora31, as it has python3
installed by default. https://pythonclock.org/
"""
ERROR: Command errored out with exit status 1:
command: /home/jcollin/workspace/teuthology/venv/bin/python3 -c
'import sys, setuptools, tokenize; sys.argv[0] =
'"'"'/tmp/pip-install-z2fktdug/functools32/setup.py'"'"';
__file__='"'"'/tmp/pip-install-z2fktdug/functools32/setup.py'"'"';f=getattr(tokenize,
'"'"'open'"'"',
open)(__file__);code=f.read().replace('"'"'\r\n'"'"',
'"'"'\n'"'"');f.close();exec(compile(code,
__file__, '"'"'exec'"'"'))'
egg_info --egg-base /tmp/pip-install-z2fktdug/functools32/pip-egg-info
cwd: /tmp/pip-install-z2fktdug/functools32/
Complete output (1 lines):
This backport is for Python 2.7 only.
""" I did the following on Fedora31 to successfully complete the setup:
$virtualenv -p /usr/bin/python2.7 myvenv $source myvenv/bin/activate $pip
install -r requirements.txt then mysqlclient fails with error:
EnvironmentError: mysql_config not found #dnf install
community-mysql-devel-8.0.17-2.fc31.x86_64 $pip install -r requirements.txt
/usr/bin/ld: cannot find -lzstd
collect2: error: ld returned 1 exit status
error: command 'gcc' failed with exit status 1
#dnf install libzstd-devel
$ pip install -r requirements.txt && python setup.py develop
Finished processing dependencies for teuthology==1.0.0
I need to run a test_ now. Hope this helps others.
Regards,
Jos Collin
1639
days inactive
1878
days old
1 comments
2 participants
participants (2)
-
Jos Collin -
Patrick Donnelly