For the benefit of our new folks and for posterity:
Many of our QA tests for CephFS are located in qa/tasks/cephfs/*.
These get run in teuthology with various cluster configurations. What
everyone will need to be able to do is develop these tests locally
without waiting for teuthology so you can rapidly find errors in your
test cases and development builds.
To do this, you need to use the qa/tasks/vstart_runner.py script. This
allows you to use a vstart cluster to execute your tests by providing
the necessary frameworks the tests expect.
On a development box*, build ceph. If you're just testing CephFS, you
can usually get away with a smaller build without rbd/rgw:
/do_cmake.sh -DWITH_PYTHON3:BOOL=ON -DWITH_BABELTRACE=OFF
-DWITH_MANPAGE=OFF -DWITH_RBD=OFF -DWITH_RADOSGW=OFF && time (cd build
&& make -j24 CMAKE_BUILD_TYPE=Debug -k)
Next, build teuthology:
git clone https://github.com/ceph/teuthology.git && cd teuthology &&
virtualenv ./venv && source venv/bin/activate && pip install --upgrade
pip && pip install -r requirements.txt && python setup.py develop
Next, start a vstart cluster:
cd ceph/build && env MDS=3 ../src/vstart.sh -d -b -l -n --without-dashboard
Finally, run vstart_runner:
python2 ../qa/tasks/vstart_runner.py --interactive
tasks.cephfs.test_snapshots.TestSnapshots
^ That's an example test. The format is based on the directory
structure of qa/tasks/cephfs/test_snapshots. The final part is the
class we're testing, TestSnapshots. This invocation of
vstart_runner.py will run every test in TestSnapshots, methods
beginning with "test_". If you want to run a specific test, then we
could do:
python2 ../qa/tasks/vstart_runner.py --interactive
tasks.cephfs.test_snapshots.TestSnapshots.test_snapclient_cache
Please give the above a try sometime soon so you know how to do it and
we can resolve any problems. This is an important skill to have for
developing CephFS.
* Hopefully you're using one of the beefy development boxes that make
compiling Ceph fast. I recommend one of the senta boxes like
senta03.front.sepia.ceph.com.
--
Patrick Donnelly
Hey Jason,
We are working towards bringing in `top` like functionality to CephFS
for displaying various client (and MDS) metrics. Since RBD has
something similar in the form of `perf image io*` via rbd cli, we
would like to understand some finer details regarding its
implementation and detail how CephFS is going forward for `fs top`
functionality.
IIUC, the `rbd_support` manager module requests object perf counters
from the OSD, thereby extracting image names from the returned list of
hot objects. I'm guessing it's done this way since there is no RBD
related active daemon to forward metrics data to the manager? OTOH,
`rbd-mirror` does make use of
`MgrClient::service_daemon_update_status()` to forward mirror daemon
status, which seems to be ok for anything that's not too bulky.
For forwarding CephFS related metrics to Ceph Manager, sticking in
blobs of metrics data in daemon status doesn't look clean (although it
might work). Therefore, for CephFS, `MMgrReport` message type is
expanded to include metrics data as part of its report update process,
as per:
https://github.com/ceph/ceph/pull/26004/commits/a75570c0e73ef67bbca8f73a974…
... and a callback function is provided to `MgrClient` (invoked
periodically) to fill in appropriate metrics data in its report. This
works well and is similar to how OSD updates PG stats to Ceph Manager.
I guess changes of this nature was not required by RBD as it can get
the required data by querying the OSDs (and were other approaches
considered regarding the same)?
Thanks,
-Venky
Hi Zheng!
I'm going over the cap handling in the kernel client, and saw this
that I didn't quite understand:
https://github.com/ceph/ceph-client/blob/daf5cc27eed99afdea8d96e71b89ba41f5…
In the event that ceph_try_drop_cap_snap returns true, we'll queue up
an extra iput for the inode. Where is that reference acquired?
Thanks,
--
Jeff Layton <jlayton(a)redhat.com>
