On 5/24/23 14:03, Emmanuel Jaep wrote:
Hi,
I inherited a ceph fs cluster. Even if I have years of experience in systems management, I fail to grasp the complete logic of it fully. From what I found on the web, the documentation is either too "high level" or too detailed.
Is this a setup based on bare metal nodes with Ceph packages installed? Or is it running with Cephadm or Rook?
Do you know any good resources to get fully acquainted with ceph (specifically, ceph fs)? Or any live training I could attend?
It depends on what level of understanding you are looking for. It definitely is useful to have a high level of understanding to begin with. Sage Weil has made some videos on Youtube. Important to understand is how OSDs, PGs and CRUSH work, basically the RADOS part, as that's the foundation of Ceph. All the other protocols are an abstraction built on top of this. For cluster maintenance (updates, upgrades, recovery, etc.) this is essential to understand. Then there are many different ways on how to run Ceph Every component has it's own intrinsic "details". For example an OSD can use BlueStore / BlueFS, and RocksDB. Each of these implementations have their own settings (and knobs you *can* but normally shouldn't change). Sometimes it's essential to understand their behavior. For CephFS, especially when running with a lot of smaller files, it makes sense to know about minimum allocation size (bluestore_min_alloc_size) of an OSD (space usage). And that RocksDB can get "degraded" (sub-optimal performance) when adding / removing PGs that hold (a lot) of OMAP (key/value) data, (when (re-)balancing for example) and might need compacting once in a while. There are many of these kind of things you don't learn in a training, but learn by experience / mailing list posts, reading tracker tickets, etc. but are essential to keep your cluster healthy and performant. The best way to get yourself familiar with Ceph is to set up a test cluster and perform all kind of operations. Try to break and fix it. This way you will learn a lot of important lessons you can later use in production. Welcome to Ceph, welcome to this list, and we are here to help you. Gr. Stefan