[ceph-users] cephfs vs rbd vs rgw

This may be too broad of a topic, or opening a can of worms, but we are running a CEPH environment and I was wondering if there's any guidance about this question: Given that some group would like to store 50-100 TBs of data on CEPH and use it from a linux environment, are there any advantages or disadvantages in terms of performance/ease of use/learning curve to using cephfs vs using a block device thru rbd vs using object storage thru rgw? Here are my general thoughts: cephfs - Until recently, you were not allowed to have multiple filesystems. Not sure about performance. rbd - Can only be mounted on one system at a time, but I guess that filesystem could then be served using NFS. rgw - A different usage model from regular linux file/directory structure. Are there advantages to forcing people to use this interface? I'm tempted to set up 3 separate areas and try them and compare the results, but I'm wondering if somebody has done some similar experiment in the past. Thanks for any help you can provide! Jorge