[ceph-users] Re: FileStore OSD, journal direct symlinked, permission troubles.

OK, trying to identify how OSDs get initialized. If i understood well: 0) systemd unit for every OSD get created following a template: /lib/systemd/system/ceph-osd@.service 1) every unit call a 'prestart' script: ExecStartPre=/usr/lib/ceph/ceph-osd-prestart.sh --cluster ${CLUSTER} --id %i 2) The prestart script, run udev: udevadm settle --timeout=5 that simply force the processing of udev queue, only to be sure there's some 'unhandled' device in the queue. 3) udev (rules in /lib/udev/rules.d/95-ceph-osd.rules), looking for GPT ID_PART_ENTRY_TYPE do two things: a) ceph-disk --log-stdout -v trigger /dev/$name (that AFAIK trigger a disk mount, for filestore) b) chown ceph:ceph /dev/$name; chmod 660 /dev/$name So, seems to me that a decent method to solve/circumvent my trouble is to: i) write a 'static' udev rule that chown ceph:ceph the partition. Very dirty. ii) modify the systemd unit and add an ExecStartPost= script that chown the partition. Dirty but probably effective. iii) modify /usr/lib/ceph/ceph-osd-prestart.sh to add the condition, something like (untested): if [ -L "$journal" -a -e "$journal" ]; then dev_journal=`readlink -f $journal` owner=`stat -c %U $dev_journal` if [ $owner != 'ceph' ]; then echo "ceph-osd(${cluster:-ceph}-$id): journal probably manually symlinked, fixing permission." 1>&2 chown ceph: $dev_journal fi fi I'm not a ceph expert, but solution iii) seems decent for me, with a little overhead (a readlinkk and a stat for every osd start). But still i don't understood why, if i have: root@capitanmarvel:~# LANG=C id ceph uid=64045(ceph) gid=64045(ceph) groups=64045(ceph),6(disk) and: brw-rw---- 1 root disk 8, 6 ago 28 14:38 /dev/sda6 (so, journal partition group-owned by 'disk' and 'ceph' user in group 'disk'), still i have permission access. The ceph-osd process reset group ownership on runtime? Thanks. -- dott. Marco Gaiarin GNUPG Key ID: 240A3D66 Associazione ``La Nostra Famiglia'' http://www.lanostrafamiglia.it/ Polo FVG - Via della Bontà, 7 - 33078 - San Vito al Tagliamento (PN) marco.gaiarin(at)lanostrafamiglia.it t +39-0434-842711 f +39-0434-842797 Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA! http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000 (cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA)