Hello Marco,
On Thu, Aug 29, 2019 at 12:55:56PM +0200, Marco Gaiarin wrote:
I've just finished a double upgrade on my ceph (PVE-based) from hammer
to jewel and from jewel to luminous.
All went well, apart that... OSD does not restart automatically,
because permission troubles on the journal:
Aug 28 14:41:55 capitanmarvel ceph-osd[6645]: starting osd.2 at - osd_data
/var/lib/ceph/osd/ceph-2 /var/lib/ceph/osd/ceph-2/journal
Aug 28 14:41:55 capitanmarvel ceph-osd[6645]: 2019-08-28 14:41:55.449886 7fa505a43e00 -1
filestore(/var/lib/ceph/osd/ceph-2) mount(1822): failed to open journal
/var/lib/ceph/osd/ceph-2/journal: (13) Permission denied
Aug 28 14:41:55 capitanmarvel ceph-osd[6645]: 2019-08-28 14:41:55.453524 7fa505a43e00 -1
osd.2 0 OSD:init: unable to mount object store
Aug 28 14:41:55 capitanmarvel ceph-osd[6645]: 2019-08-28 14:41:55.453535 7fa505a43e00 -1
#033[0;31m ** ERROR: osd init failed: (13) Permission denied#033[0m
A little fast rewind: when i've setup the cluster i've used some 'old'
servers, using a couple of SSD disks as SO and as journal.
Because servers was old, i was forced to partition the boot disk in
DOS, not GPT mode.
While creating the OSD, i've received some warnings:
WARNING:ceph-disk:Journal /dev/sdaX was not prepared with ceph-disk. Symlinking
directly.
Looking at the cluster now, seems to me that osd init scripts try to
idetify journal based on GPT partition label/info, and clearly fail.
Not that if i do, on servers that hold OSD:
for l in $(readlink -f /var/lib/ceph/osd/ceph-*/journal); do chown ceph: $l; done
OSD start flawlessy.
There's something i can do? Thanks.
Did you go through our upgrade guide(s)?
See the link [0] below, for the
permission changes. They are needed when an upgrade from Hammer to Jewel
is done.
On the wiki you can also find the upgrade guides for PVE 5.x -> 6.x and
Luminous -> Nautilus.
--
Cheers,
Alwin
[0]
https://pve.proxmox.com/wiki/Ceph_Hammer_to_Jewel#Set_permission