Hi, From a healty nautilus cluster version 14.2.9 on a CentOS7 i try to follow the upgrade procedure to the containerized octopus setup with cephadm. * https://docs.ceph.com/docs/octopus/cephadm/adoption/ Everything step went fine until i wanted to adopt the osds, then i get a error. Does anybody have a idea, what my problem could be? ```````````````````` ~# cephadm adopt --style legacy --name osd.0 INFO:cephadm:Pulling latest docker.io/ceph/ceph:v15 container... INFO:cephadm:objectstore_type is bluestore INFO:cephadm:Stopping old systemd unit ceph-osd@0... INFO:cephadm:Disabling old systemd unit ceph-osd@0... INFO:cephadm:Moving data... Traceback (most recent call last): File "/usr/sbin/cephadm", line 4282, in <module> r = args.func() File "/usr/sbin/cephadm", line 972, in _default_image return func() File "/usr/sbin/cephadm", line 2916, in command_adopt command_adopt_ceph(daemon_type, daemon_id, fsid); File "/usr/sbin/cephadm", line 2979, in command_adopt_ceph os.rmdir(data_dir_src) OSError: [Errno 39] Directory not empty: '//var/lib/ceph/osd/ceph-0' ```````````````````` Yours, bbk
Hi again, it is not the first time, just after i posted my question i find a solution :-) What i needed to do was stopping the osd first: systemctl stop ceph-osd@0 Then unmounting the tempfs: umount /var/lib/ceph/osd/ceph-0 So now the script is able to remove the folder, and adopt the osd: ```````````````````` ~# cephadm adopt --style legacy --name osd.0 INFO:cephadm:Pulling latest docker.io/ceph/ceph:v15 container... INFO:cephadm:objectstore_type is bluestore INFO:cephadm:Disabling old systemd unit ceph-osd@0... INFO:cephadm:Moving data... INFO:cephadm:Chowning content... INFO:cephadm:Chowning /var/lib/ceph/6d0ecf22-9155-4684-971a-2f6cde8628c8/osd.0/block... INFO:cephadm:Disabling host unit ceph-volume@ lvm unit... INFO:cephadm:Moving logs... INFO:cephadm:Creating new units... ```````````````````` Yours, bbk
This is a comment for documentation purposes. Note to slightly-future Zac: Add to https://docs.ceph.com/docs/octopus/cephadm/adoption/ a step directing the reader to stop the osd and unmount the tempfs as described in this email thread. CEPH DOCUMENTATION INITIATIVE On Thu, Apr 16, 2020 at 5:47 PM <bbk@nocloud.ch> wrote:
Hi again,
it is not the first time, just after i posted my question i find a solution :-)
What i needed to do was stopping the osd first:
systemctl stop ceph-osd@0
Then unmounting the tempfs:
umount /var/lib/ceph/osd/ceph-0
So now the script is able to remove the folder, and adopt the osd:
```````````````````` ~# cephadm adopt --style legacy --name osd.0 INFO:cephadm:Pulling latest docker.io/ceph/ceph:v15 container... INFO:cephadm:objectstore_type is bluestore INFO:cephadm:Disabling old systemd unit ceph-osd@0... INFO:cephadm:Moving data... INFO:cephadm:Chowning content... INFO:cephadm:Chowning /var/lib/ceph/6d0ecf22-9155-4684-971a-2f6cde8628c8/osd.0/block... INFO:cephadm:Disabling host unit ceph-volume@ lvm unit... INFO:cephadm:Moving logs... INFO:cephadm:Creating new units... ````````````````````
Yours, bbk _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
As i progressed with the migration i found out, that my problem is more of a rare case. On my 3 nodes, where i had the problem. I did once move the /var/lib/ceph to a other partition, and symlinked it back. The kernel however is mounting the tempfs at the real path (/whatever/lib/ceph is mounted). I think because of that the cephadm script couldn't unmount correctly. On the 2 other nodes, where i didn't hack around, i had no issue. But for people having similar problems... after having it migrated, the new service for the osd need to be started manually: systemctl start ceph-$CLUSTERID@osd.$ID Yours, bbk
Hi, i habe a similar issue. After migration to Cephadm, the osd services have to be started manually after every cluster reboot. Marco
Am 16.04.2020 um 15:11 schrieb bbk@nocloud.ch:
As i progressed with the migration i found out, that my problem is more of a rare case.
On my 3 nodes, where i had the problem. I did once move the /var/lib/ceph to a other partition, and symlinked it back. The kernel however is mounting the tempfs at the real path (/whatever/lib/ceph is mounted). I think because of that the cephadm script couldn't unmount correctly.
On the 2 other nodes, where i didn't hack around, i had no issue.
But for people having similar problems... after having it migrated, the new service for the osd need to be started manually:
systemctl start ceph-$CLUSTERID@osd.$ID
Yours, bbk _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi, sorry i didn't wrote very clear, what i ment was... In the workflow of * systemctl stop ceph-osd@$ID * umount /var/lib/ceph/osd/ceph-$ID * cephadm adopt --style legacy --name osd.$ID You also need to ```systemctl start ceph-$CLUSTERID@osd.$ID``` After a reboot, my osds are fine and up. I don't think this is related. Yours, bbk
participants (3)
-
bbk＠nocloud.ch
-
John Zachary Dover
-
Marco Savoca