I fixed it by starting again deleting everything and using the "--skip-mon-network”
option to cephadm bootstrap, I think the config was not finished before.
The logging is very verbose by default. I have reduced most of but can’t reduce
cluster 2020-12-23T12:26:48.142993+0000 mgr.host1.xsqlhs (mgr.34116) 238 : cluster [DBG]
pgmap v234: 1 pgs: 1
This logs every 2 seconds
I tried setting "mon_cluster_log_file_level = info” which I found on the web, but
this has no effect. Maybe because its logging to stderr/dout because its running in a
container?
On 23 Dec 2020, at 09:16, Duncan Bellamy
<a.16bit.sysop(a)me.com> wrote:
Hi,
I am trying to set up a new cluster with cephadm using a docker backend.
The initial boot strap did not finish cleanly and it errored out waiting for the mon-ip,
I used the command:
cephadm bootstrap --mon-ip 192.168.0.1
With 192.168.0.1 being the ip address of this first host.
I tried the command again but it failed as the new ceph node was actually running so it
could not bind to the ports.
After a bit of searching I was able to use "sudo cephadm shell —“ commands to change
the username and password for the dashboard and login to it.
I then used cephadm to add a new host with "sudo cephadm shell — ceph orch host add
host2”
Now in the inventory of the dashboard, and "ceph orch device ls” only devices on
host2 are listed not host1.
In the Cluster/Hosts section of the dashboard host1 has its root volume drive listed in
devices, and host2 has the root volume drive and drive for the OSD listed.
I successfully added an OSD with a drive on host2, trying the same command adjusted for
host1 I get the following in the log:
Dec 23 08:55:47 localhost systemd[1]:
var-lib-docker-overlay2-91e9dffa86c333353dd6b445021c852d7ce8da6237d0d4d95909d68ef3d4fe23\x2dinit-merged.mount:
Succeeded.
Dec 23 08:55:47 localhost systemd[24638]:
var-lib-docker-overlay2-91e9dffa86c333353dd6b445021c852d7ce8da6237d0d4d95909d68ef3d4fe23\x2dinit-merged.mount:
Succeeded.
Dec 23 08:55:47 localhost containerd[1470]:
time="2020-12-23T08:55:47.369773808Z" level=info msg="shim containerd-shim
started"
address=/containerd-shim/80f876072532ebebdfef341a5c793654e27766f2d1708991a6f25599b24b6557.sock
debug=false pid=28597
Dec 23 08:55:47 localhost bash[8745]: debug 2020-12-23T08:55:47.517+0000 ffff73d7a200 1
mon.host1(a)0(leader).osd e12 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 71303168
full_alloc: 71303168 kv_alloc: 876609536
Dec 23 08:55:47 localhost containerd[1470]:
time="2020-12-23T08:55:47.621748606Z" level=info msg="shim reaped"
id=69a786e4a61605c1e6eca5a6e0e5ed0900635a214b0f1c96a4f26ea7911a12ff
Dec 23 08:55:47 localhost dockerd[2930]: time="2020-12-23T08:55:47.631479207Z"
level=info msg="ignoring event" module=libcontainerd namespace=moby
topic=/tasks/delete type="*events.TaskDelete"
Dec 23 08:55:47 localhost systemd[24638]:
var-lib-docker-overlay2-91e9dffa86c333353dd6b445021c852d7ce8da6237d0d4d95909d68ef3d4fe23-merged.mount:
Succeeded.
Dec 23 08:55:47 localhost systemd[1]:
var-lib-docker-overlay2-91e9dffa86c333353dd6b445021c852d7ce8da6237d0d4d95909d68ef3d4fe23-merged.mount:
Succeeded.
Dec 23 08:55:47 localhost systemd[24638]:
var-lib-docker-overlay2-64bb135bc0cdab187566992dc9870068dee1430062e1a2b484381c19e03da895\x2dinit-merged.mount:
Succeeded.
Dec 23 08:55:47 localhost systemd[1]:
var-lib-docker-overlay2-64bb135bc0cdab187566992dc9870068dee1430062e1a2b484381c19e03da895\x2dinit-merged.mount:
Succeeded.
Dec 23 08:55:47 localhost containerd[1470]:
time="2020-12-23T08:55:47.972437378Z" level=info msg="shim containerd-shim
started"
address=/containerd-shim/4a61d63e1f46722ffa7a950c31145d167c5c69087d003e5928a6aa3a4831f031.sock
debug=false pid=28659
Dec 23 08:55:48 localhost bash[8745]: cluster 2020-12-23T08:55:46.892633+0000
mgr.host1.kkssvi (mgr.24098) 24278 : cluster [DBG] pgmap v24212: 1 pgs: 1
undersized+peered; 0 B data, 112 KiB used, 931 GiB / 932 GiB avail
Dec 23 08:55:48 localhost bash[8756]: debug 2020-12-23T08:55:48.889+0000 ffff93573700 0
log_channel(cluster) log [DBG] : pgmap v24213: 1 pgs: 1 undersized+peered; 0 B data, 112
KiB used, 931 GiB / 932 GiB avail
Dec 23 08:55:49 localhost bash[8756]: debug 2020-12-23T08:55:49.085+0000 ffff9056f700 0
log_channel(audit) log [DBG] : from='client.24206 -' entity='client.admin'
cmd=[{"prefix": "orch daemon add osd", "svc_arg":
"host1:/dev/nvme0n1", "target": ["mon-mgr", ""]}]:
dispatch
Dec 23 08:55:49 localhost bash[8745]: debug 2020-12-23T08:55:49.085+0000 ffff71575200 0
mon.host1@0(leader) e2 handle_command mon_command({"prefix": "osd
tree", "states": ["destroyed"], "format":
"json"} v 0) v1
Dec 23 08:55:49 localhost bash[8745]: debug 2020-12-23T08:55:49.085+0000 ffff71575200 0
log_channel(audit) log [DBG] : from='mgr.24098 192.168.0.1:0/2486989775'
entity='mgr.host1.kkssvi' cmd=[{"prefix": "osd tree",
"states": ["destroyed"], "format": "json"}]:
dispatch
Dec 23 08:55:49 localhost bash[8756]: debug 2020-12-23T08:55:49.089+0000 ffff8ed6d700 0
log_channel(cephadm) log [INF] : Found osd claims -> {}
Dec 23 08:55:49 localhost bash[8756]: debug 2020-12-23T08:55:49.089+0000 ffff8ed6d700 0
log_channel(cephadm) log [INF] : Found osd claims for drivegroup None -> {}
Dec 23 08:55:49 localhost containerd[1470]:
time="2020-12-23T08:55:49.331868093Z" level=info msg="shim reaped"
id=780a38dd49fce4a823c4c3d834abdd1cc17bbe0c0aa4f2dd7caeddf8dce1708e
Dec 23 08:55:49 localhost dockerd[2930]: time="2020-12-23T08:55:49.341765820Z"
level=info msg="ignoring event" module=libcontainerd namespace=moby
topic=/tasks/delete type="*events.TaskDelete"
Dec 23 08:55:49 localhost systemd[24638]:
var-lib-docker-overlay2-64bb135bc0cdab187566992dc9870068dee1430062e1a2b484381c19e03da895-merged.mount:
Succeeded.
Dec 23 08:55:49 localhost systemd[1]:
var-lib-docker-overlay2-64bb135bc0cdab187566992dc9870068dee1430062e1a2b484381c19e03da895-merged.mount:
Succeeded.
Dec 23 08:55:49 localhost bash[8745]: audit 2020-12-23T08:55:49.091014+0000 mon.host1
(mon.0) 1093 : audit [DBG] from='mgr.24098 192.168.0.1:0/2486989775'
entity='mgr.host1.kkssvi' cmd=[{"prefix": "osd tree",
"states": ["destroyed"], "format": "json"}]:
dispatch
Dec 23 08:55:50 localhost bash[8745]: cluster 2020-12-23T08:55:48.893433+0000
mgr.host1.kkssvi (mgr.24098) 24279 : cluster [DBG] pgmap v24213: 1 pgs: 1
undersized+peered; 0 B data, 112 KiB used, 931 GiB / 932 GiB avail
Dec 23 08:55:50 localhost bash[8745]: audit 2020-12-23T08:55:49.087597+0000
mgr.host1.kkssvi (mgr.24098) 24280 : audit [DBG] from='client.24206 -'
entity='client.admin' cmd=[{"prefix": "orch daemon add osd",
"svc_arg": "host1:/dev/nvme0n1", "target":
["mon-mgr", ""]}]: dispatch
Dec 23 08:55:50 localhost bash[8745]: cephadm 2020-12-23T08:55:49.093552+0000
mgr.host1.kkssvi (mgr.24098) 24281 : cephadm [INF] Found osd claims -> {}
Dec 23 08:55:50 localhost bash[8745]: cephadm 2020-12-23T08:55:49.093933+0000
mgr.host1.kkssvi (mgr.24098) 24282 : cephadm [INF] Found osd claims for drivegroup None
-> {}
The other problem is logging is set to debug for both hosts, I tried "sudo cephadm
shell -- ceph daemon mon.host1 config set mon_cluster_log_file_level info” which reports
success but logging remains at debug level.
If I try the same command with mon.host2 I get
INFO:cephadm:Inferring fsid ae111111-1111-1111-1111-f1111a11111a
INFO:cephadm:Inferring config
/var/lib/ceph/ae147088-4486-11eb-9044-f1337a55707a/mon.host1/config
INFO:cephadm:Using recent ceph image ceph/ceph:v15
admin_socket: exception getting command descriptions: [Errno 2] No such file or
directory
Which looks like it is trying to use the config for host1 on host2?
Thanks,
Duncan
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io