Thanks Eli
I found the problem - the old logs were owned by a user ceph (uid 1001) and
since the move to cephadm the daemons run as uid / gid 167. Thus they
didn't have permission to write to the existing logs / log directories.
Chown-ing the log directories to be owned by 167:167 and restarting the
daemons solved the problem.
Peter
On Thu, 9 Feb 2023 at 16:27, Tarrago, Eli (RIS-BCT) <
Eli.Tarrago(a)lexisnexisrisk.com> wrote:
Please include your promtai; logs, loki logs, promtail
configuration, and
your loki configuration.
*From: *Peter van Heusden <pvh(a)sanbi.ac.za>
*Date: *Wednesday, February 8, 2023 at 7:45 AM
*To: *ceph-users(a)ceph.io <ceph-users(a)ceph.io>
*Subject: *[ceph-users] OSD logs missing from Centralised Logging
You don't often get email from pvh(a)sanbi.ac.za. Learn why this is
important <https://aka.ms/LearnAboutSenderIdentification>
**** External email: use caution ****
Hi there
I am running Ceph version 17.2.5 and have deployed centralised logging as
per this guide:
https://ceph.io/en/news/blog/2022/centralized_logging/
<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fceph.io%2Fen%2Fnews%2Fblog%2F2022%2Fcentralized_logging%2F&data=05%7C01%7Celi.tarrago%40lexisnexisrisk.com%7Cc8bc3057ff78495fe07b08db09d25002%7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C638114571140998534%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=QUyBFwEmRQqSd5gYsxytVy6qEHE9GDmCkJCUre7bhxs%3D&reserved=0>
The logs from the OSDs are not, however, showing up in the Grafana
dashboard, as per this screenshot:
The Promtail daemons are running on each node including the OSDs. The Loki
server and Grafana are running on one of our monitor nodes.
Thanks for any clarifications you can provide.
Peter
------------------------------
The information contained in this e-mail message is intended only for the
personal and confidential use of the recipient(s) named above. This message
may be an attorney-client communication and/or work product and as such is
privileged and confidential. If the reader of this message is not the
intended recipient or an agent responsible for delivering it to the
intended recipient, you are hereby notified that you have received this
document in error and that any review, dissemination, distribution, or
copying of this message is strictly prohibited. If you have received this
communication in error, please notify us immediately by e-mail, and delete
the original message.