Migrated to cephadm, rgw logs to file even when rgw_ops_log_rados is true
Hi, we recently migrated to cephadm from ceph-deploy a 18.2.2 ceph cluster (Ubuntu with docker). RGWs are separate vms. We noticed syslog increased a lot due to rgw's access logs sent to it. And because we use to log ops, a huge ops log file on /var/log/ceph/cluster-id/ops-log-ceph-client.rgw.hostname-here.log. While having "rgw_ops_log_rados": "true", oplogs goes to both file and rados pool for logs. If false it doesn't log anything, as expected. How to stop dockered rgws to log to syslog and to a file on disk, but to keep opslog in logs pool? Config is: global basic log_to_journald false global advanced rgw_enable_ops_log false global advanced rgw_ops_log_rados true A few hours later after after enabling it back, after massive cleanup, it does logging ops but only to files. How to get ops logs in rados pool and access log to a file on disk but not on syslog? I have add this to daemon.json to limit access logs to accumulate on /var/log/docker/containers/rand/rand/json.log file: { "log-driver": "local", "log-opts": { "max-size": "512m", "max-file": "3" } } Thank you! Paul
Hi! Currently I have limitted the optput of rgw log to syslog from rsyslog (as suggested by Anthony), limitted docker logs from daemon.json. I still get ops logs written to both logs pool and ops log file (ops-log-ceph-client.rgw.hostname.log). How to stop logging ops log on rgw disk and keep logs on logs pool? Current config: global advanced debug_rgw 0/0 global advanced rgw_enable_ops_log true global advanced rgw_ops_log_rados true Thank you! -- Paul Jurco On Fri, Nov 22, 2024 at 6:11 PM Paul JURCO <paul.jurco@gmail.com> wrote:
Hi, we recently migrated to cephadm from ceph-deploy a 18.2.2 ceph cluster (Ubuntu with docker). RGWs are separate vms. We noticed syslog increased a lot due to rgw's access logs sent to it. And because we use to log ops, a huge ops log file on /var/log/ceph/cluster-id/ops-log-ceph-client.rgw.hostname-here.log.
While having "rgw_ops_log_rados": "true", oplogs goes to both file and rados pool for logs. If false it doesn't log anything, as expected. How to stop dockered rgws to log to syslog and to a file on disk, but to keep opslog in logs pool?
Config is: global basic log_to_journald false global advanced rgw_enable_ops_log false global advanced rgw_ops_log_rados true
A few hours later after after enabling it back, after massive cleanup, it does logging ops but only to files. How to get ops logs in rados pool and access log to a file on disk but not on syslog? I have add this to daemon.json to limit access logs to accumulate on /var/log/docker/containers/rand/rand/json.log file:
{ "log-driver": "local", "log-opts": { "max-size": "512m", "max-file": "3" } }
Thank you! Paul
I haven't played with rgw_ops yet, but have you looked at the various syslog config options? ceph config ls | grep syslog log_to_syslog err_to_syslog clog_to_syslog clog_to_syslog_level clog_to_syslog_facility mon_cluster_log_to_syslog mon_cluster_log_to_syslog_level mon_cluster_log_to_syslog_facility Maybe one of them is what you're looking for. Zitat von Paul JURCO <paul.jurco@gmail.com>:
Hi! Currently I have limitted the optput of rgw log to syslog from rsyslog (as suggested by Anthony), limitted docker logs from daemon.json. I still get ops logs written to both logs pool and ops log file (ops-log-ceph-client.rgw.hostname.log).
How to stop logging ops log on rgw disk and keep logs on logs pool? Current config: global advanced debug_rgw 0/0 global advanced rgw_enable_ops_log true global advanced rgw_ops_log_rados true
Thank you!
-- Paul Jurco
On Fri, Nov 22, 2024 at 6:11 PM Paul JURCO <paul.jurco@gmail.com> wrote:
Hi, we recently migrated to cephadm from ceph-deploy a 18.2.2 ceph cluster (Ubuntu with docker). RGWs are separate vms. We noticed syslog increased a lot due to rgw's access logs sent to it. And because we use to log ops, a huge ops log file on /var/log/ceph/cluster-id/ops-log-ceph-client.rgw.hostname-here.log.
While having "rgw_ops_log_rados": "true", oplogs goes to both file and rados pool for logs. If false it doesn't log anything, as expected. How to stop dockered rgws to log to syslog and to a file on disk, but to keep opslog in logs pool?
Config is: global basic log_to_journald false global advanced rgw_enable_ops_log false global advanced rgw_ops_log_rados true
A few hours later after after enabling it back, after massive cleanup, it does logging ops but only to files. How to get ops logs in rados pool and access log to a file on disk but not on syslog? I have add this to daemon.json to limit access logs to accumulate on /var/log/docker/containers/rand/rand/json.log file:
{ "log-driver": "local", "log-opts": { "max-size": "512m", "max-file": "3" } }
Thank you! Paul
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Hi Eugen, Yes i have played arround with some of them, with the most obvious ones. They are all false by default: :~# ceph-conf -D | grep syslog clog_to_syslog = false clog_to_syslog_facility = default=daemon audit=local0 clog_to_syslog_level = info err_to_syslog = false log_to_syslog = false mon_cluster_log_to_syslog = default=false mon_cluster_log_to_syslog_facility = daemon mon_cluster_log_to_syslog_level = info Loggically, if you turn off ops logs and it applies instantly I assume these configs from above are respected as intended. Still, I have turned on and off log_to_syslog to no avail. Interesting is the ops log file on disk is opened by the radosgw process in the container. So, inside a container, radosgw services ignores the rgw_ops_log_rados when true. Other configs like rgw_max_concurrent_requests and rgw_enable_ops_log are working as expected. -- Paul On Thu, Nov 28, 2024 at 10:30 AM Eugen Block <eblock@nde.ag> wrote:
I haven't played with rgw_ops yet, but have you looked at the various syslog config options?
ceph config ls | grep syslog log_to_syslog err_to_syslog clog_to_syslog clog_to_syslog_level clog_to_syslog_facility mon_cluster_log_to_syslog mon_cluster_log_to_syslog_level mon_cluster_log_to_syslog_facility
Maybe one of them is what you're looking for.
Zitat von Paul JURCO <paul.jurco@gmail.com>:
Hi! Currently I have limitted the optput of rgw log to syslog from rsyslog (as suggested by Anthony), limitted docker logs from daemon.json. I still get ops logs written to both logs pool and ops log file (ops-log-ceph-client.rgw.hostname.log).
How to stop logging ops log on rgw disk and keep logs on logs pool? Current config: global advanced debug_rgw 0/0 global advanced rgw_enable_ops_log true global advanced rgw_ops_log_rados true
Thank you!
-- Paul Jurco
On Fri, Nov 22, 2024 at 6:11 PM Paul JURCO <paul.jurco@gmail.com> wrote:
Hi, we recently migrated to cephadm from ceph-deploy a 18.2.2 ceph cluster (Ubuntu with docker). RGWs are separate vms. We noticed syslog increased a lot due to rgw's access logs sent to it. And because we use to log ops, a huge ops log file on /var/log/ceph/cluster-id/ops-log-ceph-client.rgw.hostname-here.log.
While having "rgw_ops_log_rados": "true", oplogs goes to both file and rados pool for logs. If false it doesn't log anything, as expected. How to stop dockered rgws to log to syslog and to a file on disk, but to keep opslog in logs pool?
Config is: global basic log_to_journald false global advanced rgw_enable_ops_log false global advanced rgw_ops_log_rados true
A few hours later after after enabling it back, after massive cleanup, it does logging ops but only to files. How to get ops logs in rados pool and access log to a file on disk but not on syslog? I have add this to daemon.json to limit access logs to accumulate on /var/log/docker/containers/rand/rand/json.log file:
{ "log-driver": "local", "log-opts": { "max-size": "512m", "max-file": "3" } }
Thank you! Paul
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (2)
-
Eugen Block
-
Paul JURCO