[ceph-users] Re: cephfs filesystem offline, ceph-mds core dumped... How to recover?

8 May 2024


      On 5/8/24 09:05, Erich Weiler wrote:
...
Actually looking again, I don't see core dumps (though I could have 
sworn I did before).  But looking at my MDS servers, they appear to be 
in a constant cycle of trying to restart, and not making it, then 
trying again, etc.  The logs look like what I sent in my previous 
email, over and over again.
I tried restarting all three of my ceph-mds daemons at the same time, 
and it still doesn't seem to help.
I didn't see any mds related logs. Could you catch more ? If not then 
could you try to enable the debug logs if you are sure the mds daemons 
are in a constant cycle of trying to restart ? Let's see what exactly it 
is trying to do and where is stuck.

Thanks
...
I hope I can recover...
-erich
On 5/7/24 5:18 PM, Erich Weiler wrote:
...
Hi All,
We just noticed that our cephfs filesystem was unreachable and we saw 
this:
# ceph -s
   cluster:
     id:     58bde08a-d7ed-11ee-9098-506b4b4da440
     health: HEALTH_WARN
             1 filesystem is degraded
             insufficient standby MDS daemons available
   services:
     mon: 5 daemons, quorum 
pr-md-01,pr-md-02,pr-store-01,pr-store-02,pr-md-03 (age 5d)
     mgr: pr-md-02.emffhz(active, since 5d), standbys: pr-md-01.jemmdf
     mds: 2/2 daemons up
     osd: 46 osds: 46 up (since 3d), 46 in (since 8w)
   data:
     volumes: 0/1 healthy, 1 recovering
     pools:   4 pools, 1313 pgs
     objects: 309.96M objects, 558 TiB
     usage:   842 TiB used, 287 TiB / 1.1 PiB avail
     pgs:     1293 active+clean
              13   active+clean+scrubbing
              7    active+clean+scrubbing+deep
   io:
     client:   1.5 KiB/s rd, 65 KiB/s wr, 1 op/s rd, 2 op/s wr
[root@pr-md-02 ~]# ceph health detail
HEALTH_WARN 1 filesystem is degraded; insufficient standby MDS 
daemons available
[WRN] FS_DEGRADED: 1 filesystem is degraded
     fs slugfs is degraded
[WRN] MDS_INSUFFICIENT_STANDBY: insufficient standby MDS daemons 
available
     have 0; want 1 more
And filesystem status is:
# ceph fs status
slugfs - 31 clients
======
RANK      STATE               MDS               ACTIVITY DNS    INOS 
   DIRS   CAPS
  0    replay(laggy)  slugfs.pr-md-02.sbblqq 0      0       0      0
  1        active     slugfs.pr-md-01.xdtppo  Reqs:    0 /s 7958   
7959    13.1k   442
        POOL           TYPE     USED  AVAIL
  cephfs_metadata    metadata   144G  3168G
cephfs_md_and_data    data       0   65.4T
    cephfs_data        data     838T   130T
MDS version: ceph version 18.2.1 
(7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)
The ceph-mds container is not running on pr-md-02.  All OSDs are 
online.   I see this in the system logs on that machine (last 1000 
lines), I see some core dumping happening in there:
# tail -1000 /var/log/messages
May  7 17:04:37 pr-md-02 ceph-mon[5348]: rocksdb: 
[db/db_impl/db_impl_files.cc:463] [JOB 12098] Try to delete WAL files 
size 318677, prev total WAL file size 318677, number of live WAL 
files 2.
May  7 17:04:37 pr-md-02 ceph-mon[5348]: rocksdb: 
[file/delete_scheduler.cc:74] Deleted file 
/var/lib/ceph/mon/ceph-pr-md-02/store.db/228432.log immediately, 
rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000
May  7 17:04:37 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:04:37.485736) 
[db/db_impl/db_impl_compaction_flush.cc:3165] [default] Manual 
compaction from level-0 to level-6 from 
'7061786F73003130313337363339' seq:72057594037927935, type:22 .. 
'7061786F73003130313337383931' seq:0, type:0; will stop at (end)
May  7 17:04:37 pr-md-02 ceph-mon[5348]: rocksdb: 
[db/compaction/compaction_job.cc:1995] [default] [JOB 12099] 
Compacting 1@0 + 1@6 files to L6, score -1.00
May  7 17:04:37 pr-md-02 ceph-mon[5348]: rocksdb: 
[db/compaction/compaction_job.cc:2001] [default]: Compaction start 
summary: Base version 12098 Base level 0, inputs: [228436(200KB)], 
[228434(37MB)]
May  7 17:04:37 pr-md-02 ceph-mon[5348]: rocksdb: EVENT_LOG_v1 
{"time_micros": 1715126677485763, "job": 12099, "event": 
"compaction_started", "compaction_reason": "ManualCompaction", 
"files_L0": [228436], "files_L6": [228434], "score": -1, 
"input_data_size": 39197377, "oldest_snapshot_seqno": -1}
May  7 17:04:37 pr-md-02 ceph-mon[5348]: rocksdb: 
[db/compaction/compaction_job.cc:1588] [default] [JOB 12099] 
Generated table #228437: 25153 keys, 38148344 bytes, temperature: 
kUnknown
May  7 17:04:37 pr-md-02 ceph-mon[5348]: rocksdb: EVENT_LOG_v1 
{"time_micros": 1715126677626467, "cf_name": "default", "job": 12099, 
"event": "table_file_creation", "file_number": 228437, "file_size": 
38148344, "file_checksum": "", "file_checksum_func_name": "Unknown", 
"smallest_seqno": 0, "largest_seqno": 0, "table_properties": 
{"data_size": 37996789, "index_size": 87588, "index_partitions": 0, 
"top_level_index_size": 0, "index_key_is_user_key": 1, 
"index_value_is_delta_encoded": 1, "filter_size": 62917, 
"raw_key_size": 697744, "raw_average_key_size": 27, "raw_value_size": 
37553381, "raw_average_value_size": 1492, "num_data_blocks": 3073, 
"num_entries": 25153, "num_filter_entries": 25153, "num_deletions": 
0, "num_merge_operands": 0, "num_range_deletions": 0, 
"format_version": 0, "fixed_key_len": 0, "filter_policy": 
"bloomfilter", "column_family_name": "default", "column_family_id": 
0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", 
"prefix_extractor_name": "nullptr", "property_collectors": "[]", 
"compression": "NoCompression", "compression_options": 
"window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; 
zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; 
use_zstd_dict_trainer=1; ", "creation_time": 1709312729, 
"oldest_key_time": 0, "file_creation_time": 1715126677, 
"slow_compression_estimated_data_size": 0, 
"fast_compression_estimated_data_size": 0, "db_id": 
"9cddaea3-96ea-4679-bc89-ab4fe1d01ddc", "db_session_id": 
"LVB8G4XNY4SWBL1V508Z", "orig_file_number": 228437, 
"seqno_to_time_mapping": "N/A"}}
May  7 17:04:37 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:04:37.626709) [db/compaction/compaction_job.cc:1663] 
[default] [JOB 12099] Compacted 1@0 + 1@6 files to L6 => 38148344 bytes
May  7 17:04:37 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:04:37.629741) [db/compaction/compaction_job.cc:865] 
[default] compacted to: base level 6 level multiplier 10.00 max bytes 
base 268435456 files[0 0 0 0 0 0 1] max score 0.00, MB/sec: 278.5 rd, 
271.0 wr, level 6, files in(1, 1) out(1 +0 blob) MB in(0.2, 37.2 +0.0 
blob) out(36.4 +0.0 blob), read-write-amplify(376.1) 
write-amplify(185.5) OK, records in: 25817, records dropped: 664 
output_compression: NoCompression
May  7 17:04:37 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:04:37.629762) EVENT_LOG_v1 {"time_micros": 
1715126677629753, "job": 12099, "event": "compaction_finished", 
"compaction_time_micros": 140760, "compaction_time_cpu_micros": 
62440, "output_level": 6, "num_output_files": 1, "total_output_size": 
38148344, "num_input_records": 25817, "num_output_records": 25153, 
"num_subcompactions": 1, "output_compression": "NoCompression", 
"num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 
0, "lsm_state": [0, 0, 0, 0, 0, 0, 1]}
May  7 17:04:37 pr-md-02 ceph-mon[5348]: rocksdb: 
[file/delete_scheduler.cc:74] Deleted file 
/var/lib/ceph/mon/ceph-pr-md-02/store.db/228436.sst immediately, 
rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000
May  7 17:04:37 pr-md-02 ceph-mon[5348]: rocksdb: EVENT_LOG_v1 
{"time_micros": 1715126677629887, "job": 12099, "event": 
"table_file_deletion", "file_number": 228436}
May  7 17:04:37 pr-md-02 ceph-mon[5348]: rocksdb: 
[file/delete_scheduler.cc:74] Deleted file 
/var/lib/ceph/mon/ceph-pr-md-02/store.db/228434.sst immediately, 
rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000
May  7 17:04:37 pr-md-02 ceph-mon[5348]: rocksdb: EVENT_LOG_v1 
{"time_micros": 1715126677632998, "job": 12099, "event": 
"table_file_deletion", "file_number": 228434}
May  7 17:04:37 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:04:37.485696) 
[db/db_impl/db_impl_compaction_flush.cc:1903] [default] Manual 
compaction starting
May  7 17:04:37 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:04:37.633030) 
[db/db_impl/db_impl_compaction_flush.cc:1903] [default] Manual 
compaction starting
May  7 17:04:37 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:04:37.633038) 
[db/db_impl/db_impl_compaction_flush.cc:1903] [default] Manual 
compaction starting
May  7 17:04:37 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:04:37.633042) 
[db/db_impl/db_impl_compaction_flush.cc:1903] [default] Manual 
compaction starting
May  7 17:04:37 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:04:37.633045) 
[db/db_impl/db_impl_compaction_flush.cc:1903] [default] Manual 
compaction starting
May  7 17:04:37 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:04:37.633049) 
[db/db_impl/db_impl_compaction_flush.cc:1903] [default] Manual 
compaction starting
May  7 17:04:38 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:04:38 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225202: 1313 pgs: 7 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1291 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s
May  7 17:04:39 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:04:40 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225203: 1313 pgs: 7 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1291 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 766 B/s rd, 33 KiB/s wr, 2 op/s
May  7 17:04:42 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5612 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:04:42 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225204: 1313 pgs: 9 active+clean+scrubbing+deep, 12 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 766 B/s rd, 33 KiB/s wr, 2 op/s
May  7 17:04:44 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225205: 1313 pgs: 9 active+clean+scrubbing+deep, 12 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 766 B/s rd, 33 KiB/s wr, 2 op/s
May  7 17:04:46 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:04:46 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:04:46] "GET /metrics HTTP/1.1" 
200 73533 "" "Prometheus/2.43.0"
May  7 17:04:46 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225206: 1313 pgs: 9 active+clean+scrubbing+deep, 12 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 766 B/s rd, 33 KiB/s wr, 2 op/s
May  7 17:04:47 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5612 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:04:47 pr-md-02 ceph-mon[5348]: from='mgr.274299 ' 
entity='mgr.pr-md-02.emffhz'
May  7 17:04:48 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225207: 1313 pgs: 6 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s
May  7 17:04:50 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225208: 1313 pgs: 6 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 766 B/s rd, 33 KiB/s wr, 2 op/s
May  7 17:04:51 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:04:52 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5612 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:04:52 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225209: 1313 pgs: 7 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1291 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 766 B/s rd, 33 KiB/s wr, 2 op/s
May  7 17:04:54 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:04:54 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225210: 1313 pgs: 7 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1291 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 766 B/s rd, 33 KiB/s wr, 2 op/s
May  7 17:04:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
scanning for idle connections..
May  7 17:04:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
cleaning up connections: []
May  7 17:04:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
scanning for idle connections..
May  7 17:04:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
cleaning up connections: []
May  7 17:04:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
scanning for idle connections..
May  7 17:04:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
cleaning up connections: []
May  7 17:04:56 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:04:56] "GET /metrics HTTP/1.1" 
200 73531 "" "Prometheus/2.43.0"
May  7 17:04:56 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:04:56 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225211: 1313 pgs: 7 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1291 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 766 B/s rd, 33 KiB/s wr, 2 op/s
May  7 17:04:57 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:04:57 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5612 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:04:58 pr-md-02 ceph-mon[5348]: from='mgr.274299 ' 
entity='mgr.pr-md-02.emffhz'
May  7 17:04:58 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225212: 1313 pgs: 7 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s
May  7 17:05:00 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225213: 1313 pgs: 7 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 766 B/s rd, 33 KiB/s wr, 2 op/s
May  7 17:05:01 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:05:01 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:05:02 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5612 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:05:02 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225214: 1313 pgs: 7 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1291 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 766 B/s rd, 33 KiB/s wr, 2 op/s
May  7 17:05:03 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:05:03 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:05:03 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev 15bb48f0-f983-4282-b9c0-19dd7e9bf550 does not exist
May  7 17:05:03 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev 4e81461d-8388-45ab-91e8-338e82465841 does not exist
May  7 17:05:03 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev e28e821d-f2eb-42d1-b9f8-a99c53e67866 does not exist
May  7 17:05:03 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:05:03 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:05:03 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:05:03 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:05:03 pr-md-02 ceph-mon[5348]: from='mgr.274299 
10.50.1.75:0/136806890' entity='mgr.pr-md-02.emffhz' cmd=[{"prefix": 
"auth get", "entity": "client.admin"}]: dispatch
May  7 17:05:03 pr-md-02 ceph-mon[5348]: from='mgr.274299 ' 
entity='mgr.pr-md-02.emffhz'
May  7 17:05:03 pr-md-02 ceph-mon[5348]: from='mgr.274299 
10.50.1.75:0/136806890' entity='mgr.pr-md-02.emffhz' cmd=[{"prefix": 
"auth get", "entity": "client.bootstrap-osd"}]: dispatch
May  7 17:05:03 pr-md-02 podman[2564001]: 2024-05-07 
17:05:03.630854043 -0700 PDT m=+0.047521077 container create 
268bc85e5ba7c5b788c4460fe5b242e07a211d4808c84158ad0d2b3e178b9f4b 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=jovial_solomon, org.label-schema.schema-version=1.0, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, 
org.label-schema.vendor=CentOS, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, 
org.label-schema.build-date=20240222, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
CEPH_POINT_RELEASE=-18.2.1, ceph=True, org.label-schema.name=CentOS 
Stream 8 Base Image, io.buildah.version=1.29.1, GIT_CLEAN=True)
May  7 17:05:03 pr-md-02 systemd[1]: Started 
libpod-conmon-268bc85e5ba7c5b788c4460fe5b242e07a211d4808c84158ad0d2b3e178b9f4b.scope.
May  7 17:05:03 pr-md-02 systemd[1]: Started libcrun container.
May  7 17:05:03 pr-md-02 podman[2564001]: 2024-05-07 
17:05:03.671554205 -0700 PDT m=+0.088221217 container init 
268bc85e5ba7c5b788c4460fe5b242e07a211d4808c84158ad0d2b3e178b9f4b 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=jovial_solomon, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume 
Abrioux <gabrioux@redhat.com>, org.label-schema.build-date=20240222, 
RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, 
org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, 
org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 
Base Image, GIT_CLEAN=True)
May  7 17:05:03 pr-md-02 jovial_solomon[2564016]: 167 167
May  7 17:05:03 pr-md-02 systemd[1]: 
libpod-268bc85e5ba7c5b788c4460fe5b242e07a211d4808c84158ad0d2b3e178b9f4b.scope: 
Deactivated successfully.
May  7 17:05:03 pr-md-02 podman[2564001]: 2024-05-07 
17:05:03.679778085 -0700 PDT m=+0.096445105 container start 
268bc85e5ba7c5b788c4460fe5b242e07a211d4808c84158ad0d2b3e178b9f4b 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=jovial_solomon, org.label-schema.vendor=CentOS, 
CEPH_POINT_RELEASE=-18.2.1, ceph=True, RELEASE=HEAD, GIT_CLEAN=True, 
GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, 
io.buildah.version=1.29.1, org.label-schema.license=GPLv2, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.name=CentOS Stream 8 Base Image, 
org.label-schema.build-date=20240222, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>)
May  7 17:05:03 pr-md-02 podman[2564001]: 2024-05-07 
17:05:03.679881368 -0700 PDT m=+0.096548394 container attach 
268bc85e5ba7c5b788c4460fe5b242e07a211d4808c84158ad0d2b3e178b9f4b 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=jovial_solomon, org.label-schema.schema-version=1.0, 
RELEASE=HEAD, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, ceph=True, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.name=CentOS Stream 8 Base Image, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
io.buildah.version=1.29.1, GIT_CLEAN=True, 
org.label-schema.build-date=20240222)
May  7 17:05:03 pr-md-02 podman[2564001]: 2024-05-07 
17:05:03.680110752 -0700 PDT m=+0.096777772 container died 
268bc85e5ba7c5b788c4460fe5b242e07a211d4808c84158ad0d2b3e178b9f4b 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=jovial_solomon, GIT_BRANCH=HEAD, ceph=True, 
org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 
Base Image, org.label-schema.build-date=20240222, 
org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
io.buildah.version=1.29.1, org.label-schema.license=GPLv2, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, 
GIT_CLEAN=True)
May  7 17:05:03 pr-md-02 podman[2564001]: 2024-05-07 
17:05:03.605825339 -0700 PDT m=+0.022492359 image pull 
5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 
quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8
May  7 17:05:03 pr-md-02 podman[2564022]: 2024-05-07 17:05:03.7398573 
-0700 PDT m=+0.053387996 container remove 
268bc85e5ba7c5b788c4460fe5b242e07a211d4808c84158ad0d2b3e178b9f4b 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=jovial_solomon, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, 
GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, 
io.buildah.version=1.29.1, ceph=True, 
org.label-schema.build-date=20240222, org.label-schema.name=CentOS 
Stream 8 Base Image, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, RELEASE=HEAD)
May  7 17:05:03 pr-md-02 systemd[1]: 
libpod-conmon-268bc85e5ba7c5b788c4460fe5b242e07a211d4808c84158ad0d2b3e178b9f4b.scope: 
Deactivated successfully.
May  7 17:05:03 pr-md-02 podman[2564043]: 2024-05-07 
17:05:03.869704758 -0700 PDT m=+0.050589984 container create 
ec12311fc2494eb8b7529762d64377e27f0c6ff34d92242ba7776aa29f35125b 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=friendly_brahmagupta, org.label-schema.vendor=CentOS, 
CEPH_POINT_RELEASE=-18.2.1, ceph=True, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, 
org.label-schema.build-date=20240222, 
org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, 
RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
io.buildah.version=1.29.1, GIT_BRANCH=HEAD)
May  7 17:05:03 pr-md-02 systemd[1]: Started 
libpod-conmon-ec12311fc2494eb8b7529762d64377e27f0c6ff34d92242ba7776aa29f35125b.scope.
May  7 17:05:03 pr-md-02 systemd[1]: Started libcrun container.
May  7 17:05:03 pr-md-02 podman[2564043]: 2024-05-07 
17:05:03.901342134 -0700 PDT m=+0.082227374 container init 
ec12311fc2494eb8b7529762d64377e27f0c6ff34d92242ba7776aa29f35125b 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=friendly_brahmagupta, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.name=CentOS Stream 8 Base Image, 
CEPH_POINT_RELEASE=-18.2.1, ceph=True, 
org.label-schema.schema-version=1.0, 
org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, 
io.buildah.version=1.29.1, GIT_BRANCH=HEAD, RELEASE=HEAD, 
GIT_CLEAN=True)
May  7 17:05:03 pr-md-02 podman[2564043]: 2024-05-07 
17:05:03.909537569 -0700 PDT m=+0.090422794 container start 
ec12311fc2494eb8b7529762d64377e27f0c6ff34d92242ba7776aa29f35125b 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=friendly_brahmagupta, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, ceph=True, 
GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, 
org.label-schema.name=CentOS Stream 8 Base Image, 
io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, 
org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, 
RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.1, 
GIT_REPO=https://github.com/ceph/ceph-container.git)
May  7 17:05:03 pr-md-02 podman[2564043]: 2024-05-07 
17:05:03.909642002 -0700 PDT m=+0.090527227 container attach 
ec12311fc2494eb8b7529762d64377e27f0c6ff34d92242ba7776aa29f35125b 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=friendly_brahmagupta, GIT_BRANCH=HEAD, 
org.label-schema.name=CentOS Stream 8 Base Image, 
io.buildah.version=1.29.1, org.label-schema.build-date=20240222, 
org.label-schema.license=GPLv2, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, 
RELEASE=HEAD, org.label-schema.schema-version=1.0, 
CEPH_POINT_RELEASE=-18.2.1, ceph=True, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>)
May  7 17:05:03 pr-md-02 podman[2564043]: 2024-05-07 
17:05:03.845778509 -0700 PDT m=+0.026663744 image pull 
5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 
quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8
May  7 17:05:04 pr-md-02 systemd[1]: 
var-lib-containers-storage-overlay-ddfb8ca9d0c69b0b759623d12dc51317ef66a0ae06c62015987327a7630ca3b2-merged.mount: 
Deactivated successfully.
May  7 17:05:04 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225215: 1313 pgs: 7 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1291 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 766 B/s rd, 33 KiB/s wr, 2 op/s
May  7 17:05:05 pr-md-02 friendly_brahmagupta[2564058]: --> passed 
data devices: 2 physical, 0 LVM
May  7 17:05:05 pr-md-02 friendly_brahmagupta[2564058]: --> relative 
data size: 1.0
May  7 17:05:05 pr-md-02 friendly_brahmagupta[2564058]: --> All data 
devices are unavailable
May  7 17:05:05 pr-md-02 systemd[1]: 
libpod-ec12311fc2494eb8b7529762d64377e27f0c6ff34d92242ba7776aa29f35125b.scope: 
Deactivated successfully.
May  7 17:05:05 pr-md-02 podman[2564043]: 2024-05-07 
17:05:05.365433795 -0700 PDT m=+1.546319027 container died 
ec12311fc2494eb8b7529762d64377e27f0c6ff34d92242ba7776aa29f35125b 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=friendly_brahmagupta, GIT_CLEAN=True, 
org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.build-date=20240222, io.buildah.version=1.29.1, 
org.label-schema.vendor=CentOS, ceph=True, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.license=GPLv2, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, 
RELEASE=HEAD)
May  7 17:05:05 pr-md-02 systemd[1]: 
var-lib-containers-storage-overlay-dfc8fb25041eaa38ebd863e2dc2eea81db94bca7926c18bf08635dd8897a3d08-merged.mount: 
Deactivated successfully.
May  7 17:05:05 pr-md-02 podman[2564087]: 2024-05-07 
17:05:05.428169761 -0700 PDT m=+0.055435773 container remove 
ec12311fc2494eb8b7529762d64377e27f0c6ff34d92242ba7776aa29f35125b 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=friendly_brahmagupta, RELEASE=HEAD, ceph=True, 
org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, org.label-schema.schema-version=1.0, 
GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, 
GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.name=CentOS Stream 8 Base Image, 
CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, 
org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2)
May  7 17:05:05 pr-md-02 systemd[1]: 
libpod-conmon-ec12311fc2494eb8b7529762d64377e27f0c6ff34d92242ba7776aa29f35125b.scope: 
Deactivated successfully.
May  7 17:05:06 pr-md-02 podman[2564224]: 2024-05-07 
17:05:06.002418626 -0700 PDT m=+0.040341047 container create 
d98b8a9e74376baf21cdc6e0712a8035e63483f1d678949a86b48646e7f35c38 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=modest_austin, GIT_CLEAN=True, RELEASE=HEAD, 
org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, 
GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base 
Image, org.label-schema.license=GPLv2, ceph=True, 
org.label-schema.build-date=20240222, 
GIT_REPO=https://github.com/ceph/ceph-container.git)
May  7 17:05:06 pr-md-02 systemd[1]: Started 
libpod-conmon-d98b8a9e74376baf21cdc6e0712a8035e63483f1d678949a86b48646e7f35c38.scope.
May  7 17:05:06 pr-md-02 systemd[1]: Started libcrun container.
May  7 17:05:06 pr-md-02 podman[2564224]: 2024-05-07 
17:05:06.032819591 -0700 PDT m=+0.070742046 container init 
d98b8a9e74376baf21cdc6e0712a8035e63483f1d678949a86b48646e7f35c38 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=modest_austin, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.schema-version=1.0, org.label-schema.name=CentOS 
Stream 8 Base Image, RELEASE=HEAD, ceph=True, 
org.label-schema.build-date=20240222, io.buildah.version=1.29.1, 
org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, 
CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True)
May  7 17:05:06 pr-md-02 modest_austin[2564240]: 167 167
May  7 17:05:06 pr-md-02 systemd[1]: 
libpod-d98b8a9e74376baf21cdc6e0712a8035e63483f1d678949a86b48646e7f35c38.scope: 
Deactivated successfully.
May  7 17:05:06 pr-md-02 podman[2564224]: 2024-05-07 
17:05:06.040138184 -0700 PDT m=+0.078060613 container start 
d98b8a9e74376baf21cdc6e0712a8035e63483f1d678949a86b48646e7f35c38 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=modest_austin, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.1, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, ceph=True, 
GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 
8 Base Image, org.label-schema.license=GPLv2, 
org.label-schema.build-date=20240222, io.buildah.version=1.29.1, 
org.label-schema.vendor=CentOS, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.schema-version=1.0)
May  7 17:05:06 pr-md-02 podman[2564224]: 2024-05-07 
17:05:06.040286008 -0700 PDT m=+0.078208464 container attach 
d98b8a9e74376baf21cdc6e0712a8035e63483f1d678949a86b48646e7f35c38 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=modest_austin, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.build-date=20240222, GIT_CLEAN=True, 
org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.license=GPLv2, RELEASE=HEAD, 
org.label-schema.name=CentOS Stream 8 Base Image, 
io.buildah.version=1.29.1, ceph=True)
May  7 17:05:06 pr-md-02 podman[2564224]: 2024-05-07 
17:05:06.040587551 -0700 PDT m=+0.078509982 container died 
d98b8a9e74376baf21cdc6e0712a8035e63483f1d678949a86b48646e7f35c38 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=modest_austin, org.label-schema.vendor=CentOS, GIT_CLEAN=True, 
org.label-schema.name=CentOS Stream 8 Base Image, 
CEPH_POINT_RELEASE=-18.2.1, ceph=True, 
org.label-schema.build-date=20240222, io.buildah.version=1.29.1, 
GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, org.label-schema.schema-version=1.0, 
GIT_BRANCH=HEAD)
May  7 17:05:06 pr-md-02 podman[2564224]: 2024-05-07 
17:05:05.987393175 -0700 PDT m=+0.025315605 image pull 
5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 
quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8
May  7 17:05:06 pr-md-02 podman[2564245]: 2024-05-07 
17:05:06.097478996 -0700 PDT m=+0.050797295 container remove 
d98b8a9e74376baf21cdc6e0712a8035e63483f1d678949a86b48646e7f35c38 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=modest_austin, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, org.label-schema.name=CentOS Stream 8 Base 
Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
RELEASE=HEAD, org.label-schema.vendor=CentOS, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
CEPH_POINT_RELEASE=-18.2.1, org.label-schema.build-date=20240222, 
GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, 
io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, 
GIT_CLEAN=True)
May  7 17:05:06 pr-md-02 systemd[1]: 
libpod-conmon-d98b8a9e74376baf21cdc6e0712a8035e63483f1d678949a86b48646e7f35c38.scope: 
Deactivated successfully.
May  7 17:05:06 pr-md-02 podman[2564266]: 2024-05-07 
17:05:06.228492423 -0700 PDT m=+0.042467581 container create 
39499009a5ee7478a058cad0a646616d3d0b62fed532ddd1b2bbdb44128c30eb 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=goofy_tesla, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
io.buildah.version=1.29.1, org.label-schema.license=GPLv2, 
RELEASE=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base 
Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, 
GIT_CLEAN=True, maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.schema-version=1.0, 
org.label-schema.build-date=20240222)
May  7 17:05:06 pr-md-02 systemd[1]: Started 
libpod-conmon-39499009a5ee7478a058cad0a646616d3d0b62fed532ddd1b2bbdb44128c30eb.scope.
May  7 17:05:06 pr-md-02 systemd[1]: Started libcrun container.
May  7 17:05:06 pr-md-02 podman[2564266]: 2024-05-07 
17:05:06.263672711 -0700 PDT m=+0.077647878 container init 
39499009a5ee7478a058cad0a646616d3d0b62fed532ddd1b2bbdb44128c30eb 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=goofy_tesla, org.label-schema.name=CentOS Stream 8 Base Image, 
RELEASE=HEAD, org.label-schema.vendor=CentOS, 
org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, 
CEPH_POINT_RELEASE=-18.2.1, org.label-schema.license=GPLv2, 
ceph=True, org.label-schema.build-date=20240222, 
io.buildah.version=1.29.1, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, 
GIT_REPO=https://github.com/ceph/ceph-container.git)
May  7 17:05:06 pr-md-02 podman[2564266]: 2024-05-07 
17:05:06.271891014 -0700 PDT m=+0.085866179 container start 
39499009a5ee7478a058cad0a646616d3d0b62fed532ddd1b2bbdb44128c30eb 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=goofy_tesla, maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
GIT_CLEAN=True, org.label-schema.license=GPLv2, 
io.buildah.version=1.29.1, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.schema-version=1.0, org.label-schema.name=CentOS 
Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, 
GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, 
ceph=True, org.label-schema.build-date=20240222, 
org.label-schema.vendor=CentOS)
May  7 17:05:06 pr-md-02 podman[2564266]: 2024-05-07 
17:05:06.27203062 -0700 PDT m=+0.086005784 container attach 
39499009a5ee7478a058cad0a646616d3d0b62fed532ddd1b2bbdb44128c30eb 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=goofy_tesla, ceph=True, org.label-schema.name=CentOS Stream 8 
Base Image, RELEASE=HEAD, GIT_CLEAN=True, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.license=GPLv2, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.schema-version=1.0, 
org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, 
CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, 
GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux <gabrioux@redhat.com>)
May  7 17:05:06 pr-md-02 podman[2564266]: 2024-05-07 
17:05:06.213038456 -0700 PDT m=+0.027013622 image pull 
5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 
quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: {
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]:    "41": [
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]:        {
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "devices": [
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "/dev/nvme0n1"
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]:            ],
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "lv_name": 
"osd-block-88070adf-ee15-406a-b8e9-98a61e294c8b",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "lv_path": 
"/dev/ceph-eac1ebc0-c00f-4558-8c72-741fc8f81abc/osd-block-88070adf-ee15-406a-b8e9-98a61e294c8b",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "lv_size": 
"1920378863616",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "lv_tags": 
"ceph.block_device=/dev/ceph-eac1ebc0-c00f-4558-8c72-741fc8f81abc/osd-block-88070adf-ee15-406a-b8e9-98a61e294c8b,ceph.block_uuid=MwSZ0h-81BS-XH33-wWY4-vk23-2TIJ-DW9XgO,ceph.cephx_lockbox_secret=AQBJFuJlxpZUJBAAHM4wPn5k5ExaR9mY0pS+YA==,ceph.cluster_fsid=58bde08a-d7ed-11ee-9098-506b4b4da440,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=1,ceph.osd_fsid=88070adf-ee15-406a-b8e9-98a61e294c8b,ceph.osd_id=41,ceph.osdspec_affinity=md_encrypted_nvme_disks,ceph.type=block,ceph.vdo=0",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "lv_uuid": 
"MwSZ0h-81BS-XH33-wWY4-vk23-2TIJ-DW9XgO",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "name": 
"osd-block-88070adf-ee15-406a-b8e9-98a61e294c8b",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "path": 
"/dev/ceph-eac1ebc0-c00f-4558-8c72-741fc8f81abc/osd-block-88070adf-ee15-406a-b8e9-98a61e294c8b",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "tags": {
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "ceph.block_device": 
"/dev/ceph-eac1ebc0-c00f-4558-8c72-741fc8f81abc/osd-block-88070adf-ee15-406a-b8e9-98a61e294c8b",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "ceph.block_uuid": 
"MwSZ0h-81BS-XH33-wWY4-vk23-2TIJ-DW9XgO",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: 
"ceph.cephx_lockbox_secret": "AQBJFuJlxpZUJBAAHM4wPn5k5ExaR9mY0pS+YA==",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "ceph.cluster_fsid": 
"58bde08a-d7ed-11ee-9098-506b4b4da440",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "ceph.cluster_name": 
"ceph",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: 
"ceph.crush_device_class": "",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "ceph.encrypted": "1",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "ceph.osd_fsid": 
"88070adf-ee15-406a-b8e9-98a61e294c8b",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "ceph.osd_id": "41",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: 
"ceph.osdspec_affinity": "md_encrypted_nvme_disks",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "ceph.type": "block",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "ceph.vdo": "0"
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]:            },
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "type": "block",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "vg_name": 
"ceph-eac1ebc0-c00f-4558-8c72-741fc8f81abc"
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]:        }
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]:    ],
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]:    "43": [
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]:        {
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "devices": [
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "/dev/nvme1n1"
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]:            ],
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "lv_name": 
"osd-block-50a79ade-482b-4c7e-84ae-0dac068ed570",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "lv_path": 
"/dev/ceph-0b5aeb1a-7dea-4260-9af9-f2bff16d642a/osd-block-50a79ade-482b-4c7e-84ae-0dac068ed570",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "lv_size": 
"1920378863616",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "lv_tags": 
"ceph.block_device=/dev/ceph-0b5aeb1a-7dea-4260-9af9-f2bff16d642a/osd-block-50a79ade-482b-4c7e-84ae-0dac068ed570,ceph.block_uuid=zN3gTU-nGwQ-DfIA-dzWZ-e7QI-OcmL-SZcgqT,ceph.cephx_lockbox_secret=AQBYFuJlTtj/DBAAt2wy+qfx2XCHFuwJZKwp6A==,ceph.cluster_fsid=58bde08a-d7ed-11ee-9098-506b4b4da440,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=1,ceph.osd_fsid=50a79ade-482b-4c7e-84ae-0dac068ed570,ceph.osd_id=43,ceph.osdspec_affinity=md_encrypted_nvme_disks,ceph.type=block,ceph.vdo=0",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "lv_uuid": 
"zN3gTU-nGwQ-DfIA-dzWZ-e7QI-OcmL-SZcgqT",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "name": 
"osd-block-50a79ade-482b-4c7e-84ae-0dac068ed570",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "path": 
"/dev/ceph-0b5aeb1a-7dea-4260-9af9-f2bff16d642a/osd-block-50a79ade-482b-4c7e-84ae-0dac068ed570",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "tags": {
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "ceph.block_device": 
"/dev/ceph-0b5aeb1a-7dea-4260-9af9-f2bff16d642a/osd-block-50a79ade-482b-4c7e-84ae-0dac068ed570",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "ceph.block_uuid": 
"zN3gTU-nGwQ-DfIA-dzWZ-e7QI-OcmL-SZcgqT",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: 
"ceph.cephx_lockbox_secret": "AQBYFuJlTtj/DBAAt2wy+qfx2XCHFuwJZKwp6A==",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "ceph.cluster_fsid": 
"58bde08a-d7ed-11ee-9098-506b4b4da440",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "ceph.cluster_name": 
"ceph",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: 
"ceph.crush_device_class": "",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "ceph.encrypted": "1",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "ceph.osd_fsid": 
"50a79ade-482b-4c7e-84ae-0dac068ed570",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "ceph.osd_id": "43",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: 
"ceph.osdspec_affinity": "md_encrypted_nvme_disks",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "ceph.type": "block",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "ceph.vdo": "0"
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]:            },
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "type": "block",
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: "vg_name": 
"ceph-0b5aeb1a-7dea-4260-9af9-f2bff16d642a"
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]:        }
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]:    ]
May  7 17:05:06 pr-md-02 goofy_tesla[2564282]: }
May  7 17:05:06 pr-md-02 systemd[1]: 
libpod-39499009a5ee7478a058cad0a646616d3d0b62fed532ddd1b2bbdb44128c30eb.scope: 
Deactivated successfully.
May  7 17:05:06 pr-md-02 podman[2564266]: 2024-05-07 
17:05:06.67755909 -0700 PDT m=+0.491534254 container died 
39499009a5ee7478a058cad0a646616d3d0b62fed532ddd1b2bbdb44128c30eb 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=goofy_tesla, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, 
GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, 
org.label-schema.schema-version=1.0, 
org.label-schema.build-date=20240222, GIT_CLEAN=True, 
org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, 
io.buildah.version=1.29.1)
May  7 17:05:06 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:05:06] "GET /metrics HTTP/1.1" 
200 73531 "" "Prometheus/2.43.0"
May  7 17:05:06 pr-md-02 podman[2564295]: 2024-05-07 
17:05:06.745980606 -0700 PDT m=+0.056378222 container remove 
39499009a5ee7478a058cad0a646616d3d0b62fed532ddd1b2bbdb44128c30eb 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=goofy_tesla, CEPH_POINT_RELEASE=-18.2.1, 
org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=HEAD, 
io.buildah.version=1.29.1, org.label-schema.build-date=20240222, 
ceph=True, maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, 
org.label-schema.name=CentOS Stream 8 Base Image, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd)
May  7 17:05:06 pr-md-02 systemd[1]: 
libpod-conmon-39499009a5ee7478a058cad0a646616d3d0b62fed532ddd1b2bbdb44128c30eb.scope: 
Deactivated successfully.
May  7 17:05:06 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225216: 1313 pgs: 7 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1291 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 766 B/s rd, 33 KiB/s wr, 2 op/s
May  7 17:05:07 pr-md-02 systemd[1]: 
var-lib-containers-storage-overlay-522d9498e68f9764cedb1b2ff926f774718727cdd52974d491a188f93a2c19f6-merged.mount: 
Deactivated successfully.
May  7 17:05:07 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5612 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:05:07 pr-md-02 podman[2564431]: 2024-05-07 
17:05:07.344365868 -0700 PDT m=+0.042884110 container create 
748094f58d93b27e8acc845f1a3e6003c9e1b03fb568f6806ffb4cd17ebe21dd 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=goofy_pare, io.buildah.version=1.29.1, maintainer=Guillaume 
Abrioux <gabrioux@redhat.com>, org.label-schema.build-date=20240222, 
ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 
Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
RELEASE=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, 
CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, 
org.label-schema.vendor=CentOS, 
GIT_REPO=https://github.com/ceph/ceph-container.git)
May  7 17:05:07 pr-md-02 systemd[1]: Started 
libpod-conmon-748094f58d93b27e8acc845f1a3e6003c9e1b03fb568f6806ffb4cd17ebe21dd.scope.
May  7 17:05:07 pr-md-02 systemd[1]: Started libcrun container.
May  7 17:05:07 pr-md-02 podman[2564431]: 2024-05-07 
17:05:07.374008883 -0700 PDT m=+0.072527132 container init 
748094f58d93b27e8acc845f1a3e6003c9e1b03fb568f6806ffb4cd17ebe21dd 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=goofy_pare, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, 
org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, ceph=True, GIT_BRANCH=HEAD, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.name=CentOS Stream 8 Base Image, 
io.buildah.version=1.29.1, org.label-schema.license=GPLv2, 
RELEASE=HEAD, org.label-schema.schema-version=1.0, 
org.label-schema.build-date=20240222)
May  7 17:05:07 pr-md-02 goofy_pare[2564446]: 167 167
May  7 17:05:07 pr-md-02 systemd[1]: 
libpod-748094f58d93b27e8acc845f1a3e6003c9e1b03fb568f6806ffb4cd17ebe21dd.scope: 
Deactivated successfully.
May  7 17:05:07 pr-md-02 podman[2564431]: 2024-05-07 
17:05:07.381560483 -0700 PDT m=+0.080078731 container start 
748094f58d93b27e8acc845f1a3e6003c9e1b03fb568f6806ffb4cd17ebe21dd 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=goofy_pare, io.buildah.version=1.29.1, GIT_CLEAN=True, 
org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 
Base Image, org.label-schema.vendor=CentOS, RELEASE=HEAD, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, 
org.label-schema.build-date=20240222, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, 
GIT_BRANCH=HEAD)
May  7 17:05:07 pr-md-02 podman[2564431]: 2024-05-07 
17:05:07.38167372 -0700 PDT m=+0.080191969 container attach 
748094f58d93b27e8acc845f1a3e6003c9e1b03fb568f6806ffb4cd17ebe21dd 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=goofy_pare, GIT_REPO=https://github.com/ceph/ceph-container.git, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, 
GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, 
org.label-schema.build-date=20240222, ceph=True, RELEASE=HEAD, 
GIT_CLEAN=True, org.label-schema.schema-version=1.0)
May  7 17:05:07 pr-md-02 podman[2564431]: 2024-05-07 
17:05:07.38196021 -0700 PDT m=+0.080478458 container died 
748094f58d93b27e8acc845f1a3e6003c9e1b03fb568f6806ffb4cd17ebe21dd 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=goofy_pare, io.buildah.version=1.29.1, maintainer=Guillaume 
Abrioux <gabrioux@redhat.com>, ceph=True, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, RELEASE=HEAD, 
org.label-schema.build-date=20240222, org.label-schema.name=CentOS 
Stream 8 Base Image, org.label-schema.license=GPLv2, 
org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, 
GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True)
May  7 17:05:07 pr-md-02 podman[2564431]: 2024-05-07 
17:05:07.330060004 -0700 PDT m=+0.028578252 image pull 
5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 
quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8
May  7 17:05:07 pr-md-02 podman[2564451]: 2024-05-07 
17:05:07.443179465 -0700 PDT m=+0.051396095 container remove 
748094f58d93b27e8acc845f1a3e6003c9e1b03fb568f6806ffb4cd17ebe21dd 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=goofy_pare, CEPH_POINT_RELEASE=-18.2.1, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.schema-version=1.0, ceph=True, 
org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, 
io.buildah.version=1.29.1, org.label-schema.license=GPLv2, 
RELEASE=HEAD, org.label-schema.build-date=20240222, 
org.label-schema.vendor=CentOS, 
GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True)
May  7 17:05:07 pr-md-02 systemd[1]: 
libpod-conmon-748094f58d93b27e8acc845f1a3e6003c9e1b03fb568f6806ffb4cd17ebe21dd.scope: 
Deactivated successfully.
May  7 17:05:07 pr-md-02 ceph-mgr[5311]: [rbd_support INFO root] 
MirrorSnapshotScheduleHandler: load_schedules
May  7 17:05:07 pr-md-02 podman[2564472]: 2024-05-07 
17:05:07.581081662 -0700 PDT m=+0.046354470 container create 
bafe02cbdc9fdd28ea041a0798aa53b45fbcc252e2e615ac26c5049a1df47575 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=quizzical_antonelli, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, org.label-schema.vendor=CentOS, 
org.label-schema.build-date=20240222, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
io.buildah.version=1.29.1, 
GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, 
GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, 
RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, 
org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1)
May  7 17:05:07 pr-md-02 systemd[1]: Started 
libpod-conmon-bafe02cbdc9fdd28ea041a0798aa53b45fbcc252e2e615ac26c5049a1df47575.scope.
May  7 17:05:07 pr-md-02 systemd[1]: Started libcrun container.
May  7 17:05:07 pr-md-02 podman[2564472]: 2024-05-07 
17:05:07.613009264 -0700 PDT m=+0.078282065 container init 
bafe02cbdc9fdd28ea041a0798aa53b45fbcc252e2e615ac26c5049a1df47575 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=quizzical_antonelli, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, 
org.label-schema.schema-version=1.0, RELEASE=HEAD, 
org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.29.1, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, 
org.label-schema.name=CentOS Stream 8 Base Image, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.license=GPLv2, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd)
May  7 17:05:07 pr-md-02 podman[2564472]: 2024-05-07 
17:05:07.620807072 -0700 PDT m=+0.086079873 container start 
bafe02cbdc9fdd28ea041a0798aa53b45fbcc252e2e615ac26c5049a1df47575 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=quizzical_antonelli, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, 
org.label-schema.schema-version=1.0, org.label-schema.name=CentOS 
Stream 8 Base Image, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, org.label-schema.license=GPLv2, ceph=True, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, 
org.label-schema.build-date=20240222, 
GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, 
RELEASE=HEAD)
May  7 17:05:07 pr-md-02 podman[2564472]: 2024-05-07 
17:05:07.620921904 -0700 PDT m=+0.086194728 container attach 
bafe02cbdc9fdd28ea041a0798aa53b45fbcc252e2e615ac26c5049a1df47575 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=quizzical_antonelli, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, 
org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, 
org.label-schema.build-date=20240222, ceph=True, maintainer=Guillaume 
Abrioux <gabrioux@redhat.com>, CEPH_POINT_RELEASE=-18.2.1, 
org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base 
Image, GIT_CLEAN=True)
May  7 17:05:07 pr-md-02 podman[2564472]: 2024-05-07 
17:05:07.560418202 -0700 PDT m=+0.025691007 image pull 
5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 
quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8
May  7 17:05:07 pr-md-02 ceph-mgr[5311]: [balancer INFO root] 
Optimize plan auto_2024-05-08_00:05:07
May  7 17:05:07 pr-md-02 ceph-mgr[5311]: [balancer INFO root] Mode 
upmap, max misplaced 0.050000
May  7 17:05:07 pr-md-02 ceph-mgr[5311]: [balancer INFO root] do_upmap
May  7 17:05:07 pr-md-02 ceph-mgr[5311]: [balancer INFO root] pools 
['.mgr', 'cephfs_md_and_data', 'cephfs_data', 'cephfs_metadata']
May  7 17:05:07 pr-md-02 ceph-mgr[5311]: [balancer INFO root] 
prepared 0/10 changes
May  7 17:05:07 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:05:08 pr-md-02 systemd[1]: 
var-lib-containers-storage-overlay-a2a58f86faa3a65838beaf8bcb779039975420b572508bf98328122f6618f26b-merged.mount: 
Deactivated successfully.
May  7 17:05:08 pr-md-02 ceph-mgr[5311]: [rbd_support INFO root] 
TrashPurgeScheduleHandler: load_schedules
May  7 17:05:08 pr-md-02 ceph-mon[5348]: from='mgr.274299 ' 
entity='mgr.pr-md-02.emffhz'
May  7 17:05:08 pr-md-02 quizzical_antonelli[2564488]: {
May  7 17:05:08 pr-md-02 quizzical_antonelli[2564488]: 
"50a79ade-482b-4c7e-84ae-0dac068ed570": {
May  7 17:05:08 pr-md-02 quizzical_antonelli[2564488]: "ceph_fsid": 
"58bde08a-d7ed-11ee-9098-506b4b4da440",
May  7 17:05:08 pr-md-02 quizzical_antonelli[2564488]: "device": 
"/dev/mapper/zN3gTU-nGwQ-DfIA-dzWZ-e7QI-OcmL-SZcgqT",
May  7 17:05:08 pr-md-02 quizzical_antonelli[2564488]: "osd_id": 43,
May  7 17:05:08 pr-md-02 quizzical_antonelli[2564488]: "osd_uuid": 
"50a79ade-482b-4c7e-84ae-0dac068ed570",
May  7 17:05:08 pr-md-02 quizzical_antonelli[2564488]: "type": 
"bluestore"
May  7 17:05:08 pr-md-02 quizzical_antonelli[2564488]:    },
May  7 17:05:08 pr-md-02 quizzical_antonelli[2564488]: 
"88070adf-ee15-406a-b8e9-98a61e294c8b": {
May  7 17:05:08 pr-md-02 quizzical_antonelli[2564488]: "ceph_fsid": 
"58bde08a-d7ed-11ee-9098-506b4b4da440",
May  7 17:05:08 pr-md-02 quizzical_antonelli[2564488]: "device": 
"/dev/mapper/MwSZ0h-81BS-XH33-wWY4-vk23-2TIJ-DW9XgO",
May  7 17:05:08 pr-md-02 quizzical_antonelli[2564488]: "osd_id": 41,
May  7 17:05:08 pr-md-02 quizzical_antonelli[2564488]: "osd_uuid": 
"88070adf-ee15-406a-b8e9-98a61e294c8b",
May  7 17:05:08 pr-md-02 quizzical_antonelli[2564488]: "type": 
"bluestore"
May  7 17:05:08 pr-md-02 quizzical_antonelli[2564488]:    }
May  7 17:05:08 pr-md-02 quizzical_antonelli[2564488]: }
May  7 17:05:08 pr-md-02 systemd[1]: 
libpod-bafe02cbdc9fdd28ea041a0798aa53b45fbcc252e2e615ac26c5049a1df47575.scope: 
Deactivated successfully.
May  7 17:05:08 pr-md-02 podman[2564472]: 2024-05-07 
17:05:08.623675315 -0700 PDT m=+1.088948125 container died 
bafe02cbdc9fdd28ea041a0798aa53b45fbcc252e2e615ac26c5049a1df47575 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=quizzical_antonelli, org.label-schema.build-date=20240222, 
GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, 
ceph=True, org.label-schema.vendor=CentOS, 
CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, RELEASE=HEAD, 
org.label-schema.name=CentOS Stream 8 Base Image, 
org.label-schema.license=GPLv2, io.buildah.version=1.29.1, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD)
May  7 17:05:08 pr-md-02 systemd[1]: 
libpod-bafe02cbdc9fdd28ea041a0798aa53b45fbcc252e2e615ac26c5049a1df47575.scope: 
Consumed 1.008s CPU time.
May  7 17:05:08 pr-md-02 systemd[1]: 
var-lib-containers-storage-overlay-b2fb74061f0c1e8a4fa96b8bf733317d61899748c44f6e0fce8e5c3cf2d425cf-merged.mount: 
Deactivated successfully.
May  7 17:05:08 pr-md-02 podman[2564569]: 2024-05-07 
17:05:08.684978208 -0700 PDT m=+0.052557214 container remove 
bafe02cbdc9fdd28ea041a0798aa53b45fbcc252e2e615ac26c5049a1df47575 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=quizzical_antonelli, org.label-schema.schema-version=1.0, 
ceph=True, org.label-schema.vendor=CentOS, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume 
Abrioux <gabrioux@redhat.com>, RELEASE=HEAD, 
org.label-schema.name=CentOS Stream 8 Base Image, 
io.buildah.version=1.29.1, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, 
org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=-18.2.1)
May  7 17:05:08 pr-md-02 systemd[1]: 
libpod-conmon-bafe02cbdc9fdd28ea041a0798aa53b45fbcc252e2e615ac26c5049a1df47575.scope: 
Deactivated successfully.
May  7 17:05:08 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225217: 1313 pgs: 6 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 766 B/s rd, 33 KiB/s wr, 2 op/s
May  7 17:05:08 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev dae056fc-5916-4778-a1c3-877b296e165d does not exist
May  7 17:05:08 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev 0dcc2973-fa02-4241-98c4-2d39053a690c does not exist
May  7 17:05:08 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev 0027459d-bcc8-4411-92f1-52408996a6c0 does not exist
May  7 17:05:08 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev b9b9bbe1-65bc-4826-812e-501f98bd0988 does not exist
May  7 17:05:08 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev cbb7bfae-1607-408d-a5d6-e7517465e941 does not exist
May  7 17:05:08 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev a1e40167-575e-4690-b910-1d9135a7bcdf does not exist
May  7 17:05:09 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:05:09 pr-md-02 ceph-mon[5348]: from='mgr.274299 ' 
entity='mgr.pr-md-02.emffhz'
May  7 17:05:09 pr-md-02 ceph-mon[5348]: from='mgr.274299 ' 
entity='mgr.pr-md-02.emffhz'
May  7 17:05:10 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225218: 1313 pgs: 6 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:05:12 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5612 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:05:12 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225219: 1313 pgs: 6 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 766 B/s rd, 33 KiB/s wr, 2 op/s
May  7 17:05:12 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:05:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler INFO root] 
_maybe_adjust
May  7 17:05:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool cephfs_metadata won't scale due to overlapping roots: {-15, -1}
May  7 17:05:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
Please See: 
https://docs.ceph.com/en/latest/rados/operations/placement-groups/#automated...
May  7 17:05:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool cephfs_data won't scale due to overlapping roots: {-15, -1, -2}
May  7 17:05:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
Please See: 
https://docs.ceph.com/en/latest/rados/operations/placement-groups/#automated...
May  7 17:05:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool cephfs_md_and_data won't scale due to overlapping roots: {-15, 
-1, -2}
May  7 17:05:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
Please See: 
https://docs.ceph.com/en/latest/rados/operations/placement-groups/#automated...
May  7 17:05:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool 1 contains an overlapping root -1... skipping scaling
May  7 17:05:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool 7 contains an overlapping root -15... skipping scaling
May  7 17:05:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool 8 contains an overlapping root -2... skipping scaling
May  7 17:05:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool 9 contains an overlapping root -2... skipping scaling
May  7 17:05:14 pr-md-02 systemd[1]: Started Session c1518 of User root.
May  7 17:05:14 pr-md-02 systemd[1]: session-c1518.scope: Deactivated 
successfully.
May  7 17:05:14 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225220: 1313 pgs: 6 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 766 B/s rd, 33 KiB/s wr, 2 op/s
May  7 17:05:16 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:05:16] "GET /metrics HTTP/1.1" 
200 73531 "" "Prometheus/2.43.0"
May  7 17:05:16 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225221: 1313 pgs: 6 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 766 B/s rd, 33 KiB/s wr, 2 op/s
May  7 17:05:17 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5612 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:05:17 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:05:18 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:05:18 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225222: 1313 pgs: 6 active+clean+scrubbing+deep, 16 
active+clean+scrubbing, 1291 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 766 B/s rd, 33 KiB/s wr, 2 op/s
May  7 17:05:18 pr-md-02 ceph-mon[5348]: from='mgr.274299 ' 
entity='mgr.pr-md-02.emffhz'
May  7 17:05:20 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225223: 1313 pgs: 6 active+clean+scrubbing+deep, 16 
active+clean+scrubbing, 1291 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 766 B/s rd, 33 KiB/s wr, 2 op/s
May  7 17:05:22 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5612 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:05:22 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225224: 1313 pgs: 7 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s
May  7 17:05:22 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:05:22 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 e5613: 46 total, 46 up, 46 in
May  7 17:05:24 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:05:24 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
scanning for idle connections..
May  7 17:05:24 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
cleaning up connections: []
May  7 17:05:24 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
scanning for idle connections..
May  7 17:05:24 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
cleaning up connections: []
May  7 17:05:24 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
scanning for idle connections..
May  7 17:05:24 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
cleaning up connections: []
May  7 17:05:24 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225226: 1313 pgs: 7 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 920 B/s rd, 39 KiB/s wr, 2 op/s
May  7 17:05:26 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:05:26] "GET /metrics HTTP/1.1" 
200 73532 "" "Prometheus/2.43.0"
May  7 17:05:26 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225227: 1313 pgs: 7 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 920 B/s rd, 39 KiB/s wr, 2 op/s
May  7 17:05:27 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:05:27 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:05:28 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225228: 1313 pgs: 7 active+clean+scrubbing+deep, 10 
active+clean+scrubbing, 1296 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 920 B/s rd, 39 KiB/s wr, 2 op/s
May  7 17:05:29 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:05:30 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:05:30 pr-md-02 ceph-mon[5348]: log_channel(audit) log [INF] 
: from='admin socket' entity='admin socket' cmd='smart' args=[json]: 
dispatch
May  7 17:05:30 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225229: 1313 pgs: 7 active+clean+scrubbing+deep, 10 
active+clean+scrubbing, 1296 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 920 B/s rd, 39 KiB/s wr, 2 op/s
May  7 17:05:31 pr-md-02 ceph-mon[5348]: log_channel(audit) log [INF] 
: from='admin socket' entity='admin socket' cmd=smart args=[json]: 
finished
May  7 17:05:31 pr-md-02 ceph-mon[5348]: from='mgr.274299 ' 
entity='mgr.pr-md-02.emffhz'
May  7 17:05:31 pr-md-02 ceph-mon[5348]: from='admin socket' 
entity='admin socket' cmd='smart' args=[json]: dispatch
May  7 17:05:31 pr-md-02 ceph-mon[5348]: from='admin socket' 
entity='admin socket' cmd=smart args=[json]: finished
May  7 17:05:31 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:05:32 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:05:32 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:05:32 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225230: 1313 pgs: 4 active+clean+scrubbing+deep, 16 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 1.8 KiB/s rd, 85 KiB/s wr, 5 op/s
May  7 17:05:32 pr-md-02 ceph-mon[5348]: from='admin socket' 
entity='admin socket' cmd='smart' args=[json]: dispatch
May  7 17:05:32 pr-md-02 ceph-mon[5348]: from='admin socket' 
entity='admin socket' cmd=smart args=[json]: finished
May  7 17:05:32 pr-md-02 ceph-mon[5348]: from='admin socket' 
entity='admin socket' cmd='smart' args=[json]: dispatch
May  7 17:05:32 pr-md-02 ceph-mon[5348]: from='admin socket' 
entity='admin socket' cmd=smart args=[json]: finished
May  7 17:05:32 pr-md-02 ceph-mon[5348]: from='mgr.274299 ' 
entity='mgr.pr-md-02.emffhz'
May  7 17:05:32 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:05:33 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:05:33 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:05:33 pr-md-02 ceph-mon[5348]: from='admin socket' 
entity='admin socket' cmd='smart' args=[json]: dispatch
May  7 17:05:33 pr-md-02 ceph-mon[5348]: from='admin socket' 
entity='admin socket' cmd=smart args=[json]: finished
May  7 17:05:33 pr-md-02 ceph-mon[5348]: from='mgr.274299 ' 
entity='mgr.pr-md-02.emffhz'
May  7 17:05:33 pr-md-02 ceph-mon[5348]: from='admin socket' 
entity='admin socket' cmd='smart' args=[json]: dispatch
May  7 17:05:33 pr-md-02 ceph-mon[5348]: from='admin socket' 
entity='admin socket' cmd=smart args=[json]: finished
May  7 17:05:34 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225231: 1313 pgs: 4 active+clean+scrubbing+deep, 16 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 1.5 KiB/s rd, 71 KiB/s wr, 4 op/s
May  7 17:05:36 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:05:36] "GET /metrics HTTP/1.1" 
200 73532 "" "Prometheus/2.43.0"
May  7 17:05:36 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225232: 1313 pgs: 4 active+clean+scrubbing+deep, 16 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 1.5 KiB/s rd, 71 KiB/s wr, 4 op/s
May  7 17:05:37 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:05:38 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:05:38 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225233: 1313 pgs: 6 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 3.0 KiB/s rd, 130 KiB/s wr, 9 op/s
May  7 17:05:39 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:05:40 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225234: 1313 pgs: 6 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 3.0 KiB/s rd, 130 KiB/s wr, 9 op/s
May  7 17:05:42 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:05:42 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225235: 1313 pgs: 6 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 3.0 KiB/s rd, 130 KiB/s wr, 9 op/s
May  7 17:05:43 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:05:44 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225236: 1313 pgs: 6 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 1.5 KiB/s rd, 60 KiB/s wr, 4 op/s
May  7 17:05:46 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:05:46] "GET /metrics HTTP/1.1" 
200 73534 "" "Prometheus/2.43.0"
May  7 17:05:46 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225237: 1313 pgs: 6 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 1.5 KiB/s rd, 60 KiB/s wr, 4 op/s
May  7 17:05:47 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:05:48 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:05:48 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225238: 1313 pgs: 6 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail; 1.5 KiB/s rd, 60 KiB/s wr, 4 op/s
May  7 17:05:50 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225239: 1313 pgs: 6 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:05:52 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:05:52 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225240: 1313 pgs: 5 active+clean+scrubbing+deep, 16 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:05:53 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:05:54 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:05:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
scanning for idle connections..
May  7 17:05:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
cleaning up connections: []
May  7 17:05:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
scanning for idle connections..
May  7 17:05:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
cleaning up connections: []
May  7 17:05:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
scanning for idle connections..
May  7 17:05:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
cleaning up connections: []
May  7 17:05:54 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225241: 1313 pgs: 5 active+clean+scrubbing+deep, 16 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:05:56 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:05:56] "GET /metrics HTTP/1.1" 
200 73532 "" "Prometheus/2.43.0"
May  7 17:05:56 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225242: 1313 pgs: 5 active+clean+scrubbing+deep, 16 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:05:57 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:05:58 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:05:58 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225243: 1313 pgs: 6 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:00 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225244: 1313 pgs: 6 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:02 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:06:02 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225245: 1313 pgs: 7 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:03 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:06:04 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225246: 1313 pgs: 7 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:06 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:06:06] "GET /metrics HTTP/1.1" 
200 73532 "" "Prometheus/2.43.0"
May  7 17:06:06 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225247: 1313 pgs: 7 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:07 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:06:07 pr-md-02 ceph-mgr[5311]: [rbd_support INFO root] 
MirrorSnapshotScheduleHandler: load_schedules
May  7 17:06:07 pr-md-02 ceph-mgr[5311]: [balancer INFO root] 
Optimize plan auto_2024-05-08_00:06:07
May  7 17:06:07 pr-md-02 ceph-mgr[5311]: [balancer INFO root] Mode 
upmap, max misplaced 0.050000
May  7 17:06:07 pr-md-02 ceph-mgr[5311]: [balancer INFO root] do_upmap
May  7 17:06:07 pr-md-02 ceph-mgr[5311]: [balancer INFO root] pools 
['cephfs_md_and_data', '.mgr', 'cephfs_data', 'cephfs_metadata']
May  7 17:06:07 pr-md-02 ceph-mgr[5311]: [balancer INFO root] 
prepared 0/10 changes
May  7 17:06:08 pr-md-02 ceph-mgr[5311]: [rbd_support INFO root] 
TrashPurgeScheduleHandler: load_schedules
May  7 17:06:08 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:06:08 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225248: 1313 pgs: 4 active+clean+scrubbing+deep, 17 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:09 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:06:09 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:06:10 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:06:10 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:06:10 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev d098a1b7-eb6e-446f-98a4-e1427262dc3c does not exist
May  7 17:06:10 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev 9289daf9-2f8a-4915-8058-687f1fc5eb11 does not exist
May  7 17:06:10 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev ca963837-183c-4bc7-990d-e76dc3a7a4b0 does not exist
May  7 17:06:10 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:06:10 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:06:10 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:06:10 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:06:10 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225249: 1313 pgs: 4 active+clean+scrubbing+deep, 17 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:11 pr-md-02 podman[2564878]: 2024-05-07 
17:06:11.181584373 -0700 PDT m=+0.039979954 container create 
64db7aaf822e2ea641ac851bb7f28b14a32618ac3311196f0121008ea7dece3c 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=nice_joliot, org.label-schema.license=GPLv2, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, 
org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, 
org.label-schema.name=CentOS Stream 8 Base Image, 
io.buildah.version=1.29.1, org.label-schema.build-date=20240222, 
CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, RELEASE=HEAD, ceph=True)
May  7 17:06:11 pr-md-02 systemd[1]: Started 
libpod-conmon-64db7aaf822e2ea641ac851bb7f28b14a32618ac3311196f0121008ea7dece3c.scope.
May  7 17:06:11 pr-md-02 systemd[1]: Started libcrun container.
May  7 17:06:11 pr-md-02 podman[2564878]: 2024-05-07 
17:06:11.211049202 -0700 PDT m=+0.069444730 container init 
64db7aaf822e2ea641ac851bb7f28b14a32618ac3311196f0121008ea7dece3c 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=nice_joliot, GIT_BRANCH=HEAD, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, 
org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=-18.2.1, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, RELEASE=HEAD, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.schema-version=1.0, GIT_CLEAN=True, 
io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, 
org.label-schema.license=GPLv2)
May  7 17:06:11 pr-md-02 nice_joliot[2564893]: 167 167
May  7 17:06:11 pr-md-02 systemd[1]: 
libpod-64db7aaf822e2ea641ac851bb7f28b14a32618ac3311196f0121008ea7dece3c.scope: 
Deactivated successfully.
May  7 17:06:11 pr-md-02 podman[2564878]: 2024-05-07 
17:06:11.218033241 -0700 PDT m=+0.076428761 container start 
64db7aaf822e2ea641ac851bb7f28b14a32618ac3311196f0121008ea7dece3c 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=nice_joliot, 
GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, 
CEPH_POINT_RELEASE=-18.2.1, org.label-schema.build-date=20240222, 
GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
io.buildah.version=1.29.1, org.label-schema.license=GPLv2, 
org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.schema-version=1.0)
May  7 17:06:11 pr-md-02 podman[2564878]: 2024-05-07 
17:06:11.218168917 -0700 PDT m=+0.076564434 container attach 
64db7aaf822e2ea641ac851bb7f28b14a32618ac3311196f0121008ea7dece3c 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=nice_joliot, io.buildah.version=1.29.1, 
org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, 
CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, GIT_CLEAN=True, 
org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, GIT_BRANCH=HEAD, org.label-schema.name=CentOS 
Stream 8 Base Image, ceph=True, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.license=GPLv2, 
GIT_REPO=https://github.com/ceph/ceph-container.git)
May  7 17:06:11 pr-md-02 podman[2564878]: 2024-05-07 
17:06:11.218487253 -0700 PDT m=+0.076882764 container died 
64db7aaf822e2ea641ac851bb7f28b14a32618ac3311196f0121008ea7dece3c 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=nice_joliot, org.label-schema.license=GPLv2, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, RELEASE=HEAD, 
io.buildah.version=1.29.1, GIT_CLEAN=True, 
org.label-schema.vendor=CentOS, 
GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, 
org.label-schema.schema-version=1.0, 
org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=-18.2.1, 
org.label-schema.name=CentOS Stream 8 Base Image, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True)
May  7 17:06:11 pr-md-02 podman[2564878]: 2024-05-07 
17:06:11.165130923 -0700 PDT m=+0.023526448 image pull 
5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 
quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8
May  7 17:06:11 pr-md-02 podman[2564898]: 2024-05-07 
17:06:11.276198352 -0700 PDT m=+0.051426463 container remove 
64db7aaf822e2ea641ac851bb7f28b14a32618ac3311196f0121008ea7dece3c 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=nice_joliot, maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.license=GPLv2, io.buildah.version=1.29.1, 
CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, 
org.label-schema.vendor=CentOS, GIT_CLEAN=True, 
org.label-schema.name=CentOS Stream 8 Base Image, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, RELEASE=HEAD, 
ceph=True)
May  7 17:06:11 pr-md-02 systemd[1]: 
libpod-conmon-64db7aaf822e2ea641ac851bb7f28b14a32618ac3311196f0121008ea7dece3c.scope: 
Deactivated successfully.
May  7 17:06:11 pr-md-02 podman[2564919]: 2024-05-07 
17:06:11.398685928 -0700 PDT m=+0.038305972 container create 
4ec999d23a0ee13ce614ae8ad5b3fd2a3acf360affe394b2507986452079dae8 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=sleepy_roentgen, CEPH_POINT_RELEASE=-18.2.1, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, GIT_BRANCH=HEAD, 
org.label-schema.name=CentOS Stream 8 Base Image, 
org.label-schema.license=GPLv2, ceph=True, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, 
RELEASE=HEAD, GIT_CLEAN=True, io.buildah.version=1.29.1, 
org.label-schema.build-date=20240222, 
GIT_REPO=https://github.com/ceph/ceph-container.git)
May  7 17:06:11 pr-md-02 systemd[1]: Started 
libpod-conmon-4ec999d23a0ee13ce614ae8ad5b3fd2a3acf360affe394b2507986452079dae8.scope.
May  7 17:06:11 pr-md-02 systemd[1]: Started libcrun container.
May  7 17:06:11 pr-md-02 podman[2564919]: 2024-05-07 17:06:11.4284462 
-0700 PDT m=+0.068066246 container init 
4ec999d23a0ee13ce614ae8ad5b3fd2a3acf360affe394b2507986452079dae8 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=sleepy_roentgen, org.label-schema.name=CentOS Stream 8 Base 
Image, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, ceph=True, 
io.buildah.version=1.29.1, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, 
GIT_CLEAN=True, org.label-schema.build-date=20240222, 
org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, org.label-schema.vendor=CentOS, 
org.label-schema.schema-version=1.0)
May  7 17:06:11 pr-md-02 podman[2564919]: 2024-05-07 
17:06:11.436328165 -0700 PDT m=+0.075948196 container start 
4ec999d23a0ee13ce614ae8ad5b3fd2a3acf360affe394b2507986452079dae8 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=sleepy_roentgen, org.label-schema.license=GPLv2, ceph=True, 
org.label-schema.schema-version=1.0, 
org.label-schema.build-date=20240222, org.label-schema.name=CentOS 
Stream 8 Base Image, io.buildah.version=1.29.1, maintainer=Guillaume 
Abrioux <gabrioux@redhat.com>, 
GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, 
CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, 
org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd)
May  7 17:06:11 pr-md-02 podman[2564919]: 2024-05-07 
17:06:11.436459593 -0700 PDT m=+0.076079634 container attach 
4ec999d23a0ee13ce614ae8ad5b3fd2a3acf360affe394b2507986452079dae8 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=sleepy_roentgen, io.buildah.version=1.29.1, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.schema-version=1.0, GIT_CLEAN=True, 
CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, 
org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume 
Abrioux <gabrioux@redhat.com>, GIT_BRANCH=HEAD, 
org.label-schema.name=CentOS Stream 8 Base Image, 
org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, 
GIT_REPO=https://github.com/ceph/ceph-container.git)
May  7 17:06:11 pr-md-02 podman[2564919]: 2024-05-07 
17:06:11.383297135 -0700 PDT m=+0.022917157 image pull 
5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 
quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8
May  7 17:06:11 pr-md-02 ceph-mon[5348]: from='mgr.274299 
10.50.1.75:0/136806890' entity='mgr.pr-md-02.emffhz' cmd=[{"prefix": 
"auth get", "entity": "client.admin"}]: dispatch
May  7 17:06:11 pr-md-02 ceph-mon[5348]: from='mgr.274299 ' 
entity='mgr.pr-md-02.emffhz'
May  7 17:06:11 pr-md-02 ceph-mon[5348]: from='mgr.274299 
10.50.1.75:0/136806890' entity='mgr.pr-md-02.emffhz' cmd=[{"prefix": 
"auth get", "entity": "client.bootstrap-osd"}]: dispatch
May  7 17:06:11 pr-md-02 ceph-mgr[5311]: log_channel(audit) log [DBG] 
: from='client.296615 -' entity='client.admin' cmd=[{"prefix": "fs 
status", "target": ["mon-mgr", ""]}]: dispatch
May  7 17:06:12 pr-md-02 sleepy_roentgen[2564934]: --> passed data 
devices: 2 physical, 0 LVM
May  7 17:06:12 pr-md-02 sleepy_roentgen[2564934]: --> relative data 
size: 1.0
May  7 17:06:12 pr-md-02 sleepy_roentgen[2564934]: --> All data 
devices are unavailable
May  7 17:06:12 pr-md-02 systemd[1]: 
libpod-4ec999d23a0ee13ce614ae8ad5b3fd2a3acf360affe394b2507986452079dae8.scope: 
Deactivated successfully.
May  7 17:06:12 pr-md-02 podman[2564919]: 2024-05-07 
17:06:12.059347046 -0700 PDT m=+0.698967087 container died 
4ec999d23a0ee13ce614ae8ad5b3fd2a3acf360affe394b2507986452079dae8 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=sleepy_roentgen, org.label-schema.schema-version=1.0, 
GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, 
org.label-schema.license=GPLv2, ceph=True, 
org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, 
org.label-schema.build-date=20240222, 
GIT_REPO=https://github.com/ceph/ceph-container.git)
May  7 17:06:12 pr-md-02 podman[2564962]: 2024-05-07 
17:06:12.115085789 -0700 PDT m=+0.048338618 container remove 
4ec999d23a0ee13ce614ae8ad5b3fd2a3acf360affe394b2507986452079dae8 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=sleepy_roentgen, org.label-schema.build-date=20240222, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
io.buildah.version=1.29.1, org.label-schema.license=GPLv2, 
GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
CEPH_POINT_RELEASE=-18.2.1, ceph=True, 
org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, 
RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True)
May  7 17:06:12 pr-md-02 systemd[1]: 
libpod-conmon-4ec999d23a0ee13ce614ae8ad5b3fd2a3acf360affe394b2507986452079dae8.scope: 
Deactivated successfully.
May  7 17:06:12 pr-md-02 systemd[1]: 
var-lib-containers-storage-overlay-5f35e0c5e28b64ec23044b996e488378fe0b7b5bd084aec04d3a8efbfd932b2a-merged.mount: 
Deactivated successfully.
May  7 17:06:12 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:06:12 pr-md-02 podman[2565099]: 2024-05-07 
17:06:12.66977777 -0700 PDT m=+0.041271914 container create 
24dd6de0e98fc7afcbfb71e44efe052816b6c5d19d57680deab9e553722d92b1 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=strange_wescoff, CEPH_POINT_RELEASE=-18.2.1, 
org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, 
org.label-schema.license=GPLv2, GIT_CLEAN=True, 
org.label-schema.build-date=20240222, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.name=CentOS Stream 8 Base Image, 
org.label-schema.schema-version=1.0, RELEASE=HEAD, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
io.buildah.version=1.29.1, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>)
May  7 17:06:12 pr-md-02 systemd[1]: Started 
libpod-conmon-24dd6de0e98fc7afcbfb71e44efe052816b6c5d19d57680deab9e553722d92b1.scope.
May  7 17:06:12 pr-md-02 systemd[1]: Started libcrun container.
May  7 17:06:12 pr-md-02 podman[2565099]: 2024-05-07 
17:06:12.698793806 -0700 PDT m=+0.070287951 container init 
24dd6de0e98fc7afcbfb71e44efe052816b6c5d19d57680deab9e553722d92b1 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=strange_wescoff, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, ceph=True, GIT_BRANCH=HEAD, 
org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, 
org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, 
GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1)
May  7 17:06:12 pr-md-02 strange_wescoff[2565115]: 167 167
May  7 17:06:12 pr-md-02 systemd[1]: 
libpod-24dd6de0e98fc7afcbfb71e44efe052816b6c5d19d57680deab9e553722d92b1.scope: 
Deactivated successfully.
May  7 17:06:12 pr-md-02 podman[2565099]: 2024-05-07 
17:06:12.705992776 -0700 PDT m=+0.077486921 container start 
24dd6de0e98fc7afcbfb71e44efe052816b6c5d19d57680deab9e553722d92b1 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=strange_wescoff, CEPH_POINT_RELEASE=-18.2.1, 
org.label-schema.build-date=20240222, org.label-schema.name=CentOS 
Stream 8 Base Image, ceph=True, RELEASE=HEAD, GIT_CLEAN=True, 
io.buildah.version=1.29.1, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>)
May  7 17:06:12 pr-md-02 podman[2565099]: 2024-05-07 
17:06:12.706102938 -0700 PDT m=+0.077597093 container attach 
24dd6de0e98fc7afcbfb71e44efe052816b6c5d19d57680deab9e553722d92b1 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=strange_wescoff, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.license=GPLv2, ceph=True, 
org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, 
GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, 
org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD)
May  7 17:06:12 pr-md-02 podman[2565099]: 2024-05-07 
17:06:12.706421977 -0700 PDT m=+0.077916136 container died 
24dd6de0e98fc7afcbfb71e44efe052816b6c5d19d57680deab9e553722d92b1 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=strange_wescoff, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, 
io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base 
Image, ceph=True, RELEASE=HEAD, GIT_BRANCH=HEAD, 
org.label-schema.schema-version=1.0, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, 
org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS)
May  7 17:06:12 pr-md-02 podman[2565099]: 2024-05-07 
17:06:12.65486152 -0700 PDT m=+0.026355674 image pull 
5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 
quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8
May  7 17:06:12 pr-md-02 podman[2565120]: 2024-05-07 
17:06:12.760218425 -0700 PDT m=+0.047515050 container remove 
24dd6de0e98fc7afcbfb71e44efe052816b6c5d19d57680deab9e553722d92b1 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=strange_wescoff, org.label-schema.vendor=CentOS, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, 
io.buildah.version=1.29.1, org.label-schema.build-date=20240222, 
GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, 
GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image)
May  7 17:06:12 pr-md-02 systemd[1]: 
libpod-conmon-24dd6de0e98fc7afcbfb71e44efe052816b6c5d19d57680deab9e553722d92b1.scope: 
Deactivated successfully.
May  7 17:06:12 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225250: 1313 pgs: 5 active+clean+scrubbing+deep, 17 
active+clean+scrubbing, 1291 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:12 pr-md-02 podman[2565141]: 2024-05-07 
17:06:12.883756999 -0700 PDT m=+0.038722013 container create 
ff87d15729739f66dfd552fbf0c2add7e0b158e09eba8b15660c6b9946c40ac4 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=affectionate_saha, org.label-schema.name=CentOS Stream 8 Base 
Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, 
RELEASE=HEAD, io.buildah.version=1.29.1, GIT_CLEAN=True, 
org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, 
GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, 
org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1)
May  7 17:06:12 pr-md-02 systemd[1]: Started 
libpod-conmon-ff87d15729739f66dfd552fbf0c2add7e0b158e09eba8b15660c6b9946c40ac4.scope.
May  7 17:06:12 pr-md-02 systemd[1]: Started libcrun container.
May  7 17:06:12 pr-md-02 podman[2565141]: 2024-05-07 
17:06:12.915479032 -0700 PDT m=+0.070443976 container init 
ff87d15729739f66dfd552fbf0c2add7e0b158e09eba8b15660c6b9946c40ac4 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=affectionate_saha, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, org.label-schema.vendor=CentOS, 
org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, 
RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, 
io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, 
ceph=True, GIT_CLEAN=True, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
CEPH_POINT_RELEASE=-18.2.1, org.label-schema.name=CentOS Stream 8 
Base Image)
May  7 17:06:12 pr-md-02 podman[2565141]: 2024-05-07 
17:06:12.922714281 -0700 PDT m=+0.077679227 container start 
ff87d15729739f66dfd552fbf0c2add7e0b158e09eba8b15660c6b9946c40ac4 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=affectionate_saha, org.label-schema.schema-version=1.0, 
GIT_CLEAN=True, org.label-schema.build-date=20240222, 
GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, 
io.buildah.version=1.29.1, org.label-schema.license=GPLv2, 
org.label-schema.name=CentOS Stream 8 Base Image, 
CEPH_POINT_RELEASE=-18.2.1, org.label-schema.vendor=CentOS, 
RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd)
May  7 17:06:12 pr-md-02 podman[2565141]: 2024-05-07 
17:06:12.922813112 -0700 PDT m=+0.077778070 container attach 
ff87d15729739f66dfd552fbf0c2add7e0b158e09eba8b15660c6b9946c40ac4 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=affectionate_saha, org.label-schema.schema-version=1.0, 
GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, 
RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, 
org.label-schema.vendor=CentOS, GIT_CLEAN=True, 
io.buildah.version=1.29.1, org.label-schema.build-date=20240222, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
CEPH_POINT_RELEASE=-18.2.1, ceph=True, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, org.label-schema.license=GPLv2)
May  7 17:06:12 pr-md-02 podman[2565141]: 2024-05-07 
17:06:12.867928578 -0700 PDT m=+0.022893523 image pull 
5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 
quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8
May  7 17:06:13 pr-md-02 systemd[1]: 
var-lib-containers-storage-overlay-dbb003174f50c19fbf330fa5c795cae4a0f3b395ec57674376442616981290d1-merged.mount: 
Deactivated successfully.
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: {
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]:    "41": [
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]:        {
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "devices": [
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "/dev/nvme0n1"
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: ],
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "lv_name": 
"osd-block-88070adf-ee15-406a-b8e9-98a61e294c8b",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "lv_path": 
"/dev/ceph-eac1ebc0-c00f-4558-8c72-741fc8f81abc/osd-block-88070adf-ee15-406a-b8e9-98a61e294c8b",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "lv_size": 
"1920378863616",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "lv_tags": 
"ceph.block_device=/dev/ceph-eac1ebc0-c00f-4558-8c72-741fc8f81abc/osd-block-88070adf-ee15-406a-b8e9-98a61e294c8b,ceph.block_uuid=MwSZ0h-81BS-XH33-wWY4-vk23-2TIJ-DW9XgO,ceph.cephx_lockbox_secret=AQBJFuJlxpZUJBAAHM4wPn5k5ExaR9mY0pS+YA==,ceph.cluster_fsid=58bde08a-d7ed-11ee-9098-506b4b4da440,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=1,ceph.osd_fsid=88070adf-ee15-406a-b8e9-98a61e294c8b,ceph.osd_id=41,ceph.osdspec_affinity=md_encrypted_nvme_disks,ceph.type=block,ceph.vdo=0",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "lv_uuid": 
"MwSZ0h-81BS-XH33-wWY4-vk23-2TIJ-DW9XgO",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "name": 
"osd-block-88070adf-ee15-406a-b8e9-98a61e294c8b",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "path": 
"/dev/ceph-eac1ebc0-c00f-4558-8c72-741fc8f81abc/osd-block-88070adf-ee15-406a-b8e9-98a61e294c8b",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "tags": {
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: 
"ceph.block_device": 
"/dev/ceph-eac1ebc0-c00f-4558-8c72-741fc8f81abc/osd-block-88070adf-ee15-406a-b8e9-98a61e294c8b",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: 
"ceph.block_uuid": "MwSZ0h-81BS-XH33-wWY4-vk23-2TIJ-DW9XgO",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: 
"ceph.cephx_lockbox_secret": "AQBJFuJlxpZUJBAAHM4wPn5k5ExaR9mY0pS+YA==",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: 
"ceph.cluster_fsid": "58bde08a-d7ed-11ee-9098-506b4b4da440",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: 
"ceph.cluster_name": "ceph",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: 
"ceph.crush_device_class": "",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: 
"ceph.encrypted": "1",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "ceph.osd_fsid": 
"88070adf-ee15-406a-b8e9-98a61e294c8b",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "ceph.osd_id": 
"41",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: 
"ceph.osdspec_affinity": "md_encrypted_nvme_disks",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "ceph.type": 
"block",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "ceph.vdo": "0"
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: },
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "type": "block",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "vg_name": 
"ceph-eac1ebc0-c00f-4558-8c72-741fc8f81abc"
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]:        }
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]:    ],
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]:    "43": [
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]:        {
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "devices": [
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "/dev/nvme1n1"
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: ],
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "lv_name": 
"osd-block-50a79ade-482b-4c7e-84ae-0dac068ed570",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "lv_path": 
"/dev/ceph-0b5aeb1a-7dea-4260-9af9-f2bff16d642a/osd-block-50a79ade-482b-4c7e-84ae-0dac068ed570",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "lv_size": 
"1920378863616",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "lv_tags": 
"ceph.block_device=/dev/ceph-0b5aeb1a-7dea-4260-9af9-f2bff16d642a/osd-block-50a79ade-482b-4c7e-84ae-0dac068ed570,ceph.block_uuid=zN3gTU-nGwQ-DfIA-dzWZ-e7QI-OcmL-SZcgqT,ceph.cephx_lockbox_secret=AQBYFuJlTtj/DBAAt2wy+qfx2XCHFuwJZKwp6A==,ceph.cluster_fsid=58bde08a-d7ed-11ee-9098-506b4b4da440,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=1,ceph.osd_fsid=50a79ade-482b-4c7e-84ae-0dac068ed570,ceph.osd_id=43,ceph.osdspec_affinity=md_encrypted_nvme_disks,ceph.type=block,ceph.vdo=0",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "lv_uuid": 
"zN3gTU-nGwQ-DfIA-dzWZ-e7QI-OcmL-SZcgqT",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "name": 
"osd-block-50a79ade-482b-4c7e-84ae-0dac068ed570",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "path": 
"/dev/ceph-0b5aeb1a-7dea-4260-9af9-f2bff16d642a/osd-block-50a79ade-482b-4c7e-84ae-0dac068ed570",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "tags": {
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: 
"ceph.block_device": 
"/dev/ceph-0b5aeb1a-7dea-4260-9af9-f2bff16d642a/osd-block-50a79ade-482b-4c7e-84ae-0dac068ed570",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: 
"ceph.block_uuid": "zN3gTU-nGwQ-DfIA-dzWZ-e7QI-OcmL-SZcgqT",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: 
"ceph.cephx_lockbox_secret": "AQBYFuJlTtj/DBAAt2wy+qfx2XCHFuwJZKwp6A==",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: 
"ceph.cluster_fsid": "58bde08a-d7ed-11ee-9098-506b4b4da440",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: 
"ceph.cluster_name": "ceph",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: 
"ceph.crush_device_class": "",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: 
"ceph.encrypted": "1",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "ceph.osd_fsid": 
"50a79ade-482b-4c7e-84ae-0dac068ed570",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "ceph.osd_id": 
"43",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: 
"ceph.osdspec_affinity": "md_encrypted_nvme_disks",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "ceph.type": 
"block",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "ceph.vdo": "0"
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: },
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "type": "block",
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: "vg_name": 
"ceph-0b5aeb1a-7dea-4260-9af9-f2bff16d642a"
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]:        }
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]:    ]
May  7 17:06:13 pr-md-02 affectionate_saha[2565157]: }
May  7 17:06:13 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:06:13 pr-md-02 systemd[1]: 
libpod-ff87d15729739f66dfd552fbf0c2add7e0b158e09eba8b15660c6b9946c40ac4.scope: 
Deactivated successfully.
May  7 17:06:13 pr-md-02 podman[2565141]: 2024-05-07 
17:06:13.298397993 -0700 PDT m=+0.453362947 container died 
ff87d15729739f66dfd552fbf0c2add7e0b158e09eba8b15660c6b9946c40ac4 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=affectionate_saha, org.label-schema.build-date=20240222, 
org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, 
org.label-schema.schema-version=1.0, RELEASE=HEAD, 
org.label-schema.name=CentOS Stream 8 Base Image, 
GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, 
io.buildah.version=1.29.1, GIT_CLEAN=True, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, maintainer=Guillaume 
Abrioux <gabrioux@redhat.com>)
May  7 17:06:13 pr-md-02 systemd[1]: 
var-lib-containers-storage-overlay-8550ac4bd2edf3f53d2004d34dad39cc51be76947b4935e496819469f7103f92-merged.mount: 
Deactivated successfully.
May  7 17:06:13 pr-md-02 podman[2565169]: 2024-05-07 
17:06:13.360359344 -0700 PDT m=+0.052612579 container remove 
ff87d15729739f66dfd552fbf0c2add7e0b158e09eba8b15660c6b9946c40ac4 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=affectionate_saha, org.label-schema.schema-version=1.0, 
GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.name=CentOS Stream 8 Base Image, 
CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, 
org.label-schema.license=GPLv2, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.vendor=CentOS, RELEASE=HEAD, ceph=True)
May  7 17:06:13 pr-md-02 systemd[1]: 
libpod-conmon-ff87d15729739f66dfd552fbf0c2add7e0b158e09eba8b15660c6b9946c40ac4.scope: 
Deactivated successfully.
May  7 17:06:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler INFO root] 
_maybe_adjust
May  7 17:06:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool cephfs_metadata won't scale due to overlapping roots: {-15, -1}
May  7 17:06:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
Please See: 
https://docs.ceph.com/en/latest/rados/operations/placement-groups/#automated...
May  7 17:06:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool cephfs_data won't scale due to overlapping roots: {-15, -1, -2}
May  7 17:06:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
Please See: 
https://docs.ceph.com/en/latest/rados/operations/placement-groups/#automated...
May  7 17:06:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool cephfs_md_and_data won't scale due to overlapping roots: {-15, 
-1, -2}
May  7 17:06:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
Please See: 
https://docs.ceph.com/en/latest/rados/operations/placement-groups/#automated...
May  7 17:06:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool 1 contains an overlapping root -1... skipping scaling
May  7 17:06:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool 7 contains an overlapping root -15... skipping scaling
May  7 17:06:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool 8 contains an overlapping root -2... skipping scaling
May  7 17:06:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool 9 contains an overlapping root -2... skipping scaling
May  7 17:06:13 pr-md-02 podman[2565305]: 2024-05-07 
17:06:13.944018291 -0700 PDT m=+0.043243095 container create 
b1570bd0cfaf1fa55ec97cd999849d15eb4c1a6b693696e342bb5598d5f12cd6 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=nifty_knuth, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
io.buildah.version=1.29.1, org.label-schema.build-date=20240222, 
ceph=True, org.label-schema.schema-version=1.0, 
CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, maintainer=Guillaume 
Abrioux <gabrioux@redhat.com>, RELEASE=HEAD, 
org.label-schema.name=CentOS Stream 8 Base Image, 
org.label-schema.vendor=CentOS)
May  7 17:06:13 pr-md-02 systemd[1]: Started 
libpod-conmon-b1570bd0cfaf1fa55ec97cd999849d15eb4c1a6b693696e342bb5598d5f12cd6.scope.
May  7 17:06:13 pr-md-02 systemd[1]: Started libcrun container.
May  7 17:06:13 pr-md-02 podman[2565305]: 2024-05-07 
17:06:13.972707239 -0700 PDT m=+0.071931967 container init 
b1570bd0cfaf1fa55ec97cd999849d15eb4c1a6b693696e342bb5598d5f12cd6 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=nifty_knuth, org.label-schema.build-date=20240222, ceph=True, 
org.label-schema.license=GPLv2, GIT_CLEAN=True, 
CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, 
io.buildah.version=1.29.1, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.schema-version=1.0, org.label-schema.name=CentOS 
Stream 8 Base Image, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, RELEASE=HEAD, org.label-schema.vendor=CentOS)
May  7 17:06:13 pr-md-02 nifty_knuth[2565321]: 167 167
May  7 17:06:13 pr-md-02 systemd[1]: 
libpod-b1570bd0cfaf1fa55ec97cd999849d15eb4c1a6b693696e342bb5598d5f12cd6.scope: 
Deactivated successfully.
May  7 17:06:13 pr-md-02 podman[2565305]: 2024-05-07 
17:06:13.979610545 -0700 PDT m=+0.078835287 container start 
b1570bd0cfaf1fa55ec97cd999849d15eb4c1a6b693696e342bb5598d5f12cd6 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=nifty_knuth, org.label-schema.schema-version=1.0, ceph=True, 
org.label-schema.license=GPLv2, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.build-date=20240222, io.buildah.version=1.29.1, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, 
RELEASE=HEAD, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 
Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, 
GIT_CLEAN=True)
May  7 17:06:13 pr-md-02 podman[2565305]: 2024-05-07 
17:06:13.979741103 -0700 PDT m=+0.078965840 container attach 
b1570bd0cfaf1fa55ec97cd999849d15eb4c1a6b693696e342bb5598d5f12cd6 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=nifty_knuth, org.label-schema.name=CentOS Stream 8 Base Image, 
org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, 
CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, io.buildah.version=1.29.1, 
org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, 
GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD)
May  7 17:06:13 pr-md-02 podman[2565305]: 2024-05-07 
17:06:13.979972502 -0700 PDT m=+0.079197239 container died 
b1570bd0cfaf1fa55ec97cd999849d15eb4c1a6b693696e342bb5598d5f12cd6 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=nifty_knuth, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, 
GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, 
RELEASE=HEAD, maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.name=CentOS Stream 8 Base Image, 
org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, 
io.buildah.version=1.29.1, ceph=True, GIT_BRANCH=HEAD, 
org.label-schema.vendor=CentOS)
May  7 17:06:14 pr-md-02 podman[2565305]: 2024-05-07 
17:06:13.927504129 -0700 PDT m=+0.026728865 image pull 
5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 
quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8
May  7 17:06:14 pr-md-02 podman[2565326]: 2024-05-07 
17:06:14.033457546 -0700 PDT m=+0.047525301 container remove 
b1570bd0cfaf1fa55ec97cd999849d15eb4c1a6b693696e342bb5598d5f12cd6 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=nifty_knuth, io.buildah.version=1.29.1, 
org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, 
ceph=True, org.label-schema.vendor=CentOS, 
GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, 
org.label-schema.build-date=20240222)
May  7 17:06:14 pr-md-02 systemd[1]: 
libpod-conmon-b1570bd0cfaf1fa55ec97cd999849d15eb4c1a6b693696e342bb5598d5f12cd6.scope: 
Deactivated successfully.
May  7 17:06:14 pr-md-02 podman[2565347]: 2024-05-07 
17:06:14.146017051 -0700 PDT m=+0.040230170 container create 
0dd718175492b3950e94f88f74b359bfb5caee9547a97927df1c979b5c46289b 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=flamboyant_poincare, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, org.label-schema.schema-version=1.0, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.name=CentOS Stream 8 Base Image, 
io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, 
org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, 
ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, 
RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, 
GIT_CLEAN=True)
May  7 17:06:14 pr-md-02 systemd[1]: Started 
libpod-conmon-0dd718175492b3950e94f88f74b359bfb5caee9547a97927df1c979b5c46289b.scope.
May  7 17:06:14 pr-md-02 systemd[1]: Started libcrun container.
May  7 17:06:14 pr-md-02 podman[2565347]: 2024-05-07 
17:06:14.175282354 -0700 PDT m=+0.069495491 container init 
0dd718175492b3950e94f88f74b359bfb5caee9547a97927df1c979b5c46289b 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=flamboyant_poincare, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, RELEASE=HEAD, io.buildah.version=1.29.1, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.name=CentOS Stream 8 Base Image, 
GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, 
org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, 
GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240222, 
org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS)
May  7 17:06:14 pr-md-02 podman[2565347]: 2024-05-07 
17:06:14.182850934 -0700 PDT m=+0.077064054 container start 
0dd718175492b3950e94f88f74b359bfb5caee9547a97927df1c979b5c46289b 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=flamboyant_poincare, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, 
org.label-schema.name=CentOS Stream 8 Base Image, 
org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, GIT_BRANCH=HEAD, 
org.label-schema.schema-version=1.0, 
org.label-schema.build-date=20240222, RELEASE=HEAD, 
io.buildah.version=1.29.1, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.license=GPLv2, ceph=True)
May  7 17:06:14 pr-md-02 podman[2565347]: 2024-05-07 
17:06:14.182936358 -0700 PDT m=+0.077149475 container attach 
0dd718175492b3950e94f88f74b359bfb5caee9547a97927df1c979b5c46289b 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=flamboyant_poincare, org.label-schema.schema-version=1.0, 
CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, 
org.label-schema.build-date=20240222, 
GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, 
RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, 
org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, ceph=True, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.license=GPLv2)
May  7 17:06:14 pr-md-02 systemd[1]: 
var-lib-containers-storage-overlay-c68f47e6650f22ede040907bfad1eaa0c96a7a2a19ef648cb1312a9ceed9a65b-merged.mount: 
Deactivated successfully.
May  7 17:06:14 pr-md-02 podman[2565347]: 2024-05-07 
17:06:14.129225093 -0700 PDT m=+0.023438214 image pull 
5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 
quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8
May  7 17:06:14 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225251: 1313 pgs: 5 active+clean+scrubbing+deep, 17 
active+clean+scrubbing, 1291 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:15 pr-md-02 flamboyant_poincare[2565363]: {
May  7 17:06:15 pr-md-02 flamboyant_poincare[2565363]: 
"50a79ade-482b-4c7e-84ae-0dac068ed570": {
May  7 17:06:15 pr-md-02 flamboyant_poincare[2565363]: "ceph_fsid": 
"58bde08a-d7ed-11ee-9098-506b4b4da440",
May  7 17:06:15 pr-md-02 flamboyant_poincare[2565363]: "device": 
"/dev/mapper/zN3gTU-nGwQ-DfIA-dzWZ-e7QI-OcmL-SZcgqT",
May  7 17:06:15 pr-md-02 flamboyant_poincare[2565363]: "osd_id": 43,
May  7 17:06:15 pr-md-02 flamboyant_poincare[2565363]: "osd_uuid": 
"50a79ade-482b-4c7e-84ae-0dac068ed570",
May  7 17:06:15 pr-md-02 flamboyant_poincare[2565363]: "type": 
"bluestore"
May  7 17:06:15 pr-md-02 flamboyant_poincare[2565363]:    },
May  7 17:06:15 pr-md-02 flamboyant_poincare[2565363]: 
"88070adf-ee15-406a-b8e9-98a61e294c8b": {
May  7 17:06:15 pr-md-02 flamboyant_poincare[2565363]: "ceph_fsid": 
"58bde08a-d7ed-11ee-9098-506b4b4da440",
May  7 17:06:15 pr-md-02 flamboyant_poincare[2565363]: "device": 
"/dev/mapper/MwSZ0h-81BS-XH33-wWY4-vk23-2TIJ-DW9XgO",
May  7 17:06:15 pr-md-02 flamboyant_poincare[2565363]: "osd_id": 41,
May  7 17:06:15 pr-md-02 flamboyant_poincare[2565363]: "osd_uuid": 
"88070adf-ee15-406a-b8e9-98a61e294c8b",
May  7 17:06:15 pr-md-02 flamboyant_poincare[2565363]: "type": 
"bluestore"
May  7 17:06:15 pr-md-02 flamboyant_poincare[2565363]:    }
May  7 17:06:15 pr-md-02 flamboyant_poincare[2565363]: }
May  7 17:06:15 pr-md-02 systemd[1]: 
libpod-0dd718175492b3950e94f88f74b359bfb5caee9547a97927df1c979b5c46289b.scope: 
Deactivated successfully.
May  7 17:06:15 pr-md-02 podman[2565347]: 2024-05-07 
17:06:15.160506671 -0700 PDT m=+1.054719792 container died 
0dd718175492b3950e94f88f74b359bfb5caee9547a97927df1c979b5c46289b 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=flamboyant_poincare, RELEASE=HEAD, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
io.buildah.version=1.29.1, ceph=True, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.schema-version=1.0, 
org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, 
GIT_CLEAN=True, org.label-schema.vendor=CentOS, 
CEPH_POINT_RELEASE=-18.2.1, org.label-schema.name=CentOS Stream 8 
Base Image, GIT_BRANCH=HEAD)
May  7 17:06:15 pr-md-02 systemd[1]: 
var-lib-containers-storage-overlay-b053c5cb16b427d8a3ee8d426e731633a8d8b6337b3c37ef50f558eadad5bd38-merged.mount: 
Deactivated successfully.
May  7 17:06:15 pr-md-02 podman[2565444]: 2024-05-07 
17:06:15.220598278 -0700 PDT m=+0.052865840 container remove 
0dd718175492b3950e94f88f74b359bfb5caee9547a97927df1c979b5c46289b 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=flamboyant_poincare, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, 
org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, 
CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, org.label-schema.license=GPLv2, 
org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, 
io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, 
org.label-schema.build-date=20240222)
May  7 17:06:15 pr-md-02 systemd[1]: 
libpod-conmon-0dd718175492b3950e94f88f74b359bfb5caee9547a97927df1c979b5c46289b.scope: 
Deactivated successfully.
May  7 17:06:15 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev 0ef17f53-4942-4371-9efc-a1a6b11d0994 does not exist
May  7 17:06:15 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev 8e261f50-3ae9-48c5-9371-cad82ccaaa9b does not exist
May  7 17:06:15 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev 54b062b8-01ba-45e7-8809-24bb3b472492 does not exist
May  7 17:06:15 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev 953528e8-a621-4cb5-9c5d-ed9bc75deada does not exist
May  7 17:06:15 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev 1c93f903-df81-4e2a-be93-75dc1a37dfb1 does not exist
May  7 17:06:15 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev bcd9783c-8512-42cf-b704-cce9f827345e does not exist
May  7 17:06:15 pr-md-02 ceph-mon[5348]: from='mgr.274299 ' 
entity='mgr.pr-md-02.emffhz'
May  7 17:06:15 pr-md-02 ceph-mon[5348]: from='mgr.274299 ' 
entity='mgr.pr-md-02.emffhz'
May  7 17:06:15 pr-md-02 ceph-mgr[5311]: log_channel(audit) log [DBG] 
: from='client.296625 -' entity='client.admin' cmd=[{"prefix": "fs 
status", "target": ["mon-mgr", ""]}]: dispatch
May  7 17:06:16 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:06:16] "GET /metrics HTTP/1.1" 
200 73532 "" "Prometheus/2.43.0"
May  7 17:06:16 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225252: 1313 pgs: 5 active+clean+scrubbing+deep, 17 
active+clean+scrubbing, 1291 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:17 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:06:18 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:06:18 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225253: 1313 pgs: 6 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:20 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225254: 1313 pgs: 6 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:22 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:06:22 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225255: 1313 pgs: 7 active+clean+scrubbing+deep, 13 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:23 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:06:24 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:06:24 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
scanning for idle connections..
May  7 17:06:24 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
cleaning up connections: []
May  7 17:06:24 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
scanning for idle connections..
May  7 17:06:24 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
cleaning up connections: []
May  7 17:06:24 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
scanning for idle connections..
May  7 17:06:24 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
cleaning up connections: []
May  7 17:06:24 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225256: 1313 pgs: 7 active+clean+scrubbing+deep, 13 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:26 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:06:26] "GET /metrics HTTP/1.1" 
200 73533 "" "Prometheus/2.43.0"
May  7 17:06:26 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225257: 1313 pgs: 7 active+clean+scrubbing+deep, 13 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:27 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:06:28 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:06:28 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225258: 1313 pgs: 8 active+clean+scrubbing+deep, 13 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:30 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225259: 1313 pgs: 8 active+clean+scrubbing+deep, 13 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:32 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:06:32 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225260: 1313 pgs: 7 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:33 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:06:34 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225261: 1313 pgs: 7 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:36 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:06:36] "GET /metrics HTTP/1.1" 
200 73533 "" "Prometheus/2.43.0"
May  7 17:06:36 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225262: 1313 pgs: 7 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:37 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:06:38 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:06:38 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225263: 1313 pgs: 6 active+clean+scrubbing+deep, 13 
active+clean+scrubbing, 1294 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:39 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:06:40 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225264: 1313 pgs: 6 active+clean+scrubbing+deep, 13 
active+clean+scrubbing, 1294 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:42 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:06:42 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225265: 1313 pgs: 6 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:43 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:06:44 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225266: 1313 pgs: 6 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:46 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:06:46] "GET /metrics HTTP/1.1" 
200 73531 "" "Prometheus/2.43.0"
May  7 17:06:46 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225267: 1313 pgs: 6 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:47 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:06:48 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:06:48 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225268: 1313 pgs: 6 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:50 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225269: 1313 pgs: 6 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:52 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:06:52 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225270: 1313 pgs: 5 active+clean+scrubbing+deep, 16 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:53 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:06:54 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:06:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
scanning for idle connections..
May  7 17:06:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
cleaning up connections: []
May  7 17:06:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
scanning for idle connections..
May  7 17:06:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
cleaning up connections: []
May  7 17:06:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
scanning for idle connections..
May  7 17:06:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
cleaning up connections: []
May  7 17:06:54 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225271: 1313 pgs: 5 active+clean+scrubbing+deep, 16 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:56 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:06:56] "GET /metrics HTTP/1.1" 
200 73533 "" "Prometheus/2.43.0"
May  7 17:06:56 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225272: 1313 pgs: 5 active+clean+scrubbing+deep, 16 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:57 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:06:58 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:06:58 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225273: 1313 pgs: 7 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: 
[db/db_impl/db_impl_write.cc:2098] [default] New memtable created 
with log file: #228438. Immutable memtables: 0.
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:06:59.708981) 
[db/db_impl/db_impl_compaction_flush.cc:2832] Calling 
FlushMemTableToOutputFile with column family [default], flush slots 
available 1, compaction slots available 1, flush slots scheduled 1, 
compaction slots scheduled 0
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: 
[db/flush_job.cc:856] [default] [JOB 12100] Flushing memtable with 
next log file: 228438
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: EVENT_LOG_v1 
{"time_micros": 1715126819709068, "job": 12100, "event": 
"flush_started", "num_memtables": 1, "num_entries": 3587, 
"num_deletes": 921, "total_data_size": 2910092, "memory_usage": 
2992096, "flush_reason": "Manual Compaction"}
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: 
[db/flush_job.cc:885] [default] [JOB 12100] Level-0 flush table 
#228439: started
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: EVENT_LOG_v1 
{"time_micros": 1715126819721834, "cf_name": "default", "job": 12100, 
"event": "table_file_creation", "file_number": 228439, "file_size": 
1847607, "file_checksum": "", "file_checksum_func_name": "Unknown", 
"smallest_seqno": 141701983, "largest_seqno": 141705565, 
"table_properties": {"data_size": 1834448, "index_size": 6860, 
"index_partitions": 0, "top_level_index_size": 0, 
"index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, 
"filter_size": 5253, "raw_key_size": 55412, "raw_average_key_size": 
26, "raw_value_size": 1798552, "raw_average_value_size": 857, 
"num_data_blocks": 265, "num_entries": 2097, "num_filter_entries": 
2097, "num_deletions": 921, "num_merge_operands": 0, 
"num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, 
"filter_policy": "bloomfilter", "column_family_name": "default", 
"column_family_id": 0, "comparator": "leveldb.BytewiseComparator", 
"merge_operator": "", "prefix_extractor_name": "nullptr", 
"property_collectors": "[]", "compression": "NoCompression", 
"compression_options": "window_bits=-14; level=32767; strategy=0; 
max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; 
max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 
1715126677, "oldest_key_time": 1715126677, "file_creation_time": 
1715126819, "slow_compression_estimated_data_size": 0, 
"fast_compression_estimated_data_size": 0, "db_id": 
"9cddaea3-96ea-4679-bc89-ab4fe1d01ddc", "db_session_id": 
"LVB8G4XNY4SWBL1V508Z", "orig_file_number": 228439, 
"seqno_to_time_mapping": "N/A"}}
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: 
[db/flush_job.cc:1019] [default] [JOB 12100] Flush lasted 12882 
microseconds, and 5541 cpu microseconds.
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:06:59.721870) [db/flush_job.cc:967] [default] [JOB 
12100] Level-0 flush table #228439: 1847607 bytes OK
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:06:59.721889) [db/memtable_list.cc:519] [default] 
Level-0 commit table #228439 started
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:06:59.724483) [db/memtable_list.cc:722] [default] 
Level-0 commit table #228439: memtable #1 done
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:06:59.724523) EVENT_LOG_v1 {"time_micros": 
1715126819724510, "job": 12100, "event": "flush_finished", 
"output_compression": "NoCompression", "lsm_state": [1, 0, 0, 0, 0, 
0, 1], "immutable_memtables": 0}
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:06:59.724561) 
[db/db_impl/db_impl_compaction_flush.cc:299] [default] Level summary: 
base level 6 level multiplier 10.00 max bytes base 268435456 files[1 
0 0 0 0 0 1] max score 0.25
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: 
[db/db_impl/db_impl_files.cc:463] [JOB 12100] Try to delete WAL files 
size 2890782, prev total WAL file size 2890782, number of live WAL 
files 2.
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: 
[file/delete_scheduler.cc:74] Deleted file 
/var/lib/ceph/mon/ceph-pr-md-02/store.db/228435.log immediately, 
rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:06:59.725271) 
[db/db_impl/db_impl_compaction_flush.cc:3165] [default] Manual 
compaction from level-0 to level-6 from 
'7061786F73003130313337383930' seq:72057594037927935, type:22 .. 
'7061786F73003130313338313432' seq:0, type:0; will stop at (end)
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: 
[db/compaction/compaction_job.cc:1995] [default] [JOB 12101] 
Compacting 1@0 + 1@6 files to L6, score -1.00
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: 
[db/compaction/compaction_job.cc:2001] [default]: Compaction start 
summary: Base version 12100 Base level 0, inputs: [228439(1804KB)], 
[228437(36MB)]
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: EVENT_LOG_v1 
{"time_micros": 1715126819725328, "job": 12101, "event": 
"compaction_started", "compaction_reason": "ManualCompaction", 
"files_L0": [228439], "files_L6": [228437], "score": -1, 
"input_data_size": 39995951, "oldest_snapshot_seqno": -1}
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: 
[db/compaction/compaction_job.cc:1588] [default] [JOB 12101] 
Generated table #228440: 25379 keys, 38251292 bytes, temperature: 
kUnknown
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: EVENT_LOG_v1 
{"time_micros": 1715126819875843, "cf_name": "default", "job": 12101, 
"event": "table_file_creation", "file_number": 228440, "file_size": 
38251292, "file_checksum": "", "file_checksum_func_name": "Unknown", 
"smallest_seqno": 0, "largest_seqno": 0, "table_properties": 
{"data_size": 38096937, "index_size": 89812, "index_partitions": 0, 
"top_level_index_size": 0, "index_key_is_user_key": 1, 
"index_value_is_delta_encoded": 1, "filter_size": 63493, 
"raw_key_size": 702456, "raw_average_key_size": 27, "raw_value_size": 
37648320, "raw_average_value_size": 1483, "num_data_blocks": 3163, 
"num_entries": 25379, "num_filter_entries": 25379, "num_deletions": 
0, "num_merge_operands": 0, "num_range_deletions": 0, 
"format_version": 0, "fixed_key_len": 0, "filter_policy": 
"bloomfilter", "column_family_name": "default", "column_family_id": 
0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", 
"prefix_extractor_name": "nullptr", "property_collectors": "[]", 
"compression": "NoCompression", "compression_options": 
"window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; 
zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; 
use_zstd_dict_trainer=1; ", "creation_time": 1709312729, 
"oldest_key_time": 0, "file_creation_time": 1715126819, 
"slow_compression_estimated_data_size": 0, 
"fast_compression_estimated_data_size": 0, "db_id": 
"9cddaea3-96ea-4679-bc89-ab4fe1d01ddc", "db_session_id": 
"LVB8G4XNY4SWBL1V508Z", "orig_file_number": 228440, 
"seqno_to_time_mapping": "N/A"}}
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:06:59.876095) [db/compaction/compaction_job.cc:1663] 
[default] [JOB 12101] Compacted 1@0 + 1@6 files to L6 => 38251292 bytes
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:06:59.878629) [db/compaction/compaction_job.cc:865] 
[default] compacted to: base level 6 level multiplier 10.00 max bytes 
base 268435456 files[0 0 0 0 0 0 1] max score 0.00, MB/sec: 265.6 rd, 
254.0 wr, level 6, files in(1, 1) out(1 +0 blob) MB in(1.8, 36.4 +0.0 
blob) out(36.5 +0.0 blob), read-write-amplify(42.4) 
write-amplify(20.7) OK, records in: 27250, records dropped: 1871 
output_compression: NoCompression
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:06:59.878669) EVENT_LOG_v1 {"time_micros": 
1715126819878658, "job": 12101, "event": "compaction_finished", 
"compaction_time_micros": 150577, "compaction_time_cpu_micros": 
64114, "output_level": 6, "num_output_files": 1, "total_output_size": 
38251292, "num_input_records": 27250, "num_output_records": 25379, 
"num_subcompactions": 1, "output_compression": "NoCompression", 
"num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 
0, "lsm_state": [0, 0, 0, 0, 0, 0, 1]}
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: 
[file/delete_scheduler.cc:74] Deleted file 
/var/lib/ceph/mon/ceph-pr-md-02/store.db/228439.sst immediately, 
rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: EVENT_LOG_v1 
{"time_micros": 1715126819878937, "job": 12101, "event": 
"table_file_deletion", "file_number": 228439}
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: 
[file/delete_scheduler.cc:74] Deleted file 
/var/lib/ceph/mon/ceph-pr-md-02/store.db/228437.sst immediately, 
rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: EVENT_LOG_v1 
{"time_micros": 1715126819882652, "job": 12101, "event": 
"table_file_deletion", "file_number": 228437}
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:06:59.725225) 
[db/db_impl/db_impl_compaction_flush.cc:1903] [default] Manual 
compaction starting
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:06:59.882737) 
[db/db_impl/db_impl_compaction_flush.cc:1903] [default] Manual 
compaction starting
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:06:59.882741) 
[db/db_impl/db_impl_compaction_flush.cc:1903] [default] Manual 
compaction starting
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:06:59.882743) 
[db/db_impl/db_impl_compaction_flush.cc:1903] [default] Manual 
compaction starting
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:06:59.882744) 
[db/db_impl/db_impl_compaction_flush.cc:1903] [default] Manual 
compaction starting
May  7 17:06:59 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:06:59.882746) 
[db/db_impl/db_impl_compaction_flush.cc:1903] [default] Manual 
compaction starting
May  7 17:07:00 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225274: 1313 pgs: 7 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:02 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:07:02 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225275: 1313 pgs: 5 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:03 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:07:04 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225276: 1313 pgs: 5 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:06 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:07:06] "GET /metrics HTTP/1.1" 
200 73533 "" "Prometheus/2.43.0"
May  7 17:07:06 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225277: 1313 pgs: 5 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:07 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:07:07 pr-md-02 ceph-mgr[5311]: [rbd_support INFO root] 
MirrorSnapshotScheduleHandler: load_schedules
May  7 17:07:07 pr-md-02 ceph-mgr[5311]: [balancer INFO root] 
Optimize plan auto_2024-05-08_00:07:07
May  7 17:07:07 pr-md-02 ceph-mgr[5311]: [balancer INFO root] Mode 
upmap, max misplaced 0.050000
May  7 17:07:07 pr-md-02 ceph-mgr[5311]: [balancer INFO root] do_upmap
May  7 17:07:07 pr-md-02 ceph-mgr[5311]: [balancer INFO root] pools 
['cephfs_data', 'cephfs_md_and_data', '.mgr', 'cephfs_metadata']
May  7 17:07:07 pr-md-02 ceph-mgr[5311]: [balancer INFO root] 
prepared 0/10 changes
May  7 17:07:08 pr-md-02 ceph-mgr[5311]: [rbd_support INFO root] 
TrashPurgeScheduleHandler: load_schedules
May  7 17:07:08 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:07:08 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225278: 1313 pgs: 5 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:09 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:07:10 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225279: 1313 pgs: 5 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:12 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5613 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:07:12 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225280: 1313 pgs: 6 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:13 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:07:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler INFO root] 
_maybe_adjust
May  7 17:07:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool cephfs_metadata won't scale due to overlapping roots: {-15, -1}
May  7 17:07:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
Please See: 
https://docs.ceph.com/en/latest/rados/operations/placement-groups/#automated...
May  7 17:07:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool cephfs_data won't scale due to overlapping roots: {-15, -1, -2}
May  7 17:07:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
Please See: 
https://docs.ceph.com/en/latest/rados/operations/placement-groups/#automated...
May  7 17:07:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool cephfs_md_and_data won't scale due to overlapping roots: {-15, 
-1, -2}
May  7 17:07:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
Please See: 
https://docs.ceph.com/en/latest/rados/operations/placement-groups/#automated...
May  7 17:07:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool 1 contains an overlapping root -1... skipping scaling
May  7 17:07:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool 7 contains an overlapping root -15... skipping scaling
May  7 17:07:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool 8 contains an overlapping root -2... skipping scaling
May  7 17:07:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool 9 contains an overlapping root -2... skipping scaling
May  7 17:07:14 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225281: 1313 pgs: 6 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:15 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5614 e5614: 46 total, 46 up, 46 in
May  7 17:07:15 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:07:16 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:07:16] "GET /metrics HTTP/1.1" 
200 73532 "" "Prometheus/2.43.0"
May  7 17:07:16 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225283: 1313 pgs: 6 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:17 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:07:17 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:07:17 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev d807ba79-465d-4d89-ab03-9bff6275dd10 does not exist
May  7 17:07:17 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev 9e26e8f6-fba9-425c-923b-ba9da32693a7 does not exist
May  7 17:07:17 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev 1f435daa-e8f0-4964-b7fb-07ea14b95e9f does not exist
May  7 17:07:17 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:07:17 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:07:17 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:07:17 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:07:17 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5614 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:07:17 pr-md-02 podman[2565736]: 2024-05-07 
17:07:17.694558647 -0700 PDT m=+0.039788036 container create 
877f3f2f39644710f6a08499d230d3af827df70612c20d5e409ec36b554c4839 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=stoic_mayer, org.label-schema.build-date=20240222, 
org.label-schema.vendor=CentOS, 
GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, 
org.label-schema.schema-version=1.0, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=HEAD, 
io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base 
Image, ceph=True, maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
CEPH_POINT_RELEASE=-18.2.1)
May  7 17:07:17 pr-md-02 systemd[1]: Started 
libpod-conmon-877f3f2f39644710f6a08499d230d3af827df70612c20d5e409ec36b554c4839.scope.
May  7 17:07:17 pr-md-02 systemd[1]: Started libcrun container.
May  7 17:07:17 pr-md-02 podman[2565736]: 2024-05-07 
17:07:17.722272344 -0700 PDT m=+0.067501679 container init 
877f3f2f39644710f6a08499d230d3af827df70612c20d5e409ec36b554c4839 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=stoic_mayer, GIT_CLEAN=True, 
org.label-schema.build-date=20240222, io.buildah.version=1.29.1, 
RELEASE=HEAD, org.label-schema.schema-version=1.0, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.name=CentOS Stream 8 Base Image, 
org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, GIT_BRANCH=HEAD, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.license=GPLv2, ceph=True)
May  7 17:07:17 pr-md-02 stoic_mayer[2565752]: 167 167
May  7 17:07:17 pr-md-02 systemd[1]: 
libpod-877f3f2f39644710f6a08499d230d3af827df70612c20d5e409ec36b554c4839.scope: 
Deactivated successfully.
May  7 17:07:17 pr-md-02 podman[2565736]: 2024-05-07 
17:07:17.728419093 -0700 PDT m=+0.073648421 container start 
877f3f2f39644710f6a08499d230d3af827df70612c20d5e409ec36b554c4839 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=stoic_mayer, org.label-schema.license=GPLv2, GIT_CLEAN=True, 
io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, 
CEPH_POINT_RELEASE=-18.2.1, org.label-schema.vendor=CentOS, 
org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.build-date=20240222, 
GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, 
RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd)
May  7 17:07:17 pr-md-02 podman[2565736]: 2024-05-07 
17:07:17.728522035 -0700 PDT m=+0.073751360 container attach 
877f3f2f39644710f6a08499d230d3af827df70612c20d5e409ec36b554c4839 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=stoic_mayer, org.label-schema.schema-version=1.0, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, 
org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, 
io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, 
GIT_CLEAN=True, org.label-schema.vendor=CentOS, 
org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, org.label-schema.name=CentOS Stream 8 Base 
Image, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git)
May  7 17:07:17 pr-md-02 podman[2565736]: 2024-05-07 
17:07:17.728769341 -0700 PDT m=+0.073998662 container died 
877f3f2f39644710f6a08499d230d3af827df70612c20d5e409ec36b554c4839 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=stoic_mayer, ceph=True, org.label-schema.schema-version=1.0, 
CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
io.buildah.version=1.29.1, org.label-schema.license=GPLv2, 
org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 
Base Image, GIT_BRANCH=HEAD, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.build-date=20240222, RELEASE=HEAD)
May  7 17:07:17 pr-md-02 podman[2565736]: 2024-05-07 
17:07:17.678321769 -0700 PDT m=+0.023551090 image pull 
5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 
quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8
May  7 17:07:17 pr-md-02 podman[2565757]: 2024-05-07 
17:07:17.78259101 -0700 PDT m=+0.046708660 container remove 
877f3f2f39644710f6a08499d230d3af827df70612c20d5e409ec36b554c4839 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=stoic_mayer, org.label-schema.name=CentOS Stream 8 Base Image, 
org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, 
ceph=True, maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.build-date=20240222, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
io.buildah.version=1.29.1, GIT_CLEAN=True, RELEASE=HEAD, 
org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, 
CEPH_POINT_RELEASE=-18.2.1, 
GIT_REPO=https://github.com/ceph/ceph-container.git)
May  7 17:07:17 pr-md-02 systemd[1]: 
libpod-conmon-877f3f2f39644710f6a08499d230d3af827df70612c20d5e409ec36b554c4839.scope: 
Deactivated successfully.
May  7 17:07:17 pr-md-02 podman[2565778]: 2024-05-07 
17:07:17.90851263 -0700 PDT m=+0.039669496 container create 
263083cabee2763b55db565f8fc223d2af29c999d42e972cc67055e08986f350 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=intelligent_taussig, org.label-schema.build-date=20240222, 
RELEASE=HEAD, io.buildah.version=1.29.1, 
org.label-schema.vendor=CentOS, 
GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, 
CEPH_POINT_RELEASE=-18.2.1, org.label-schema.name=CentOS Stream 8 
Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.schema-version=1.0, GIT_CLEAN=True, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd)
May  7 17:07:17 pr-md-02 systemd[1]: Started 
libpod-conmon-263083cabee2763b55db565f8fc223d2af29c999d42e972cc67055e08986f350.scope.
May  7 17:07:17 pr-md-02 systemd[1]: Started libcrun container.
May  7 17:07:17 pr-md-02 podman[2565778]: 2024-05-07 
17:07:17.936966849 -0700 PDT m=+0.068123675 container init 
263083cabee2763b55db565f8fc223d2af29c999d42e972cc67055e08986f350 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=intelligent_taussig, org.label-schema.schema-version=1.0, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.name=CentOS Stream 8 Base Image, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume 
Abrioux <gabrioux@redhat.com>, GIT_CLEAN=True, 
org.label-schema.build-date=20240222, RELEASE=HEAD, 
org.label-schema.license=GPLv2, ceph=True, 
CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1)
May  7 17:07:17 pr-md-02 podman[2565778]: 2024-05-07 
17:07:17.943702066 -0700 PDT m=+0.074858891 container start 
263083cabee2763b55db565f8fc223d2af29c999d42e972cc67055e08986f350 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=intelligent_taussig, org.label-schema.license=GPLv2, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, ceph=True, 
CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, 
org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, 
GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 
8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
io.buildah.version=1.29.1, org.label-schema.schema-version=1.0)
May  7 17:07:17 pr-md-02 podman[2565778]: 2024-05-07 
17:07:17.943794994 -0700 PDT m=+0.074951827 container attach 
263083cabee2763b55db565f8fc223d2af29c999d42e972cc67055e08986f350 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=intelligent_taussig, CEPH_POINT_RELEASE=-18.2.1, 
org.label-schema.vendor=CentOS, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, org.label-schema.schema-version=1.0, 
ceph=True, RELEASE=HEAD, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, 
org.label-schema.build-date=20240222, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, 
org.label-schema.name=CentOS Stream 8 Base Image)
May  7 17:07:17 pr-md-02 podman[2565778]: 2024-05-07 
17:07:17.892929482 -0700 PDT m=+0.024086307 image pull 
5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 
quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8
May  7 17:07:18 pr-md-02 ceph-mon[5348]: from='mgr.274299 
10.50.1.75:0/136806890' entity='mgr.pr-md-02.emffhz' cmd=[{"prefix": 
"auth get", "entity": "client.admin"}]: dispatch
May  7 17:07:18 pr-md-02 ceph-mon[5348]: from='mgr.274299 ' 
entity='mgr.pr-md-02.emffhz'
May  7 17:07:18 pr-md-02 ceph-mon[5348]: from='mgr.274299 
10.50.1.75:0/136806890' entity='mgr.pr-md-02.emffhz' cmd=[{"prefix": 
"auth get", "entity": "client.bootstrap-osd"}]: dispatch
May  7 17:07:18 pr-md-02 intelligent_taussig[2565794]: --> passed 
data devices: 2 physical, 0 LVM
May  7 17:07:18 pr-md-02 intelligent_taussig[2565794]: --> relative 
data size: 1.0
May  7 17:07:18 pr-md-02 intelligent_taussig[2565794]: --> All data 
devices are unavailable
May  7 17:07:18 pr-md-02 systemd[1]: 
libpod-263083cabee2763b55db565f8fc223d2af29c999d42e972cc67055e08986f350.scope: 
Deactivated successfully.
May  7 17:07:18 pr-md-02 podman[2565778]: 2024-05-07 
17:07:18.556739188 -0700 PDT m=+0.687896012 container died 
263083cabee2763b55db565f8fc223d2af29c999d42e972cc67055e08986f350 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=intelligent_taussig, org.label-schema.vendor=CentOS, 
org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, 
ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.schema-version=1.0, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, RELEASE=HEAD, 
org.label-schema.name=CentOS Stream 8 Base Image)
May  7 17:07:18 pr-md-02 podman[2565822]: 2024-05-07 
17:07:18.614711286 -0700 PDT m=+0.047414226 container remove 
263083cabee2763b55db565f8fc223d2af29c999d42e972cc67055e08986f350 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=intelligent_taussig, org.label-schema.license=GPLv2, 
org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, 
GIT_CLEAN=True, org.label-schema.build-date=20240222, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.schema-version=1.0, RELEASE=HEAD, 
org.label-schema.name=CentOS Stream 8 Base Image, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, 
GIT_REPO=https://github.com/ceph/ceph-container.git)
May  7 17:07:18 pr-md-02 systemd[1]: 
libpod-conmon-263083cabee2763b55db565f8fc223d2af29c999d42e972cc67055e08986f350.scope: 
Deactivated successfully.
May  7 17:07:18 pr-md-02 systemd[1]: 
var-lib-containers-storage-overlay-2f63fd9ffd2763a7ee3f75befaf99b27a75325cd025cc2e74e72d7bd150afc42-merged.mount: 
Deactivated successfully.
May  7 17:07:18 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:07:18 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225284: 1313 pgs: 3 active+clean+scrubbing+deep, 8 
active+clean+scrubbing, 1302 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:19 pr-md-02 podman[2565959]: 2024-05-07 
17:07:19.171860525 -0700 PDT m=+0.039261474 container create 
eb695a8059499edec16f6205fa8eabfecd83baf7f28b29860ac05430368bb96e 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=xenodochial_wu, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, 
RELEASE=HEAD, org.label-schema.schema-version=1.0, 
org.label-schema.build-date=20240222, org.label-schema.name=CentOS 
Stream 8 Base Image, io.buildah.version=1.29.1, 
GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, 
CEPH_POINT_RELEASE=-18.2.1, org.label-schema.vendor=CentOS, 
GIT_CLEAN=True, maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.license=GPLv2)
May  7 17:07:19 pr-md-02 systemd[1]: Started 
libpod-conmon-eb695a8059499edec16f6205fa8eabfecd83baf7f28b29860ac05430368bb96e.scope.
May  7 17:07:19 pr-md-02 systemd[1]: Started libcrun container.
May  7 17:07:19 pr-md-02 podman[2565959]: 2024-05-07 
17:07:19.199519439 -0700 PDT m=+0.066920407 container init 
eb695a8059499edec16f6205fa8eabfecd83baf7f28b29860ac05430368bb96e 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=xenodochial_wu, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, GIT_BRANCH=HEAD, GIT_CLEAN=True, 
org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, 
RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
io.buildah.version=1.29.1, org.label-schema.license=GPLv2, 
org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, 
org.label-schema.build-date=20240222, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
CEPH_POINT_RELEASE=-18.2.1)
May  7 17:07:19 pr-md-02 xenodochial_wu[2565974]: 167 167
May  7 17:07:19 pr-md-02 systemd[1]: 
libpod-eb695a8059499edec16f6205fa8eabfecd83baf7f28b29860ac05430368bb96e.scope: 
Deactivated successfully.
May  7 17:07:19 pr-md-02 podman[2565959]: 2024-05-07 
17:07:19.205924433 -0700 PDT m=+0.073325392 container start 
eb695a8059499edec16f6205fa8eabfecd83baf7f28b29860ac05430368bb96e 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=xenodochial_wu, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, 
org.label-schema.schema-version=1.0, org.label-schema.name=CentOS 
Stream 8 Base Image, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, org.label-schema.build-date=20240222, 
CEPH_POINT_RELEASE=-18.2.1, ceph=True, 
org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.license=GPLv2, RELEASE=HEAD, GIT_CLEAN=True)
May  7 17:07:19 pr-md-02 podman[2565959]: 2024-05-07 17:07:19.2060646 
-0700 PDT m=+0.073465559 container attach 
eb695a8059499edec16f6205fa8eabfecd83baf7f28b29860ac05430368bb96e 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=xenodochial_wu, GIT_CLEAN=True, 
GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, 
RELEASE=HEAD, io.buildah.version=1.29.1, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, 
ceph=True, org.label-schema.vendor=CentOS, 
org.label-schema.name=CentOS Stream 8 Base Image)
May  7 17:07:19 pr-md-02 podman[2565959]: 2024-05-07 
17:07:19.206298065 -0700 PDT m=+0.073699031 container died 
eb695a8059499edec16f6205fa8eabfecd83baf7f28b29860ac05430368bb96e 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=xenodochial_wu, org.label-schema.schema-version=1.0, 
org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, ceph=True, org.label-schema.vendor=CentOS, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.license=GPLv2, GIT_CLEAN=True, 
org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, 
CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1)
May  7 17:07:19 pr-md-02 podman[2565959]: 2024-05-07 
17:07:19.158156321 -0700 PDT m=+0.025557273 image pull 
5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 
quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8
May  7 17:07:19 pr-md-02 podman[2565979]: 2024-05-07 
17:07:19.25994464 -0700 PDT m=+0.046894141 container remove 
eb695a8059499edec16f6205fa8eabfecd83baf7f28b29860ac05430368bb96e 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=xenodochial_wu, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, 
GIT_CLEAN=True, io.buildah.version=1.29.1, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, 
CEPH_POINT_RELEASE=-18.2.1, ceph=True, 
org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0)
May  7 17:07:19 pr-md-02 systemd[1]: 
libpod-conmon-eb695a8059499edec16f6205fa8eabfecd83baf7f28b29860ac05430368bb96e.scope: 
Deactivated successfully.
May  7 17:07:19 pr-md-02 podman[2566000]: 2024-05-07 
17:07:19.388083595 -0700 PDT m=+0.043236647 container create 
a9fae2fc56591b62da497f4015a6e693da36864c0ee3f0259fc0d5a2e796184d 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=agitated_mayer, io.buildah.version=1.29.1, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, 
ceph=True, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, 
org.label-schema.name=CentOS Stream 8 Base Image, 
org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, RELEASE=HEAD, 
org.label-schema.build-date=20240222, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.vendor=CentOS)
May  7 17:07:19 pr-md-02 systemd[1]: Started 
libpod-conmon-a9fae2fc56591b62da497f4015a6e693da36864c0ee3f0259fc0d5a2e796184d.scope.
May  7 17:07:19 pr-md-02 systemd[1]: Started libcrun container.
May  7 17:07:19 pr-md-02 podman[2566000]: 2024-05-07 
17:07:19.417053374 -0700 PDT m=+0.072206356 container init 
a9fae2fc56591b62da497f4015a6e693da36864c0ee3f0259fc0d5a2e796184d 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=agitated_mayer, RELEASE=HEAD, GIT_CLEAN=True, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, 
io.buildah.version=1.29.1, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, GIT_BRANCH=HEAD, org.label-schema.name=CentOS 
Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240222)
May  7 17:07:19 pr-md-02 podman[2566000]: 2024-05-07 
17:07:19.423715752 -0700 PDT m=+0.078868733 container start 
a9fae2fc56591b62da497f4015a6e693da36864c0ee3f0259fc0d5a2e796184d 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=agitated_mayer, org.label-schema.vendor=CentOS, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, 
org.label-schema.name=CentOS Stream 8 Base Image, 
CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, GIT_CLEAN=True, ceph=True, 
org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.license=GPLv2, org.label-schema.build-date=20240222)
May  7 17:07:19 pr-md-02 podman[2566000]: 2024-05-07 
17:07:19.423810643 -0700 PDT m=+0.078963634 container attach 
a9fae2fc56591b62da497f4015a6e693da36864c0ee3f0259fc0d5a2e796184d 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=agitated_mayer, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, org.label-schema.schema-version=1.0, 
RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.name=CentOS Stream 8 Base Image, 
org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, 
GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, 
CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, 
org.label-schema.build-date=20240222)
May  7 17:07:19 pr-md-02 podman[2566000]: 2024-05-07 
17:07:19.371099332 -0700 PDT m=+0.026252317 image pull 
5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 
quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8
May  7 17:07:19 pr-md-02 systemd[1]: 
var-lib-containers-storage-overlay-91f5899d00d5794e67bbdd45fa72d22c74b0b977efec0238aa842ac272b635ce-merged.mount: 
Deactivated successfully.
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: {
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]:    "41": [
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]:        {
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "devices": [
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "/dev/nvme0n1"
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]:            ],
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "lv_name": 
"osd-block-88070adf-ee15-406a-b8e9-98a61e294c8b",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "lv_path": 
"/dev/ceph-eac1ebc0-c00f-4558-8c72-741fc8f81abc/osd-block-88070adf-ee15-406a-b8e9-98a61e294c8b",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "lv_size": 
"1920378863616",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "lv_tags": 
"ceph.block_device=/dev/ceph-eac1ebc0-c00f-4558-8c72-741fc8f81abc/osd-block-88070adf-ee15-406a-b8e9-98a61e294c8b,ceph.block_uuid=MwSZ0h-81BS-XH33-wWY4-vk23-2TIJ-DW9XgO,ceph.cephx_lockbox_secret=AQBJFuJlxpZUJBAAHM4wPn5k5ExaR9mY0pS+YA==,ceph.cluster_fsid=58bde08a-d7ed-11ee-9098-506b4b4da440,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=1,ceph.osd_fsid=88070adf-ee15-406a-b8e9-98a61e294c8b,ceph.osd_id=41,ceph.osdspec_affinity=md_encrypted_nvme_disks,ceph.type=block,ceph.vdo=0",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "lv_uuid": 
"MwSZ0h-81BS-XH33-wWY4-vk23-2TIJ-DW9XgO",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "name": 
"osd-block-88070adf-ee15-406a-b8e9-98a61e294c8b",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "path": 
"/dev/ceph-eac1ebc0-c00f-4558-8c72-741fc8f81abc/osd-block-88070adf-ee15-406a-b8e9-98a61e294c8b",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "tags": {
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: 
"ceph.block_device": 
"/dev/ceph-eac1ebc0-c00f-4558-8c72-741fc8f81abc/osd-block-88070adf-ee15-406a-b8e9-98a61e294c8b",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "ceph.block_uuid": 
"MwSZ0h-81BS-XH33-wWY4-vk23-2TIJ-DW9XgO",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: 
"ceph.cephx_lockbox_secret": "AQBJFuJlxpZUJBAAHM4wPn5k5ExaR9mY0pS+YA==",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: 
"ceph.cluster_fsid": "58bde08a-d7ed-11ee-9098-506b4b4da440",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: 
"ceph.cluster_name": "ceph",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: 
"ceph.crush_device_class": "",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "ceph.encrypted": "1",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "ceph.osd_fsid": 
"88070adf-ee15-406a-b8e9-98a61e294c8b",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "ceph.osd_id": "41",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: 
"ceph.osdspec_affinity": "md_encrypted_nvme_disks",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "ceph.type": "block",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "ceph.vdo": "0"
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]:            },
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "type": "block",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "vg_name": 
"ceph-eac1ebc0-c00f-4558-8c72-741fc8f81abc"
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]:        }
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]:    ],
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]:    "43": [
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]:        {
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "devices": [
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "/dev/nvme1n1"
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]:            ],
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "lv_name": 
"osd-block-50a79ade-482b-4c7e-84ae-0dac068ed570",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "lv_path": 
"/dev/ceph-0b5aeb1a-7dea-4260-9af9-f2bff16d642a/osd-block-50a79ade-482b-4c7e-84ae-0dac068ed570",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "lv_size": 
"1920378863616",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "lv_tags": 
"ceph.block_device=/dev/ceph-0b5aeb1a-7dea-4260-9af9-f2bff16d642a/osd-block-50a79ade-482b-4c7e-84ae-0dac068ed570,ceph.block_uuid=zN3gTU-nGwQ-DfIA-dzWZ-e7QI-OcmL-SZcgqT,ceph.cephx_lockbox_secret=AQBYFuJlTtj/DBAAt2wy+qfx2XCHFuwJZKwp6A==,ceph.cluster_fsid=58bde08a-d7ed-11ee-9098-506b4b4da440,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=1,ceph.osd_fsid=50a79ade-482b-4c7e-84ae-0dac068ed570,ceph.osd_id=43,ceph.osdspec_affinity=md_encrypted_nvme_disks,ceph.type=block,ceph.vdo=0",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "lv_uuid": 
"zN3gTU-nGwQ-DfIA-dzWZ-e7QI-OcmL-SZcgqT",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "name": 
"osd-block-50a79ade-482b-4c7e-84ae-0dac068ed570",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "path": 
"/dev/ceph-0b5aeb1a-7dea-4260-9af9-f2bff16d642a/osd-block-50a79ade-482b-4c7e-84ae-0dac068ed570",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "tags": {
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: 
"ceph.block_device": 
"/dev/ceph-0b5aeb1a-7dea-4260-9af9-f2bff16d642a/osd-block-50a79ade-482b-4c7e-84ae-0dac068ed570",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "ceph.block_uuid": 
"zN3gTU-nGwQ-DfIA-dzWZ-e7QI-OcmL-SZcgqT",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: 
"ceph.cephx_lockbox_secret": "AQBYFuJlTtj/DBAAt2wy+qfx2XCHFuwJZKwp6A==",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: 
"ceph.cluster_fsid": "58bde08a-d7ed-11ee-9098-506b4b4da440",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: 
"ceph.cluster_name": "ceph",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: 
"ceph.crush_device_class": "",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "ceph.encrypted": "1",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "ceph.osd_fsid": 
"50a79ade-482b-4c7e-84ae-0dac068ed570",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "ceph.osd_id": "43",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: 
"ceph.osdspec_affinity": "md_encrypted_nvme_disks",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "ceph.type": "block",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "ceph.vdo": "0"
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]:            },
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "type": "block",
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: "vg_name": 
"ceph-0b5aeb1a-7dea-4260-9af9-f2bff16d642a"
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]:        }
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]:    ]
May  7 17:07:19 pr-md-02 agitated_mayer[2566015]: }
May  7 17:07:19 pr-md-02 systemd[1]: 
libpod-a9fae2fc56591b62da497f4015a6e693da36864c0ee3f0259fc0d5a2e796184d.scope: 
Deactivated successfully.
May  7 17:07:19 pr-md-02 podman[2566000]: 2024-05-07 
17:07:19.801695942 -0700 PDT m=+0.456848932 container died 
a9fae2fc56591b62da497f4015a6e693da36864c0ee3f0259fc0d5a2e796184d 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=agitated_mayer, GIT_BRANCH=HEAD, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
CEPH_POINT_RELEASE=-18.2.1, org.label-schema.license=GPLv2, 
org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=HEAD, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
io.buildah.version=1.29.1, ceph=True, 
org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.name=CentOS Stream 8 Base Image)
May  7 17:07:19 pr-md-02 systemd[1]: 
var-lib-containers-storage-overlay-68bc380ea9ec1cff282d08f862c7b661ba0571a20622225a5a3994e5fc88985d-merged.mount: 
Deactivated successfully.
May  7 17:07:19 pr-md-02 podman[2566028]: 2024-05-07 
17:07:19.854983202 -0700 PDT m=+0.045986655 container remove 
a9fae2fc56591b62da497f4015a6e693da36864c0ee3f0259fc0d5a2e796184d 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=agitated_mayer, org.label-schema.name=CentOS Stream 8 Base 
Image, RELEASE=HEAD, org.label-schema.vendor=CentOS, ceph=True, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.license=GPLv2, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.schema-version=1.0, GIT_CLEAN=True, 
io.buildah.version=1.29.1, org.label-schema.build-date=20240222)
May  7 17:07:19 pr-md-02 systemd[1]: 
libpod-conmon-a9fae2fc56591b62da497f4015a6e693da36864c0ee3f0259fc0d5a2e796184d.scope: 
Deactivated successfully.
May  7 17:07:20 pr-md-02 podman[2566165]: 2024-05-07 
17:07:20.420915201 -0700 PDT m=+0.041065865 container create 
f1af63ba0752e42553bce21029758749d47f2dccd9f6503a004e15a9c340bb0a 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=practical_wu, org.label-schema.schema-version=1.0, 
GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, 
RELEASE=HEAD, org.label-schema.license=GPLv2, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.build-date=20240222, io.buildah.version=1.29.1, 
GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image)
May  7 17:07:20 pr-md-02 systemd[1]: Started 
libpod-conmon-f1af63ba0752e42553bce21029758749d47f2dccd9f6503a004e15a9c340bb0a.scope.
May  7 17:07:20 pr-md-02 systemd[1]: Started libcrun container.
May  7 17:07:20 pr-md-02 podman[2566165]: 2024-05-07 
17:07:20.45143613 -0700 PDT m=+0.071586769 container init 
f1af63ba0752e42553bce21029758749d47f2dccd9f6503a004e15a9c340bb0a 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=practical_wu, org.label-schema.name=CentOS Stream 8 Base Image, 
RELEASE=HEAD, ceph=True, org.label-schema.schema-version=1.0, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.build-date=20240222, GIT_CLEAN=True, 
io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, 
GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2)
May  7 17:07:20 pr-md-02 practical_wu[2566180]: 167 167
May  7 17:07:20 pr-md-02 systemd[1]: 
libpod-f1af63ba0752e42553bce21029758749d47f2dccd9f6503a004e15a9c340bb0a.scope: 
Deactivated successfully.
May  7 17:07:20 pr-md-02 podman[2566165]: 2024-05-07 
17:07:20.457513493 -0700 PDT m=+0.077664122 container start 
f1af63ba0752e42553bce21029758749d47f2dccd9f6503a004e15a9c340bb0a 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=practical_wu, org.label-schema.vendor=CentOS, 
org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, 
ceph=True, org.label-schema.build-date=20240222, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, 
org.label-schema.name=CentOS Stream 8 Base Image, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, RELEASE=HEAD, 
io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True)
May  7 17:07:20 pr-md-02 podman[2566165]: 2024-05-07 
17:07:20.457638676 -0700 PDT m=+0.077789308 container attach 
f1af63ba0752e42553bce21029758749d47f2dccd9f6503a004e15a9c340bb0a 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=practical_wu, 
GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, 
RELEASE=HEAD, org.label-schema.build-date=20240222, 
io.buildah.version=1.29.1, org.label-schema.license=GPLv2, 
org.label-schema.schema-version=1.0, GIT_CLEAN=True, 
org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 
Base Image, maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
CEPH_POINT_RELEASE=-18.2.1, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD)
May  7 17:07:20 pr-md-02 podman[2566165]: 2024-05-07 
17:07:20.457913249 -0700 PDT m=+0.078063874 container died 
f1af63ba0752e42553bce21029758749d47f2dccd9f6503a004e15a9c340bb0a 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=practical_wu, RELEASE=HEAD, org.label-schema.license=GPLv2, 
org.label-schema.build-date=20240222, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
CEPH_POINT_RELEASE=-18.2.1, org.label-schema.name=CentOS Stream 8 
Base Image, GIT_BRANCH=HEAD)
May  7 17:07:20 pr-md-02 podman[2566165]: 2024-05-07 
17:07:20.405495878 -0700 PDT m=+0.025646501 image pull 
5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 
quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8
May  7 17:07:20 pr-md-02 podman[2566185]: 2024-05-07 
17:07:20.515553442 -0700 PDT m=+0.045837844 container remove 
f1af63ba0752e42553bce21029758749d47f2dccd9f6503a004e15a9c340bb0a 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=practical_wu, ceph=True, org.label-schema.name=CentOS Stream 8 
Base Image, org.label-schema.license=GPLv2, 
org.label-schema.build-date=20240222, RELEASE=HEAD, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, GIT_BRANCH=HEAD, 
io.buildah.version=1.29.1, GIT_CLEAN=True, 
CEPH_POINT_RELEASE=-18.2.1, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS)
May  7 17:07:20 pr-md-02 systemd[1]: 
libpod-conmon-f1af63ba0752e42553bce21029758749d47f2dccd9f6503a004e15a9c340bb0a.scope: 
Deactivated successfully.
May  7 17:07:20 pr-md-02 podman[2566207]: 2024-05-07 
17:07:20.631476902 -0700 PDT m=+0.043759539 container create 
c8952c7d94b5440b33b6562bed1ec270208bbfa94a25d32c81278e71b821495f 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=quirky_clarke, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, 
org.label-schema.build-date=20240222, GIT_CLEAN=True, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.vendor=CentOS, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
org.label-schema.name=CentOS Stream 8 Base Image, 
io.buildah.version=1.29.1, RELEASE=HEAD, ceph=True, 
CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0)
May  7 17:07:20 pr-md-02 systemd[1]: Started 
libpod-conmon-c8952c7d94b5440b33b6562bed1ec270208bbfa94a25d32c81278e71b821495f.scope.
May  7 17:07:20 pr-md-02 systemd[1]: Started libcrun container.
May  7 17:07:20 pr-md-02 podman[2566207]: 2024-05-07 
17:07:20.660235834 -0700 PDT m=+0.072518401 container init 
c8952c7d94b5440b33b6562bed1ec270208bbfa94a25d32c81278e71b821495f 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=quirky_clarke, org.label-schema.vendor=CentOS, 
org.label-schema.build-date=20240222, io.buildah.version=1.29.1, 
ceph=True, CEPH_POINT_RELEASE=-18.2.1, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, 
org.label-schema.schema-version=1.0, org.label-schema.name=CentOS 
Stream 8 Base Image, maintainer=Guillaume Abrioux 
<gabrioux@redhat.com>, 
GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, 
GIT_CLEAN=True)
May  7 17:07:20 pr-md-02 podman[2566207]: 2024-05-07 
17:07:20.666915701 -0700 PDT m=+0.079198260 container start 
c8952c7d94b5440b33b6562bed1ec270208bbfa94a25d32c81278e71b821495f 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=quirky_clarke, ceph=True, CEPH_POINT_RELEASE=-18.2.1, 
org.label-schema.schema-version=1.0, org.label-schema.name=CentOS 
Stream 8 Base Image, org.label-schema.build-date=20240222, 
io.buildah.version=1.29.1, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, 
org.label-schema.vendor=CentOS, 
GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, 
RELEASE=HEAD)
May  7 17:07:20 pr-md-02 podman[2566207]: 2024-05-07 
17:07:20.667002215 -0700 PDT m=+0.079284787 container attach 
c8952c7d94b5440b33b6562bed1ec270208bbfa94a25d32c81278e71b821495f 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=quirky_clarke, org.label-schema.name=CentOS Stream 8 Base Image, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, RELEASE=HEAD, 
ceph=True, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, 
org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, 
GIT_BRANCH=HEAD, org.label-schema.build-date=20240222)
May  7 17:07:20 pr-md-02 systemd[1]: 
var-lib-containers-storage-overlay-dbb2679257a24186947b435237af458658d281389b531640688ba39cfa7cac79-merged.mount: 
Deactivated successfully.
May  7 17:07:20 pr-md-02 podman[2566207]: 2024-05-07 17:07:20.6134279 
-0700 PDT m=+0.025710470 image pull 
5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 
quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8
May  7 17:07:20 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225285: 1313 pgs: 3 active+clean+scrubbing+deep, 8 
active+clean+scrubbing, 1302 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:21 pr-md-02 quirky_clarke[2566222]: {
May  7 17:07:21 pr-md-02 quirky_clarke[2566222]: 
"50a79ade-482b-4c7e-84ae-0dac068ed570": {
May  7 17:07:21 pr-md-02 quirky_clarke[2566222]: "ceph_fsid": 
"58bde08a-d7ed-11ee-9098-506b4b4da440",
May  7 17:07:21 pr-md-02 quirky_clarke[2566222]: "device": 
"/dev/mapper/zN3gTU-nGwQ-DfIA-dzWZ-e7QI-OcmL-SZcgqT",
May  7 17:07:21 pr-md-02 quirky_clarke[2566222]: "osd_id": 43,
May  7 17:07:21 pr-md-02 quirky_clarke[2566222]: "osd_uuid": 
"50a79ade-482b-4c7e-84ae-0dac068ed570",
May  7 17:07:21 pr-md-02 quirky_clarke[2566222]:        "type": 
"bluestore"
May  7 17:07:21 pr-md-02 quirky_clarke[2566222]:    },
May  7 17:07:21 pr-md-02 quirky_clarke[2566222]: 
"88070adf-ee15-406a-b8e9-98a61e294c8b": {
May  7 17:07:21 pr-md-02 quirky_clarke[2566222]: "ceph_fsid": 
"58bde08a-d7ed-11ee-9098-506b4b4da440",
May  7 17:07:21 pr-md-02 quirky_clarke[2566222]: "device": 
"/dev/mapper/MwSZ0h-81BS-XH33-wWY4-vk23-2TIJ-DW9XgO",
May  7 17:07:21 pr-md-02 quirky_clarke[2566222]: "osd_id": 41,
May  7 17:07:21 pr-md-02 quirky_clarke[2566222]: "osd_uuid": 
"88070adf-ee15-406a-b8e9-98a61e294c8b",
May  7 17:07:21 pr-md-02 quirky_clarke[2566222]:        "type": 
"bluestore"
May  7 17:07:21 pr-md-02 quirky_clarke[2566222]:    }
May  7 17:07:21 pr-md-02 quirky_clarke[2566222]: }
May  7 17:07:21 pr-md-02 systemd[1]: 
libpod-c8952c7d94b5440b33b6562bed1ec270208bbfa94a25d32c81278e71b821495f.scope: 
Deactivated successfully.
May  7 17:07:21 pr-md-02 podman[2566207]: 2024-05-07 
17:07:21.655826716 -0700 PDT m=+1.068109291 container died 
c8952c7d94b5440b33b6562bed1ec270208bbfa94a25d32c81278e71b821495f 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=quirky_clarke, org.label-schema.name=CentOS Stream 8 Base Image, 
GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, 
ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, 
CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, 
io.buildah.version=1.29.1, org.label-schema.schema-version=1.0)
May  7 17:07:21 pr-md-02 systemd[1]: 
var-lib-containers-storage-overlay-e573f9cf164334d9b913d82ba760fa7f00e98646ea067755c99a3842969d30d8-merged.mount: 
Deactivated successfully.
May  7 17:07:21 pr-md-02 podman[2566303]: 2024-05-07 
17:07:21.711419562 -0700 PDT m=+0.047850993 container remove 
c8952c7d94b5440b33b6562bed1ec270208bbfa94a25d32c81278e71b821495f 
(image=quay.io/ceph/ceph@sha256:632158bcaab42d1034c1fa4a9a4d90a2d910f8d87f7b8d97a9e58af7a17db9e8, 
name=quirky_clarke, org.label-schema.name=CentOS Stream 8 Base Image, 
GIT_REPO=https://github.com/ceph/ceph-container.git, 
GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, 
maintainer=Guillaume Abrioux <gabrioux@redhat.com>, 
org.label-schema.vendor=CentOS, ceph=True, 
org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, 
org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=HEAD, 
io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1)
May  7 17:07:21 pr-md-02 systemd[1]: 
libpod-conmon-c8952c7d94b5440b33b6562bed1ec270208bbfa94a25d32c81278e71b821495f.scope: 
Deactivated successfully.
May  7 17:07:21 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev 57f8febe-215e-403c-821d-95da6e3c5073 does not exist
May  7 17:07:21 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev 2637522f-3bd6-453c-976c-f7da78d12399 does not exist
May  7 17:07:21 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev b0cc7927-d150-4870-8ec7-6ef7d847ffd2 does not exist
May  7 17:07:21 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev 9b053170-587d-42ee-929e-25112b3abba1 does not exist
May  7 17:07:21 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev 352cfbfb-4b0a-420c-959b-e42615f9913d does not exist
May  7 17:07:21 pr-md-02 ceph-mgr[5311]: [progress WARNING root] 
complete: ev 144dbb62-1eeb-49ba-b928-eee4a1756251 does not exist
May  7 17:07:22 pr-md-02 ceph-mon[5348]: from='mgr.274299 ' 
entity='mgr.pr-md-02.emffhz'
May  7 17:07:22 pr-md-02 ceph-mon[5348]: from='mgr.274299 ' 
entity='mgr.pr-md-02.emffhz'
May  7 17:07:22 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5614 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:07:22 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225286: 1313 pgs: 5 active+clean+scrubbing+deep, 10 
active+clean+scrubbing, 1298 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:23 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:07:24 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:07:24 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
scanning for idle connections..
May  7 17:07:24 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
cleaning up connections: []
May  7 17:07:24 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
scanning for idle connections..
May  7 17:07:24 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
cleaning up connections: []
May  7 17:07:24 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
scanning for idle connections..
May  7 17:07:24 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
cleaning up connections: []
May  7 17:07:24 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225287: 1313 pgs: 5 active+clean+scrubbing+deep, 10 
active+clean+scrubbing, 1298 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:26 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:07:26] "GET /metrics HTTP/1.1" 
200 73532 "" "Prometheus/2.43.0"
May  7 17:07:26 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225288: 1313 pgs: 5 active+clean+scrubbing+deep, 10 
active+clean+scrubbing, 1298 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:27 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5614 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:07:28 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:07:29 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225289: 1313 pgs: 6 active+clean+scrubbing+deep, 11 
active+clean+scrubbing, 1296 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:31 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225290: 1313 pgs: 6 active+clean+scrubbing+deep, 11 
active+clean+scrubbing, 1296 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:32 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5614 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:07:33 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225291: 1313 pgs: 4 active+clean+scrubbing+deep, 16 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:33 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:07:35 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225292: 1313 pgs: 4 active+clean+scrubbing+deep, 16 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:36 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:07:36] "GET /metrics HTTP/1.1" 
200 73532 "" "Prometheus/2.43.0"
May  7 17:07:37 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225293: 1313 pgs: 4 active+clean+scrubbing+deep, 16 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:37 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5614 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:07:38 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:07:39 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225294: 1313 pgs: 6 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:39 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:07:41 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225295: 1313 pgs: 6 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:42 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5614 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:07:43 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225296: 1313 pgs: 7 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1291 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:43 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:07:45 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225297: 1313 pgs: 7 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1291 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:46 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:07:46] "GET /metrics HTTP/1.1" 
200 73538 "" "Prometheus/2.43.0"
May  7 17:07:47 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225298: 1313 pgs: 7 active+clean+scrubbing+deep, 15 
active+clean+scrubbing, 1291 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:47 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5614 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:07:48 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:07:49 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225299: 1313 pgs: 8 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1291 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:51 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225300: 1313 pgs: 8 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1291 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:52 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5614 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:07:53 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225301: 1313 pgs: 8 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1291 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:53 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:07:54 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:07:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
scanning for idle connections..
May  7 17:07:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
cleaning up connections: []
May  7 17:07:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
scanning for idle connections..
May  7 17:07:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
cleaning up connections: []
May  7 17:07:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
scanning for idle connections..
May  7 17:07:54 pr-md-02 ceph-mgr[5311]: [volumes INFO mgr_util] 
cleaning up connections: []
May  7 17:07:55 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225302: 1313 pgs: 8 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1291 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:56 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:07:56] "GET /metrics HTTP/1.1" 
200 73534 "" "Prometheus/2.43.0"
May  7 17:07:57 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225303: 1313 pgs: 8 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1291 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:07:57 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5614 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:07:59 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:07:59 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225304: 1313 pgs: 8 active+clean+scrubbing+deep, 13 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:08:01 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225305: 1313 pgs: 8 active+clean+scrubbing+deep, 13 
active+clean+scrubbing, 1292 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:08:02 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5614 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:08:03 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225306: 1313 pgs: 6 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:08:04 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:08:05 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225307: 1313 pgs: 6 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:08:06 pr-md-02 ceph-osd[6259]: log_channel(cluster) log 
[DBG] : purged_snaps scrub starts
May  7 17:08:06 pr-md-02 ceph-osd[6259]: log_channel(cluster) log 
[DBG] : purged_snaps scrub ok
May  7 17:08:06 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:08:06] "GET /metrics HTTP/1.1" 
200 73534 "" "Prometheus/2.43.0"
May  7 17:08:07 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225308: 1313 pgs: 6 active+clean+scrubbing+deep, 14 
active+clean+scrubbing, 1293 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:08:07 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5615 e5615: 46 total, 46 up, 46 in
May  7 17:08:07 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5615 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:08:07 pr-md-02 ceph-mgr[5311]: [rbd_support INFO root] 
MirrorSnapshotScheduleHandler: load_schedules
May  7 17:08:07 pr-md-02 ceph-mgr[5311]: [balancer INFO root] 
Optimize plan auto_2024-05-08_00:08:07
May  7 17:08:07 pr-md-02 ceph-mgr[5311]: [balancer INFO root] Mode 
upmap, max misplaced 0.050000
May  7 17:08:07 pr-md-02 ceph-mgr[5311]: [balancer INFO root] do_upmap
May  7 17:08:07 pr-md-02 ceph-mgr[5311]: [balancer INFO root] pools 
['cephfs_md_and_data', 'cephfs_data', '.mgr', 'cephfs_metadata']
May  7 17:08:07 pr-md-02 ceph-mgr[5311]: [balancer INFO root] 
prepared 0/10 changes
May  7 17:08:08 pr-md-02 ceph-mgr[5311]: [rbd_support INFO root] 
TrashPurgeScheduleHandler: load_schedules
May  7 17:08:09 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225310: 1313 pgs: 3 active+clean+scrubbing+deep, 2 
active+clean+scrubbing, 1308 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:08:09 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:08:09 pr-md-02 ceph-mgr[5311]: [stats WARNING root] cmdtag 
not found in client metadata
May  7 17:08:11 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225311: 1313 pgs: 3 active+clean+scrubbing+deep, 2 
active+clean+scrubbing, 1308 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:08:12 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5615 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:08:13 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225312: 1313 pgs: 4 active+clean+scrubbing+deep, 10 
active+clean+scrubbing, 1299 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:08:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler INFO root] 
_maybe_adjust
May  7 17:08:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool cephfs_metadata won't scale due to overlapping roots: {-15, -1}
May  7 17:08:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
Please See: 
https://docs.ceph.com/en/latest/rados/operations/placement-groups/#automated...
May  7 17:08:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool cephfs_data won't scale due to overlapping roots: {-15, -1, -2}
May  7 17:08:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
Please See: 
https://docs.ceph.com/en/latest/rados/operations/placement-groups/#automated...
May  7 17:08:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool cephfs_md_and_data won't scale due to overlapping roots: {-15, 
-1, -2}
May  7 17:08:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
Please See: 
https://docs.ceph.com/en/latest/rados/operations/placement-groups/#automated...
May  7 17:08:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool 1 contains an overlapping root -1... skipping scaling
May  7 17:08:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool 7 contains an overlapping root -15... skipping scaling
May  7 17:08:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool 8 contains an overlapping root -2... skipping scaling
May  7 17:08:13 pr-md-02 ceph-mgr[5311]: [pg_autoscaler WARNING root] 
pool 9 contains an overlapping root -2... skipping scaling
May  7 17:08:14 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-ceph-exporter-pr-md-02[5226]: 
system:0
May  7 17:08:15 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225313: 1313 pgs: 4 active+clean+scrubbing+deep, 10 
active+clean+scrubbing, 1299 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:08:16 pr-md-02 
ceph-58bde08a-d7ed-11ee-9098-506b4b4da440-mgr-pr-md-02-emffhz[5306]: 
::ffff:10.50.1.74 - - [08/May/2024:00:08:16] "GET /metrics HTTP/1.1" 
200 73536 "" "Prometheus/2.43.0"
May  7 17:08:17 pr-md-02 ceph-mgr[5311]: log_channel(cluster) log 
[DBG] : pgmap v225314: 1313 pgs: 4 active+clean+scrubbing+deep, 10 
active+clean+scrubbing, 1299 active+clean; 558 TiB data, 842 TiB 
used, 287 TiB / 1.1 PiB avail
May  7 17:08:17 pr-md-02 ceph-mon[5348]: mon.pr-md-02@1(peon).osd 
e5615 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 339738624 
full_alloc: 356515840 kv_alloc: 318767104
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: 
[db/db_impl/db_impl_write.cc:2098] [default] New memtable created 
with log file: #228441. Immutable memtables: 0.
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:08:18.105603) 
[db/db_impl/db_impl_compaction_flush.cc:2832] Calling 
FlushMemTableToOutputFile with column family [default], flush slots 
available 1, compaction slots available 1, flush slots scheduled 1, 
compaction slots scheduled 0
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: 
[db/flush_job.cc:856] [default] [JOB 12102] Flushing memtable with 
next log file: 228441
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: EVENT_LOG_v1 
{"time_micros": 1715126898105644, "job": 12102, "event": 
"flush_started", "num_memtables": 1, "num_entries": 2046, 
"num_deletes": 617, "total_data_size": 1704550, "memory_usage": 
1744008, "flush_reason": "Manual Compaction"}
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: 
[db/flush_job.cc:885] [default] [JOB 12102] Level-0 flush table 
#228442: started
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: EVENT_LOG_v1 
{"time_micros": 1715126898113765, "cf_name": "default", "job": 12102, 
"event": "table_file_creation", "file_number": 228442, "file_size": 
847046, "file_checksum": "", "file_checksum_func_name": "Unknown", 
"smallest_seqno": 141705570, "largest_seqno": 141707611, 
"table_properties": {"data_size": 839933, "index_size": 2991, 
"index_partitions": 0, "top_level_index_size": 0, 
"index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, 
"filter_size": 3077, "raw_key_size": 32187, "raw_average_key_size": 
26, "raw_value_size": 819966, "raw_average_value_size": 670, 
"num_data_blocks": 115, "num_entries": 1223, "num_filter_entries": 
1223, "num_deletions": 617, "num_merge_operands": 0, 
"num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, 
"filter_policy": "bloomfilter", "column_family_name": "default", 
"column_family_id": 0, "comparator": "leveldb.BytewiseComparator", 
"merge_operator": "", "prefix_extractor_name": "nullptr", 
"property_collectors": "[]", "compression": "NoCompression", 
"compression_options": "window_bits=-14; level=32767; strategy=0; 
max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; 
max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 
1715126820, "oldest_key_time": 1715126820, "file_creation_time": 
1715126898, "slow_compression_estimated_data_size": 0, 
"fast_compression_estimated_data_size": 0, "db_id": 
"9cddaea3-96ea-4679-bc89-ab4fe1d01ddc", "db_session_id": 
"LVB8G4XNY4SWBL1V508Z", "orig_file_number": 228442, 
"seqno_to_time_mapping": "N/A"}}
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: 
[db/flush_job.cc:1019] [default] [JOB 12102] Flush lasted 8207 
microseconds, and 2745 cpu microseconds.
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:08:18.113810) [db/flush_job.cc:967] [default] [JOB 
12102] Level-0 flush table #228442: 847046 bytes OK
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:08:18.113830) [db/memtable_list.cc:519] [default] 
Level-0 commit table #228442 started
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:08:18.116583) [db/memtable_list.cc:722] [default] 
Level-0 commit table #228442: memtable #1 done
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:08:18.116634) EVENT_LOG_v1 {"time_micros": 
1715126898116612, "job": 12102, "event": "flush_finished", 
"output_compression": "NoCompression", "lsm_state": [1, 0, 0, 0, 0, 
0, 1], "immutable_memtables": 0}
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:08:18.116650) 
[db/db_impl/db_impl_compaction_flush.cc:299] [default] Level summary: 
base level 6 level multiplier 10.00 max bytes base 268435456 files[1 
0 0 0 0 0 1] max score 0.25
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: 
[db/db_impl/db_impl_files.cc:463] [JOB 12102] Try to delete WAL files 
size 1693223, prev total WAL file size 1693550, number of live WAL 
files 2.
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: 
[file/delete_scheduler.cc:74] Deleted file 
/var/lib/ceph/mon/ceph-pr-md-02/store.db/228438.log immediately, 
rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:08:18.117035) 
[db/db_impl/db_impl_compaction_flush.cc:3165] [default] Manual 
compaction from level-0 to level-6 from 
'6D6772737461740032393030383037' seq:72057594037927935, type:22 .. 
'6D6772737461740032393031303630' seq:0, type:0; will stop at (end)
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: 
[db/compaction/compaction_job.cc:1995] [default] [JOB 12103] 
Compacting 1@0 + 1@6 files to L6, score -1.00
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: 
[db/compaction/compaction_job.cc:2001] [default]: Compaction start 
summary: Base version 12102 Base level 0, inputs: [228442(827KB)], 
[228440(36MB)]
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: EVENT_LOG_v1 
{"time_micros": 1715126898117062, "job": 12103, "event": 
"compaction_started", "compaction_reason": "ManualCompaction", 
"files_L0": [228442], "files_L6": [228440], "score": -1, 
"input_data_size": 39098338, "oldest_snapshot_seqno": -1}
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: 
[db/compaction/compaction_job.cc:1588] [default] [JOB 12103] 
Generated table #228443: 25379 keys, 37258556 bytes, temperature: 
kUnknown
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: EVENT_LOG_v1 
{"time_micros": 1715126898320688, "cf_name": "default", "job": 12103, 
"event": "table_file_creation", "file_number": 228443, "file_size": 
37258556, "file_checksum": "", "file_checksum_func_name": "Unknown", 
"smallest_seqno": 0, "largest_seqno": 0, "table_properties": 
{"data_size": 37107602, "index_size": 86411, "index_partitions": 0, 
"top_level_index_size": 0, "index_key_is_user_key": 1, 
"index_value_is_delta_encoded": 1, "filter_size": 63493, 
"raw_key_size": 702042, "raw_average_key_size": 27, "raw_value_size": 
36662378, "raw_average_value_size": 1444, "num_data_blocks": 3038, 
"num_entries": 25379, "num_filter_entries": 25379, "num_deletions": 
0, "num_merge_operands": 0, "num_range_deletions": 0, 
"format_version": 0, "fixed_key_len": 0, "filter_policy": 
"bloomfilter", "column_family_name": "default", "column_family_id": 
0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", 
"prefix_extractor_name": "nullptr", "property_collectors": "[]", 
"compression": "NoCompression", "compression_options": 
"window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; 
zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; 
use_zstd_dict_trainer=1; ", "creation_time": 1709312729, 
"oldest_key_time": 0, "file_creation_time": 1715126898, 
"slow_compression_estimated_data_size": 0, 
"fast_compression_estimated_data_size": 0, "db_id": 
"9cddaea3-96ea-4679-bc89-ab4fe1d01ddc", "db_session_id": 
"LVB8G4XNY4SWBL1V508Z", "orig_file_number": 228443, 
"seqno_to_time_mapping": "N/A"}}
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:08:18.321010) [db/compaction/compaction_job.cc:1663] 
[default] [JOB 12103] Compacted 1@0 + 1@6 files to L6 => 37258556 bytes
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:08:18.323275) [db/compaction/compaction_job.cc:865] 
[default] compacted to: base level 6 level multiplier 10.00 max bytes 
base 268435456 files[0 0 0 0 0 0 1] max score 0.00, MB/sec: 191.9 rd, 
182.9 wr, level 6, files in(1, 1) out(1 +0 blob) MB in(0.8, 36.5 +0.0 
blob) out(35.5 +0.0 blob), read-write-amplify(90.1) 
write-amplify(44.0) OK, records in: 26602, records dropped: 1223 
output_compression: NoCompression
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:08:18.323329) EVENT_LOG_v1 {"time_micros": 
1715126898323308, "job": 12103, "event": "compaction_finished", 
"compaction_time_micros": 203764, "compaction_time_cpu_micros": 
65018, "output_level": 6, "num_output_files": 1, "total_output_size": 
37258556, "num_input_records": 26602, "num_output_records": 25379, 
"num_subcompactions": 1, "output_compression": "NoCompression", 
"num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 
0, "lsm_state": [0, 0, 0, 0, 0, 0, 1]}
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: 
[file/delete_scheduler.cc:74] Deleted file 
/var/lib/ceph/mon/ceph-pr-md-02/store.db/228442.sst immediately, 
rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: EVENT_LOG_v1 
{"time_micros": 1715126898323660, "job": 12103, "event": 
"table_file_deletion", "file_number": 228442}
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: 
[file/delete_scheduler.cc:74] Deleted file 
/var/lib/ceph/mon/ceph-pr-md-02/store.db/228440.sst immediately, 
rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: EVENT_LOG_v1 
{"time_micros": 1715126898326958, "job": 12103, "event": 
"table_file_deletion", "file_number": 228440}
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:08:18.116998) 
[db/db_impl/db_impl_compaction_flush.cc:1903] [default] Manual 
compaction starting
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:08:18.326990) 
[db/db_impl/db_impl_compaction_flush.cc:1903] [default] Manual 
compaction starting
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:08:18.326994) 
[db/db_impl/db_impl_compaction_flush.cc:1903] [default] Manual 
compaction starting
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:08:18.327002) 
[db/db_impl/db_impl_compaction_flush.cc:1903] [default] Manual 
compaction starting
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:08:18.327006) 
[db/db_impl/db_impl_compaction_flush.cc:1903] [default] Manual 
compaction starting
May  7 17:08:18 pr-md-02 ceph-mon[5348]: rocksdb: (Original Log Time 
2024/05/08-00:08:18.327008) 
[db/db_impl/db_impl_compaction_flush.cc:1903] [default] Manual 
compaction starting
I have not updated any packages or anything, I'm not sure how this 
came about.  I did set the following two items earlier today, 
although I'm not sure if it would make any difference?
ceph config set mds mds_recall_max_caps 3000
ceph config set mds mds_cache_memory_limit 16000000000
I tried to restart the MDS daemons alike so:
ceph orch daemon restart mds.slugfs.pr-md-02.sbblqq
And they seem to start for a moment then stop again.
I then tried to restart the one remaining one that was online, then 
it seemed to die as well, and now I see this:
# ceph fs status
slugfs - 0 clients
======
RANK  STATE            MDS            ACTIVITY   DNS    INOS DIRS   CAPS
  0    replay  slugfs.pr-md-01.xdtppo               0 0      0      0
  1    failed
        POOL           TYPE     USED  AVAIL
  cephfs_metadata    metadata   144G  3168G
cephfs_md_and_data    data       0   65.4T
    cephfs_data        data     838T   130T
MDS version: ceph version 18.2.1 
(7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)
But the ceph-mds on pr-md-01 doesn't exist now.
Anyone know how to recover from this?  I'm kind of panicking. :(
Thanks much,
erich

ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-leave@ceph.io

[ceph-users] Re: cephfs filesystem offline, ceph-mds core dumped... How to recover?

Xiubo Li