Hi. I found a default rocksdb option in bluestore that I can't find in facebook rocksdb. recycle_log_file_num this config if a boolean config in facebook rocksdb but in default Ceph configs the value of this is 4. Can someone tell what it means?
I believe it is the number of WALs that should be reused and should be equal to write_buffer_number but don't quote me. On Sat, Jun 20, 2020 at 11:35 AM Seena Fallah <seenafallah@gmail.com> wrote:
Hi. I found a default rocksdb option in bluestore that I can't find in facebook rocksdb. recycle_log_file_num this config if a boolean config in facebook rocksdb but in default Ceph configs the value of this is 4. Can someone tell what it means? _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
also see: https://github.com/facebook/rocksdb/wiki/WAL-Performance " Options.recycle_log_file_num = true will keep a pool of WAL files and try to reuse them. When writing to an existing log file, random writes are used from size 0. Before writes hit the end of the file, the file size doesn't change, so the I/O for metadata might be avoided (also depends on file system mount options). Assuming most WAL files will have similar sizes, I/O needed for metadata will be minimal. " On Sat, Jun 20, 2020 at 12:43 PM Frank R <frankaritchie@gmail.com> wrote:
I believe it is the number of WALs that should be reused and should be equal to write_buffer_number but don't quote me.
On Sat, Jun 20, 2020 at 11:35 AM Seena Fallah <seenafallah@gmail.com> wrote:
Hi. I found a default rocksdb option in bluestore that I can't find in facebook rocksdb. recycle_log_file_num this config if a boolean config in facebook rocksdb but in default Ceph configs the value of this is 4. Can someone tell what it means? _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Yes here I found it’s a boolean (= true) Will this true convert to 1? On Sat, Jun 20, 2020 at 9:15 PM Frank R <frankaritchie@gmail.com> wrote:
also see:
https://github.com/facebook/rocksdb/wiki/WAL-Performance
" Options.recycle_log_file_num = true will keep a pool of WAL files and try to reuse them. When writing to an existing log file, random writes are used from size 0. Before writes hit the end of the file, the file size doesn't change, so the I/O for metadata might be avoided (also depends on file system mount options). Assuming most WAL files will have similar sizes, I/O needed for metadata will be minimal. "
On Sat, Jun 20, 2020 at 12:43 PM Frank R <frankaritchie@gmail.com> wrote:
I believe it is the number of WALs that should be reused and should be equal to write_buffer_number but don't quote me.
On Sat, Jun 20, 2020 at 11:35 AM Seena Fallah <seenafallah@gmail.com>
wrote:
Hi. I found a default rocksdb option in bluestore that I can't find in facebook rocksdb. recycle_log_file_num this config if a boolean config in facebook rocksdb but in default Ceph configs the value of this is 4. Can someone tell what it means? _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
With ceph I have always used it to set the number of WALs to recycle, ie to recycle 8 WALs I use: " recycle_log_file_num=8 " On Sat, Jun 20, 2020 at 12:47 PM Seena Fallah <seenafallah@gmail.com> wrote:
Yes here I found it’s a boolean (= true) Will this true convert to 1?
On Sat, Jun 20, 2020 at 9:15 PM Frank R <frankaritchie@gmail.com> wrote:
also see:
https://github.com/facebook/rocksdb/wiki/WAL-Performance
" Options.recycle_log_file_num = true will keep a pool of WAL files and try to reuse them. When writing to an existing log file, random writes are used from size 0. Before writes hit the end of the file, the file size doesn't change, so the I/O for metadata might be avoided (also depends on file system mount options). Assuming most WAL files will have similar sizes, I/O needed for metadata will be minimal. "
On Sat, Jun 20, 2020 at 12:43 PM Frank R <frankaritchie@gmail.com> wrote:
I believe it is the number of WALs that should be reused and should be equal to write_buffer_number but don't quote me.
On Sat, Jun 20, 2020 at 11:35 AM Seena Fallah <seenafallah@gmail.com> wrote:
Hi. I found a default rocksdb option in bluestore that I can't find in facebook rocksdb. recycle_log_file_num this config if a boolean config in facebook rocksdb but in default Ceph configs the value of this is 4. Can someone tell what it means? _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Do you have any reason for this value? :)) On Sat, Jun 20, 2020 at 9:24 PM Frank R <frankaritchie@gmail.com> wrote:
With ceph I have always used it to set the number of WALs to recycle, ie to recycle 8 WALs I use:
" recycle_log_file_num=8 "
On Sat, Jun 20, 2020 at 12:47 PM Seena Fallah <seenafallah@gmail.com> wrote:
Yes here I found it’s a boolean (= true) Will this true convert to 1?
On Sat, Jun 20, 2020 at 9:15 PM Frank R <frankaritchie@gmail.com> wrote:
also see:
https://github.com/facebook/rocksdb/wiki/WAL-Performance
" Options.recycle_log_file_num = true will keep a pool of WAL files and try to reuse them. When writing to an existing log file, random writes are used from size 0. Before writes hit the end of the file, the file size doesn't change, so the I/O for metadata might be avoided (also depends on file system mount options). Assuming most WAL files will have similar sizes, I/O needed for metadata will be minimal. "
On Sat, Jun 20, 2020 at 12:43 PM Frank R <frankaritchie@gmail.com>
wrote:
I believe it is the number of WALs that should be reused and should be equal to write_buffer_number but don't quote me.
On Sat, Jun 20, 2020 at 11:35 AM Seena Fallah <seenafallah@gmail.com>
wrote:
Hi. I found a default rocksdb option in bluestore that I can't find
in
facebook rocksdb. recycle_log_file_num this config if a boolean config in facebook rocksdb but in default Ceph configs the value of this is 4. Can someone tell what it means? _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
I would use 8 if I have max_write_buffer_number set to 8 as each write_buffer will have it's own WAL. If max_write_buffer_number is 1 I would use recycle_log_file_num=1. RocksDB tuning can get somewhat complex, see: https://github.com/facebook/rocksdb/wiki/RocksDB-Tuning-Guide On Sat, Jun 20, 2020 at 12:55 PM Seena Fallah <seenafallah@gmail.com> wrote:
Do you have any reason for this value? :))
On Sat, Jun 20, 2020 at 9:24 PM Frank R <frankaritchie@gmail.com> wrote:
With ceph I have always used it to set the number of WALs to recycle, ie to recycle 8 WALs I use:
" recycle_log_file_num=8 "
On Sat, Jun 20, 2020 at 12:47 PM Seena Fallah <seenafallah@gmail.com> wrote:
Yes here I found it’s a boolean (= true) Will this true convert to 1?
On Sat, Jun 20, 2020 at 9:15 PM Frank R <frankaritchie@gmail.com> wrote:
also see:
https://github.com/facebook/rocksdb/wiki/WAL-Performance
" Options.recycle_log_file_num = true will keep a pool of WAL files and try to reuse them. When writing to an existing log file, random writes are used from size 0. Before writes hit the end of the file, the file size doesn't change, so the I/O for metadata might be avoided (also depends on file system mount options). Assuming most WAL files will have similar sizes, I/O needed for metadata will be minimal. "
On Sat, Jun 20, 2020 at 12:43 PM Frank R <frankaritchie@gmail.com> wrote:
I believe it is the number of WALs that should be reused and should be equal to write_buffer_number but don't quote me.
On Sat, Jun 20, 2020 at 11:35 AM Seena Fallah <seenafallah@gmail.com> wrote:
Hi. I found a default rocksdb option in bluestore that I can't find in facebook rocksdb. recycle_log_file_num this config if a boolean config in facebook rocksdb but in default Ceph configs the value of this is 4. Can someone tell what it means? _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (2)
-
Frank R
-
Seena Fallah