Re: Nautilus OSD memory consumption?
On Thu, 27 Feb 2020 at 06:27, Anthony D'Atri <aad@dreamsnake.net> wrote:
If the heap stats reported by telling the OSD `heap stats` is large, telling each `heap release` may be useful. I suspect a TCMALLOC shortcoming.
osd.158 tcmalloc heap stats:------------------------------------------------ MALLOC: 5722761448 ( 5457.7 MiB) Bytes in use by application MALLOC: + 0 ( 0.0 MiB) Bytes in page heap freelist MALLOC: + 311621760 ( 297.2 MiB) Bytes in central cache freelist MALLOC: + 26242992 ( 25.0 MiB) Bytes in transfer cache freelist MALLOC: + 62721768 ( 59.8 MiB) Bytes in thread cache freelists MALLOC: + 113340608 ( 108.1 MiB) Bytes in malloc metadata MALLOC: ------------ MALLOC: = 6236688576 ( 5947.8 MiB) Actual memory used (physical + swap) MALLOC: + 21415870464 (20423.8 MiB) Bytes released to OS (aka unmapped) MALLOC: ------------ MALLOC: = 27652559040 (26371.5 MiB) Virtual address space used MALLOC: MALLOC: 394518 Spans in use MALLOC: 37 Thread heaps in use MALLOC: 8192 Tcmalloc page size ------------------------------------------------ Call ReleaseFreeMemory() to release freelist memory to the OS (via madvise()). Bytes released to the OS take up virtual address space but no physical memory.
On Thu, 27 Feb 2020 at 13:08, Nigel Williams <nigel.williams@tpac.org.au> wrote
On Thu, 27 Feb 2020 at 06:27, Anthony D'Atri <aad@dreamsnake.net> wrote:
If the heap stats reported by telling the OSD `heap stats` is large, telling each `heap release` may be useful. I suspect a TCMALLOC shortcoming.
heap release seemingly had no impact: root@cnx-11:~# ceph tell osd.158 heap release osd.158 releasing free RAM back to system. root@cnx-11:~# ceph tell osd.158 heap stats osd.158 tcmalloc heap stats:------------------------------------------------ MALLOC: 5722636664 ( 5457.5 MiB) Bytes in use by application MALLOC: + 0 ( 0.0 MiB) Bytes in page heap freelist MALLOC: + 315545536 ( 300.9 MiB) Bytes in central cache freelist MALLOC: + 25843216 ( 24.6 MiB) Bytes in transfer cache freelist MALLOC: + 56307896 ( 53.7 MiB) Bytes in thread cache freelists MALLOC: + 113340608 ( 108.1 MiB) Bytes in malloc metadata MALLOC: ------------ MALLOC: = 6233673920 ( 5944.9 MiB) Actual memory used (physical + swap) MALLOC: + 21418885120 (20426.6 MiB) Bytes released to OS (aka unmapped) MALLOC: ------------ MALLOC: = 27652559040 (26371.5 MiB) Virtual address space used MALLOC: MALLOC: 394287 Spans in use MALLOC: 37 Thread heaps in use MALLOC: 8192 Tcmalloc page size ------------------------------------------------ Call ReleaseFreeMemory() to release freelist memory to the OS (via madvise()). Bytes released to the OS take up virtual address space but no physical memory. root@cnx-11:~#
Hi Nigel, Any update to this issue? Br, Xu Yun
2020年2月27日 上午10:10，Nigel Williams <nigel.williams@tpac.org.au> 写道：
On Thu, 27 Feb 2020 at 13:08, Nigel Williams <nigel.williams@tpac.org.au> wrote
On Thu, 27 Feb 2020 at 06:27, Anthony D'Atri <aad@dreamsnake.net> wrote:
If the heap stats reported by telling the OSD `heap stats` is large, telling each `heap release` may be useful. I suspect a TCMALLOC shortcoming.
heap release seemingly had no impact:
root@cnx-11:~# ceph tell osd.158 heap release osd.158 releasing free RAM back to system. root@cnx-11:~# ceph tell osd.158 heap stats osd.158 tcmalloc heap stats:------------------------------------------------ MALLOC: 5722636664 ( 5457.5 MiB) Bytes in use by application MALLOC: + 0 ( 0.0 MiB) Bytes in page heap freelist MALLOC: + 315545536 ( 300.9 MiB) Bytes in central cache freelist MALLOC: + 25843216 ( 24.6 MiB) Bytes in transfer cache freelist MALLOC: + 56307896 ( 53.7 MiB) Bytes in thread cache freelists MALLOC: + 113340608 ( 108.1 MiB) Bytes in malloc metadata MALLOC: ------------ MALLOC: = 6233673920 ( 5944.9 MiB) Actual memory used (physical + swap) MALLOC: + 21418885120 (20426.6 MiB) Bytes released to OS (aka unmapped) MALLOC: ------------ MALLOC: = 27652559040 (26371.5 MiB) Virtual address space used MALLOC: MALLOC: 394287 Spans in use MALLOC: 37 Thread heaps in use MALLOC: 8192 Tcmalloc page size ------------------------------------------------ Call ReleaseFreeMemory() to release freelist memory to the OS (via madvise()). Bytes released to the OS take up virtual address space but no physical memory. root@cnx-11:~# _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
participants (2)
-
Nigel Williams
-
XuYun