We are constantly running into this issue. The mem_per_process does help
but it is just a heuristic and needs to be tuned quite frequently.
The last time this occurred, I did give it some thought. What if we tuned
the build system a little? We could identify the biggest memory hogs and
build them sequentially (post-parallel build?). I know ceph-dencoder was
super-memory-expensive (it required more than 4G of RAM even on 32-bit
architectures so we couldn't even build on 32-bit architectures anymore)
and it might be a first candidate on the 'sequential' list. Do we have any
other culprits?
What do you think?
Regards,
Boris
On Wed, Jan 27, 2021 at 8:34 PM Ken Dreyer <kdreyer(a)redhat.com> wrote:
Hi folks,
I'm seeing some of our internal Red Hat builders going OOM and killing
ceph builds. This is happening across architectures.
Upstream our braggi builders have 48 vCPUs and 256GB of RAM. That's not
small.
What is the minimum memory and CPU requirement for building pacific?
Internally, to use one ppc64le example, we're running with 14Gb RAM
and 16 CPUs, and the RPM spec file chooses -j5, hitting OOM. We tuned
mem_per_process from 2500 to 2700 a while back to alleviate this, but
we're still hitting OOM consistently with the pacific branch now.
- Ken
_______________________________________________
Dev mailing list -- dev(a)ceph.io
To unsubscribe send an email to dev-leave(a)ceph.io