8 views
-/https://github.com/berriai/litellm/issues/30195
GitHub · issue

#30195 [Feature]: Using libjemalloc2 for efficient memory management and garbage collection

  • State: open
  • Author: @Swipe4057
  • Labels: enhancement, proxy

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

In our production environment, we add libjemalloc2 as a memory manager during image build. Memory consumption at service startup and after applying migrations decreased from 1.2 GB to 700 MB. Uncontrolled memory accumulation during long-running service operation has been completely eliminated; memory is now efficiently cleaned and freed (effective flushes occur approximately every hour according to graphs). No impact on performance—only efficient garbage collection and OOM protection.

### Motivation, pitch

Install libjemalloc2 + specify the following commands in Dockerfile: ``` ENV LD_PRELOAD="/usr/lib/x86_64-linux-gnu/libjemalloc.so.2" ENV MALLOC_CONF="background_thread:true,metadata_thp:auto,dirty_decay_ms:30000,muzzy_decay_ms:30000" ```

### What part of LiteLLM is this about?

Proxy

### LiteLLM is hiring a founding backend engineer, are you interested in joining us and shipping to all our users?

No

### Twitter / LinkedIn details

_No response_

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h3m36s ago — entered · #import:https:::github.com:berriai:litellm post #3013
The left issue requires container-image changes, runtime-level allocator integration, platform compatibility checks, and long-running memory/regression validation. The right issue is narrower and is more likely addressed through certificate configuration or HTTP-client troubleshooting.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search