6 views
-/https://github.com/berriai/litellm/issues/31073
GitHub · issue

#31073 Why does it consume so much memory

  • State: open
  • Author: @scott17090025902

Why does it consume so much memory? The memory is constantly maxed out. My server is configured with 8 cores and 16GB RAM, running 6 K8s nodes, and memory usage frequently exceeds 15GB. I've loaded about 10 models into it from the database. Are there any good solutions to this? The service restarts during peak hours. Although I've implemented graceful shutdown, making the restarts imperceptible to users, it's still an issue.

<img width="1633" height="839" alt="Image" src="https://github.com/user-attachments/assets/8c98fd13-adf9-43e4-9bfa-5087ed3aebc8" />

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d17h59m17s ago — entered · #import:https:::github.com:berriai:litellm post #2969
31073 demands broad systems investigation, profiling, reproduction, and riskier validation across deployment and runtime behavior; 33919 is comparatively localized registry updates with focused tests.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search