#33955 [Bug]: CheckBatchCost fully buffers batch output (incl. base64 images) in memory, unlocked across workers ā OOM on image-generation batches
**Check for existing issues:** Searched existing issues ā related but distinct: #33316, #30635, #28294, #27944, #27891 (all cost-attribution bugs ā wrong IDs, 403s, cost showing 0 ā not the memory/concurrency issue below).
**Issue classification:** Out of Memory (OOM) events or pod restarts.
**What happened?**
Running litellm proxy v1.91.0 with `num_workers=2` per pod (4 replicas, 8 worker processes total) and a client submitting Vertex AI image-generation batches (`vertex_ai/gemini-*-image*`) via `/v1/files` + `/v1/batches`. 4 replicas were OOM-killed within a 15-minute window, each showing the same pattern: memory climbs to a steady ~4-4.5GB plateau over 1.5-2h, then a fast spike (confirmed via `container_memory_max_usage_bytes`, up to 5.59GB) crosses the container's memory limit in under a minute ā faster than any periodic metrics scrape resolves.
No recent litellm version upgrade preceded this incident ā the deployment had been stable on v1.91.0.
Root cause, traced through the source: 1. `CheckBatchCost`'s output-file cost-tracking path (`enterprise/litellm_enterprise/proxy/common_utils/check_batch_cost.py`) fully downloads a completed batch's GCS output file, transforms iā¦