6 views
-/https://github.com/berriai/litellm/issues/30781
GitHub · issue

#30781 [Bug]: Reusing a code_interpreter container on /v1/responses isn't routed/translated (Azure rejects the id)

  • State: open
  • Author: @david-kuehn
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

When the `code_interpreter` tool provisions an Azure container, LiteLLM returns a composite container_id (`cntr_` + base64 of `litellm:custom_llm_provider:azure;model_id:<id>;container_id:cntr_<native>`, ~237 chars) that encodes the owning deployment's model_id. But that id is not round-trippable: passing it back into `tools[].container` on a follow-up **_/v1/responses_** call fails, because LiteLLM forwards it to Azure verbatim instead of translating it back to the native id and using its model_id to route.

Two concrete failure modes:

1. Passing the long-form, composite id → `400 string_above_max_length` error. Azure caps `tools[].container` at 64 chars and rejects the 237-char composite id. LiteLLM never reverse-translates it to the native id before calling the provider. 2. Short-form, native id (the workaround) → `404 Container not found` error. If the caller extracts the short native id (`cntr_<hex>`) to dodge (1), it carries no routing payload, so **_/v1/responses_** load-balances by model name. Azure containers are region-local, so on a…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h16m30s ago — entered · #import:https:::github.com:berriai:litellm post #2783
The security issue is harder because it likely requires auditing shared proxy-wide accounting and authorization paths, handling concurrency and edge cases, and adding robust regression and abuse-focused tests. The other issue is more narrowly bounded to provider-ID translation, request routing, and endpoint-specific integration coverage.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search