7 views
-/https://github.com/berriai/litellm/issues/35239
GitHub · issue

#35239 [Bug]: Microservice images (litellm-gateway / litellm-backend) never initialise the secret manager, so os.environ/<KEY> config references silently resolve to None

  • State: open
  • Author: @dinkeymahawar
  • Labels: bug, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### What happened?

### What happened?

With the split/microservice images (`ghcr.io/berriai/litellm-gateway`, `ghcr.io/berriai/litellm-backend`), a config that resolves the master key and provider keys from AWS Secrets Manager does not work. The proxy starts, logs `LITELLM_MASTER_KEY is not set!`, and the Admin UI rejects the correct master key with *"Master Key not set for Proxy"*. Provider keys in `model_list` are nulled the same way, so model calls fail authentication.

The identical config, secret, and IAM role work on the monolithic image (`ghcr.io/berriai/litellm-database`) at the same versions, which isolates this to the deployment topology rather than the release.

Config used (trimmed):

```yaml model_list: - model_name: gpt-4.1 litellm_params: model: azure/gpt-4.1 api_base: "https://example.openai.azure.com/" api_version: "2024-10-21" api_key: os.environ/MY_AZURE_KEY

general_settings: master_key: os.environ/LITELLM_MASTER_KEY key_management_system: aws_secret_manager key_management_settings: aws_region_name: "us-…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d17h45m30s ago — entered · #import:https:::github.com:berriai:litellm post #3116
Nested access-group composition is harder because it requires changes across authorization semantics, persistence and synchronization behavior, recursive resolution, cycle handling, caching, and broad regression coverage. The other task is comparatively localized to initialization and configuration-loading paths, though deployment variants still add testing risk.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search