18 views
-/https://github.com/berriai/litellm/issues/22966
GitHub · issue

#22966 [Feature]: Key-level tag routing enforcement (allowed_tags or enforced_tags on virtual keys)

  • State: open
  • Author: @MiloParigi
  • Labels: enhancement, proxy, llm translation

### Check for existing issues

- [x] I have searched the existing issues and checked that my issue is not a duplicate.

### The Feature

Add an `allowed_tags` field to virtual keys that enforces tag-based deployment routing at the key level, independently of what tags the caller sends in the request body.

When `allowed_tags` is set on a key, requests using that key are validated against that whitelist before routing. If the caller sends tags not present in `allowed_tags`, the request is rejected with a 401. If the caller sends no tags, the `allowed_tags` are injected as the default. This ensures a key scoped to `["region:eu"]` can never - accidentally or deliberately - route to a non-EU deployment.

An alternative simpler design would be `enforced_tags`, where the key always forces specific tags regardless of what the caller sends. `allowed_tags` is more flexible and covers a broader set of use cases, which is why I explored it here - but I'm open to discussing which approach makes more sense.

Note that this is also distinct from the existing `metadata.tags` field on keys, which is used solely for spend tracking. `allowed_tags` would operate at the routing layer, not the logging …

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (6 events)
#0 of 0 · 31d19h5m9s ago — entered · #import:https:::github.com:berriai:litellm post #1786
The left issue is harder because it introduces a new key-level authorization concept spanning key schemas, request validation, tag precedence, routing integration, backward compatibility, and comprehensive security-focused tests. The right issue is a narrower behavioral defect in the interaction between two existing selection mechanisms, likely requiring targeted control-flow changes and regression coverage.
The right-side feature spans key configuration, authentication, routing precedence, validation, persistence, compatibility, and broad test coverage. The left-side bug is more localized to provider-specific request translation and regression tests, despite requiring careful schema handling.
Key-level authorization and routing changes span configuration, request validation, precedence rules, persistence, compatibility, and broad test coverage, making them substantially larger and riskier than a localized streaming response-framing fix.
The right-side work is broader: it requires coordinated changes across virtual-key configuration, request validation, routing behavior, precedence rules, compatibility, and extensive proxy-level testing. The left-side work is a more localized async streaming error-path fix with targeted regression coverage.
The right issue is harder because it requires coordinated changes across virtual-key configuration, request validation, tag precedence, routing behavior, authentication responses, and compatibility tests. The left issue is serious but comparatively localized to metadata copying/serialization and spend-queue failure handling.
#0 of 0 · 31d17h58m36s ago — current · #import:https:::github.com:berriai:litellm post #2891
The right-side change has substantially greater scope and risk: it affects credential data modeling, authorization semantics, routing precedence, backward compatibility, and integration testing. The left-side work is comparatively narrower, centered on diagnosing and correcting authentication configuration behavior.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search