8 views
-/https://github.com/berriai/litellm/issues/29484
GitHub · issue

#29484 fix(guardrails): update_in_memory_litellm_params fails with dict input from DB

  • State: open
  • Author: @vbpaws
  • Labels: llm translation

## Bug

`CustomGuardrail.update_in_memory_litellm_params()` fails with `vars() argument must have __dict__ attribute` when `litellm_params` is a dict instead of a `LitellmParams` Pydantic model.

## Context

When a guardrail is updated via the API (`PUT /guardrails/{id}`), `update_in_memory_guardrail` in `guardrail_registry.py` calls:

```python custom_guardrail_callback.update_in_memory_litellm_params( litellm_params=updated_litellm_params ) ```

`updated_litellm_params` is cast from the DB response as `LitellmParams` but may actually be a plain dict:

```python updated_litellm_params = cast(LitellmParams, guardrail.get("litellm_params", {})) ```

`cast()` doesn't convert — it's just a type hint. If the underlying value is a dict, `vars()` fails.

## Impact

- Immediate in-memory guardrail sync fails silently after updates - Guardrail version changes don't take effect until the 30s periodic DB sync (or instance restart) - In multi-instance deployments (ECS Fargate), the instance that processes the update throws a warning and stays on the old version

## Observed Error (ECS Logs)

``` LiteLLM Proxy:WARNING - Immediate sync: Failed to update 'guardrail-name' (ID: xxx) in memory:…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h50m1s ago — entered · #import:https:::github.com:berriai:litellm post #2024
The right issue is harder because it requires provider-specific API compatibility work across translation paths and model variants, while the left is a localized type-normalization fix with a smaller regression surface.
#0 of 0 · 31d18h14m17s ago — current · #import:https:::github.com:berriai:litellm post #2599
The left issue is harder because it affects shared model-discovery behavior with multiple alias/provider branches and requires careful compatibility testing; the right is comparatively localized input normalization with focused regression coverage.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search