8 views
-/https://github.com/berriai/litellm/issues/33986
GitHub · issue

#33986 [Bug]: Managed Bedrock batches cannot be cancelled through /v1/batches/{id}/cancel

  • State: open
  • Author: @jimmychen-p72
  • Labels: llm translation

## What happened

A LiteLLM managed Bedrock batch can be created and reach `InProgress`, but `POST /v1/batches/{managed_id}/cancel` fails before reaching AWS:

```text LiteLLM doesn't support bedrock for `cancel_batch`. Only openai, azure, and vertex_ai are supported. ```

The equivalent provider operation is AWS Bedrock `StopModelInvocationJob`.

## Reproduction

1. Configure a managed Bedrock batch model with `model_info.mode: batch`, an S3 staging bucket, and `aws_batch_role_arn`. 2. `POST /v1/files` with a valid 100-record JSONL file. 3. `POST /v1/batches` using the managed input file and wait for the provider job to reach `InProgress`. 4. `POST /v1/batches/{managed_batch_id}/cancel`.

Expected: LiteLLM calls `StopModelInvocationJob` and returns an OpenAI-compatible batch resource with `status=cancelling`, then `cancelled` when Bedrock reports `Stopped`.

Actual: LiteLLM returns HTTP 400 without calling Bedrock.

## Root cause

`litellm.cancel_batch` / `acancel_batch` only dispatches OpenAI, Azure, and Vertex batch cancellation. Bedrock has a retrieve handler for `GetModelInvocationJob`, but no corresponding cancellation handler for `StopModelInvocationJob`.

## Proposed behavi…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (2 events)
#0 of 0 · 31d18h3m15s ago — entered · #import:https:::github.com:berriai:litellm post #2810
The left requires coordinated provider integration, sync/async dispatch changes, lifecycle-state handling, and regression coverage. The right appears primarily diagnostic and may depend on external rate-limit behavior, with no comparable implementation scope.
#0 of 0 · 31d17h26m20s ago — current · #import:https:::github.com:berriai:litellm post #3445
The left issue is harder because it requires a secure, cross-provider change to credential resolution and endpoint validation, with substantial regression risk across existing authentication flows. The right issue is comparatively localized to adding and wiring one provider-specific batch cancellation operation and its status mapping.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search