Skip to content

Commit 3990f61

Browse files
committed
Refactor Anthropic Configurations and Add Support for anthropic_beta Headers
- Renamed `AmazonAnthropicClaude3Config` and `AmazonAnthropicClaude3MessagesConfig` to `AmazonAnthropicClaudeConfig` and `AmazonAnthropicClaudeMessagesConfig` respectively for consistency. - Implemented `get_anthropic_beta_from_headers` function to extract and handle `anthropic-beta` headers across various transformations. - Updated request transformations in `AmazonConverseConfig` and `AmazonInvokeConfig` to include `anthropic_beta` parameters based on user headers. - Added tests to ensure proper handling of `anthropic_beta` headers in different scenarios.
1 parent 37e57a0 commit 3990f61

File tree

11 files changed

+389
-13
lines changed

11 files changed

+389
-13
lines changed

docs/my-website/docs/providers/bedrock.md

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -584,6 +584,143 @@ Same as [Anthropic API response](../providers/anthropic#usage---thinking--reason
584584
Same as [Anthropic API response](../providers/anthropic#usage---thinking--reasoning_content).
585585

586586

587+
## Usage - Anthropic Beta Features
588+
589+
LiteLLM supports Anthropic's beta features on AWS Bedrock through the `anthropic-beta` header. This enables access to experimental features like:
590+
591+
- **1M Context Window** - Up to 1 million tokens of context (Claude Sonnet 4)
592+
- **Computer Use Tools** - AI that can interact with computer interfaces
593+
- **Token-Efficient Tools** - More efficient tool usage patterns
594+
- **Extended Output** - Up to 128K output tokens
595+
- **Enhanced Thinking** - Advanced reasoning capabilities
596+
597+
### Supported Beta Features
598+
599+
| Beta Feature | Header Value | Compatible Models | Description |
600+
|--------------|-------------|------------------|-------------|
601+
| 1M Context Window | `context-1m-2025-08-07` | Claude Sonnet 4 | Enable 1 million token context window |
602+
| Computer Use (Latest) | `computer-use-2025-01-24` | Claude 3.7 Sonnet | Latest computer use tools |
603+
| Computer Use (Legacy) | `computer-use-2024-10-22` | Claude 3.5 Sonnet v2 | Computer use tools for Claude 3.5 |
604+
| Token-Efficient Tools | `token-efficient-tools-2025-02-19` | Claude 3.7 Sonnet | More efficient tool usage |
605+
| Interleaved Thinking | `interleaved-thinking-2025-05-14` | Claude 4 models | Enhanced thinking capabilities |
606+
| Extended Output | `output-128k-2025-02-19` | Claude 3.7 Sonnet | Up to 128K output tokens |
607+
| Developer Thinking | `dev-full-thinking-2025-05-14` | Claude 4 models | Raw thinking mode for developers |
608+
609+
<Tabs>
610+
<TabItem value="sdk" label="SDK">
611+
612+
**Single Beta Feature**
613+
614+
```python
615+
from litellm import completion
616+
import os
617+
618+
# set env
619+
os.environ["AWS_ACCESS_KEY_ID"] = ""
620+
os.environ["AWS_SECRET_ACCESS_KEY"] = ""
621+
os.environ["AWS_REGION_NAME"] = ""
622+
623+
# Use 1M context window with Claude Sonnet 4
624+
response = completion(
625+
model="bedrock/anthropic.claude-sonnet-4-20250115-v1:0",
626+
messages=[{"role": "user", "content": "Hello! Testing 1M context window."}],
627+
max_tokens=100,
628+
extra_headers={
629+
"anthropic-beta": "context-1m-2025-08-07" # 👈 Enable 1M context
630+
}
631+
)
632+
```
633+
634+
**Multiple Beta Features**
635+
636+
```python
637+
from litellm import completion
638+
639+
# Combine multiple beta features (comma-separated)
640+
response = completion(
641+
model="bedrock/converse/anthropic.claude-3-5-sonnet-20241022-v2:0",
642+
messages=[{"role": "user", "content": "Testing multiple beta features"}],
643+
max_tokens=100,
644+
extra_headers={
645+
"anthropic-beta": "computer-use-2024-10-22,context-1m-2025-08-07"
646+
}
647+
)
648+
```
649+
650+
**Computer Use Tools with Beta Features**
651+
652+
```python
653+
from litellm import completion
654+
655+
# Computer use tools automatically add computer-use-2024-10-22
656+
# You can add additional beta features
657+
response = completion(
658+
model="bedrock/converse/anthropic.claude-3-5-sonnet-20241022-v2:0",
659+
messages=[{"role": "user", "content": "Take a screenshot"}],
660+
tools=[{
661+
"type": "computer_20241022",
662+
"name": "computer",
663+
"display_width_px": 1920,
664+
"display_height_px": 1080
665+
}],
666+
extra_headers={
667+
"anthropic-beta": "context-1m-2025-08-07" # Additional beta feature
668+
}
669+
)
670+
```
671+
672+
</TabItem>
673+
<TabItem value="proxy" label="PROXY">
674+
675+
**Set on YAML Config**
676+
677+
```yaml
678+
model_list:
679+
- model_name: claude-sonnet-4-1m
680+
litellm_params:
681+
model: bedrock/anthropic.claude-sonnet-4-20250115-v1:0
682+
extra_headers:
683+
anthropic-beta: "context-1m-2025-08-07" # 👈 Enable 1M context
684+
685+
- model_name: claude-computer-use
686+
litellm_params:
687+
model: bedrock/converse/anthropic.claude-3-5-sonnet-20241022-v2:0
688+
extra_headers:
689+
anthropic-beta: "computer-use-2024-10-22,context-1m-2025-08-07"
690+
```
691+
692+
**Set on Request**
693+
694+
```python
695+
import openai
696+
697+
client = openai.OpenAI(
698+
api_key="anything",
699+
base_url="http://0.0.0.0:4000"
700+
)
701+
702+
response = client.chat.completions.create(
703+
model="claude-sonnet-4-1m",
704+
messages=[{
705+
"role": "user",
706+
"content": "Testing 1M context window"
707+
}],
708+
extra_headers={
709+
"anthropic-beta": "context-1m-2025-08-07"
710+
}
711+
)
712+
```
713+
714+
</TabItem>
715+
</Tabs>
716+
717+
:::info
718+
719+
Beta features may require special access or permissions in your AWS account. Some features are only available in specific AWS regions. Check the [AWS Bedrock documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages-request-response.html) for availability and access requirements.
720+
721+
:::
722+
723+
587724
## Usage - Structured Output / JSON mode
588725

589726
<Tabs>

litellm/__init__.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1041,7 +1041,7 @@ def add_known_models():
10411041
AnthropicMessagesConfig,
10421042
)
10431043
from .llms.bedrock.messages.invoke_transformations.anthropic_claude3_transformation import (
1044-
AmazonAnthropicClaude3MessagesConfig,
1044+
AmazonAnthropicClaudeMessagesConfig,
10451045
)
10461046
from .llms.together_ai.chat import TogetherAIConfig
10471047
from .llms.together_ai.completion.transformation import TogetherAITextCompletionConfig
@@ -1101,7 +1101,7 @@ def add_known_models():
11011101
AmazonAnthropicConfig,
11021102
)
11031103
from .llms.bedrock.chat.invoke_transformations.anthropic_claude3_transformation import (
1104-
AmazonAnthropicClaude3Config,
1104+
AmazonAnthropicClaudeConfig,
11051105
)
11061106
from .llms.bedrock.chat.invoke_transformations.amazon_cohere_transformation import (
11071107
AmazonCohereConfig,

litellm/llms/bedrock/chat/converse_handler.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,6 +119,7 @@ async def async_streaming(
119119
messages=messages,
120120
optional_params=optional_params,
121121
litellm_params=litellm_params,
122+
headers=headers,
122123
)
123124
data = json.dumps(request_data)
124125

@@ -185,6 +186,7 @@ async def async_completion(
185186
messages=messages,
186187
optional_params=optional_params,
187188
litellm_params=litellm_params,
189+
headers=headers,
188190
)
189191
data = json.dumps(request_data)
190192
prepped = self.get_request_headers(
@@ -390,6 +392,7 @@ def completion( # noqa: PLR0915
390392
messages=messages,
391393
optional_params=optional_params,
392394
litellm_params=litellm_params,
395+
headers=extra_headers,
393396
)
394397
data = json.dumps(_data)
395398
prepped = self.get_request_headers(

litellm/llms/bedrock/chat/converse_transformation.py

Lines changed: 26 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@
4747
)
4848
from litellm.utils import add_dummy_tool, has_tool_call_blocks, supports_reasoning
4949

50-
from ..common_utils import BedrockError, BedrockModelInfo, get_bedrock_tool_name
50+
from ..common_utils import BedrockError, BedrockModelInfo, get_bedrock_tool_name, get_anthropic_beta_from_headers
5151

5252
# Computer use tool prefixes supported by Bedrock
5353
BEDROCK_COMPUTER_USE_TOOLS = [
@@ -593,12 +593,14 @@ def _handle_top_k_value(self, model: str, inference_params: dict) -> dict:
593593

594594
return {}
595595

596+
596597
def _transform_request_helper(
597598
self,
598599
model: str,
599600
system_content_blocks: List[SystemContentBlock],
600601
optional_params: dict,
601602
messages: Optional[List[AllMessageValues]] = None,
603+
headers: Optional[dict] = None,
602604
) -> CommonRequestObject:
603605
## VALIDATE REQUEST
604606
"""
@@ -651,6 +653,12 @@ def _transform_request_helper(
651653
# Initialize bedrock_tools
652654
bedrock_tools: List[ToolBlock] = []
653655

656+
# Collect anthropic_beta values from user headers
657+
anthropic_beta_list = []
658+
if headers:
659+
user_betas = get_anthropic_beta_from_headers(headers)
660+
anthropic_beta_list.extend(user_betas)
661+
654662
# Only separate tools if computer use tools are actually present
655663
if original_tools and self.is_computer_use_tool_used(original_tools, model):
656664
# Separate computer use tools from regular function tools
@@ -663,14 +671,25 @@ def _transform_request_helper(
663671

664672
# Add computer use tools and anthropic_beta if needed (only when computer use tools are present)
665673
if computer_use_tools:
666-
additional_request_params["anthropic_beta"] = ["computer-use-2024-10-22"]
674+
anthropic_beta_list.append("computer-use-2024-10-22")
667675
# Transform computer use tools to proper Bedrock format
668676
transformed_computer_tools = self._transform_computer_use_tools(computer_use_tools)
669677
additional_request_params["tools"] = transformed_computer_tools
670678
else:
671679
# No computer use tools, process all tools as regular tools
672680
bedrock_tools = _bedrock_tools_pt(original_tools)
673681

682+
# Set anthropic_beta in additional_request_params if we have any beta features
683+
if anthropic_beta_list:
684+
# Remove duplicates while preserving order
685+
unique_betas = []
686+
seen = set()
687+
for beta in anthropic_beta_list:
688+
if beta not in seen:
689+
unique_betas.append(beta)
690+
seen.add(beta)
691+
additional_request_params["anthropic_beta"] = unique_betas
692+
674693
bedrock_tool_config: Optional[ToolConfigBlock] = None
675694
if len(bedrock_tools) > 0:
676695
tool_choice_values: ToolChoiceValuesBlock = inference_params.pop(
@@ -708,6 +727,7 @@ async def _async_transform_request(
708727
messages: List[AllMessageValues],
709728
optional_params: dict,
710729
litellm_params: dict,
730+
headers: Optional[dict] = None,
711731
) -> RequestObject:
712732
messages, system_content_blocks = self._transform_system_message(messages)
713733
## TRANSFORMATION ##
@@ -717,6 +737,7 @@ async def _async_transform_request(
717737
system_content_blocks=system_content_blocks,
718738
optional_params=optional_params,
719739
messages=messages,
740+
headers=headers,
720741
)
721742

722743
bedrock_messages = (
@@ -747,6 +768,7 @@ def transform_request(
747768
messages=messages,
748769
optional_params=optional_params,
749770
litellm_params=litellm_params,
771+
headers=headers,
750772
),
751773
)
752774

@@ -756,6 +778,7 @@ def _transform_request(
756778
messages: List[AllMessageValues],
757779
optional_params: dict,
758780
litellm_params: dict,
781+
headers: Optional[dict] = None,
759782
) -> RequestObject:
760783
messages, system_content_blocks = self._transform_system_message(messages)
761784

@@ -764,6 +787,7 @@ def _transform_request(
764787
system_content_blocks=system_content_blocks,
765788
optional_params=optional_params,
766789
messages=messages,
790+
headers=headers,
767791
)
768792

769793
## TRANSFORMATION ##

litellm/llms/bedrock/chat/invoke_handler.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -831,7 +831,7 @@ def completion( # noqa: PLR0915
831831
model=model, messages=messages, custom_llm_provider="anthropic_xml"
832832
) # type: ignore
833833
## LOAD CONFIG
834-
config = litellm.AmazonAnthropicClaude3Config.get_config()
834+
config = litellm.AmazonAnthropicClaudeConfig.get_config()
835835
for k, v in config.items():
836836
if (
837837
k not in inference_params

litellm/llms/bedrock/chat/invoke_transformations/anthropic_claude3_transformation.py

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
from litellm.llms.bedrock.chat.invoke_transformations.base_invoke_transformation import (
77
AmazonInvokeConfig,
88
)
9+
from litellm.llms.bedrock.common_utils import get_anthropic_beta_from_headers
910
from litellm.types.llms.openai import AllMessageValues
1011
from litellm.types.utils import ModelResponse
1112

@@ -17,13 +18,22 @@
1718
LiteLLMLoggingObj = Any
1819

1920

20-
class AmazonAnthropicClaude3Config(AmazonInvokeConfig, AnthropicConfig):
21+
class AmazonAnthropicClaudeConfig(AmazonInvokeConfig, AnthropicConfig):
2122
"""
2223
Reference:
2324
https://us-west-2.console.aws.amazon.com/bedrock/home?region=us-west-2#/providers?model=claude
2425
https://docs.anthropic.com/claude/docs/models-overview#model-comparison
26+
https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages-request-response.html
2527
26-
Supported Params for the Amazon / Anthropic Claude 3 models:
28+
Supported Params for the Amazon / Anthropic Claude models (Claude 3, Claude 4, etc.):
29+
Supports anthropic_beta parameter for beta features like:
30+
- computer-use-2025-01-24 (Claude 3.7 Sonnet)
31+
- computer-use-2024-10-22 (Claude 3.5 Sonnet v2)
32+
- token-efficient-tools-2025-02-19 (Claude 3.7 Sonnet)
33+
- interleaved-thinking-2025-05-14 (Claude 4 models)
34+
- output-128k-2025-02-19 (Claude 3.7 Sonnet)
35+
- dev-full-thinking-2025-05-14 (Claude 4 models)
36+
- context-1m-2025-08-07 (Claude Sonnet 4)
2737
"""
2838

2939
anthropic_version: str = "bedrock-2023-05-31"
@@ -50,6 +60,7 @@ def map_openai_params(
5060
drop_params,
5161
)
5262

63+
5364
def transform_request(
5465
self,
5566
model: str,
@@ -72,6 +83,11 @@ def transform_request(
7283
if "anthropic_version" not in _anthropic_request:
7384
_anthropic_request["anthropic_version"] = self.anthropic_version
7485

86+
# Handle anthropic_beta from user headers
87+
anthropic_beta_list = get_anthropic_beta_from_headers(headers)
88+
if anthropic_beta_list:
89+
_anthropic_request["anthropic_beta"] = anthropic_beta_list
90+
7591
return _anthropic_request
7692

7793
def transform_response(

litellm/llms/bedrock/chat/invoke_transformations/base_invoke_transformation.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -190,7 +190,7 @@ def transform_request(
190190
] = True # cohere requires stream = True in inference params
191191
request_data = {"prompt": prompt, **inference_params}
192192
elif provider == "anthropic":
193-
return litellm.AmazonAnthropicClaude3Config().transform_request(
193+
return litellm.AmazonAnthropicClaudeConfig().transform_request(
194194
model=model,
195195
messages=messages,
196196
optional_params=optional_params,
@@ -293,7 +293,7 @@ def transform_response( # noqa: PLR0915
293293
completion_response["generations"][0]["finish_reason"]
294294
)
295295
elif provider == "anthropic":
296-
return litellm.AmazonAnthropicClaude3Config().transform_response(
296+
return litellm.AmazonAnthropicClaudeConfig().transform_response(
297297
model=model,
298298
raw_response=raw_response,
299299
model_response=model_response,

litellm/llms/bedrock/common_utils.py

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -524,3 +524,25 @@ def _parse_message_from_event(self, event) -> Optional[str]:
524524
return None
525525

526526
return chunk.decode() # type: ignore[no-any-return]
527+
528+
529+
def get_anthropic_beta_from_headers(headers: dict) -> List[str]:
530+
"""
531+
Extract anthropic-beta header values and convert them to a list.
532+
Supports comma-separated values from user headers.
533+
534+
Used by both converse and invoke transformations for consistent handling
535+
of anthropic-beta headers that should be passed to AWS Bedrock.
536+
537+
Args:
538+
headers (dict): Request headers dictionary
539+
540+
Returns:
541+
List[str]: List of anthropic beta feature strings, empty list if no header
542+
"""
543+
anthropic_beta_header = headers.get("anthropic-beta")
544+
if not anthropic_beta_header:
545+
return []
546+
547+
# Split comma-separated values and strip whitespace
548+
return [beta.strip() for beta in anthropic_beta_header.split(",")]

0 commit comments

Comments
 (0)