Skip to content

Commit a2e3b94

Browse files
Sameerliteishaan-jaff
Sameerlite
and
ishaan-jaff
authored
Vector store files Stable Release (#16643)
* Add support for vector store files endpoints (#16490) * Add base code for vector store integration * fix azure related tests and linting error * fix mypy errors * Add vector store files documentation * fix mapped tests * Add bytedance and ideogram support in fal ai (#16636) * Add fal ai flux pro v1.1 support (#16578) * Add fal ai flux pro v1.1 support * Add tests and docs --------- Co-authored-by: Ishaan Jaffer <[email protected]>
1 parent 59d17cc commit a2e3b94

File tree

35 files changed

+4066
-21
lines changed

35 files changed

+4066
-21
lines changed

‎docs/my-website/docs/providers/fal_ai.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,10 +31,14 @@ Get your API key from [fal.ai](https://fal.ai/).
3131

3232
| Model Name | Description | Documentation |
3333
|------------|-------------|---------------|
34+
| `fal_ai/fal-ai/flux-pro/v1.1` | FLUX Pro v1.1 - Balanced speed and quality | [Docs ↗](https://fal.ai/models/fal-ai/flux-pro/v1.1) |
3435
| `fal_ai/flux/schnell` | Flux Schnell - Low-latency generation with `image_size` support | [Docs ↗](https://fal.ai/models/fal-ai/flux/schnell) |
36+
| `fal_ai/fal-ai/bytedance/seedream/v3/text-to-image` | ByteDance Seedream v3 - Text-to-image with `image_size` control | [Docs ↗](https://fal.ai/models/fal-ai/bytedance/seedream/v3/text-to-image) |
37+
| `fal_ai/fal-ai/bytedance/dreamina/v3.1/text-to-image` | ByteDance Dreamina v3.1 - Text-to-image with `image_size` control | [Docs ↗](https://fal.ai/models/fal-ai/bytedance/dreamina/v3.1/text-to-image) |
3538
| `fal_ai/fal-ai/flux-pro/v1.1-ultra` | FLUX Pro v1.1 Ultra - High-quality image generation | [Docs ↗](https://fal.ai/models/fal-ai/flux-pro/v1.1-ultra) |
3639
| `fal_ai/fal-ai/imagen4/preview` | Google's Imagen 4 - Highest quality model | [Docs ↗](https://fal.ai/models/fal-ai/imagen4/preview) |
3740
| `fal_ai/fal-ai/recraft/v3/text-to-image` | Recraft v3 - Multiple style options | [Docs ↗](https://fal.ai/models/fal-ai/recraft/v3/text-to-image) |
41+
| `fal_ai/fal-ai/ideogram/v3` | Ideogram v3 - Lettering-first creative model (Balanced: $0.06/image) | [Docs ↗](https://fal.ai/models/fal-ai/ideogram/v3) |
3842
| `fal_ai/fal-ai/stable-diffusion-v35-medium` | Stable Diffusion v3.5 Medium | [Docs ↗](https://fal.ai/models/fal-ai/stable-diffusion-v35-medium) |
3943
| `fal_ai/bria/text-to-image/3.2` | Bria 3.2 - Commercial-grade generation | [Docs ↗](https://fal.ai/models/bria/text-to-image/3.2) |
4044

‎docs/my-website/docs/vector_store_files.md‎

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
# /vector_stores/{vector_store_id}/files
2+
3+
Vector store files represent the individual files that live inside a vector store.
4+
5+
| Feature | Supported |
6+
|---------|-----------|
7+
| Logging | ✅ (full request/response logging) |
8+
| Supported Providers | `openai` |
9+
10+
11+
## Supported operations
12+
13+
| Operation | Description | OpenAI Python Client | LiteLLM Proxy |
14+
|-----------|-------------|----------------------|---------------|
15+
| Create vector store file | Attach a file to a vector store with optional chunking overrides | ✅ | ✅ |
16+
| List vector store files | Paginated listing with filters | ✅ | ✅ |
17+
| Retrieve vector store file | Fetch metadata for a single file | ✅ | ✅ |
18+
| Delete vector store file | Remove a file from a store (file object persists) | ✅ | ✅ |
19+
| Retrieve vector store file content | Stream processed chunks | ❌ | ✅ |
20+
| Update vector store file attributes | Patch custom attributes | ❌ | ✅ |
21+
22+
:::note
23+
Vector store support currently works **only with OpenAI vector stores and OpenAI-uploaded file IDs**.
24+
:::
25+
26+
27+
## Create vector store file
28+
29+
`POST http://localhost:4000/v1/vector_stores/{vector_store_id}/files`
30+
31+
```python
32+
from openai import OpenAI
33+
34+
client = OpenAI(
35+
base_url="http://localhost:4000", # LiteLLM proxy or OpenAI base
36+
api_key="sk-1234"
37+
)
38+
39+
vector_store_file = client.vector_stores.files.create(
40+
vector_store_id="vs_69172088a18c8191ab3e2621aa87d1ee",
41+
file_id="file-NDbEDJTfqVh7S4Ugi3CGYw",
42+
chunking_strategy={
43+
"type": "static",
44+
"static": {
45+
"max_chunk_size_tokens": 800,
46+
"chunk_overlap_tokens": 400,
47+
},
48+
},
49+
)
50+
51+
print(vector_store_file)
52+
```
53+
54+
## List vector store files
55+
56+
`GET http://localhost:4000/v1/vector_stores/{vector_store_id}/files`
57+
58+
Parameters:
59+
60+
- `vector_store_id` (path, required)
61+
- `after` / `before` (query, optional) – pagination cursors
62+
- `filter` (query, optional) – `in_progress`, `completed`, `failed`, `cancelled`
63+
- `limit` (query, optional, default `20`, range `1-100`)
64+
- `order` (query, optional, default `desc`)
65+
66+
```python
67+
vector_store_files = client.vector_stores.files.list(
68+
vector_store_id="vs_abc123"
69+
)
70+
print(vector_store_files)
71+
```
72+
73+
## Retrieve vector store file
74+
75+
`GET http://localhost:4000/v1/vector_stores/{vector_store_id}/files/{file_id}`
76+
77+
```python
78+
vector_store_file = client.vector_stores.files.retrieve(
79+
vector_store_id="vs_abc123",
80+
file_id="file-abc123"
81+
)
82+
print(vector_store_file)
83+
```
84+
85+
## Delete vector store file
86+
87+
`DELETE http://localhost:4000/v1/vector_stores/{vector_store_id}/files/{file_id}`
88+
89+
```python
90+
deleted_vector_store_file = client.vector_stores.files.delete(
91+
vector_store_id="vs_abc123",
92+
file_id="file-abc123"
93+
)
94+
print(deleted_vector_store_file)
95+
```
96+
97+
## Proxy-only endpoints
98+
99+
When you need raw content chunks or attribute updates, call the LiteLLM Proxy directly.
100+
101+
### Retrieve file content
102+
103+
```bash
104+
curl -X GET "http://localhost:4000/v1/vector_stores/{vector_store_id}/files/{file_id}/content" \
105+
-H "Authorization: Bearer sk-1234"
106+
```
107+
108+
### Update file attributes
109+
110+
```bash
111+
curl -X POST "http://localhost:4000/v1/vector_stores/{vector_store_id}/files/{file_id}" \
112+
-H "Authorization: Bearer sk-1234" \
113+
-H "Content-Type: application/json" \
114+
-d '{
115+
"attributes": {
116+
"category": "support-faq",
117+
"language": "en"
118+
}
119+
}'
120+
```

‎docs/my-website/sidebars.js‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -368,6 +368,7 @@ const sidebars = {
368368
]
369369
},
370370
"videos",
371+
"vector_store_files",
371372
{
372373
type: "category",
373374
label: "/mcp - Model Context Protocol",

‎litellm/__init__.py‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1386,6 +1386,20 @@ def add_known_models():
13861386
from .realtime_api.main import _arealtime
13871387
from .fine_tuning.main import *
13881388
from .files.main import *
1389+
from .vector_store_files.main import (
1390+
acreate as avector_store_file_create,
1391+
adelete as avector_store_file_delete,
1392+
alist as avector_store_file_list,
1393+
aretrieve as avector_store_file_retrieve,
1394+
aretrieve_content as avector_store_file_content,
1395+
aupdate as avector_store_file_update,
1396+
create as vector_store_file_create,
1397+
delete as vector_store_file_delete,
1398+
list as vector_store_file_list,
1399+
retrieve as vector_store_file_retrieve,
1400+
retrieve_content as vector_store_file_content,
1401+
update as vector_store_file_update,
1402+
)
13891403
from .scheduler import *
13901404
from .cost_calculator import response_cost_calculator, cost_per_token
13911405

‎litellm/llms/base_llm/vector_store_files/transformation.py‎

Lines changed: 226 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,226 @@
1+
from abc import ABC, abstractmethod
2+
from typing import TYPE_CHECKING, Any, Dict, Optional, Tuple, Union
3+
4+
import httpx
5+
6+
from litellm.types.router import GenericLiteLLMParams
7+
from litellm.types.vector_store_files import (
8+
VectorStoreFileAuthCredentials,
9+
VectorStoreFileChunkingStrategy,
10+
VectorStoreFileContentResponse,
11+
VectorStoreFileCreateRequest,
12+
VectorStoreFileDeleteResponse,
13+
VectorStoreFileListQueryParams,
14+
VectorStoreFileListResponse,
15+
VectorStoreFileObject,
16+
VectorStoreFileUpdateRequest,
17+
)
18+
19+
if TYPE_CHECKING:
20+
from litellm.litellm_core_utils.litellm_logging import Logging as _LiteLLMLoggingObj
21+
22+
from ..chat.transformation import BaseLLMException as _BaseLLMException
23+
24+
LiteLLMLoggingObj = _LiteLLMLoggingObj
25+
BaseLLMException = _BaseLLMException
26+
else:
27+
LiteLLMLoggingObj = Any
28+
BaseLLMException = Any
29+
30+
31+
class BaseVectorStoreFilesConfig(ABC):
32+
"""Base configuration contract for provider-specific vector store file implementations."""
33+
34+
def get_supported_openai_params(
35+
self,
36+
operation: str,
37+
) -> Tuple[str, ...]:
38+
"""Return the set of OpenAI params supported for the given operation."""
39+
40+
return tuple()
41+
42+
def map_openai_params(
43+
self,
44+
*,
45+
operation: str,
46+
non_default_params: Dict[str, Any],
47+
optional_params: Dict[str, Any],
48+
drop_params: bool,
49+
) -> Dict[str, Any]:
50+
"""Map non-default OpenAI params to provider-specific params."""
51+
52+
return optional_params
53+
54+
@abstractmethod
55+
def get_auth_credentials(
56+
self, litellm_params: Dict[str, Any]
57+
) -> VectorStoreFileAuthCredentials:
58+
...
59+
60+
@abstractmethod
61+
def get_vector_store_file_endpoints_by_type(self) -> Dict[
62+
str, Tuple[Tuple[str, str], ...]
63+
]:
64+
...
65+
66+
@abstractmethod
67+
def validate_environment(
68+
self,
69+
*,
70+
headers: Dict[str, str],
71+
litellm_params: Optional[GenericLiteLLMParams],
72+
) -> Dict[str, str]:
73+
return {}
74+
75+
@abstractmethod
76+
def get_complete_url(
77+
self,
78+
*,
79+
api_base: Optional[str],
80+
vector_store_id: str,
81+
litellm_params: Dict[str, Any],
82+
) -> str:
83+
if api_base is None:
84+
raise ValueError("api_base is required")
85+
return api_base
86+
87+
@abstractmethod
88+
def transform_create_vector_store_file_request(
89+
self,
90+
*,
91+
vector_store_id: str,
92+
create_request: VectorStoreFileCreateRequest,
93+
api_base: str,
94+
) -> Tuple[str, Dict[str, Any]]:
95+
...
96+
97+
@abstractmethod
98+
def transform_create_vector_store_file_response(
99+
self,
100+
*,
101+
response: httpx.Response,
102+
) -> VectorStoreFileObject:
103+
...
104+
105+
@abstractmethod
106+
def transform_list_vector_store_files_request(
107+
self,
108+
*,
109+
vector_store_id: str,
110+
query_params: VectorStoreFileListQueryParams,
111+
api_base: str,
112+
) -> Tuple[str, Dict[str, Any]]:
113+
...
114+
115+
@abstractmethod
116+
def transform_list_vector_store_files_response(
117+
self,
118+
*,
119+
response: httpx.Response,
120+
) -> VectorStoreFileListResponse:
121+
...
122+
123+
@abstractmethod
124+
def transform_retrieve_vector_store_file_request(
125+
self,
126+
*,
127+
vector_store_id: str,
128+
file_id: str,
129+
api_base: str,
130+
) -> Tuple[str, Dict[str, Any]]:
131+
...
132+
133+
@abstractmethod
134+
def transform_retrieve_vector_store_file_response(
135+
self,
136+
*,
137+
response: httpx.Response,
138+
) -> VectorStoreFileObject:
139+
...
140+
141+
@abstractmethod
142+
def transform_retrieve_vector_store_file_content_request(
143+
self,
144+
*,
145+
vector_store_id: str,
146+
file_id: str,
147+
api_base: str,
148+
) -> Tuple[str, Dict[str, Any]]:
149+
...
150+
151+
@abstractmethod
152+
def transform_retrieve_vector_store_file_content_response(
153+
self,
154+
*,
155+
response: httpx.Response,
156+
) -> VectorStoreFileContentResponse:
157+
...
158+
159+
@abstractmethod
160+
def transform_update_vector_store_file_request(
161+
self,
162+
*,
163+
vector_store_id: str,
164+
file_id: str,
165+
update_request: VectorStoreFileUpdateRequest,
166+
api_base: str,
167+
) -> Tuple[str, Dict[str, Any]]:
168+
...
169+
170+
@abstractmethod
171+
def transform_update_vector_store_file_response(
172+
self,
173+
*,
174+
response: httpx.Response,
175+
) -> VectorStoreFileObject:
176+
...
177+
178+
@abstractmethod
179+
def transform_delete_vector_store_file_request(
180+
self,
181+
*,
182+
vector_store_id: str,
183+
file_id: str,
184+
api_base: str,
185+
) -> Tuple[str, Dict[str, Any]]:
186+
...
187+
188+
@abstractmethod
189+
def transform_delete_vector_store_file_response(
190+
self,
191+
*,
192+
response: httpx.Response,
193+
) -> VectorStoreFileDeleteResponse:
194+
...
195+
196+
def get_error_class(
197+
self,
198+
*,
199+
error_message: str,
200+
status_code: int,
201+
headers: Union[Dict[str, Any], httpx.Headers],
202+
) -> BaseLLMException:
203+
from ..chat.transformation import BaseLLMException
204+
205+
raise BaseLLMException(
206+
status_code=status_code,
207+
message=error_message,
208+
headers=headers,
209+
)
210+
211+
def sign_request(
212+
self,
213+
*,
214+
headers: Dict[str, str],
215+
optional_params: Dict[str, Any],
216+
request_data: Dict[str, Any],
217+
api_base: str,
218+
api_key: Optional[str] = None,
219+
) -> Tuple[Dict[str, str], Optional[bytes]]:
220+
return headers, None
221+
222+
def prepare_chunking_strategy(
223+
self,
224+
chunking_strategy: Optional[VectorStoreFileChunkingStrategy],
225+
) -> Optional[VectorStoreFileChunkingStrategy]:
226+
return chunking_strategy

0 commit comments

Comments
 (0)