Skip to content

Commit 9f7fb2b

Browse files
authored
Merge branch 'main' into feat/claude-block-v2
2 parents 2a42fa5 + 1ec227f commit 9f7fb2b

File tree

2 files changed

+116
-1
lines changed

2 files changed

+116
-1
lines changed

docs/foundation/sam3.md

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -179,3 +179,116 @@ The Workflow block allows you to:
179179
- **Unified Architecture**: Handles both detection and segmentation in a single model.
180180

181181
For more technical details, refer to the [official SAM 3 paper](https://ai.meta.com/research/publications/sam-3-segment-anything-with-concepts/).
182+
183+
184+
## How to use SAM 3 taking advantage of hot SAM3 instances maintained by Roboflow
185+
186+
In below examples we are taking advantage of the serverless infrastructure which handles GPU provisioning automatically, making it ideal for applications that need on-demand segmentation without managing infrastructure.
187+
188+
### 1. SAM3 Concept Segmentation workflow
189+
190+
This example demonstrates using SAM3 with the workflow approach which allows you to combine SAM3's concept segmentation with visualization in a single pipeline. Here, we're segmenting all dogs in an image and automatically visualizing the results with polygon overlays.
191+
If you have created a workflow in Roboflow platform you can use `workspace_name` and `workflow_id` instead of `specification` to run it.
192+
193+
```python
194+
import base64
195+
196+
import cv2 as cv
197+
import numpy as np
198+
199+
from inference_sdk import InferenceHTTPClient
200+
201+
# 2. Connect to your workflow
202+
client = InferenceHTTPClient(
203+
api_url="https://serverless.roboflow.com",
204+
api_key="<YOUR_ROBOFLOW_API_KEY>"
205+
)
206+
207+
# 3. Run your workflow on an image
208+
workflow_spec = {
209+
"version": "1.0",
210+
"inputs": [
211+
{
212+
"type": "InferenceImage",
213+
"name": "image"
214+
}
215+
],
216+
"steps": [
217+
{
218+
"type": "roboflow_core/sam3@v1",
219+
"name": "sam",
220+
"images": "$inputs.image",
221+
"class_names": "dog"
222+
},
223+
{
224+
"type": "roboflow_core/polygon_visualization@v1",
225+
"name": "polygon_visualization",
226+
"image": "$inputs.image",
227+
"predictions": "$steps.sam.predictions"
228+
}
229+
],
230+
"outputs": [
231+
{
232+
"type": "JsonField",
233+
"name": "output",
234+
"coordinates_system": "own",
235+
"selector": "$steps.polygon_visualization.image"
236+
}
237+
]
238+
}
239+
240+
result = client.run_workflow(
241+
specification=workflow_spec,
242+
images={
243+
"image": "https://media.roboflow.com/inference/dog.jpeg" # Path or url to your image file
244+
},
245+
use_cache=True # Speeds up repeated requests
246+
)
247+
248+
# 4. Display the result
249+
nparr = np.frombuffer(base64.b64decode(result[0]["output"]), np.uint8)
250+
img = cv.imdecode(nparr, cv.IMREAD_COLOR)
251+
252+
cv.imshow("result", img)
253+
cv.waitKey(0)
254+
cv.destroyAllWindows()
255+
```
256+
257+
### 2. SAM3 raw API
258+
259+
For direct API access to SAM3 without workflows, you can use Roboflow's serverless endpoint.
260+
This approach gives you raw segmentation results that you can process however you need.
261+
The example below shows how to segment a dog and draw the resulting polygon directly on the image using OpenCV.
262+
263+
264+
```python
265+
import requests
266+
import cv2 as cv
267+
import numpy as np
268+
269+
response = requests.post(
270+
"https://serverless.roboflow.com/sam3/concept_segment?api_key=<YOUR_ROBOFLOW_API_KEY>",
271+
headers={
272+
"Content-Type": "application/json"
273+
},
274+
json={
275+
"format": "polygon",
276+
"image": {
277+
"type": "url",
278+
"value": "https://media.roboflow.com/dog.jpeg"
279+
},
280+
"prompts": [
281+
{ "text": "dog" }
282+
]
283+
}
284+
)
285+
286+
img_req = requests.get("https://media.roboflow.com/dog.jpeg")
287+
img_arr = np.asarray(bytearray(img_req.content), dtype=np.uint8)
288+
img = cv.imdecode(img_arr, -1)
289+
polygon_arr = np.array(response.json()["prompt_results"][0]["predictions"][0]["masks"][0])
290+
cv.polylines(img, [polygon_arr], True, (0, 200, 200), 3)
291+
cv.imshow("result", img)
292+
cv.waitKey(0)
293+
cv.destroyAllWindows()
294+
```

docs/workflows/batch_processing/about.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -300,7 +300,9 @@ potential costs.
300300

301301
## Known limitations
302302

303-
* Batch Processing service cannot run Custom Python blocks.
303+
<div style="border: 2px solid #059669; background: #D1FAE5; color: #065F46; padding: 1em; border-radius: 8px; text-align: center; margin-bottom: 1em; font-weight: bold;">
304+
🚀 <b>Update:</b> Batch Processing service <b>now supports running Custom Python blocks.</b>
305+
</div>
304306

305307
* Certain Workflow blocks requiring access to env variables and local storage (like File Sink and Environment
306308
Secret Store) are blacklisted and will not execute.

0 commit comments

Comments
 (0)