You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: packages/orchestration/README.md
+53-12Lines changed: 53 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -109,7 +109,6 @@ Optionally, define `model_version` (default: `latest`) and `model_params` for cu
109
109
110
110
Use the orchestration client with templating to pass a prompt containing placeholders that will be replaced with input parameters during a chat completion request.
111
111
This allows for variations in the prompt based on the input parameters.
112
-
Set custom request configuration when calling `chatCompletion` function.
`getContent()` is a convenience method that parses the response and returns the model's output as a string.
137
+
You can use the following convenience methods for handling chat completion responses:
138
+
139
+
-`getContent()` parses the response and returns the model's output as a string.
140
+
-`getFinishReason()` retrieves the `finish_reason` explaining why chat completion request stopped.
141
+
-`getTokenUsage()` provides token usage details, including `total_tokens`, `prompt_tokens`, and `completion_tokens`.
142
+
143
+
#### Structured Outputs
144
+
145
+
Setting `response_format` under `templating` guarantees that the model's output aligns with the schema type specified by developers.
146
+
It is useful when the model is not calling a tool, but rather, responding to the user in a structured way.
137
147
138
-
To retrieve the `finish_reason` for stopping the chat completion request, use the convenience method `getFinishReason()`:
148
+
The example below demonstrates how to use `response_format` to return a JSON Schema, with `strict: true` ensuring the outputs conform precisely to the schema.
139
149
140
150
```ts
141
-
const finishReason =response.getFinishReason();
151
+
templating: {
152
+
template: [
153
+
{ role: 'user', content: 'What is the capital of {{?country}}?' }
154
+
],
155
+
response_format: {
156
+
type: 'json_schema',
157
+
json_schema: {
158
+
name: 'capital_response',
159
+
strict: true,
160
+
schema: {
161
+
type: 'object',
162
+
properties: {
163
+
country_name: {
164
+
type: "string",
165
+
description: "The name of the country provided by the user."
166
+
},
167
+
capital: {
168
+
type: "string",
169
+
description: "The capital city of the country."
170
+
}
171
+
},
172
+
required: ["country_name", "capital"]
173
+
}
174
+
}
175
+
}
176
+
}
142
177
```
143
178
144
-
Use the `getTokenUsage()` convenience method to retrieve the token usage details of the chat completion request:
179
+
You can also initialize `json_schema` using a Zod schema, as shown below:
145
180
146
181
```ts
147
-
const tokenUsage =response.getTokenUsage();
148
-
149
-
console.log(
150
-
`Total tokens consumed by the request: ${tokenUsage.total_tokens}\n`+
0 commit comments