Skip to content

Commit 603d2c3

Browse files
authored
feat: Add Response Format Sample Code (#538)
1 parent bf899d4 commit 603d2c3

File tree

8 files changed

+192
-140
lines changed

8 files changed

+192
-140
lines changed

packages/orchestration/README.md

Lines changed: 53 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,6 @@ Optionally, define `model_version` (default: `latest`) and `model_params` for cu
109109

110110
Use the orchestration client with templating to pass a prompt containing placeholders that will be replaced with input parameters during a chat completion request.
111111
This allows for variations in the prompt based on the input parameters.
112-
Set custom request configuration when calling `chatCompletion` function.
113112

114113
```ts
115114
import { OrchestrationClient } from '@sap-ai-sdk/orchestration';
@@ -131,26 +130,68 @@ const response = await orchestrationClient.chatCompletion({
131130
});
132131

133132
const responseContent = response.getContent();
133+
const finishReason = response.getFinishReason();
134+
const tokenUsage = response.getTokenUsage();
134135
```
135136

136-
`getContent()` is a convenience method that parses the response and returns the model's output as a string.
137+
You can use the following convenience methods for handling chat completion responses:
138+
139+
- `getContent()` parses the response and returns the model's output as a string.
140+
- `getFinishReason()` retrieves the `finish_reason` explaining why chat completion request stopped.
141+
- `getTokenUsage()` provides token usage details, including `total_tokens`, `prompt_tokens`, and `completion_tokens`.
142+
143+
#### Structured Outputs
144+
145+
Setting `response_format` under `templating` guarantees that the model's output aligns with the schema type specified by developers.
146+
It is useful when the model is not calling a tool, but rather, responding to the user in a structured way.
137147

138-
To retrieve the `finish_reason` for stopping the chat completion request, use the convenience method `getFinishReason()`:
148+
The example below demonstrates how to use `response_format` to return a JSON Schema, with `strict: true` ensuring the outputs conform precisely to the schema.
139149

140150
```ts
141-
const finishReason = response.getFinishReason();
151+
templating: {
152+
template: [
153+
{ role: 'user', content: 'What is the capital of {{?country}}?' }
154+
],
155+
response_format: {
156+
type: 'json_schema',
157+
json_schema: {
158+
name: 'capital_response',
159+
strict: true,
160+
schema: {
161+
type: 'object',
162+
properties: {
163+
country_name: {
164+
type: "string",
165+
description: "The name of the country provided by the user."
166+
},
167+
capital: {
168+
type: "string",
169+
description: "The capital city of the country."
170+
}
171+
},
172+
required: ["country_name", "capital"]
173+
}
174+
}
175+
}
176+
}
142177
```
143178

144-
Use the `getTokenUsage()` convenience method to retrieve the token usage details of the chat completion request:
179+
You can also initialize `json_schema` using a Zod schema, as shown below:
145180

146181
```ts
147-
const tokenUsage = response.getTokenUsage();
148-
149-
console.log(
150-
`Total tokens consumed by the request: ${tokenUsage.total_tokens}\n` +
151-
`Input prompt tokens consumed: ${tokenUsage.prompt_tokens}\n` +
152-
`Output text completion tokens consumed: ${tokenUsage.completion_tokens}\n`
153-
);
182+
const countryCapitalSchema = z.object({
183+
country_name: z.string(),
184+
capital: z.string()
185+
}).strict();
186+
187+
response_format: {
188+
type: 'json_schema',
189+
json_schema: {
190+
name: 'capital_response',
191+
strict: true,
192+
schema: zodToJsonSchema(countryCapitalSchema)
193+
}
194+
}
154195
```
155196

156197
### Prompt Registry

0 commit comments

Comments
 (0)