|
| 1 | +# @sap-ai-sdk/langchain |
| 2 | + |
| 3 | +SAP Cloud SDK for AI is the official Software Development Kit (SDK) for **SAP AI Core**, **SAP Generative AI Hub**, and **Orchestration Service**. |
| 4 | + |
| 5 | +This package provides LangChain model clients built on top of the foundation model clients of the SAP Cloud SDK for AI. |
| 6 | + |
| 7 | +### Table of Contents |
| 8 | + |
| 9 | +- [Installation](#installation) |
| 10 | +- [Prerequisites](#prerequisites) |
| 11 | +- [Relationship between Models and Deployment ID](#relationship-between-models-and-deployment-id) |
| 12 | +- [Usage](#usage) |
| 13 | + - [Client Initialization](#client-initialization) |
| 14 | + - [Chat Client](#chat-client) |
| 15 | + - [Embedding Client](#embedding-client) |
| 16 | +- [Local Testing](#local-testing) |
| 17 | + |
| 18 | +## Installation |
| 19 | + |
| 20 | +``` |
| 21 | +$ npm install @sap-ai-sdk/langchain |
| 22 | +``` |
| 23 | + |
| 24 | +## Prerequisites |
| 25 | + |
| 26 | +- [Enable the AI Core service in SAP BTP](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/initial-setup). |
| 27 | +- Use the same `@langchain/core` version as the `@sap-ai-sdk/langchain` package, to see which langchain version this package is currently using, check our [package.json](./package.json). |
| 28 | +- Configure the project with **Node.js v20 or higher** and **native ESM** support. |
| 29 | +- Ensure a deployed OpenAI model is available in the SAP Generative AI Hub. |
| 30 | + - Use the [`DeploymentApi`](https://github.com/SAP/ai-sdk-js/blob/main/packages/ai-api/README.md#create-a-deployment) from `@sap-ai-sdk/ai-api` [to deploy a model](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). |
| 31 | + Alternatively, you can also create deployments using the [SAP AI Launchpad](https://help.sap.com/docs/sap-ai-core/generative-ai-hub/activate-generative-ai-hub-for-sap-ai-launchpad?locale=en-US&q=launchpad). |
| 32 | + - Once deployment is complete, access the model via the `deploymentUrl`. |
| 33 | + |
| 34 | +> **Accessing the AI Core Service via the SDK** |
| 35 | +> |
| 36 | +> The SDK automatically retrieves the `AI Core` service credentials and resolves the access token needed for authentication. |
| 37 | +> |
| 38 | +> - In Cloud Foundry, it's accessed from the `VCAP_SERVICES` environment variable. |
| 39 | +> - In Kubernetes / Kyma environments, you have to mount the service binding as a secret instead, for more information refer to [this documentation](https://www.npmjs.com/package/@sap/xsenv#usage-in-kubernetes). |
| 40 | +
|
| 41 | +## Relationship between Models and Deployment ID |
| 42 | + |
| 43 | +SAP AI Core manages access to generative AI models through the global AI scenario `foundation-models`. |
| 44 | +Creating a deployment for a model requires access to this scenario. |
| 45 | + |
| 46 | +Each model, model version, and resource group allows for a one-time deployment. |
| 47 | +After deployment completion, the response includes a `deploymentUrl` and an `id`, which is the deployment ID. |
| 48 | +For more information, see [here](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-deployment-for-generative-ai-model-in-sap-ai-core). |
| 49 | + |
| 50 | +[Resource groups](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/resource-groups?q=resource+group) represent a virtual collection of related resources within the scope of one SAP AI Core tenant. |
| 51 | + |
| 52 | +Consequently, each deployment ID and resource group uniquely map to a combination of model and model version within the `foundation-models` scenario. |
| 53 | + |
| 54 | +## Usage |
| 55 | + |
| 56 | +This sub-package offers a Azure OpenAI chat and embedding client. |
| 57 | +Both clients comply with [LangChain's interface](https://js.langchain.com/docs/introduction). |
| 58 | + |
| 59 | +### Client Initialization |
| 60 | + |
| 61 | +To initialize a client, provide the model name: |
| 62 | + |
| 63 | +```ts |
| 64 | +import { |
| 65 | + AzureOpenAiChatClient, |
| 66 | + AzureOpenAiEmbeddingClient |
| 67 | +} from '@sap-ai-sdk/langchain'; |
| 68 | + |
| 69 | +// For a chat client |
| 70 | +const chatClient = new AzureOpenAiChatClient({ modelName: 'gpt-4o' }); |
| 71 | +// For an embedding client |
| 72 | +const embeddingClient = new AzureOpenAiEmbeddingClient({ modelName: 'gpt-4o' }); |
| 73 | +``` |
| 74 | + |
| 75 | +In addition to the default parameters of the model vendor (e.g., OpenAI) and LangChain, additional parameters can be used to help narrow down the search for the desired model: |
| 76 | + |
| 77 | +```ts |
| 78 | +const chatClient = new AzureOpenAiChatClient({ |
| 79 | + modelName: 'gpt-4o', |
| 80 | + modelVersion: '24-07-2021', |
| 81 | + resourceGroup: 'my-resource-group' |
| 82 | +}); |
| 83 | +``` |
| 84 | + |
| 85 | +**Do not pass a `deployment ID` to initialize the client.** |
| 86 | +For the LangChain model clients, initialization is done using the model name, model version and resource group. |
| 87 | + |
| 88 | +An important note is that LangChain clients by default attempt 6 retries with exponential backoff in case of a failure. |
| 89 | +Especially in testing environments you might want to reduce this number to speed up the process: |
| 90 | + |
| 91 | +```ts |
| 92 | +const embeddingClient = new AzureOpenAiEmbeddingClient({ |
| 93 | + modelName: 'gpt-4o', |
| 94 | + maxRetries: 0 |
| 95 | +}); |
| 96 | +``` |
| 97 | + |
| 98 | +#### Custom Destination |
| 99 | + |
| 100 | +When initializing the `AzureOpenAiChatClient` and `AzureOpenAiEmbeddingClient` clients, it is possible to provide a custom destination. |
| 101 | +For example, when targeting a destination with the name `my-destination`, the following code can be used: |
| 102 | + |
| 103 | +```ts |
| 104 | +const chatClient = new AzureOpenAiChatClient( |
| 105 | + { |
| 106 | + modelName: 'gpt-4o', |
| 107 | + modelVersion: '24-07-2021', |
| 108 | + resourceGroup: 'my-resource-group' |
| 109 | + }, |
| 110 | + { |
| 111 | + destinationName: 'my-destination' |
| 112 | + } |
| 113 | +); |
| 114 | +``` |
| 115 | + |
| 116 | +By default, the fetched destination is cached. |
| 117 | +To disable caching, set the `useCache` parameter to `false` together with the `destinationName` parameter. |
| 118 | + |
| 119 | +### Chat Client |
| 120 | + |
| 121 | +The chat client allows you to interact with Azure OpenAI chat models, accessible via the generative AI hub of SAP AI Core. |
| 122 | +To invoke the client, pass a prompt: |
| 123 | + |
| 124 | +```ts |
| 125 | +const response = await chatClient.invoke("What's the capital of France?"); |
| 126 | +``` |
| 127 | + |
| 128 | +#### Advanced Example with Templating and Output Parsing |
| 129 | + |
| 130 | +```ts |
| 131 | +import { AzureOpenAiChatClient } from '@sap-ai-sdk/langchain'; |
| 132 | +import { StringOutputParser } from '@langchain/core/output_parsers'; |
| 133 | +import { ChatPromptTemplate } from '@langchain/core/prompts'; |
| 134 | + |
| 135 | +// initialize the client |
| 136 | +const client = new AzureOpenAiChatClient({ modelName: 'gpt-35-turbo' }); |
| 137 | + |
| 138 | +// create a prompt template |
| 139 | +const promptTemplate = ChatPromptTemplate.fromMessages([ |
| 140 | + ['system', 'Answer the following in {language}:'], |
| 141 | + ['user', '{text}'] |
| 142 | +]); |
| 143 | +// create an output parser |
| 144 | +const parser = new StringOutputParser(); |
| 145 | + |
| 146 | +// chain together template, client, and parser |
| 147 | +const llmChain = promptTemplate.pipe(client).pipe(parser); |
| 148 | + |
| 149 | +// invoke the chain |
| 150 | +return llmChain.invoke({ |
| 151 | + language: 'german', |
| 152 | + text: 'What is the capital of France?' |
| 153 | +}); |
| 154 | +``` |
| 155 | + |
| 156 | +### Embedding Client |
| 157 | + |
| 158 | +Embedding clients allow embedding either text or document chunks (represented as arrays of strings). |
| 159 | +While you can use them standalone, they are usually used in combination with other LangChain utilities, like a text splitter for preprocessing and a vector store for storage and retrieval of the relevant embeddings. |
| 160 | +For a complete example how to implement RAG with our LangChain client, take a look at our [sample code](https://github.com/SAP/ai-sdk-js/blob/main/sample-code/src/langchain-azure-openai.ts). |
| 161 | + |
| 162 | +#### Embed Text |
| 163 | + |
| 164 | +```ts |
| 165 | +const embeddedText = await embeddingClient.embedQuery( |
| 166 | + 'Paris is the capital of France.' |
| 167 | +); |
| 168 | +``` |
| 169 | + |
| 170 | +#### Embed Document Chunks |
| 171 | + |
| 172 | +```ts |
| 173 | +const embeddedDocuments = await embeddingClient.embedDocuments([ |
| 174 | + 'Page 1: Paris is the capital of France.', |
| 175 | + 'Page 2: It is a beautiful city.' |
| 176 | +]); |
| 177 | +``` |
| 178 | + |
| 179 | +#### Preprocess, embed, and store documents |
| 180 | + |
| 181 | +```ts |
| 182 | +// Create a text splitter and split the document |
| 183 | +const textSplitter = new RecursiveCharacterTextSplitter({ |
| 184 | + chunkSize: 2000, |
| 185 | + chunkOverlap: 200 |
| 186 | +}); |
| 187 | +const splits = await textSplitter.splitDocuments(docs); |
| 188 | + |
| 189 | +// Initialize the embedding client |
| 190 | +const embeddingClient = new AzureOpenAiEmbeddingClient({ |
| 191 | + modelName: 'text-embedding-ada-002' |
| 192 | +}); |
| 193 | + |
| 194 | +// Create a vector store from the document |
| 195 | +const vectorStore = await MemoryVectorStore.fromDocuments( |
| 196 | + splits, |
| 197 | + embeddingClient |
| 198 | +); |
| 199 | + |
| 200 | +// Create a retriever for the vector store |
| 201 | +const retriever = vectorStore.asRetriever(); |
| 202 | +``` |
| 203 | + |
| 204 | +## Local Testing |
| 205 | + |
| 206 | +For local testing instructions, refer to this [section](https://github.com/SAP/ai-sdk-js/blob/main/README.md#local-testing). |
0 commit comments