You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: help/ingestion/batch-ingestion/partial.md
+100Lines changed: 100 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -115,6 +115,106 @@ The **[!UICONTROL Error diagnostics]** toggle only appears when the **[!UICONTRO
115
115
116
116
**[!UICONTROL Error threshold]** allows you to set the percentage of acceptable errors before the entire batch will fail. By default, this value is set to 5%.
117
117
118
+
## Enable partial ingestion and error diagnostics for an existing dataflow
119
+
120
+
If a dataflow in Experience Platform was created without enabling partial ingestion or error diagnostics, you can still enable these features without recreating the flow. By enabling partial ingestion and robust error diagnostics, you can greatly enhance the reliability and ease of troubleshooting in your data ingestion workflows. Read the sections below to learn how to enable partial ingestion and error diagnostics for an existing dataflow using the [!DNL Flow Service] API.
121
+
122
+
By default, dataflows may not have partial ingestion or error diagnostics enabled. These features are helpful for identifying and isolating issues during data ingestion. Using the [!DNL Flow Service] API, you can retrieve your current dataflow configuration and apply the necessary changes using a PATCH request.
123
+
124
+
Follow the steps below to enable partial ingestion and error diagnostics for an existing dataflow.
125
+
126
+
### Retrieve flow details
127
+
128
+
To retrieve your dataflow configurations, make a GET request to the `/flows/{FLOW_ID}` endpoint and provide the ID of your dataflow. For more information on retrieving dataflow details, refer to the [Update dataflows using the [!DNL Flow Service] API](../../sources/tutorials/api/update-dataflows.md) guide.
129
+
130
+
Make sure to save the value of the `etag` field returned in the response. This is necessary for the update request to ensure version consistency.
131
+
132
+
### Update flow configuration
133
+
134
+
Next, make a PATCH request to the `/flows/` endpoint and provide the ID of the dataflow that you want to enable partial ingestion and error diagnostics for.
135
+
136
+
>[!IMPORTANT]
137
+
>
138
+
>- Include the previously saved `etag` value in the request header using the If-Match key.
139
+
>- You can modify the `partialIngestionPercent` value to suit your specific needs.
A successful response returns your dataflow details, confirming that partial ingestion and error diagnostics are now enabled in the `options` section.
210
+
211
+
```json
212
+
"options": {
213
+
"partialIngestionPercent": 10,
214
+
"errorDiagnosticsEnabled": true
215
+
}
216
+
```
217
+
118
218
## Next steps {#next-steps}
119
219
120
220
This tutorial covered how to create or modify a dataset to enable partial batch ingestion. For more information on batch ingestion, please read the [batch ingestion developer guide](./api-overview.md).
0 commit comments