Skip to content

Commit 601fa61

Browse files
authored
[TI_MISP] - Updated transform to sort by '@timestamp' to ensure the latest documents are selected for each unique key (#15816)
Existing sorting logic in the transform was by event.ingested field. This however created issues with bulk ingestion where the order of ingestion is not gauaranteed by elasticsearch, which can sometimes lead to recent events being not updated in the destination index of the transform. Sorting by '@timestamp' helps mitigate this issue.
1 parent 28f9488 commit 601fa61

File tree

3 files changed

+8
-3
lines changed

3 files changed

+8
-3
lines changed

packages/ti_misp/changelog.yml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,9 @@
11
# newer versions go on top
2+
- version: "1.38.1"
3+
changes:
4+
- description: Updated transform to sort by `@timestamp` to ensure the latest documents are selected for each unique key.
5+
type: bugfix
6+
link: https://github.com/elastic/integrations/pull/15816
27
- version: "1.38.0"
38
changes:
49
- description: Update Kibana constraint to support 9.0.0.

packages/ti_misp/elasticsearch/transform/latest_ioc/transform.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ latest:
1818
- event.dataset
1919
- misp.attribute.uuid
2020
- misp.attribute.Event.uuid
21-
sort: "event.ingested"
21+
sort: "@timestamp"
2222
description: Latest MISP IoC data
2323
frequency: 30s
2424
sync:
@@ -33,4 +33,4 @@ retention_policy:
3333
_meta:
3434
managed: true
3535
# Bump this version to delete, reinstall, and restart the transform during package.
36-
fleet_transform_version: 0.3.0
36+
fleet_transform_version: 0.4.0

packages/ti_misp/manifest.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
name: ti_misp
22
title: MISP
3-
version: "1.38.0"
3+
version: "1.38.1"
44
description: Ingest threat intelligence indicators from MISP platform with Elastic Agent.
55
type: integration
66
format_version: "3.0.2"

0 commit comments

Comments
 (0)