Skip to content

Commit 06671c8

Browse files
sharadcrestsharadcrest
authored
airlock_digital: Add server activities datastream (#15106)
The PR includes server activities data stream and associated dashboard. Airlock Digital server activities fields are mapped to their corresponding ECS fields where possible. Test samples were derived from documentation, which were subsequently sanitized.
1 parent 42ff22b commit 06671c8

26 files changed

+1818
-18
lines changed

‎packages/airlock_digital/_dev/build/docs/README.md‎

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,21 +8,22 @@ The Airlock Digital integration for Elastic allows you to collect logs from, [Ai
88

99
### Compatibility
1010

11-
The Airlock Digital integration is compatible with `v6.1.x` and `v1` version of Airlock Digital REST API.
11+
The Airlock Digital integration is compatible with version `v6.1.x` of Airlock Digital and `v1` of the REST API.
1212

1313
### How it works
1414

15-
This integration periodically queries the Airlock Digital REST API to retrieve Agent and Execution Histories logs.
15+
This integration periodically queries the Airlock Digital REST API to retrieve Agent, Execution Histories and Server Activities logs.
1616

1717
## What data does this integration collect?
1818

1919
This integration collects log messages of the following types:
2020

2121
- `Agent`: Collects agent logs via [Airlock Digital REST API](https://api.airlockdigital.com/#35ef50c6-1df4-4330-a433-1915ccf380cf).
2222
- `Execution Histories`: Collects executions history logs via [Airlock Digital REST API](https://api.airlockdigital.com/#3634a82d-eb6b-44b7-b662-dddc37d4d9d6).
23+
- `Server Activities`: Collects server activity logs via [Airlock Digital REST API](https://api.airlockdigital.com/#290b4657-17d4-4048-982e-43df95200624).
2324

2425
### Supported use cases
25-
Integrating Airlock Digital agent and execution history logs with Elastic SIEM provides SOC teams with deep visibility into endpoint activity and policy enforcement. Dashboards surface insights into agent health, host and user patterns, OS distribution, group and policy metrics, storage availability, trusted configurations, and execution behaviors such as blocked or untrusted runs and policy violations. This enables faster investigations, stronger compliance, proactive resource management, and improved overall endpoint security.
26+
Integrating Airlock Digital agent, execution history, and server activity logs with Elastic SIEM delivers comprehensive visibility into endpoint and system operations. Dashboards highlight agent health, user and host activity, OS distribution, policy enforcement, storage status, trusted configurations, and execution behaviors such as blocked or untrusted runs and policy violations. At the same time, monitoring server-side activities across tasks, users, and root actions enables quick identification of unauthorized changes, policy misuse, and suspicious behavior. Together, these insights empower SOC teams to accelerate investigations, strengthen compliance, optimize resource management, and maintain stronger endpoint and system security.
2627

2728
## What do I need to use this integration?
2829

@@ -32,7 +33,7 @@ Integrating Airlock Digital agent and execution history logs with Elastic SIEM p
3233

3334
1. In order to make the API calls, the User Group to which a user belongs should contain required permissions. You can follow the below steps for that:
3435
2. Go to the **Settings** and navigate to **Users** tab.
35-
3. Under **User Group Management** for the respective user group provide **agent/find**, **group/policies** and **logging/exechistories** roles in the REST API Roles section and click on save.
36+
3. Under **User Group Management** for the respective user group provide **agent/find**, **group/policies**, **logging/exechistories** and **logging/svractivities** roles in the REST API Roles section and click on save.
3637

3738
#### Generate Client API key for Authentication:
3839

@@ -97,6 +98,10 @@ For more information on architectures that can be used for scaling this integrat
9798

9899
{{fields "execution_histories"}}
99100

101+
#### Server Activities
102+
103+
{{fields "server_activities"}}
104+
100105
### Example event
101106

102107
#### Agent
@@ -107,6 +112,10 @@ For more information on architectures that can be used for scaling this integrat
107112

108113
{{event "execution_histories"}}
109114

115+
#### Server Activities
116+
117+
{{event "server_activities"}}
118+
110119
### Inputs used
111120

112121
These inputs can be used in this integration:
@@ -138,3 +147,4 @@ These integration datasets use the following APIs:
138147
- Blocklist Browser Execution
139148
- Trusted Installer Execution
140149
- Trusted Browser Metadata Execution
150+
- `Server Activities`: [Airlock Digital REST API](https://api.airlockdigital.com/#290b4657-17d4-4048-982e-43df95200624).

‎packages/airlock_digital/_dev/deploy/docker/files/config.yml‎

Lines changed: 42 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
rules:
22
# We’re unable to test the pagination functionality with the current rule
3-
# since a fixed batch size of 10,000 is being used in data-collection.
3+
# since a fixed batch size of 10,000 is being used in data collection.
44
- path: /v1/agent/find
55
methods: ['POST']
66
responses:
@@ -541,3 +541,44 @@ rules:
541541
}
542542
}
543543
` }}
544+
- path: /v1/logging/svractivities
545+
methods: ['POST']
546+
request_body: '{}'
547+
responses:
548+
- status_code: 200
549+
headers:
550+
Content-Type:
551+
- application/json
552+
X-ApiKey:
553+
- my-secret-api-key
554+
body: |
555+
{{ minify_json `
556+
{
557+
"error": "Success",
558+
"response": {
559+
"svractivities": [
560+
{
561+
"checkpoint": "firstcheckpoint",
562+
"datetime": "2024-01-25T05:23:31.77Z",
563+
"task": "Repository Add",
564+
"user": "SYSTEM",
565+
"description": "a.out added to repository"
566+
},
567+
{
568+
"checkpoint": "secondcheckpoint",
569+
"datetime": "2024-01-25T05:23:31.88Z",
570+
"task": "Repository Add",
571+
"user": "SYSTEM",
572+
"description": "file.exe added to repository"
573+
},
574+
{
575+
"checkpoint": "thirdcheckpoint",
576+
"datetime": "2024-01-25T05:23:30.88Z",
577+
"task": "Client Checkin",
578+
"user": "SYSTEM",
579+
"description": "DESKTOP-GG4CEJM checks in"
580+
}
581+
]
582+
}
583+
}
584+
` }}

‎packages/airlock_digital/changelog.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,9 @@
11
# newer versions go on top
2+
- version: '0.3.0'
3+
changes:
4+
- description: Add server activities data stream.
5+
type: enhancement
6+
link: https://github.com/elastic/integrations/pull/15106
27
- version: '0.2.0'
38
changes:
49
- description: Add execution histories data stream.

‎packages/airlock_digital/data_stream/server_activities/_dev/test/pipeline/test-common-config.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
fields:
2+
tags:
3+
- preserve_duplicate_custom_fields

‎packages/airlock_digital/data_stream/server_activities/_dev/test/pipeline/test-server-activities.log‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
{"checkpoint":"65b1f05387bacb6955a32a3c","datetime":"2024-01-25T05:23:31.77Z","task":"Repository Add","user":"SYSTEM","description":"a.out added to repository"}
2+
{"checkpoint":"65b1f05387bacb6955a32a40","datetime":"2024-01-25T05:23:31.88Z","task":"Repository Add","user":"SYSTEM","description":"file.exe added to repository"}

‎packages/airlock_digital/data_stream/server_activities/_dev/test/pipeline/test-server-activities.log-expected.json‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
{
2+
"expected": [
3+
{
4+
"@timestamp": "2024-01-25T05:23:31.770Z",
5+
"airlock_digital": {
6+
"server_activities": {
7+
"checkpoint": "65b1f05387bacb6955a32a3c",
8+
"datetime": "2024-01-25T05:23:31.770Z",
9+
"description": "a.out added to repository",
10+
"task": "Repository Add",
11+
"user": "SYSTEM"
12+
}
13+
},
14+
"ecs": {
15+
"version": "8.17.0"
16+
},
17+
"event": {
18+
"action": "repository-add",
19+
"category": [
20+
"host"
21+
],
22+
"id": "65b1f05387bacb6955a32a3c",
23+
"kind": "event",
24+
"original": "{\"checkpoint\":\"65b1f05387bacb6955a32a3c\",\"datetime\":\"2024-01-25T05:23:31.77Z\",\"task\":\"Repository Add\",\"user\":\"SYSTEM\",\"description\":\"a.out added to repository\"}",
25+
"type": [
26+
"info"
27+
]
28+
},
29+
"message": "a.out added to repository",
30+
"related": {
31+
"user": [
32+
"SYSTEM"
33+
]
34+
},
35+
"tags": [
36+
"preserve_duplicate_custom_fields"
37+
],
38+
"user": {
39+
"name": "SYSTEM"
40+
}
41+
},
42+
{
43+
"@timestamp": "2024-01-25T05:23:31.880Z",
44+
"airlock_digital": {
45+
"server_activities": {
46+
"checkpoint": "65b1f05387bacb6955a32a40",
47+
"datetime": "2024-01-25T05:23:31.880Z",
48+
"description": "file.exe added to repository",
49+
"task": "Repository Add",
50+
"user": "SYSTEM"
51+
}
52+
},
53+
"ecs": {
54+
"version": "8.17.0"
55+
},
56+
"event": {
57+
"action": "repository-add",
58+
"category": [
59+
"host"
60+
],
61+
"id": "65b1f05387bacb6955a32a40",
62+
"kind": "event",
63+
"original": "{\"checkpoint\":\"65b1f05387bacb6955a32a40\",\"datetime\":\"2024-01-25T05:23:31.88Z\",\"task\":\"Repository Add\",\"user\":\"SYSTEM\",\"description\":\"file.exe added to repository\"}",
64+
"type": [
65+
"info"
66+
]
67+
},
68+
"message": "file.exe added to repository",
69+
"related": {
70+
"user": [
71+
"SYSTEM"
72+
]
73+
},
74+
"tags": [
75+
"preserve_duplicate_custom_fields"
76+
],
77+
"user": {
78+
"name": "SYSTEM"
79+
}
80+
}
81+
]
82+
}

‎packages/airlock_digital/data_stream/server_activities/_dev/test/system/test-common-config.yml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
input: cel
2+
service: airlock_digital
3+
vars:
4+
url: http://{{Hostname}}:{{Port}}
5+
api_key: my-secret-api-key
6+
data_stream:
7+
vars:
8+
preserve_original_event: true
9+
preserve_duplicate_custom_fields: true
10+
assert:
11+
hit_count: 3

‎packages/airlock_digital/data_stream/server_activities/agent/stream/cel.yml.hbs‎

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
config_version: 2
2+
interval: {{interval}}
3+
resource.tracer:
4+
enabled: {{enable_request_tracer}}
5+
filename: "../../logs/cel/http-request-trace-*.ndjson"
6+
maxbackups: 5
7+
{{#if proxy_url}}
8+
resource.proxy_url: {{proxy_url}}
9+
{{/if}}
10+
{{#if ssl}}
11+
resource.ssl: {{ssl}}
12+
{{/if}}
13+
{{#if http_client_timeout}}
14+
resource.timeout: {{http_client_timeout}}
15+
{{/if}}
16+
resource.url: {{url}}
17+
18+
state:
19+
api_key: {{api_key}}
20+
{{!--
21+
The API does not support specifying a batch_size parameter; however, the documentation states
22+
that a maximum of 10,000 events can be returned in a single response.
23+
Reference Link: https://api.airlockdigital.com/#8643f2cc-4930-44d5-a7cb-21395b33ab62
24+
--}}
25+
max_batch_size: 10000
26+
redact:
27+
fields:
28+
- api_key
29+
program: |
30+
state.with(
31+
post_request(
32+
state.url.trim_right("/") + "/v1/logging/svractivities",
33+
"application/json",
34+
{
35+
?"checkpoint": state.?cursor.checkpoint,
36+
}.encode_json()
37+
).with(
38+
{
39+
"Header": {
40+
"Content-Type": ["application/json"],
41+
"X-ApiKey": [state.api_key],
42+
},
43+
}
44+
).do_request().as(resp, (resp.StatusCode == 200) ?
45+
resp.Body.decode_json().as(body,
46+
{
47+
"events": body.response.svractivities.map(e,
48+
{
49+
"message": e.encode_json(),
50+
}
51+
),
52+
"want_more": size(body.response.svractivities) >= state.max_batch_size,
53+
"cursor": {
54+
?"checkpoint": (has(body.?response.svractivities) && size(body.response.svractivities) > 0) ?
55+
optional.of(body.response.svractivities[size(body.response.svractivities) - 1].checkpoint)
56+
:
57+
state.?cursor.checkpoint,
58+
},
59+
}
60+
)
61+
:
62+
{
63+
"events": {
64+
"error": {
65+
"code": string(resp.StatusCode),
66+
"id": string(resp.Status),
67+
"message": "POST " + state.url.trim_right("/") + "/v1/logging/svractivities " + (
68+
(size(resp.Body) != 0) ?
69+
string(resp.Body)
70+
:
71+
string(resp.Status) + " (" + string(resp.StatusCode) + ")"
72+
),
73+
},
74+
},
75+
"want_more": false,
76+
}
77+
)
78+
)
79+
tags:
80+
{{#if preserve_original_event}}
81+
- preserve_original_event
82+
{{/if}}
83+
{{#if preserve_duplicate_custom_fields}}
84+
- preserve_duplicate_custom_fields
85+
{{/if}}
86+
{{#each tags as |tag|}}
87+
- {{tag}}
88+
{{/each}}
89+
{{#contains "forwarded" tags}}
90+
publisher_pipeline.disable_host: true
91+
{{/contains}}
92+
{{#if processors}}
93+
processors:
94+
{{processors}}
95+
{{/if}}

0 commit comments

Comments
 (0)