[island_browser][compromised_credential] Add island_browser compromised credential datastream

packages/island_browser/_dev/build/docs/README.md

@@ -12,25 +12,24 @@ The Island Browser integration is compatible with `v1` version of Island Browser
 
 ### How it works
 
-This integration periodically queries the Island Browser API to retrieve details for devices and users, and audit events.
+This integration periodically queries the Island Browser API to retrieve details for devices, users and compromised credentials, and to log audit events.
 
 ## What data does this integration collect?
 
 This integration collects log messages of the following types:
 
 - `Audit`: Collects all timeline audits from the Island Browser via [Audit API endpoint](https://documentation.island.io/apidocs/get-all-timeline-audits-that-match-the-specified-simple-filter).
+- `Compromised Credential`: Collects a list of all compromised credentials from the Island Browser via [Compromised Credential API endpoint](https://documentation.island.io/apidocs/get-a-list-of-all-compromised-credentials).
 - `Device`: Collects a list of all devices from the Island Browser via [Device API endpoint](https://documentation.island.io/apidocs/get-a-list-of-all-devices-1).
 - `User`: Collects all the users from the Island Browser via [User API endpoint](https://documentation.island.io/apidocs/get-all-browser-users-that-match-the-specified-simple-filter).
 
->**Note:** Device and user data streams currently do not have an ILM policy applied. A policy will be introduced in an upcoming release. Until then, full sync will be performed, which may result in higher storage costs.
-
 ### Supported use cases
 
-Integrating Island Browser User, Device, and Audit endpoint data with Elastic SIEM provides unified visibility into identity activity, device posture, and security events across the environment.
+Integrating Island Browser User, Device, Audit, and Compromised Credential endpoint data with Elastic SIEM provides unified visibility into identity activity, device posture, account exposure, and security events across the environment. This integration enables analysts to correlate user behavior, device health, and credential risks within a single view, strengthening both detection and response capabilities.
 
-Dashboards track total and active users, login trends, and group distributions, alongside device insights such as active, archived, and jailbroken states, OS platform distribution, policy updates, browser update status, Windows license status, and MDM provider compliance.
+Dashboards track total and active users, login trends, and group distributions, alongside device insights such as active, archived, and jailbroken states, OS platform distribution, policy updates, browser update status, Windows license status, and MDM provider compliance. Compromised Credential visualizations highlight account risks with timelines of exposed records, unresolved credential counts, breach source breakdowns, and distributions by status. Additional charts surface top impacted domains and most affected users, enabling security teams to quickly assess exposure, prioritize remediation, and mitigate identity-based threats.
 
-Audit visualizations further enhance oversight by showing event activity over time, verdicts and reasons, top rules, users, source IPs, event types, geographic distributions, and compatibility modes. Saved searches and tables consolidate essential attributes—including verified emails, device and host IDs, IPs, MACs, users, and organizations—adding valuable investigative context. Together, these insights enable analysts to monitor user behavior, track device health, analyze audit activity, detect anomalies, and strengthen compliance, identity management, and endpoint security oversight.
+Audit dashboards further enhance oversight by showing event activity over time, verdicts and reasons, top rules, users, source IPs, event types, geographic distributions, and compatibility modes. Saved searches and tables consolidate essential attributes—including verified emails, device and host IDs, IPs, MACs, users, and organizations—adding valuable investigative context. Together, these insights allow organizations to monitor user behavior, track device health, detect compromised accounts, analyze audit activity, and strengthen compliance, identity management, and endpoint security oversight.
 
 ## What do I need to use this integration?
 
@@ -123,6 +122,10 @@ For more information on architectures that can be used for scaling this integrat
 
 {{fields "audit"}}
 
+#### Compromised Credential
+
+{{fields "compromised_credential"}}
+
 ### Example event
 
 #### User
@@ -137,6 +140,10 @@ For more information on architectures that can be used for scaling this integrat
 
 {{event "audit"}}
 
+#### Compromised Credential
+
+{{event "compromised_credential"}}
+
 ### Inputs used
 
 These inputs can be used in this integration:
@@ -150,3 +157,8 @@ This integration dataset uses the following APIs:
 - `User`: [Island Browser API](https://documentation.island.io/apidocs/get-all-browser-users-that-match-the-specified-simple-filter).
 - `Device`: [Island Browser API](https://documentation.island.io/apidocs/get-a-list-of-all-devices-1).
 - `Audit`: [Island Browser API](https://documentation.island.io/apidocs/get-all-timeline-audits-that-match-the-specified-simple-filter).
+- `Compromised Credential`: [Island Browser API](https://documentation.island.io/apidocs/get-a-list-of-all-compromised-credentials).
+
+#### ILM Policy
+
+To facilitate user and device data, source data stream-backed indices `.ds-logs-island_browser.user-*` and `.ds-logs-island_browser.device-*` are allowed to contain duplicates from each polling interval. ILM policy `logs-island_browser.user-default_policy` and `logs-island_browser.device-default_policy` is added to these source indices, so it doesn't lead to unbounded growth. This means that in these source indices data will be deleted after `30 days` from ingested date.

packages/island_browser/_dev/deploy/docker/files/config.yml

@@ -835,3 +835,130 @@ rules:
             "events": []
           }
           `}}
+  - path: /api/external/v1/compromised-credentials
+    methods: ['POST']
+    request_body: /.*"limit":2,"offset":0,"sortBy":"CompromisedDate","sortDirection":"Asc".*/
+    request_headers:
+      Content-Type:
+        - "application/json"
+      Api-Key:
+        - "xxxx"
+    responses:
+      - status_code: 200
+        body: |
+          {{ minify_json `
+          {
+            "compromisedCredentials": [
+              {
+                "breachSource": "Ransomware Attack - April 2025",
+                "compromisedDate": "2024-09-13T00:00:00Z",
+                "createdDate": "2024-09-21T09:46:00Z",
+                "email": "[email protected]",
+                "id": "cc-10364-ae99d-20364",
+                "impactedDomain": "enterprise.io",
+                "status": "Unresolved",
+                "tenantId": "tenant-005-tech",
+                "updatedDate": "2024-09-21T14:40:00Z",
+                "username": "john.doe364"
+              },
+              {
+                "breachSource": "Data Leak - January 2025",
+                "compromisedDate": "2024-09-14T00:00:00Z",
+                "createdDate": "2024-10-03T04:41:00Z",
+                "email": "[email protected]",
+                "id": "cc-10363-edb53-20363",
+                "impactedDomain": "business.net",
+                "status": "Investigating",
+                "tenantId": "tenant-004-biz",
+                "updatedDate": "2024-10-03T09:25:00Z",
+                "username": "emily.mitchell363"
+              }
+            ]
+          }
+          `}}
+  - path: /api/external/v1/compromised-credentials
+    methods: ['POST']
+    request_body: /.*"limit":2,"offset":2,"sortBy":"CompromisedDate","sortDirection":"Asc".*/
+    request_headers:
+      Content-Type:
+        - "application/json"
+      Api-Key:
+        - "xxxx"
+    responses:
+      - status_code: 200
+        body: |
+          {{ minify_json `
+          {
+            "compromisedCredentials": [
+              {
+                "breachSource": "Phishing Campaign - March 2025",
+                "compromisedDate": "2024-09-15T00:00:00Z",
+                "createdDate": "2024-09-18T15:36:00Z",
+                "email": "[email protected]",
+                "id": "cc-10362-9cee6-20362",
+                "impactedDomain": "mycompany.org",
+                "status": "In Progress",
+                "tenantId": "tenant-003-corp",
+                "updatedDate": "2024-09-18T19:28:00Z",
+                "username": "joseph.carter362"
+              },
+              {
+                "breachSource": "Corporate Breach - Q2 2025",
+                "compromisedDate": "2024-09-16T00:00:00Z",
+                "createdDate": "2024-10-12T03:43:00Z",
+                "email": "[email protected]",
+                "id": "cc-10361-1758b-20361",
+                "impactedDomain": "testcorp.com",
+                "status": "Resolved",
+                "tenantId": "tenant-002-secure",
+                "updatedDate": "2024-10-12T11:05:00Z",
+                "username": "abigail.nelson361"
+              }
+            ]
+          }
+          `}}
+  - path: /api/external/v1/compromised-credentials
+    methods: ['POST']
+    request_body: /.*"limit":2,"offset":4,"sortBy":"CompromisedDate","sortDirection":"Asc".*/
+    request_headers:
+      Content-Type:
+        - "application/json"
+      Api-Key:
+        - "xxxx"
+    responses:
+      - status_code: 200
+        body: |
+          {{ minify_json `
+          {
+            "compromisedCredentials": [
+              {
+                "breachSource": "DarkWeb Dump - May 2025",
+                "compromisedDate": "2024-09-17T00:00:00Z",
+                "createdDate": "2024-09-23T16:47:00Z",
+                "email": "[email protected]",
+                "id": "cc-10360-08b91-20360",
+                "impactedDomain": "example.com",
+                "status": "Unresolved",
+                "tenantId": "tenant-001-island",
+                "updatedDate": "2024-09-24T16:06:00Z",
+                "username": "christopher.gonzalez360"
+              }
+            ]
+          }
+          `}}
+  - path: /api/external/v1/compromised-credentials
+    methods: ['POST']
+    request_body: /.*"limit":2,"offset":5,"sortBy":"CompromisedDate","sortDirection":"Asc".*/
+    request_headers:
+      Content-Type:
+        - "application/json"
+      Api-Key:
+        - "xxxx"
+    responses:
+      - status_code: 200
+        body: |
+          {{ minify_json `
+          {
+            "compromisedCredentials": []
+          }
+          `}}

packages/island_browser/changelog.yml

@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: 0.4.0
+  changes:
+    - description: Add compromised credential data stream and add ILM policy for user and device data streams.
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/15372
 - version: 0.3.1
   changes:
     - description: Remove ILM policy from user and device data streams.

packages/island_browser/data_stream/compromised_credential/_dev/test/pipeline/test-common-config.yml

@@ -0,0 +1,3 @@
+fields:
+  tags:
+    - preserve_duplicate_custom_fields

packages/island_browser/data_stream/compromised_credential/_dev/test/pipeline/test-compromised-credential.log

@@ -0,0 +1,5 @@
+{"breachSource":"Ransomware Attack - April 2025","compromisedDate":"2024-09-13T00:00:00Z","createdDate":"2024-09-21T09:46:00Z","email":"[email protected]","id":"cc-10364-ae99d-20364","impactedDomain":"enterprise.io","status":"Unresolved","tenantId":"tenant-005-tech","updatedDate":"2024-09-21T14:40:00Z","username":"john.doe364"}
+{"breachSource":"Data Leak - January 2025","compromisedDate":"2024-09-14T00:00:00Z","createdDate":"2024-10-03T04:41:00Z","email":"[email protected]","id":"cc-10363-edb53-20363","impactedDomain":"business.net","status":"Investigating","tenantId":"tenant-004-biz","updatedDate":"2024-10-03T09:25:00Z","username":"emily.mitchell363"}
+{"breachSource":"Phishing Campaign - March 2025","compromisedDate":"2024-09-15T00:00:00Z","createdDate":"2024-09-18T15:36:00Z","email":"[email protected]","id":"cc-10362-9cee6-20362","impactedDomain":"mycompany.org","status":"In Progress","tenantId":"tenant-003-corp","updatedDate":"2024-09-18T19:28:00Z","username":"joseph.carter362"}
+{"breachSource":"Corporate Breach - Q2 2025","compromisedDate":"2024-09-16T00:00:00Z","createdDate":"2024-10-12T03:43:00Z","email":"[email protected]","id":"cc-10361-1758b-20361","impactedDomain":"testcorp.com","status":"Resolved","tenantId":"tenant-002-secure","updatedDate":"2024-10-12T11:05:00Z","username":"abigail.nelson361"}
+{"breachSource":"DarkWeb Dump - May 2025","compromisedDate":"2024-09-17T00:00:00Z","createdDate":"2024-09-23T16:47:00Z","email":"[email protected]","id":"cc-10360-08b91-20360","impactedDomain":"example.com","status":"Unresolved","tenantId":"tenant-001-island","updatedDate":"2024-09-24T16:06:00Z","username":"christopher.gonzalez360"}