diff --git a/config/_default/menus/main.en.yaml b/config/_default/menus/main.en.yaml index 30214525f31..bb35a262359 100644 --- a/config/_default/menus/main.en.yaml +++ b/config/_default/menus/main.en.yaml @@ -6707,7 +6707,7 @@ menu: parent: cloud_siem_ingest_and_enrich identifier: cloud_siem_content_packs weight: 101 - - name: Threat Intelligence + - name: Bring Your Own Threat Intelligence url: security/cloud_siem/ingest_and_enrich/threat_intelligence parent: cloud_siem_ingest_and_enrich identifier: cloud_siem_threat_intelligence @@ -6802,11 +6802,16 @@ menu: parent: cloud_siem_triage_and_investigate identifier: cloud_siem_entities_and_risk_scoring weight: 302 + - name: IOC Explorer + url: security/cloud_siem/triage_and_investigate/ioc_explorer + parent: cloud_siem_triage_and_investigate + identifier: cloud_siem_ioc_explorer + weight: 303 - name: Investigator url: security/cloud_siem/triage_and_investigate/investigator parent: cloud_siem_triage_and_investigate identifier: cloud_siem_investigator - weight: 303 + weight: 304 - name: Respond and Report url: security/cloud_siem/respond_and_report parent: cloud_siem diff --git a/content/en/security/cloud_siem/ingest_and_enrich/threat_intelligence.md b/content/en/security/cloud_siem/ingest_and_enrich/threat_intelligence.md index 8428f83814f..047d1a5ad04 100644 --- a/content/en/security/cloud_siem/ingest_and_enrich/threat_intelligence.md +++ b/content/en/security/cloud_siem/ingest_and_enrich/threat_intelligence.md @@ -1,5 +1,5 @@ --- -title: Threat Intelligence +title: Bring Your Own Threat Intelligence disable_toc: false aliases: - /security/cloud_siem/threat_intelligence @@ -7,14 +7,15 @@ further_reading: - link: "security/cloud_siem/detection_rules" tag: "Documentation" text: "Create custom detection rules" +- link: /security/cloud_siem/triage_and_investigate/ioc_explorer/ + tag: documentation + text: IOC Explorer --- ## Overview Datadog provides built-in [threat intelligence][1] for Cloud SIEM logs. This article explains how to extend that functionality by enriching logs with your own custom threat intelligence feeds. -## Bring your own threat intelligence - Cloud SIEM supports enriching and searching logs using threat intelligence indicators of compromise (IOCs) stored in Datadog reference tables. [Reference Tables][7] allow you to combine metadata with information already in Datadog. ### How bring your own threat intelligence works @@ -38,21 +39,21 @@ When Cloud SIEM processes a log, the log's IP and domain attributes are evaluate - Cloud SIEM evaluates logs in real time and uses both [Datadog-curated threat intelligence][10] and your own reference tables. - Reference tables are the mechanism for storing and joining your custom IoCs with logs and detections. -### Storing indicators of compromise in reference tables +### Store indicators of compromise in reference tables Threat intelligence is supported in the CSV format, and requires a table for each Indicator type (for example, IP address or domain) and requires the following columns: #### CSV structure for IP address -| Field | Data | Description | Required | Example | -|-------------------|-------|-------------------------------------------------------------------------------------------------|----------|----------------------------------| -| ip_address | text | The primary key for the reference table in the IPv4 dot notation format. | true | 192.0.2.1 | -| additional_data | json | Additional data to enrich the logs. | false | `{"ref":"hxxp://example.org"}` | -| category | text | The threat intel [category][8]. This is used by some out-of-the-box detection rules. | true | Malware | -| intention | text | The threat intel [intent][9]. This is used by some out-of-the-box detection rules. | true | malicious | -| source | text | The name of the source and the link to its site, such as your team and your team's wiki. | true | `{"name":"internal_security_team", "url":"https://teamwiki.example.org"}` | +| Field | Data | Description | Required | Example | +|-----------------|------|------------------------------------------------------------------------------------------|----------|---------------------------------------------------------------------------| +| ip_address | text | The primary key for the reference table in the IPv4 dot notation format. | true | 192.0.2.1 | +| additional_data | json | Additional data to enrich the logs. | false | `{"ref":"hxxp://example.org"}` | +| category | text | The threat intel [category][8]. This is used by some out-of-the-box detection rules. | true | Malware | +| intention | text | The threat intel [intent][9]. This is used by some out-of-the-box detection rules. | true | malicious | +| source | text | The name of the source and the link to its site, such as your team and your team's wiki. | true | `{"name":"internal_security_team", "url":"https://teamwiki.example.org"}` | -
JSON in a CSV requires double quoting. The following is an example CSV.
+
JSON in a CSV requires double quoting. The following is an example CSV:
``` ip_address,additional_data,category,intention,source @@ -63,15 +64,15 @@ ip_address,additional_data,category,intention,source #### CSV structure for domain -| Field | Data | Description | Required | Example | -|-------------------|-------|-------------------------------------------------------------------------------------------------|----------|----------------------------------| -| domain | text | The primary key for the reference table. | true | mal-domain.com | -| additional_data | json | Additional data to enrich the trace. | false | `{"ref":"hxxp://example.org"}` | -| category | text | The threat intel [category][8]. This is used by some out-of-the-box detection rules. | true | Phishing | -| intention | text | The threat intel [intent][9]. This is used by some out-of-the-box detection rules. | true | malicious | -| source | text | The name of the source and the link to its site, such as your team and your team's wiki. | true | `{"name":"internal_security_team", "url":"https://teamwiki.example.org"}` | +| Field | Data | Description | Required | Example | +|-----------------|------|------------------------------------------------------------------------------------------|----------|---------------------------------------------------------------------------| +| domain | text | The primary key for the reference table. | true | mal-domain.com | +| additional_data | json | Additional data to enrich the trace. | false | `{"ref":"hxxp://example.org"}` | +| category | text | The threat intel [category][8]. This is used by some out-of-the-box detection rules. | true | Phishing | +| intention | text | The threat intel [intent][9]. This is used by some out-of-the-box detection rules. | true | malicious | +| source | text | The name of the source and the link to its site, such as your team and your team's wiki. | true | `{"name":"internal_security_team", "url":"https://teamwiki.example.org"}` | -### Uploading and enabling your own threat intelligence +### Upload and enable your own threat intelligence Datadog supports creating reference tables either by a manual upload or by periodically retrieving the data from Amazon S3, Azure storage, or Google Cloud storage. @@ -110,15 +111,15 @@ In Datadog Event Management, it may appear that data has been fetched from the c - The update replaces the entire table with the new data. In case of a duplicated primary key, the rows with the duplicated key are not written, and an error is shown in the reference table detail page. -## Threat intelligence in the user interface +## View threat intelligence data in Datadog To enable Cloud SIEM threat intelligence data for reference tables: 1. Navigate to [Threat Intelligence][3]. -1. For the table you want to see Cloud SIEM threat intelligence data, click the dropdown menu in the **Enabled** column and select Cloud SIEM. +1. For the table you want to see Cloud SIEM threat intelligence data for, click the dropdown menu in the **Enabled** column and select Cloud SIEM. -After applying a reference table to Cloud SIEM, all incoming logs are evaluated against the table using a specific Indicator of Compromise (IoC) key, such as an IP address or domain. If a match is found, the log is enriched with relevant Threat Intelligence (TI) attributes from the table, which enhances detection, investigation, and response. +After applying a reference table to Cloud SIEM, all incoming logs are evaluated against the table using a specific Indicator of Compromise (IoC) key, such as an IP address or domain. If a match is found, the log is enriched with relevant Threat Intelligence (TI) attributes from the table, which enhances detection, investigation, and response. A threat intelligence reference table can be shared across multiple security products. -A threat intelligence reference table can be shared across multiple security products. +You can view your threat intelligence data in the [IOC Explorer][11]. ## Further reading @@ -133,4 +134,5 @@ A threat intelligence reference table can be shared across multiple security pro [7]: /reference_tables/ [8]: /security/threat_intelligence/#threat-intelligence-categories [9]: /security/threat_intelligence/#threat-intelligence-intents -[10]: /security/threat_intelligence#threat-intelligence-sources \ No newline at end of file +[10]: /security/threat_intelligence#threat-intelligence-sources +[11]: /security/cloud_siem/triage_and_investigate/ioc_explorer/ \ No newline at end of file diff --git a/content/en/security/cloud_siem/triage_and_investigate/ioc_explorer.md b/content/en/security/cloud_siem/triage_and_investigate/ioc_explorer.md new file mode 100644 index 00000000000..0b09bfd64b3 --- /dev/null +++ b/content/en/security/cloud_siem/triage_and_investigate/ioc_explorer.md @@ -0,0 +1,74 @@ +--- +title: IOC Explorer +further_reading: +- link: /security/threat_intelligence/ + tag: documentation + text: Threat Intelligence +- link: /security/cloud_siem/ingest_and_enrich/threat_intelligence + tag: documentation + text: Bring Your Own Threat Intelligence +--- + +{{< callout url="" btn_hidden="true" header="false" >}} +The IOC Explorer is in Preview. +{{< /callout >}} + +## Overview + +Indicators of Compromise (IOC) are evidence that your systems have experienced a security breach. With the [IOC Explorer][1], you can view more details about compromises, and see related signals and logs. + +{{< img src="security/security_monitoring/ioc_explorer.png" alt="The IOC Explorer, showing an IP address that has been flagged as an indicator of compromise" style="width:100%;" >}} + +## Prerequisites + +To view data in the IOC Explorer, all of the following must be true: +- Your organization must subscribe to Cloud SIEM. +- The indicator of compromise must be in a threat feed that was available to Datadog at the time of the log acquisition. + - For more information on the threat intelligence feeds the IOC Explorer displays content from, see [Threat intelligence sources][2]. +- A log that has a matching entity in threat intelligence must be acquired. +- The time frame for the Explorer is fixed to the last 30 days. The log must be from within that time frame. + +## Use the IOC Explorer + +To access the IOC Explorer in Datadog, go to **Security** > **Cloud SIEM** > **Investigate** > [**IOC Explorer**][1]. + +### Query and filter indicators of compromise + +You can write custom queries or apply filters to determine which indicators of compromise you can see in the explorer. You can query or filter by: +- Severity score +- [Entity type][3] +- [Threat intelligence source][2] +- [Threat intelligence category][4] + +Additionally, you can click a column heading in the Explorer to sort by that column's values. + +### Get more context on an indicator of compromise + +Click an indicator of compromise to open a side panel that contains additional information about it: +- When the indicator was first and last seen in a threat intelligence feed +
This is distinct from the first or last time the indicator was seen in a log.
+- Any categories and ratings assigned to it, and the threat intelligence feeds associated with those ratings +- A breakdown of the indicator's severity score +- Signal matches, which you can view in Signals Explorer +- Related logs, which you can view in Log Explorer + +## Understand severity scoring + +It's important to have proper context for the severity score for an indicator, so you can properly prioritize investigations. For example, [IP addresses][5] can be volatile and require frequent reassessments as a result. + +In the IOC Explorer side panel, you can see the factors that contribute to the severity score. Severity score starts from a base score based on classification, and increases or decreases based on additional factors: +- **Classification**: The base score associated with the indicator's category and intent +- **Corroboration**: Whether the indicator appears on multiple threat intelligent feeds +- **Persistence**: How long threat intelligence feeds have been reporting on the indicator +- **Hosting Type**: Used for IP and domain entity types; evaluates whether the hosting infrastructure type is commonly used for attacks +- **Signal Activity**: Whether the indicator has been observed in Signals + +## Further reading + +{{< partial name="whats-next/whats-next.html" >}} + +[1]: https://app.datadoghq.com/security/siem/ioc-explorer +[2]: /security/threat_intelligence/#threat-intelligence-sources +[3]: /security/threat_intelligence/#entity-types +[4]: /security/threat_intelligence/#threat-intelligence-categories +[5]: /security/threat_intelligence/#ip-addresses-dynamic-and-transient \ No newline at end of file diff --git a/content/en/security/threat_intelligence.md b/content/en/security/threat_intelligence.md index 4d0f34e1e6f..34cdc2866fe 100644 --- a/content/en/security/threat_intelligence.md +++ b/content/en/security/threat_intelligence.md @@ -26,6 +26,7 @@ products: {{< product-availability >}} ## Overview + Threat Intelligence is reputation information that helps responders make informed decisions on attacks and compromises. Datadog curates commercial, open-source, and in-house threat intelligence indicators of compromise into categories and intents. Threat intelligence is updated at least once per day, per source. This data is used to enrich your logs and traces with relevant reputation information. @@ -34,45 +35,50 @@ Datadog curates commercial, open-source, and in-house threat intelligence indica Datadog Security supports enriching and searching traces with threat intelligence indicators of compromise stored in Datadog reference tables. [Reference Tables][2] allow you to combine metadata with information already in Datadog. +The amount of time threat intelligence persists as an available enrichment varies depending on source: +- OEM- and integration-based feeds: 24 hours + - Datadog reacquires threat intelligence from these providers at least once a day, and depends on the provider to persist threat intelligence information +- Bring your own threat intelligence: however long you configure the intelligence to be available + For more information, see the [Bring Your Own Threat Intelligence][3] guide. -## Threat Intelligence Lifecycle +## Threat intelligence lifecycle -Datadog collects threat intelligence across the following entity types. Each entity type has unique characteristics and a useful timeframe. This timeframe, or lifecycle, requires consideration when assessing the importance of a threat intelligence match on your data. +Datadog collects threat intelligence across the following entity types. Each entity type has unique characteristics and a useful time frame. This time frame, or lifecycle, requires consideration when assessing the importance of a threat intelligence match on your data. -### File Hashes: Unique Digital Fingerprints +### File hashes: Unique digital fingerprints File hashes function as unique digital fingerprints for specific files. When a file hash is marked as malware, it signifies the file's exact content is harmful. The immutability of a hash, which is tied to its file's content, ensures its consistent identification. As a result, a file hash tagged as malware retains this identification, provided the identification was a true positive. -### Application Packages: Malware Risk in Distribution +### Application packages: Malware risk in distribution Unlike immutable file hashes, application packages can vary in content and security, even under the same version number. Malicious actors may upload harmful packages mimicking legitimate ones, or they might compromise existing packages by introducing malware. The lifecycle of malicious packages is frequently long-lived, but not immutable. -### Domains: Temporary Signatures +### Domains: Temporary signatures Unlike file hashes, domains identified as malicious are subject to change. They may undergo processes such as remediation, reassignment, or repurposing by various entities. While the lifecycle of malicious or suspicious domains is somewhat prolonged compared to IP addresses, it remains temporary and variable. -### IP Addresses: Dynamic and Transient +### IP addresses: Dynamic and transient IP addresses represent the most volatile element in threat intelligence, often changing reputations within a 24-hour cycle. Given their dynamic nature, particularly in residential and mobile networks where multiple hosts may be involved, it's crucial to regularly reassess their status. Not all hosts connected to a low-reputation IP address are inherently malicious, underscoring the need for correlation. -## Best Practices in Threat Intelligence +## Best practices in threat intelligence With threat intelligence, reputation is key, but it must be weighed alongside other evidence. Relying solely on IP and domain intelligence for blocking traffic is not recommended, with few exceptions. A balanced, evidence-based approach is essential. Threat intelligence used in [Detection Rules][1] should reference the Datadog keys such as category (`@threat_intel.results.category`) and intent (`@threat_intel.results.intention`). Other keys should not be used. -## Transparency in Threat Intelligence +## Transparency in threat intelligence Datadog ensures transparency by providing external links to external threat intelligence sources associated with a detection. Threat intelligence curated by Datadog is ingested into the Datadog platform for enrichment and detection. Datadog does not send customer data to threat intelligence sources. The detections and enrichments are accessible in the UI and event JSON. -## Threat Intelligence Facets +## Threat intelligence facets Sources, categories, and intents are available as facets and filters on relevant product explorers. -### Threat Intelligence Sources +### Threat intelligence sources | Source | Category | Source Use Cases | Primary Products | |--------|------------|-----------|------------------| @@ -86,7 +92,7 @@ Sources, categories, and intents are available as facets and filters on relevant | [Threatfox](https://threatfox.abuse.ch/) | malware | Identify hosts communicating with known malware infrastructure | Cloud SIEM, and Workload Protection | -### Threat Intelligence Categories +### Threat intelligence categories | Category | Intention | Entity Types | Product Use Cases | Primary Products | |----------|----------|--------------|----------|------------------| @@ -100,28 +106,29 @@ Sources, categories, and intents are available as facets and filters on relevant | corp_vpn | benign | IP addresses | IPs associated to corporate VPNs | AAP and Client SIEM | | cryptomining | malicious | IP addresses | IP addresses associated with cryptomining activities | AAP, CWS, and Cloud SIEM | -### Threat Intelligence Intents -| Intent | Use Case | -|--------|----------| -| benign | Corporate VPNs and informational enrichments | -| suspicious | Low reputation | -| malicious | Malicious reputation | +### Threat intelligence intents + +| Intent | Use Case | +|------------|----------------------------------------------| +| benign | Corporate VPNs and informational enrichments | +| suspicious | Low reputation | +| malicious | Malicious reputation | -## Entity Types +## Entity types | Entity Type | Example | Use Cases | |-------------|---------|-----------------------------| | IP addresses | 128.66.0.1 | Identify IP addresses associated with attacks, command and control, and scanning activity | | domains | example.com, subdomain.example.com | Domains associated with malicious use. Often used with malware as a command and control | -| application packages versions | (example_package, 1.0.0) | Identify malicious packages downloaded from PyPi | -| file hashes [SHA1, SHA256, ssdeep (Workload Protection only)] | 5f7afeeee13aaee6874a59a510b75767156f75d14db0cd4e1725ee619730ccc8 | Identify a distinct file associated with malware or compromise |
+| application package versions | (example_package, 1.0.0) | Identify malicious packages downloaded from PyPi | +| file hashes [SHA1, SHA256, ssdeep (Workload Protection only)] | 5f7afeeee13aaee6874a59a510b75767156f75d14db0cd4e1725ee619730ccc8 | Identify a distinct file associated with malware or compromise | -**Note**: Threat intelligence sources and categories are not configurable at this time. +
Threat intelligence sources and categories are not configurable.
-## Further Reading +## Further reading {{< partial name="whats-next/whats-next.html" >}} -[1]:/security/detection_rules/ +[1]: /security/detection_rules/ [2]: /integrations/guide/reference-tables [3]: /security/guide/byoti_guide diff --git a/static/images/security/security_monitoring/ioc_explorer.png b/static/images/security/security_monitoring/ioc_explorer.png new file mode 100644 index 00000000000..2a31bcad7b9 Binary files /dev/null and b/static/images/security/security_monitoring/ioc_explorer.png differ