Skip to content

Commit a6a91e7

Browse files
oswcaboswcab
authored
KFLUXINFRA-2558 - Add network policies to ESO (staging only) (#9370)
This commit introduces NetworkPolicy resources to enhance the security posture of the external-secrets-operator component by implementing a zero-trust network model. The policies include: - Default deny-all policy as a security baseline - Allow egress to Kubernetes API server and DNS for operator functionality - Allow egress to external secret providers (AWS, Vault, etc.) - Allow webhook ingress from the API server and ArgoCD for CRD validation - Allow metrics scraping from monitoring namespaces - Allow health check probes from kubelet - Allow intra-namespace communication between ESO components These policies were already tested in a OpenShift cluster but are currently deployed only to staging environments for additional validation before rolling out to production. Implements: KFLUXINFRA-2558 KFLUXINFRA-2559
1 parent 301ca5f commit a6a91e7

File tree

5 files changed

+259
-0
lines changed

5 files changed

+259
-0
lines changed

‎components/external-secrets-operator/staging/kustomization.yaml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,7 @@ apiVersion: kustomize.config.k8s.io/v1beta1
22
kind: Kustomization
33
resources:
44
- ../base
5+
- networkpolicy-default-deny.yaml
6+
- networkpolicy-allow-egress.yaml
7+
- networkpolicy-allow-ingress.yaml
8+
- networkpolicy-allow-same-ns.yaml

‎components/external-secrets-operator/staging/networkpolicy-allow-egress.yaml‎

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
# Network Policy: DNS Egress
2+
#
3+
# Purpose: Allow the external-secrets pods to communicate with DNS (for name resolution)
4+
apiVersion: networking.k8s.io/v1
5+
kind: NetworkPolicy
6+
metadata:
7+
name: allow-dns-egress
8+
namespace: external-secrets-operator
9+
spec:
10+
# Selects ALL ESO Pods: main-controller, webhook, and cert-controller
11+
podSelector:
12+
matchLabels:
13+
app.kubernetes.io/instance: external-secrets-operator
14+
policyTypes:
15+
- Egress
16+
egress:
17+
# Allow DNS queries to both the DNS pods and the DNS service (ClusterIP)
18+
# The DNS service is typically at 172.30.0.10 in the service network
19+
- to:
20+
- namespaceSelector:
21+
matchLabels:
22+
kubernetes.io/metadata.name: openshift-dns
23+
- ipBlock:
24+
cidr: 172.30.0.10/32
25+
ports:
26+
- protocol: UDP
27+
port: 53
28+
- protocol: TCP
29+
port: 53
30+
---
31+
# Network Policy: Main Controller Egress
32+
#
33+
# Purpose: Allow the main external-secrets controller to communicate with:
34+
# - Kubernetes API server (for watching CRDs, updating satus)
35+
# - External secret providers (AWS, GCP, Vault, etc.)
36+
# - OpenShift internal services
37+
apiVersion: networking.k8s.io/v1
38+
kind: NetworkPolicy
39+
metadata:
40+
name: allow-main-controller-egress
41+
namespace: external-secrets-operator
42+
spec:
43+
podSelector:
44+
matchLabels:
45+
app.kubernetes.io/instance: external-secrets-operator
46+
app.kubernetes.io/name: external-secrets
47+
policyTypes:
48+
- Egress
49+
egress:
50+
- to:
51+
- ipBlock:
52+
cidr: 0.0.0.0/0
53+
---
54+
# Network Policy: Cert Controller and Webhook Egress
55+
#
56+
# Purpose: Allow the cert-controller and webhook to communicate with
57+
# Kubernetes API server:
58+
# - cert-controller: for managing webhook certificates
59+
# - webhook: for CRD validation and cert management
60+
apiVersion: networking.k8s.io/v1
61+
kind: NetworkPolicy
62+
metadata:
63+
name: allow-cert-controller-webhook-egress
64+
namespace: external-secrets-operator
65+
spec:
66+
podSelector:
67+
matchExpressions:
68+
- key: app.kubernetes.io/name
69+
operator: In
70+
values:
71+
- external-secrets-cert-controller
72+
- external-secrets-webhook
73+
matchLabels:
74+
app.kubernetes.io/instance: external-secrets-operator
75+
policyTypes:
76+
- Egress
77+
egress:
78+
# Allow egress to Kubernetes API server
79+
# As kube-apiserver uses host network, support multiple cluster
80+
# configurations with different network CIDRs
81+
- to:
82+
# kubernetes default service
83+
- ipBlock:
84+
cidr: 172.30.0.1/32
85+
# Aggregated node network for internal clusters (10.28.x.x - 10.29.x.x)
86+
- ipBlock:
87+
cidr: 10.28.0.0/15
88+
# Aggregated node network for public clusters (10.200 - 10.210)
89+
- ipBlock:
90+
cidr: 10.192.0.0/11
91+
ports:
92+
- protocol: TCP
93+
port: 443
94+
- protocol: TCP
95+
port: 6443
96+

‎components/external-secrets-operator/staging/networkpolicy-allow-ingress.yaml‎

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
---
2+
# Network Policy: Webhook Ingress
3+
#
4+
# Purpose: Allow incoming traffic to the validating webhook on port 10250 from:
5+
# - Kubernetes API server (required for CRD validation)
6+
apiVersion: networking.k8s.io/v1
7+
kind: NetworkPolicy
8+
metadata:
9+
name: allow-webhook-ingress
10+
namespace: external-secrets-operator
11+
spec:
12+
podSelector:
13+
matchLabels:
14+
app.kubernetes.io/instance: external-secrets-operator
15+
app.kubernetes.io/name: external-secrets-webhook
16+
policyTypes:
17+
- Ingress
18+
ingress:
19+
- from:
20+
# Allow webhook traffic from Kubernetes API server
21+
# The kube-apiserver uses host network (node IPs) but the call
22+
# is NATed and comes from a pod IP assigned to interface ovn-k8s-mp0
23+
# - 192.168.0.0/16 (pod CIDR in most clusters)
24+
# - 10.128.0.0/14 (pod CIDR in stg-rh01, prd-rh01 and prd-p01 clusters)
25+
- ipBlock:
26+
cidr: 10.128.0.0/14
27+
- ipBlock:
28+
cidr: 192.168.0.0/16
29+
ports:
30+
- protocol: TCP
31+
port: 10250 # Webhook port
32+
---
33+
# Network Policy: Metrics Ingress
34+
#
35+
# Purpose: Allow Prometheus/monitoring systems to scrape metrics from all ESO components
36+
# Port: 8080 (metrics endpoint)
37+
#
38+
# Allows traffic from:
39+
# - appstudio-workload-monitoring namespace
40+
# - openshift-monitoring namespace
41+
apiVersion: networking.k8s.io/v1
42+
kind: NetworkPolicy
43+
metadata:
44+
name: allow-metrics-ingress
45+
namespace: external-secrets-operator
46+
spec:
47+
# Selects ALL ESO Pods (Controller, Webhook, Cert Controller)
48+
podSelector:
49+
matchLabels:
50+
app.kubernetes.io/instance: external-secrets-operator
51+
policyTypes:
52+
- Ingress
53+
ingress:
54+
- from:
55+
# Allow Prometheus to scrape metrics
56+
- namespaceSelector:
57+
matchLabels:
58+
kubernetes.io/metadata.name: appstudio-workload-monitoring
59+
# Also allow from openshift-monitoring if it exists
60+
- namespaceSelector:
61+
matchLabels:
62+
kubernetes.io/metadata.name: openshift-monitoring
63+
ports:
64+
- protocol: TCP
65+
port: 8080 # Metrics port
66+
---
67+
# Network Policy: Health Checks
68+
#
69+
# Purpose: Allow liveness/readiness probes on all ESO components
70+
# Port: 8081 (health check endpoint)
71+
apiVersion: networking.k8s.io/v1
72+
kind: NetworkPolicy
73+
metadata:
74+
name: allow-health-checks-ingress
75+
namespace: external-secrets-operator
76+
spec:
77+
# Selects ALL ESO Pods (Controller, Webhook, Cert Controller)
78+
podSelector:
79+
matchLabels:
80+
app.kubernetes.io/instance: external-secrets-operator
81+
policyTypes:
82+
- Ingress
83+
ingress:
84+
# Allow health checks on port 8081
85+
- from:
86+
- ipBlock:
87+
cidr: 10.128.0.0/14
88+
- ipBlock:
89+
cidr: 192.168.0.0/16
90+
ports:
91+
- protocol: TCP
92+
port: 8081
93+

‎components/external-secrets-operator/staging/networkpolicy-allow-same-ns.yaml‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
---
2+
# Network Policy: Intra-namespace Communication
3+
#
4+
# Purpose: Allow all ESO components to communicate with each other
5+
# within the same namespace, including localhost communication
6+
#
7+
# Applies to: Main controller, webhook, and cert-controller
8+
# Note: This provides a fallback for any internal coordination between ESO components
9+
# that is not explicitly covered by more specific policies
10+
apiVersion: networking.k8s.io/v1
11+
kind: NetworkPolicy
12+
metadata:
13+
name: allow-same-namespace
14+
namespace: external-secrets-operator
15+
spec:
16+
podSelector:
17+
matchLabels:
18+
app.kubernetes.io/instance: external-secrets-operator
19+
policyTypes:
20+
- Ingress
21+
- Egress
22+
ingress:
23+
- from:
24+
- podSelector:
25+
matchLabels:
26+
app.kubernetes.io/instance: external-secrets-operator
27+
# Allow pods to communicate with themselves (localhost)
28+
# This is needed for health checks and internal probe endpoints
29+
- from:
30+
- podSelector: {}
31+
ports:
32+
- protocol: TCP
33+
port: 8081 # Health check port
34+
egress:
35+
- to:
36+
- podSelector:
37+
matchLabels:
38+
app.kubernetes.io/instance: external-secrets-operator
39+
# Allow pods to reach their own localhost
40+
- to:
41+
- podSelector: {}
42+
ports:
43+
- protocol: TCP
44+
port: 8081 # Health check port

‎components/external-secrets-operator/staging/networkpolicy-default-deny.yaml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
---
2+
# Network Policy: Default Deny All
3+
#
4+
# Purpose: Implements a "deny all / permit by exception" security model
5+
#
6+
# Behavior:
7+
# - Blocks ALL ingress traffic by default
8+
# - Blocks ALL egress traffic by default
9+
# - Other NetworkPolicies selectively allow specific traffic patterns
10+
#
11+
# Security: Aligns with ESO threat model recommendations
12+
# Reference: https://external-secrets.io/latest/guides/threat-model/
13+
apiVersion: networking.k8s.io/v1
14+
kind: NetworkPolicy
15+
metadata:
16+
name: default-deny-all
17+
namespace: external-secrets-operator
18+
spec:
19+
podSelector: {}
20+
policyTypes:
21+
- Ingress
22+
- Egress

0 commit comments

Comments
 (0)