Skip to content

Commit 4630bc4

Browse files
committed
[Feature] Add Webhook
Signed-off-by: yandongxiao <[email protected]>
1 parent ef31be2 commit 4630bc4

40 files changed

+2429
-562
lines changed

CLAUDE.md

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Development Commands
6+
7+
### Build and Development
8+
- `make build` - Build the operator binary (output: bin/sroperator)
9+
- `make run` - Run the controller locally from your host
10+
- `make docker` - Build Docker image (requires IMG environment variable)
11+
- `make test` - Run all tests with coverage
12+
- `make fmt` - Format Go code (fails if formatting needed)
13+
- `make vet` - Run go vet static analysis
14+
- `make tidy` - Clean up go.mod dependencies
15+
16+
### Code Quality and Testing Workflow
17+
When making code changes, always run these commands in order:
18+
1. `golangci-lint run` - Check code quality and style
19+
2. `make build` - Verify code compiles successfully
20+
3. `go test ./pkg/path/to/your/package` - Run unit tests for modified packages
21+
4. `make test` - Run full test suite if needed
22+
23+
**Important**: All three checks must pass before committing code changes.
24+
25+
### Code Generation
26+
- `make generate` - Generate DeepCopy methods for API types
27+
- `make manifests` - Generate CRDs, webhooks, and RBAC manifests
28+
- `make crd-all` - Generate all CRDs and API documentation
29+
- `make gen-api` - Generate API reference documentation (NOTE: This automatically runs during `make build` - no need to run separately)
30+
31+
### Kubernetes Deployment
32+
- `make install` - Install CRDs into current Kubernetes cluster
33+
- `make deploy` - Deploy operator to current Kubernetes cluster
34+
- `make uninstall` - Remove CRDs from cluster
35+
- `make undeploy` - Remove operator from cluster
36+
37+
## Architecture
38+
39+
### Core Components
40+
This is a Kubernetes operator for StarRocks, an OLAP database system. The operator manages three main component types:
41+
- **FE (Frontend)** - Query coordination and metadata management
42+
- **BE (Backend)** - Data storage and compute for OLAP workloads
43+
- **CN (Compute Node)** - Elastic compute nodes, can be auto-scaled
44+
45+
### Code Structure
46+
- `pkg/apis/starrocks/v1/` - CRD definitions for StarRocksCluster and StarRocksWarehouse
47+
- `pkg/controllers/` - Main reconciliation controllers for the CRDs
48+
- `pkg/subcontrollers/` - Component-specific controllers (fe/, be/, cn/, feproxy/)
49+
- `pkg/k8sutils/` - Kubernetes resource management utilities and templates
50+
- `pkg/common/` - Shared utilities (hashing, logging, resource utils)
51+
- `cmd/main.go` - Operator entry point
52+
- `config/crd/` - CRD definitions and patches (only CRDs remain after kustomize removal)
53+
- `deploy/` - Generated YAML files for direct application (generated by scripts/operator.sh)
54+
- `helm-charts/` - Helm charts for operator and StarRocks deployment
55+
- `charts/kube-starrocks/` - Parent chart composed of two sub-charts
56+
- `charts/kube-starrocks/charts/operator/` - Operator deployment chart
57+
- `charts/kube-starrocks/charts/starrocks/` - StarRocks cluster deployment chart
58+
- `examples/` - Sample StarRocks cluster configurations
59+
60+
### Key Patterns
61+
- Uses Kubebuilder v3 framework for operator development
62+
- Employs subcontroller pattern where main controllers delegate to component-specific subcontrollers
63+
- Extensive use of Kubernetes templates in `pkg/k8sutils/templates/` for generating resources
64+
- Auto-scaling support specifically for CN (Compute Node) components
65+
- Support for both direct YAML application and Helm-based deployment
66+
- Kustomize support has been removed - only Helm charts and direct YAML files are supported
67+
68+
### Testing
69+
- Unit tests are co-located with source files (`*_test.go`)
70+
- Integration tests use controller-runtime's envtest framework
71+
- Test coverage reports generated during `make test`
72+
- Uses go-sqlmock for database interaction testing
73+
74+
### Important Files
75+
- `pkg/apis/starrocks/v1/starrockscluster_types.go` - Main CRD specification
76+
- `pkg/controllers/starrockscluster_controller.go` - Primary reconciliation logic
77+
- `pkg/subcontrollers/subcontroller.go` - Base interface for component controllers
78+
- `pkg/webhook/starrockscluster_webhook.go` - Validating admission webhook for StarRocksCluster
79+
- `scripts/operator.sh` - Generates `deploy/operator.yaml` using `helm template`
80+
- `scripts/create-parent-chart-values.sh` - Generates parent chart values.yaml
81+
82+
### Webhook Configuration
83+
- The operator includes a validating admission webhook that validates StarRocksCluster resources
84+
- Webhook validates cluster configuration, resource requirements, HA settings, and autoscaling policies
85+
- Webhook can be enabled/disabled through Helm values (`starrocksOperator.webhook.enabled`)
86+
- Supports both automatic self-signed certificate generation and custom certificates
87+
88+
#### Local Development with Webhooks
89+
- By default, webhooks are disabled when running locally (`make run`)
90+
- To enable webhooks locally: `make run -- --enable-webhooks`
91+
- The operator automatically generates self-signed certificates when webhooks are enabled
92+
- No manual certificate generation needed for local development
93+
94+
#### Production Deployment with Webhooks
95+
- Enable webhooks in Helm values: `starrocksOperator.webhook.enabled=true`
96+
- For custom certificates, provide secret name in `starrocksOperator.webhook.tls.secretName`
97+
- Recommended to use cert-manager for automatic certificate management
98+
99+
#### Webhook Certificate Management
100+
- The operator automatically generates self-signed certificates when webhooks are enabled and no custom certificate is provided
101+
- Certificates are stored in an emptyDir volume mounted at the path specified by `starrocksOperator.webhook.tls.certDir`
102+
- The operator dynamically updates the ValidatingAdmissionWebhook configuration with the CA bundle after startup
103+
- This ensures proper TLS verification between Kubernetes API server and the webhook endpoint
104+
- For production use, consider using cert-manager or providing your own certificates via `starrocksOperator.webhook.tls.secretName`

Makefile

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -155,10 +155,6 @@ ENVTEST = $(GOBIN)/setup-envtest
155155
envtest: ## Download envtest-setup locally if necessary.
156156
$(call go-get-tool,$(ENVTEST),sigs.k8s.io/controller-runtime/tools/setup-envtest@latest)
157157

158-
.PHONY: gen-api
159-
gen-api:
160-
cd scripts && ./gen-api-reference-docs.sh
161-
162158
.PHONY: crd-all
163159
crd-all: generate manifests gen-api
164160

cmd/config/config.go

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,9 @@ var (
88

99
// VolumeNameWithHash decides whether adding a hash to the volume name
1010
VolumeNameWithHash bool
11+
12+
// WebhookCertValidityDays is the validity period in days for self-signed certificates
13+
WebhookCertValidityDays int
1114
)
1215

1316
func GetServiceDomainSuffix() string {

cmd/main.go

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,27 +17,69 @@ limitations under the License.
1717
package main
1818

1919
import (
20+
"context"
2021
"flag"
22+
"fmt"
2123
"os"
2224
"time"
2325

26+
"github.com/go-logr/logr"
27+
admissionregistrationv1 "k8s.io/api/admissionregistration/v1"
28+
"k8s.io/apimachinery/pkg/types"
2429
ctrl "sigs.k8s.io/controller-runtime"
30+
"sigs.k8s.io/controller-runtime/pkg/client"
2531
"sigs.k8s.io/controller-runtime/pkg/healthz"
2632
"sigs.k8s.io/controller-runtime/pkg/log/zap"
33+
"sigs.k8s.io/controller-runtime/pkg/webhook"
2734

2835
"github.com/StarRocks/starrocks-kubernetes-operator/cmd/config"
2936
srapi "github.com/StarRocks/starrocks-kubernetes-operator/pkg/apis/starrocks/v1"
3037
"github.com/StarRocks/starrocks-kubernetes-operator/pkg/controllers"
3138
"github.com/StarRocks/starrocks-kubernetes-operator/pkg/k8sutils"
39+
srwebhook "github.com/StarRocks/starrocks-kubernetes-operator/pkg/webhook"
3240
)
3341

3442
var (
3543
_metricsAddr string
3644
_enableLeaderElection bool
3745
_probeAddr string
3846
_namespace string
47+
_enableWebhooks bool
3948
)
4049

50+
// caBundleUpdater updates the ValidatingAdmissionWebhook with CA bundle after manager starts
51+
type caBundleUpdater struct {
52+
client client.Client
53+
caCert []byte
54+
logger logr.Logger
55+
}
56+
57+
// Start implements the Runnable interface
58+
func (c *caBundleUpdater) Start(ctx context.Context) error {
59+
c.logger.Info("updating webhook CA bundle")
60+
61+
webhookName := "starrockscluster-validating-webhook"
62+
63+
// Get the existing ValidatingAdmissionWebhook
64+
webhook := &admissionregistrationv1.ValidatingWebhookConfiguration{}
65+
if err := c.client.Get(ctx, types.NamespacedName{Name: webhookName}, webhook); err != nil {
66+
return fmt.Errorf("failed to get ValidatingAdmissionWebhook: %v", err)
67+
}
68+
69+
// Update caBundle for all webhooks
70+
for i := range webhook.Webhooks {
71+
webhook.Webhooks[i].ClientConfig.CABundle = c.caCert
72+
}
73+
74+
// Update the webhook configuration
75+
if err := c.client.Update(ctx, webhook); err != nil {
76+
return fmt.Errorf("failed to update ValidatingAdmissionWebhook: %v", err)
77+
}
78+
79+
c.logger.Info("webhook CA bundle updated successfully")
80+
return nil
81+
}
82+
4183
func main() {
4284
flag.StringVar(&_metricsAddr, "metrics-bind-address", ":8080", "The address the metric endpoint binds to.")
4385
flag.StringVar(&_probeAddr, "health-probe-bind-address", ":8081", "The address the probe endpoint binds to.")
@@ -48,6 +90,10 @@ func main() {
4890
"restricts the manager's cache to watch objects in the desired namespace. Defaults to all namespaces.")
4991
flag.StringVar(&config.DNSDomainSuffix, "dns-domain-suffix", "cluster.local", "The suffix of the dns domain in k8s")
5092
flag.BoolVar(&config.VolumeNameWithHash, "volume-name-with-hash", true, "Add a hash to the volume name")
93+
flag.BoolVar(&_enableWebhooks, "enable-webhooks", false, "Enable admission webhooks. "+
94+
"Requires TLS certificates to be configured. Disabled by default for local development.")
95+
flag.IntVar(&config.WebhookCertValidityDays, "webhook-cert-validity-days", 365,
96+
"Validity period in days for self-signed webhook certificates")
5197

5298
// Set up logger.
5399
opts := zap.Options{}
@@ -86,6 +132,41 @@ func main() {
86132
os.Exit(1)
87133
}
88134

135+
// setup webhooks if enabled
136+
if _enableWebhooks {
137+
// Get webhook configuration from environment variables
138+
webhookNamespace := os.Getenv("WEBHOOK_NAMESPACE")
139+
webhookServiceName := os.Getenv("WEBHOOK_SERVICE_NAME")
140+
webhookCertDir := os.Getenv("WEBHOOK_CERT_DIR")
141+
142+
// Generate self-signed certificates if not provided
143+
caCert, err := srwebhook.GenerateSelfSignedCerts(webhookCertDir, webhookServiceName, webhookNamespace, config.WebhookCertValidityDays)
144+
if err != nil {
145+
logger.Error(err, "unable to generate self-signed certificates")
146+
os.Exit(1)
147+
}
148+
logger.Info("self-signed certificates generated for webhook server", "validityDays",
149+
config.WebhookCertValidityDays, "namespace", webhookNamespace, "serviceName", webhookServiceName, "certDir", webhookCertDir)
150+
151+
// Add CA bundle updater as a runnable that starts after the manager cache is ready
152+
if err := mgr.Add(&caBundleUpdater{
153+
client: mgr.GetClient(),
154+
caCert: caCert,
155+
logger: logger.WithName("ca-bundle-updater"),
156+
}); err != nil {
157+
logger.Error(err, "unable to add CA bundle updater")
158+
os.Exit(1)
159+
}
160+
161+
if err := setupWebhooks(mgr); err != nil {
162+
logger.Error(err, "unable to set up webhooks")
163+
os.Exit(1)
164+
}
165+
logger.Info("webhooks enabled")
166+
} else {
167+
logger.Info("webhooks disabled - use --enable-webhooks to enable")
168+
}
169+
89170
// +kubebuilder:scaffold:builder
90171

91172
if err := mgr.AddHealthzCheck("healthz", healthz.Ping); err != nil {
@@ -107,3 +188,22 @@ func main() {
107188
os.Exit(1)
108189
}
109190
}
191+
192+
// setupWebhooks sets up webhooks for the manager
193+
func setupWebhooks(mgr ctrl.Manager) error {
194+
// Set up the webhook server
195+
hookServer := mgr.GetWebhookServer()
196+
if hookServer == nil {
197+
return fmt.Errorf("failed to get webhook server from manager")
198+
}
199+
200+
// Register the StarRocksCluster validating webhook
201+
validator := &srwebhook.StarRocksClusterValidator{
202+
Client: mgr.GetClient(),
203+
}
204+
205+
hookServer.Register("/validate-starrocks-com-v1-starrockscluster",
206+
&webhook.Admission{Handler: validator})
207+
208+
return nil
209+
}

config/default/kustomization.yaml

Lines changed: 0 additions & 74 deletions
This file was deleted.

config/default/manager_auth_proxy_patch.yaml

Lines changed: 0 additions & 34 deletions
This file was deleted.

0 commit comments

Comments
 (0)