feat: Claude Code Monitor — lanes, pipelines and a merged workspace
Internal SmartGift build of a Claude Code monitoring dashboard. Lanes: a durable unit of parallel agent work, one per working directory, tracked across session restarts. Managed lanes are git worktrees the dashboard provisions and can reset or remove behind a three-check destroy guard and a counted preflight; adopted lanes are directories you already own and are never destroyable. Pipelines: a lane moves through pipeline stages. A stage the agent declares with evidence renders green; a stage inferred from the tool-event stream renders dashed amber and never counts as done. Detection is forward-only within a 30-minute window, and never writes the declared stage. Workspace: one page at /run with a lane grid, the selected lane's pipeline, and a full Claude console behind a disclosure.
This commit is contained in:
@@ -0,0 +1,559 @@
|
||||
# Deployments
|
||||
|
||||
Production-ready, cloud-agnostic deployment infrastructure for the Claude Code Agent Monitor. Supports AWS, GCP, Azure, and OCI with Helm, Kustomize, and Terraform deployment methods, blue-green and canary release strategies, and full observability.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
> **User-facing guide:** See [DEPLOYMENT.md](../DEPLOYMENT.md) in the project root for the step-by-step deployment guide with commands and workflows.
|
||||
>
|
||||
> This README is the **technical reference** for the infrastructure code in this directory.
|
||||
|
||||
---
|
||||
|
||||
## Infrastructure Architecture
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "deployments/"
|
||||
direction TB
|
||||
|
||||
subgraph "Infrastructure Provisioning"
|
||||
TF["terraform/<br/>Cloud resource provisioning<br/>AWS · GCP · Azure · OCI"]
|
||||
end
|
||||
|
||||
subgraph "Application Deployment"
|
||||
HELM["helm/<br/>Parameterized Helm chart<br/>12 templates · 4 value sets"]
|
||||
KUST["kubernetes/<br/>Kustomize base + overlays<br/>11 resources · 3 envs"]
|
||||
end
|
||||
|
||||
subgraph "Operations"
|
||||
SCRIPTS["scripts/<br/>7 operational scripts<br/>deploy · rollback · backup"]
|
||||
CI["ci/<br/>GitHub Actions + GitLab CI<br/>Build · Scan · Deploy"]
|
||||
end
|
||||
|
||||
subgraph "Observability"
|
||||
MON["monitoring/<br/>Prometheus · Grafana · Alertmanager · Coralogix<br/>13 rules · 16 panels · OTel Collector"]
|
||||
end
|
||||
end
|
||||
|
||||
TF -->|"Provisions cloud infra"| HELM & KUST
|
||||
SCRIPTS -->|"Orchestrates"| HELM & KUST & TF
|
||||
CI -->|"Automates"| SCRIPTS
|
||||
MON -->|"Monitors"| HELM & KUST
|
||||
|
||||
style TF fill:#7b42bc,color:#fff
|
||||
style HELM fill:#0f1689,color:#fff
|
||||
style KUST fill:#326ce5,color:#fff
|
||||
style SCRIPTS fill:#4caf50,color:#fff
|
||||
style CI fill:#2088ff,color:#fff
|
||||
style MON fill:#e6522c,color:#fff
|
||||
```
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
deployments/
|
||||
├── terraform/ # Infrastructure as Code (HashiCorp Terraform)
|
||||
│ ├── main.tf # Root module — orchestrates all child modules
|
||||
│ ├── variables.tf # Input variables with validation
|
||||
│ ├── outputs.tf # Exported values (URLs, IDs, endpoints)
|
||||
│ ├── versions.tf # Terraform + provider version constraints
|
||||
│ ├── backend.tf # State backends (S3, GCS, Azure Blob, OCI S3)
|
||||
│ ├── modules/ # Reusable, cloud-agnostic modules
|
||||
│ │ ├── networking/ # VPC, subnets, security groups, NAT
|
||||
│ │ ├── compute/ # Container orchestration (ECS/Cloud Run/ACI/OKE)
|
||||
│ │ ├── database/ # Persistent storage for SQLite (EFS/Filestore/Azure Files/FSS)
|
||||
│ │ ├── loadbalancer/ # Application LB with WebSocket + blue-green weighted routing
|
||||
│ │ ├── monitoring/ # Metrics, logs, alerts, dashboards
|
||||
│ │ └── secrets/ # Vault integration or cloud-native secret stores
|
||||
│ ├── providers/ # Cloud-specific root configurations
|
||||
│ │ ├── aws/ # ECS Fargate + ALB + EFS + CloudWatch
|
||||
│ │ ├── gcp/ # Cloud Run + GCLB + Filestore + Cloud Monitoring
|
||||
│ │ ├── azure/ # ACI + App Gateway + Azure Files + Azure Monitor
|
||||
│ │ └── oci/ # OKE + LBaaS + FSS + OCI Monitoring
|
||||
│ └── environments/ # Per-environment variable overrides
|
||||
│ ├── dev/ # 1 replica, 256 CPU, monitoring off
|
||||
│ ├── staging/ # 2 replicas, 512 CPU, monitoring on
|
||||
│ └── production/ # 3 replicas, 1024 CPU, HA, blue-green
|
||||
├── kubernetes/ # Kubernetes-native manifests (Kustomize)
|
||||
│ ├── base/ # 11 shared base resources
|
||||
│ ├── overlays/ # Environment-specific patches
|
||||
│ │ ├── dev/
|
||||
│ │ ├── staging/
|
||||
│ │ └── production/
|
||||
│ ├── strategies/ # Advanced deployment patterns
|
||||
│ │ ├── blue-green/ # Zero-downtime slot switching
|
||||
│ │ └── canary/ # Progressive traffic shifting
|
||||
│ └── components/ # Optional add-ons (Kustomize components)
|
||||
│ ├── mcp-sidecar/ # MCP server as a sidecar container
|
||||
│ └── monitoring/ # Prometheus ServiceMonitor
|
||||
├── helm/ # Helm chart (alternative to Kustomize)
|
||||
│ └── agent-monitor/
|
||||
│ ├── templates/ # Kubernetes resource templates
|
||||
│ ├── values.yaml # Default values
|
||||
│ ├── values-dev.yaml
|
||||
│ ├── values-staging.yaml
|
||||
│ └── values-production.yaml
|
||||
├── scripts/ # Operational shell scripts
|
||||
│ ├── deploy.sh # Main deployment orchestrator
|
||||
│ ├── rollback.sh # Rollback to previous revision
|
||||
│ ├── blue-green-switch.sh # Switch active blue/green slot
|
||||
│ ├── health-check.sh # Comprehensive health verification
|
||||
│ ├── db-backup.sh # SQLite backup (local + cloud upload)
|
||||
│ ├── db-restore.sh # SQLite restore from backup
|
||||
│ └── teardown.sh # Full environment teardown
|
||||
├── monitoring/ # Observability stack configs
|
||||
│ ├── prometheus/ # Scrape config + alert rules
|
||||
│ ├── grafana/ # Dashboards + datasources
|
||||
│ ├── alertmanager/ # Alert routing (Slack, PagerDuty, email)
|
||||
│ └── coralogix/ # Full-stack observability (logs, metrics, traces, SLOs)
|
||||
└── ci/ # CI/CD pipeline definitions
|
||||
├── github-actions/ # GitHub Actions workflows
|
||||
└── gitlab-ci/ # GitLab CI pipeline
|
||||
```
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph Internet
|
||||
USER["Users / API Clients"]
|
||||
end
|
||||
|
||||
subgraph Cloud["Cloud Provider (AWS / GCP / Azure / OCI)"]
|
||||
LB["Load Balancer<br/>TLS termination<br/>WebSocket upgrade<br/>Blue/Green routing"]
|
||||
|
||||
subgraph Cluster["Container Cluster"]
|
||||
subgraph Blue["Blue Slot"]
|
||||
B1["agent-monitor:blue"]
|
||||
B_MCP["mcp-sidecar:blue"]
|
||||
end
|
||||
subgraph Green["Green Slot"]
|
||||
G1["agent-monitor:green"]
|
||||
G_MCP["mcp-sidecar:green"]
|
||||
end
|
||||
end
|
||||
|
||||
PV["Persistent Volume<br/>(EFS / Filestore / Azure Files / FSS)"]
|
||||
SECRETS["Secret Store<br/>(Vault / Secrets Manager)"]
|
||||
MON["Monitoring<br/>(Prometheus / Grafana)"]
|
||||
OTEL["OTel Collector<br/>(Coralogix)"]
|
||||
end
|
||||
|
||||
USER -->|HTTPS + WSS| LB
|
||||
LB -->|active slot| Blue
|
||||
LB -.->|standby| Green
|
||||
B1 --> PV
|
||||
G1 --> PV
|
||||
B1 --> SECRETS
|
||||
B_MCP -->|localhost:4820| B1
|
||||
G_MCP -->|localhost:4820| G1
|
||||
MON -->|scrape /api/health| Blue
|
||||
MON -->|scrape /api/health| Green
|
||||
Blue -->|logs + metrics| OTEL
|
||||
Green -->|logs + metrics| OTEL
|
||||
|
||||
style Blue fill:#2563eb,stroke:#3b82f6,color:#fff
|
||||
style Green fill:#16a34a,stroke:#22c55e,color:#fff
|
||||
style LB fill:#7c3aed,stroke:#a78bfa,color:#fff
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Option A: Helm (recommended for Kubernetes)
|
||||
|
||||
```bash
|
||||
# Dev
|
||||
helm install agent-monitor ./deployments/helm/agent-monitor \
|
||||
-f ./deployments/helm/agent-monitor/values-dev.yaml \
|
||||
-n agent-monitor --create-namespace
|
||||
|
||||
# Production
|
||||
helm install agent-monitor ./deployments/helm/agent-monitor \
|
||||
-f ./deployments/helm/agent-monitor/values-production.yaml \
|
||||
-n agent-monitor --create-namespace
|
||||
```
|
||||
|
||||
### Option B: Kustomize
|
||||
|
||||
```bash
|
||||
# Dev
|
||||
kubectl apply -k ./deployments/kubernetes/overlays/dev
|
||||
|
||||
# Production
|
||||
kubectl apply -k ./deployments/kubernetes/overlays/production
|
||||
```
|
||||
|
||||
### Option C: Terraform (full infra + app)
|
||||
|
||||
```bash
|
||||
cd deployments/terraform/providers/aws # or gcp, azure, oci
|
||||
terraform init
|
||||
terraform plan -var-file=../../environments/production/terraform.tfvars
|
||||
terraform apply -var-file=../../environments/production/terraform.tfvars
|
||||
```
|
||||
|
||||
### Option D: Script orchestrator
|
||||
|
||||
```bash
|
||||
./deployments/scripts/deploy.sh --env production --method helm --strategy rolling
|
||||
```
|
||||
|
||||
## Deployment Strategies
|
||||
|
||||
### Rolling Update (default)
|
||||
|
||||
Zero-downtime rolling replacement. One pod at a time is replaced with the new version.
|
||||
|
||||
```bash
|
||||
./deployments/scripts/deploy.sh --env production --method helm --strategy rolling
|
||||
```
|
||||
|
||||
### Blue-Green
|
||||
|
||||
Two identical environments. Traffic switches instantly from blue to green after validation.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Ops as Operator
|
||||
participant LB as Load Balancer
|
||||
participant Blue as Blue Slot (current)
|
||||
participant Green as Green Slot (new)
|
||||
|
||||
Ops->>Green: Deploy new version
|
||||
Ops->>Green: Run health checks
|
||||
Green-->>Ops: Healthy ✔
|
||||
Ops->>LB: Switch traffic → Green
|
||||
LB-->>Blue: Drain connections
|
||||
LB-->>Green: Route all traffic
|
||||
Note over Blue: Keep as rollback target
|
||||
```
|
||||
|
||||
```bash
|
||||
# Deploy to inactive slot
|
||||
./deployments/scripts/deploy.sh --env production --method helm --strategy blue-green
|
||||
|
||||
# Switch traffic
|
||||
./deployments/scripts/blue-green-switch.sh --env production --target green
|
||||
|
||||
# Instant rollback
|
||||
./deployments/scripts/blue-green-switch.sh --env production --target blue
|
||||
```
|
||||
|
||||
### Canary
|
||||
|
||||
Progressive traffic shifting with automated analysis. Rolls back on metric degradation.
|
||||
|
||||
```bash
|
||||
./deployments/scripts/deploy.sh --env production --method helm --strategy canary
|
||||
```
|
||||
|
||||
## Cloud Provider Comparison
|
||||
|
||||
| Feature | AWS | GCP | Azure | OCI |
|
||||
|---|---|---|---|---|
|
||||
| Compute | ECS Fargate | Cloud Run / GKE | ACI / AKS | OKE |
|
||||
| Load Balancer | ALB | GCLB | App Gateway | LBaaS |
|
||||
| Persistent Storage | EFS | Filestore | Azure Files | FSS |
|
||||
| Secrets | Secrets Manager | Secret Manager | Key Vault | Vault |
|
||||
| Monitoring | CloudWatch | Cloud Monitoring | Azure Monitor | OCI Monitoring |
|
||||
| DNS | Route 53 | Cloud DNS | Azure DNS | OCI DNS |
|
||||
| TLS Certs | ACM | Managed Certs | App Gateway Certs | Certificates |
|
||||
|
||||
## Operations
|
||||
|
||||
### Health Checks
|
||||
|
||||
```bash
|
||||
./deployments/scripts/health-check.sh --url https://monitor.example.com
|
||||
./deployments/scripts/health-check.sh --url http://localhost:4820 --retries 30
|
||||
```
|
||||
|
||||
### Backup & Restore
|
||||
|
||||
```bash
|
||||
# Backup SQLite database
|
||||
./deployments/scripts/db-backup.sh --env production --output ./backups/
|
||||
./deployments/scripts/db-backup.sh --env production --upload s3://my-bucket/backups/
|
||||
|
||||
# Restore from backup
|
||||
./deployments/scripts/db-restore.sh --env production --input ./backups/dashboard-20240101.db
|
||||
```
|
||||
|
||||
### Rollback
|
||||
|
||||
```bash
|
||||
# Helm rollback
|
||||
./deployments/scripts/rollback.sh --env production --method helm --revision 3
|
||||
|
||||
# Kubernetes rollback
|
||||
./deployments/scripts/rollback.sh --env production --method kustomize
|
||||
```
|
||||
|
||||
### Teardown
|
||||
|
||||
```bash
|
||||
./deployments/scripts/teardown.sh --env dev --method helm
|
||||
```
|
||||
|
||||
## Monitoring
|
||||
|
||||
The monitoring stack provides:
|
||||
|
||||
- **Prometheus** scrape configuration and alert rules
|
||||
- **Grafana** dashboard with request rate, latency, errors, WebSocket connections, resource usage
|
||||
- **Alertmanager** routing to Slack, PagerDuty, and email
|
||||
- **Coralogix** full-stack observability with log analytics (DataPrime), metrics, distributed tracing, SLO tracking, and error budget management via OpenTelemetry Collector
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
APP["agent-monitor pods"] -->|metrics| PROM["Prometheus"]
|
||||
APP -->|"logs + metrics"| OTEL["OTel Collector"]
|
||||
PROM -->|query| GRAF["Grafana Dashboards"]
|
||||
PROM -->|evaluate rules| AM["Alertmanager"]
|
||||
OTEL -->|"OTLP gRPC"| CX["Coralogix"]
|
||||
AM -->|critical| PD["PagerDuty"]
|
||||
AM -->|warning| SLACK["Slack"]
|
||||
AM -->|info| EMAIL["Email"]
|
||||
CX -->|alerts| PD
|
||||
CX -->|alerts| SLACK
|
||||
|
||||
style PROM fill:#e6522c,stroke:#e6522c,color:#fff
|
||||
style GRAF fill:#f46800,stroke:#f46800,color:#fff
|
||||
style AM fill:#e6522c,stroke:#e6522c,color:#fff
|
||||
style CX fill:#1a1a2e,stroke:#1a1a2e,color:#fff
|
||||
style OTEL fill:#4f46e5,stroke:#4f46e5,color:#fff
|
||||
```
|
||||
|
||||
Deploy the monitoring stack:
|
||||
|
||||
```bash
|
||||
# Apply Prometheus rules
|
||||
kubectl apply -f ./deployments/monitoring/prometheus/rules/
|
||||
|
||||
# Import Grafana dashboard
|
||||
# Upload monitoring/grafana/dashboards/agent-monitor.json via Grafana UI or API
|
||||
|
||||
# Apply Alertmanager config
|
||||
kubectl create secret generic alertmanager-config \
|
||||
--from-file=./deployments/monitoring/alertmanager/alertmanager.yaml
|
||||
|
||||
# Deploy Coralogix OTel Collector (optional)
|
||||
helm repo add coralogix https://cgx.jfrog.io/artifactory/coralogix-charts-virtual
|
||||
kubectl create secret generic coralogix-keys \
|
||||
--namespace agent-monitor \
|
||||
--from-literal=PRIVATE_KEY=<YOUR_CORALOGIX_KEY>
|
||||
helm install coralogix-otel coralogix/opentelemetry \
|
||||
--namespace agent-monitor \
|
||||
-f ./deployments/monitoring/coralogix/values.yaml
|
||||
```
|
||||
|
||||
## CI/CD
|
||||
|
||||
### GitHub Actions
|
||||
|
||||
Three workflows are provided:
|
||||
|
||||
| Workflow | Trigger | Purpose |
|
||||
|---|---|---|
|
||||
| `ci.yaml` | Push/PR to main | Lint, test, build, security scan |
|
||||
| `deploy.yaml` | Tag `v*` or manual | Build → staging (auto) → production (manual) |
|
||||
| `rollback.yaml` | Manual dispatch | Rollback to a specific revision |
|
||||
|
||||
### GitLab CI
|
||||
|
||||
Single `.gitlab-ci.yml` covering all stages from test through production deploy.
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `IMAGE_REGISTRY` | — | Container image registry URL |
|
||||
| `IMAGE_TAG` | `latest` | Container image tag |
|
||||
| `DASHBOARD_PORT` | `4820` | Dashboard API + UI port |
|
||||
| `NODE_ENV` | `production` | Node.js environment |
|
||||
| `MCP_TRANSPORT` | `stdio` | MCP transport mode (stdio/http/repl) |
|
||||
| `MCP_HTTP_PORT` | `8819` | MCP HTTP server port |
|
||||
| `TLS_CERT_ARN` | — | TLS certificate ARN/ID (cloud-specific) |
|
||||
| `DOMAIN` | — | Public domain for ingress/DNS |
|
||||
|
||||
---
|
||||
|
||||
## Terraform Module Reference
|
||||
|
||||
The Terraform infrastructure is organized as reusable modules that work across all four cloud providers.
|
||||
|
||||
### Module Dependency Chain
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
NET[networking/] --> DB[database/]
|
||||
NET --> COMP[compute/]
|
||||
NET --> LB[loadbalancer/]
|
||||
DB --> COMP
|
||||
COMP --> LB
|
||||
COMP --> MON[monitoring/]
|
||||
LB --> MON
|
||||
|
||||
style NET fill:#42a5f5,color:#fff
|
||||
style DB fill:#66bb6a,color:#fff
|
||||
style COMP fill:#ffa726,color:#fff
|
||||
style LB fill:#ab47bc,color:#fff
|
||||
style MON fill:#ef5350,color:#fff
|
||||
```
|
||||
|
||||
### networking/
|
||||
|
||||
Provisions the cloud network foundation.
|
||||
|
||||
| Output | Description |
|
||||
|--------|-------------|
|
||||
| `vpc_id` | VPC / VNet / VCN identifier |
|
||||
| `public_subnet_ids` | Subnets for load balancers |
|
||||
| `private_subnet_ids` | Subnets for containers |
|
||||
| `storage_security_group_ids` | SG allowing NFS (port 2049) |
|
||||
|
||||
### database/
|
||||
|
||||
Provisions persistent storage for SQLite data.
|
||||
|
||||
| Provider | Service | Encryption |
|
||||
|----------|---------|:----------:|
|
||||
| AWS | EFS (Elastic File System) | AES-256 at rest + TLS in transit |
|
||||
| GCP | Filestore (NFS) | Google-managed |
|
||||
| Azure | Azure Files (SMB/NFS) | SSE with platform key |
|
||||
| OCI | File Storage Service (NFS) | Oracle-managed |
|
||||
|
||||
### compute/
|
||||
|
||||
Provisions dual blue/green container slots with auto-scaling.
|
||||
|
||||
| Provider | Service | Container Runtime |
|
||||
|----------|---------|-------------------|
|
||||
| AWS | ECS Fargate | Docker |
|
||||
| GCP | Cloud Run v2 | Docker |
|
||||
| Azure | Container Instances | Docker |
|
||||
| OCI | Container Instances / OKE | Docker |
|
||||
|
||||
### loadbalancer/
|
||||
|
||||
Provisions the application load balancer with TLS termination and WebSocket support.
|
||||
|
||||
| Feature | Implementation |
|
||||
|---------|---------------|
|
||||
| TLS | TLS 1.3 minimum policy |
|
||||
| WebSocket | Sticky sessions (cookie/ClientIP) |
|
||||
| Blue-green | Weighted target groups (0-100) |
|
||||
| Health checks | HTTP GET `/api/health` every 30s |
|
||||
| Idle timeout | 300s (for long-lived WebSocket) |
|
||||
|
||||
### monitoring/
|
||||
|
||||
Provisions cloud-native monitoring and alerting, with optional Coralogix full-stack observability.
|
||||
|
||||
| Provider | Metrics | Alarms | Logs |
|
||||
|----------|---------|--------|------|
|
||||
| AWS | CloudWatch | SNS → Email | CloudWatch Logs |
|
||||
| GCP | Cloud Monitoring | Notification Channel | Cloud Logging |
|
||||
| Azure | Azure Monitor | Action Group | Log Analytics |
|
||||
| OCI | OCI Monitoring | Notification Topic | OCI Logging |
|
||||
| Coralogix | PromQL + Recording Rules | Coralogix Alerts → PagerDuty/Slack | DataPrime Log Analytics |
|
||||
|
||||
### Root Variables
|
||||
|
||||
Key variables defined in `terraform/variables.tf`:
|
||||
|
||||
| Variable | Type | Validation | Description |
|
||||
|----------|------|-----------|-------------|
|
||||
| `cloud_provider` | string | `aws\|gcp\|azure\|oci` | Target cloud |
|
||||
| `environment` | string | `dev\|staging\|production` | Deployment tier |
|
||||
| `vpc_cidr` | string | Valid CIDR | Network address space |
|
||||
| `cpu` | number | `256\|512\|1024\|2048\|4096` | CPU units per container |
|
||||
| `deployment_strategy` | string | `rolling\|blue-green\|canary` | Release strategy |
|
||||
| `blue_weight` / `green_weight` | number | `0-100` | Traffic distribution |
|
||||
|
||||
---
|
||||
|
||||
## Kubernetes Security Posture
|
||||
|
||||
All Kubernetes manifests enforce the **Restricted Pod Security Standard**:
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "Namespace"
|
||||
NS["pod-security.kubernetes.io/enforce: restricted"]
|
||||
end
|
||||
|
||||
subgraph "Pod Security Context"
|
||||
PSC1[runAsNonRoot: true]
|
||||
PSC2[runAsUser: 1000]
|
||||
PSC3[fsGroup: 1000]
|
||||
PSC4["seccompProfile: RuntimeDefault"]
|
||||
end
|
||||
|
||||
subgraph "Container Security Context"
|
||||
CSC1[readOnlyRootFilesystem: true]
|
||||
CSC2[allowPrivilegeEscalation: false]
|
||||
CSC3["capabilities.drop: ALL"]
|
||||
CSC4[automountServiceAccountToken: false]
|
||||
end
|
||||
|
||||
NS --> PSC1 & PSC2 & PSC3 & PSC4
|
||||
PSC1 --> CSC1 & CSC2 & CSC3 & CSC4
|
||||
|
||||
style NS fill:#f44336,color:#fff
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Data Flow
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant User as Browser
|
||||
participant LB as Load Balancer
|
||||
participant App as Dashboard Pod
|
||||
participant DB as SQLite (PV)
|
||||
participant WS as WebSocket
|
||||
participant Hook as Claude Code Hook
|
||||
|
||||
Hook->>App: POST /api/hooks/event
|
||||
App->>DB: INSERT event
|
||||
App->>WS: broadcast(new_event)
|
||||
WS->>User: WebSocket message
|
||||
|
||||
User->>LB: GET /api/sessions
|
||||
LB->>App: Forward (sticky session)
|
||||
App->>DB: SELECT sessions
|
||||
App->>LB: JSON response
|
||||
LB->>User: HTTPS response
|
||||
|
||||
User->>LB: WSS upgrade
|
||||
LB->>App: WebSocket handshake
|
||||
App->>User: Real-time events
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [DEPLOYMENT.md](../DEPLOYMENT.md) — Step-by-step deployment guide with workflows
|
||||
- [terraform/README.md](./terraform/README.md) — Terraform module details
|
||||
- [kubernetes/README.md](./kubernetes/README.md) — Kustomize overlay guide
|
||||
@@ -0,0 +1,250 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# GitHub Actions CI Pipeline – Claude Code Agent Monitor
|
||||
#
|
||||
# Triggers on push to main and PRs. Runs linting, tests, builds Docker
|
||||
# images, and scans for security vulnerabilities.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
name: CI
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write
|
||||
security-events: write
|
||||
|
||||
concurrency:
|
||||
group: ci-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
NODE_VERSION: "22"
|
||||
REGISTRY: ghcr.io
|
||||
IMAGE_NAME: ${{ github.repository }}/agent-monitor
|
||||
MCP_IMAGE_NAME: ${{ github.repository }}/agent-monitor-mcp
|
||||
|
||||
jobs:
|
||||
# ── Lint & Format Check ─────────────────────────────────────────────────
|
||||
lint:
|
||||
name: Lint & Format
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: ${{ env.NODE_VERSION }}
|
||||
cache: "npm"
|
||||
|
||||
- name: Install dependencies
|
||||
run: npm ci
|
||||
|
||||
- name: Check formatting
|
||||
run: npm run format:check
|
||||
|
||||
# ── Server Tests ────────────────────────────────────────────────────────
|
||||
test-server:
|
||||
name: Server Tests
|
||||
runs-on: ubuntu-latest
|
||||
needs: lint
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: ${{ env.NODE_VERSION }}
|
||||
cache: "npm"
|
||||
|
||||
- name: Install dependencies
|
||||
run: npm ci
|
||||
|
||||
- name: Run server tests
|
||||
run: npm run test:server
|
||||
|
||||
# ── Client Tests ────────────────────────────────────────────────────────
|
||||
test-client:
|
||||
name: Client Tests
|
||||
runs-on: ubuntu-latest
|
||||
needs: lint
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: ${{ env.NODE_VERSION }}
|
||||
cache: "npm"
|
||||
|
||||
- name: Install root dependencies
|
||||
run: npm ci
|
||||
|
||||
- name: Install client dependencies
|
||||
run: cd client && npm ci
|
||||
|
||||
- name: Run client tests
|
||||
run: npm run test:client
|
||||
|
||||
# ── MCP Tests ───────────────────────────────────────────────────────────
|
||||
test-mcp:
|
||||
name: MCP Sidecar Tests
|
||||
runs-on: ubuntu-latest
|
||||
needs: lint
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: ${{ env.NODE_VERSION }}
|
||||
cache: "npm"
|
||||
|
||||
- name: Install MCP dependencies
|
||||
run: npm run mcp:install
|
||||
|
||||
- name: Type check MCP
|
||||
run: npm run mcp:typecheck
|
||||
|
||||
- name: Run MCP tests
|
||||
run: npm run mcp:test
|
||||
|
||||
# ── Build Docker Images ────────────────────────────────────────────────
|
||||
build-image:
|
||||
name: Build Docker Images
|
||||
runs-on: ubuntu-latest
|
||||
needs: [test-server, test-client, test-mcp]
|
||||
# Only push images on main branch
|
||||
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
||||
outputs:
|
||||
image-tag: ${{ steps.meta.outputs.version }}
|
||||
image-digest: ${{ steps.build-app.outputs.digest }}
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v3
|
||||
|
||||
- name: Login to Container Registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Extract metadata (app)
|
||||
id: meta
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
|
||||
tags: |
|
||||
type=ref,event=branch
|
||||
type=sha,prefix=sha-
|
||||
type=raw,value=latest,enable={{is_default_branch}}
|
||||
type=semver,pattern={{version}},enable=${{ startsWith(github.ref, 'refs/tags/v') }}
|
||||
|
||||
- name: Build & push app image
|
||||
id: build-app
|
||||
uses: docker/build-push-action@v5
|
||||
with:
|
||||
context: .
|
||||
file: ./Dockerfile
|
||||
push: true
|
||||
tags: ${{ steps.meta.outputs.tags }}
|
||||
labels: ${{ steps.meta.outputs.labels }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
platforms: linux/amd64,linux/arm64
|
||||
|
||||
- name: Extract metadata (MCP)
|
||||
id: meta-mcp
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: ${{ env.REGISTRY }}/${{ env.MCP_IMAGE_NAME }}
|
||||
tags: |
|
||||
type=ref,event=branch
|
||||
type=sha,prefix=sha-
|
||||
type=raw,value=latest,enable={{is_default_branch}}
|
||||
|
||||
- name: Build & push MCP image
|
||||
id: build-mcp
|
||||
uses: docker/build-push-action@v5
|
||||
with:
|
||||
context: .
|
||||
file: ./mcp/Dockerfile
|
||||
push: true
|
||||
tags: ${{ steps.meta-mcp.outputs.tags }}
|
||||
labels: ${{ steps.meta-mcp.outputs.labels }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
platforms: linux/amd64,linux/arm64
|
||||
|
||||
# ── Security Scan ──────────────────────────────────────────────────────
|
||||
security-scan:
|
||||
name: Security Scan
|
||||
runs-on: ubuntu-latest
|
||||
needs: build-image
|
||||
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Login to Container Registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Run Trivy vulnerability scanner (app)
|
||||
uses: aquasecurity/trivy-action@master
|
||||
with:
|
||||
image-ref: "${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ needs.build-image.outputs.image-tag }}"
|
||||
format: "sarif"
|
||||
output: "trivy-app-results.sarif"
|
||||
severity: "CRITICAL,HIGH"
|
||||
exit-code: "1"
|
||||
|
||||
- name: Run Trivy vulnerability scanner (MCP)
|
||||
uses: aquasecurity/trivy-action@master
|
||||
with:
|
||||
image-ref: "${{ env.REGISTRY }}/${{ env.MCP_IMAGE_NAME }}:${{ needs.build-image.outputs.image-tag }}"
|
||||
format: "sarif"
|
||||
output: "trivy-mcp-results.sarif"
|
||||
severity: "CRITICAL,HIGH"
|
||||
exit-code: "1"
|
||||
|
||||
- name: Upload Trivy SARIF (app)
|
||||
uses: github/codeql-action/upload-sarif@v3
|
||||
if: always()
|
||||
with:
|
||||
sarif_file: "trivy-app-results.sarif"
|
||||
category: "trivy-app"
|
||||
|
||||
- name: Upload Trivy SARIF (MCP)
|
||||
uses: github/codeql-action/upload-sarif@v3
|
||||
if: always()
|
||||
with:
|
||||
sarif_file: "trivy-mcp-results.sarif"
|
||||
category: "trivy-mcp"
|
||||
|
||||
- name: Run npm audit
|
||||
run: npm audit --production --audit-level=high
|
||||
|
||||
- name: Trivy filesystem scan (IaC)
|
||||
uses: aquasecurity/trivy-action@master
|
||||
with:
|
||||
scan-type: "fs"
|
||||
scan-ref: "./deployments"
|
||||
format: "table"
|
||||
severity: "CRITICAL,HIGH"
|
||||
exit-code: "1"
|
||||
@@ -0,0 +1,335 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# GitHub Actions Deploy Pipeline – Claude Code Agent Monitor
|
||||
#
|
||||
# Triggers on version tags and manual dispatch. Deploys to staging
|
||||
# automatically and to production after manual approval.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
name: Deploy
|
||||
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- "v*"
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
environment:
|
||||
description: "Target environment"
|
||||
required: true
|
||||
type: choice
|
||||
options:
|
||||
- staging
|
||||
- production
|
||||
image_tag:
|
||||
description: "Image tag to deploy (default: latest from main)"
|
||||
required: false
|
||||
type: string
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
packages: read
|
||||
id-token: write # For OIDC cloud auth
|
||||
|
||||
concurrency:
|
||||
group: deploy-${{ github.event.inputs.environment || 'staging' }}
|
||||
cancel-in-progress: false
|
||||
|
||||
env:
|
||||
REGISTRY: ghcr.io
|
||||
IMAGE_NAME: ${{ github.repository }}/agent-monitor
|
||||
MCP_IMAGE_NAME: ${{ github.repository }}/agent-monitor-mcp
|
||||
HELM_CHART_PATH: deployments/helm/agent-monitor
|
||||
|
||||
jobs:
|
||||
# ── Resolve image tag ──────────────────────────────────────────────────
|
||||
prepare:
|
||||
name: Prepare Deployment
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
image-tag: ${{ steps.resolve.outputs.tag }}
|
||||
version: ${{ steps.resolve.outputs.version }}
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Resolve image tag
|
||||
id: resolve
|
||||
run: |
|
||||
if [[ -n "${{ github.event.inputs.image_tag }}" ]]; then
|
||||
TAG="${{ github.event.inputs.image_tag }}"
|
||||
elif [[ "${{ github.ref_type }}" == "tag" ]]; then
|
||||
TAG="${{ github.ref_name }}"
|
||||
else
|
||||
TAG="sha-$(git rev-parse --short HEAD)"
|
||||
fi
|
||||
echo "tag=${TAG}" >> "$GITHUB_OUTPUT"
|
||||
echo "version=${TAG#v}" >> "$GITHUB_OUTPUT"
|
||||
echo "Resolved image tag: ${TAG}"
|
||||
|
||||
# ── Build (if triggered by tag) ────────────────────────────────────────
|
||||
build:
|
||||
name: Build Images
|
||||
runs-on: ubuntu-latest
|
||||
needs: prepare
|
||||
if: github.ref_type == 'tag'
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v3
|
||||
|
||||
- name: Login to Container Registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Build & push app image
|
||||
uses: docker/build-push-action@v5
|
||||
with:
|
||||
context: .
|
||||
file: ./Dockerfile
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ needs.prepare.outputs.image-tag }}
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
platforms: linux/amd64,linux/arm64
|
||||
|
||||
- name: Build & push MCP image
|
||||
uses: docker/build-push-action@v5
|
||||
with:
|
||||
context: .
|
||||
file: ./mcp/Dockerfile
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.REGISTRY }}/${{ env.MCP_IMAGE_NAME }}:${{ needs.prepare.outputs.image-tag }}
|
||||
${{ env.REGISTRY }}/${{ env.MCP_IMAGE_NAME }}:latest
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
platforms: linux/amd64,linux/arm64
|
||||
|
||||
# ── Deploy to Staging ──────────────────────────────────────────────────
|
||||
deploy-staging:
|
||||
name: Deploy to Staging
|
||||
runs-on: ubuntu-latest
|
||||
needs: [prepare, build]
|
||||
if: |
|
||||
always() &&
|
||||
needs.prepare.result == 'success' &&
|
||||
(needs.build.result == 'success' || needs.build.result == 'skipped') &&
|
||||
(github.event.inputs.environment == 'staging' || github.event.inputs.environment == '' || github.ref_type == 'tag')
|
||||
environment:
|
||||
name: staging
|
||||
url: https://staging.agent-monitor.example.com
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Configure AWS credentials (OIDC)
|
||||
uses: aws-actions/configure-aws-credentials@v4
|
||||
with:
|
||||
role-to-assume: ${{ secrets.AWS_ROLE_ARN_STAGING }}
|
||||
aws-region: ${{ vars.AWS_REGION || 'us-west-2' }}
|
||||
|
||||
- name: Setup kubectl
|
||||
uses: azure/setup-kubectl@v3
|
||||
with:
|
||||
version: "v1.29.0"
|
||||
|
||||
- name: Setup Helm
|
||||
uses: azure/setup-helm@v4
|
||||
with:
|
||||
version: "v3.14.0"
|
||||
|
||||
- name: Update kubeconfig
|
||||
run: |
|
||||
aws eks update-kubeconfig \
|
||||
--region ${{ vars.AWS_REGION || 'us-west-2' }} \
|
||||
--name ${{ vars.EKS_CLUSTER_STAGING || 'agent-monitor-staging' }}
|
||||
|
||||
- name: Deploy to staging via Helm
|
||||
run: |
|
||||
helm upgrade --install agent-monitor ${{ env.HELM_CHART_PATH }} \
|
||||
--namespace agent-monitor-staging \
|
||||
--create-namespace \
|
||||
--set image.repository=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} \
|
||||
--set image.tag=${{ needs.prepare.outputs.image-tag }} \
|
||||
--set mcp.image.repository=${{ env.REGISTRY }}/${{ env.MCP_IMAGE_NAME }} \
|
||||
--set mcp.image.tag=${{ needs.prepare.outputs.image-tag }} \
|
||||
--set environment=staging \
|
||||
--set ingress.host=staging.agent-monitor.example.com \
|
||||
--values ${{ env.HELM_CHART_PATH }}/values-staging.yaml \
|
||||
--wait \
|
||||
--atomic \
|
||||
--timeout 600s
|
||||
|
||||
- name: Health check
|
||||
run: |
|
||||
echo "Waiting for pods to be ready..."
|
||||
kubectl wait --for=condition=ready pod \
|
||||
-l app.kubernetes.io/name=agent-monitor \
|
||||
-n agent-monitor-staging \
|
||||
--timeout=300s
|
||||
|
||||
# Port forward and check health
|
||||
kubectl port-forward svc/agent-monitor 14820:4820 -n agent-monitor-staging &
|
||||
PF_PID=$!
|
||||
sleep 5
|
||||
|
||||
for i in $(seq 1 10); do
|
||||
if curl -sf http://localhost:14820/api/health | grep -q '"status":"ok"'; then
|
||||
echo "✔ Health check passed"
|
||||
kill $PF_PID 2>/dev/null || true
|
||||
exit 0
|
||||
fi
|
||||
echo "Attempt $i/10..."
|
||||
sleep 5
|
||||
done
|
||||
|
||||
kill $PF_PID 2>/dev/null || true
|
||||
echo "✖ Health check failed"
|
||||
exit 1
|
||||
|
||||
- name: Notify Slack (staging)
|
||||
if: always()
|
||||
uses: slackapi/slack-github-action@v1.26.0
|
||||
with:
|
||||
payload: |
|
||||
{
|
||||
"text": "${{ job.status == 'success' && '✅' || '❌' }} Staging deployment ${{ job.status }}: `${{ needs.prepare.outputs.image-tag }}`",
|
||||
"blocks": [
|
||||
{
|
||||
"type": "section",
|
||||
"text": {
|
||||
"type": "mrkdwn",
|
||||
"text": "${{ job.status == 'success' && ':white_check_mark:' || ':x:' }} *Staging Deployment ${{ job.status }}*\n*Image:* `${{ needs.prepare.outputs.image-tag }}`\n*Commit:* `${{ github.sha }}`\n*Actor:* ${{ github.actor }}"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
env:
|
||||
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
|
||||
SLACK_WEBHOOK_TYPE: INCOMING_WEBHOOK
|
||||
|
||||
# ── Deploy to Production ───────────────────────────────────────────────
|
||||
deploy-production:
|
||||
name: Deploy to Production
|
||||
runs-on: ubuntu-latest
|
||||
needs: [prepare, deploy-staging]
|
||||
if: |
|
||||
always() &&
|
||||
needs.prepare.result == 'success' &&
|
||||
needs.deploy-staging.result == 'success' &&
|
||||
(github.event.inputs.environment == 'production' || github.ref_type == 'tag')
|
||||
environment:
|
||||
name: production
|
||||
url: https://agent-monitor.example.com
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Configure AWS credentials (OIDC)
|
||||
uses: aws-actions/configure-aws-credentials@v4
|
||||
with:
|
||||
role-to-assume: ${{ secrets.AWS_ROLE_ARN_PRODUCTION }}
|
||||
aws-region: ${{ vars.AWS_REGION || 'us-west-2' }}
|
||||
|
||||
- name: Setup kubectl
|
||||
uses: azure/setup-kubectl@v3
|
||||
with:
|
||||
version: "v1.29.0"
|
||||
|
||||
- name: Setup Helm
|
||||
uses: azure/setup-helm@v4
|
||||
with:
|
||||
version: "v3.14.0"
|
||||
|
||||
- name: Update kubeconfig
|
||||
run: |
|
||||
aws eks update-kubeconfig \
|
||||
--region ${{ vars.AWS_REGION || 'us-west-2' }} \
|
||||
--name ${{ vars.EKS_CLUSTER_PRODUCTION || 'agent-monitor-production' }}
|
||||
|
||||
- name: Create database backup
|
||||
run: |
|
||||
chmod +x deployments/scripts/db-backup.sh
|
||||
# Find a running pod to backup from
|
||||
POD=$(kubectl get pods -n agent-monitor-production \
|
||||
-l app.kubernetes.io/name=agent-monitor \
|
||||
--field-selector=status.phase=Running \
|
||||
-o jsonpath='{.items[0].metadata.name}' 2>/dev/null || echo "")
|
||||
|
||||
if [[ -n "$POD" ]]; then
|
||||
echo "Backing up database from pod: $POD"
|
||||
kubectl exec "$POD" -n agent-monitor-production -- \
|
||||
sh -c "cp /app/data/dashboard.db /tmp/pre-deploy-backup.db 2>/dev/null || true"
|
||||
echo "Pre-deploy backup created"
|
||||
else
|
||||
echo "⚠ No running pods found – skipping backup"
|
||||
fi
|
||||
|
||||
- name: Deploy to production via Helm
|
||||
run: |
|
||||
helm upgrade --install agent-monitor ${{ env.HELM_CHART_PATH }} \
|
||||
--namespace agent-monitor-production \
|
||||
--create-namespace \
|
||||
--set image.repository=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} \
|
||||
--set image.tag=${{ needs.prepare.outputs.image-tag }} \
|
||||
--set mcp.image.repository=${{ env.REGISTRY }}/${{ env.MCP_IMAGE_NAME }} \
|
||||
--set mcp.image.tag=${{ needs.prepare.outputs.image-tag }} \
|
||||
--set environment=production \
|
||||
--set ingress.host=agent-monitor.example.com \
|
||||
--values ${{ env.HELM_CHART_PATH }}/values-production.yaml \
|
||||
--wait \
|
||||
--atomic \
|
||||
--timeout 600s
|
||||
|
||||
- name: Health check
|
||||
run: |
|
||||
echo "Waiting for pods to be ready..."
|
||||
kubectl wait --for=condition=ready pod \
|
||||
-l app.kubernetes.io/name=agent-monitor \
|
||||
-n agent-monitor-production \
|
||||
--timeout=300s
|
||||
|
||||
kubectl port-forward svc/agent-monitor 14820:4820 -n agent-monitor-production &
|
||||
PF_PID=$!
|
||||
sleep 5
|
||||
|
||||
for i in $(seq 1 15); do
|
||||
if curl -sf http://localhost:14820/api/health | grep -q '"status":"ok"'; then
|
||||
echo "✔ Production health check passed"
|
||||
kill $PF_PID 2>/dev/null || true
|
||||
exit 0
|
||||
fi
|
||||
echo "Attempt $i/15..."
|
||||
sleep 5
|
||||
done
|
||||
|
||||
kill $PF_PID 2>/dev/null || true
|
||||
echo "✖ Production health check failed!"
|
||||
exit 1
|
||||
|
||||
- name: Notify Slack (production)
|
||||
if: always()
|
||||
uses: slackapi/slack-github-action@v1.26.0
|
||||
with:
|
||||
payload: |
|
||||
{
|
||||
"text": "${{ job.status == 'success' && '🚀' || '🚨' }} Production deployment ${{ job.status }}: `${{ needs.prepare.outputs.image-tag }}`",
|
||||
"blocks": [
|
||||
{
|
||||
"type": "section",
|
||||
"text": {
|
||||
"type": "mrkdwn",
|
||||
"text": "${{ job.status == 'success' && ':rocket:' || ':rotating_light:' }} *Production Deployment ${{ job.status }}*\n*Image:* `${{ needs.prepare.outputs.image-tag }}`\n*Version:* `${{ needs.prepare.outputs.version }}`\n*Commit:* `${{ github.sha }}`\n*Actor:* ${{ github.actor }}\n*Workflow:* <${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}|View Run>"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
env:
|
||||
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
|
||||
SLACK_WEBHOOK_TYPE: INCOMING_WEBHOOK
|
||||
@@ -0,0 +1,160 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# GitHub Actions Rollback Pipeline – Claude Code Agent Monitor
|
||||
#
|
||||
# Manual workflow to roll back a Helm deployment to a previous revision.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
name: Rollback
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
environment:
|
||||
description: "Target environment to rollback"
|
||||
required: true
|
||||
type: choice
|
||||
options:
|
||||
- staging
|
||||
- production
|
||||
revision:
|
||||
description: "Helm revision number (leave empty for previous)"
|
||||
required: false
|
||||
type: string
|
||||
reason:
|
||||
description: "Reason for rollback"
|
||||
required: true
|
||||
type: string
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
|
||||
concurrency:
|
||||
group: deploy-${{ github.event.inputs.environment }}
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
rollback:
|
||||
name: Rollback ${{ github.event.inputs.environment }}
|
||||
runs-on: ubuntu-latest
|
||||
environment:
|
||||
name: ${{ github.event.inputs.environment }}
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Configure AWS credentials (OIDC)
|
||||
uses: aws-actions/configure-aws-credentials@v4
|
||||
with:
|
||||
role-to-assume: ${{ github.event.inputs.environment == 'production' && secrets.AWS_ROLE_ARN_PRODUCTION || secrets.AWS_ROLE_ARN_STAGING }}
|
||||
aws-region: ${{ vars.AWS_REGION || 'us-west-2' }}
|
||||
|
||||
- name: Setup kubectl
|
||||
uses: azure/setup-kubectl@v3
|
||||
with:
|
||||
version: "v1.29.0"
|
||||
|
||||
- name: Setup Helm
|
||||
uses: azure/setup-helm@v4
|
||||
with:
|
||||
version: "v3.14.0"
|
||||
|
||||
- name: Update kubeconfig
|
||||
run: |
|
||||
CLUSTER_NAME="${{ github.event.inputs.environment == 'production' && vars.EKS_CLUSTER_PRODUCTION || vars.EKS_CLUSTER_STAGING }}"
|
||||
CLUSTER_NAME="${CLUSTER_NAME:-agent-monitor-${{ github.event.inputs.environment }}}"
|
||||
aws eks update-kubeconfig \
|
||||
--region ${{ vars.AWS_REGION || 'us-west-2' }} \
|
||||
--name "${CLUSTER_NAME}"
|
||||
|
||||
- name: Show Helm history
|
||||
run: |
|
||||
NAMESPACE="agent-monitor-${{ github.event.inputs.environment }}"
|
||||
echo "## Current Helm History"
|
||||
helm history agent-monitor -n "${NAMESPACE}" --max 10 || echo "No history found"
|
||||
|
||||
- name: Execute rollback
|
||||
run: |
|
||||
NAMESPACE="agent-monitor-${{ github.event.inputs.environment }}"
|
||||
REVISION="${{ github.event.inputs.revision }}"
|
||||
|
||||
echo "Rolling back in namespace: ${NAMESPACE}"
|
||||
|
||||
ROLLBACK_ARGS="helm rollback agent-monitor"
|
||||
if [[ -n "${REVISION}" ]]; then
|
||||
ROLLBACK_ARGS="${ROLLBACK_ARGS} ${REVISION}"
|
||||
echo "Target revision: ${REVISION}"
|
||||
else
|
||||
echo "Target revision: previous"
|
||||
fi
|
||||
|
||||
${ROLLBACK_ARGS} -n "${NAMESPACE}" --wait --timeout 300s
|
||||
|
||||
echo "✔ Rollback command succeeded"
|
||||
|
||||
- name: Health check after rollback
|
||||
run: |
|
||||
NAMESPACE="agent-monitor-${{ github.event.inputs.environment }}"
|
||||
|
||||
echo "Waiting for pods to be ready..."
|
||||
kubectl wait --for=condition=ready pod \
|
||||
-l app.kubernetes.io/name=agent-monitor \
|
||||
-n "${NAMESPACE}" \
|
||||
--timeout=300s
|
||||
|
||||
kubectl port-forward svc/agent-monitor 14820:4820 -n "${NAMESPACE}" &
|
||||
PF_PID=$!
|
||||
sleep 5
|
||||
|
||||
HEALTHY=false
|
||||
for i in $(seq 1 10); do
|
||||
if curl -sf http://localhost:14820/api/health | grep -q '"status":"ok"'; then
|
||||
echo "✔ Health check passed after rollback"
|
||||
HEALTHY=true
|
||||
break
|
||||
fi
|
||||
echo "Attempt $i/10..."
|
||||
sleep 5
|
||||
done
|
||||
|
||||
kill $PF_PID 2>/dev/null || true
|
||||
|
||||
if [[ "$HEALTHY" != true ]]; then
|
||||
echo "✖ Health check failed after rollback!"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Show post-rollback status
|
||||
if: always()
|
||||
run: |
|
||||
NAMESPACE="agent-monitor-${{ github.event.inputs.environment }}"
|
||||
echo "## Post-Rollback Status"
|
||||
echo ""
|
||||
echo "### Helm Status"
|
||||
helm status agent-monitor -n "${NAMESPACE}" || true
|
||||
echo ""
|
||||
echo "### Pod Status"
|
||||
kubectl get pods -n "${NAMESPACE}" -l app.kubernetes.io/name=agent-monitor || true
|
||||
echo ""
|
||||
echo "### Recent Events"
|
||||
kubectl get events -n "${NAMESPACE}" --sort-by='.lastTimestamp' | tail -20 || true
|
||||
|
||||
- name: Notify Slack
|
||||
if: always()
|
||||
uses: slackapi/slack-github-action@v1.26.0
|
||||
with:
|
||||
payload: |
|
||||
{
|
||||
"text": "${{ job.status == 'success' && '⏪' || '🚨' }} Rollback ${{ job.status }} on ${{ github.event.inputs.environment }}",
|
||||
"blocks": [
|
||||
{
|
||||
"type": "section",
|
||||
"text": {
|
||||
"type": "mrkdwn",
|
||||
"text": "${{ job.status == 'success' && ':rewind:' || ':rotating_light:' }} *Rollback ${{ job.status }}*\n*Environment:* `${{ github.event.inputs.environment }}`\n*Revision:* `${{ github.event.inputs.revision || 'previous' }}`\n*Reason:* ${{ github.event.inputs.reason }}\n*Actor:* ${{ github.actor }}\n*Workflow:* <${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}|View Run>"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
env:
|
||||
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
|
||||
SLACK_WEBHOOK_TYPE: INCOMING_WEBHOOK
|
||||
@@ -0,0 +1,323 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# GitLab CI/CD Pipeline – Claude Code Agent Monitor
|
||||
#
|
||||
# Stages: test → build → deploy-staging → deploy-production
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
# ── Global settings ─────────────────────────────────────────────────────────
|
||||
default:
|
||||
image: node:22-alpine
|
||||
interruptible: true
|
||||
retry:
|
||||
max: 1
|
||||
when:
|
||||
- runner_system_failure
|
||||
- stuck_or_timeout_failure
|
||||
|
||||
variables:
|
||||
NODE_VERSION: "22"
|
||||
REGISTRY: "${CI_REGISTRY}"
|
||||
IMAGE_NAME: "${CI_REGISTRY_IMAGE}/agent-monitor"
|
||||
MCP_IMAGE_NAME: "${CI_REGISTRY_IMAGE}/agent-monitor-mcp"
|
||||
HELM_CHART_PATH: "deployments/helm/agent-monitor"
|
||||
APP_NAME: "agent-monitor"
|
||||
# Kaniko cache
|
||||
KANIKO_CACHE_ARGS: "--cache=true --cache-repo=${CI_REGISTRY_IMAGE}/cache"
|
||||
|
||||
stages:
|
||||
- test
|
||||
- build
|
||||
- deploy-staging
|
||||
- deploy-production
|
||||
- rollback
|
||||
|
||||
# ── Cache configuration ────────────────────────────────────────────────────
|
||||
.node_cache: &node_cache
|
||||
cache:
|
||||
key:
|
||||
files:
|
||||
- package-lock.json
|
||||
paths:
|
||||
- node_modules/
|
||||
policy: pull-push
|
||||
|
||||
# ── Test stage ──────────────────────────────────────────────────────────────
|
||||
lint:
|
||||
stage: test
|
||||
<<: *node_cache
|
||||
script:
|
||||
- npm ci --prefer-offline
|
||||
- npm run format:check
|
||||
rules:
|
||||
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
|
||||
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
|
||||
|
||||
test:server:
|
||||
stage: test
|
||||
<<: *node_cache
|
||||
script:
|
||||
- npm ci --prefer-offline
|
||||
- npm run test:server
|
||||
rules:
|
||||
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
|
||||
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
|
||||
artifacts:
|
||||
when: on_failure
|
||||
paths:
|
||||
- server/__tests__/
|
||||
expire_in: 7 days
|
||||
|
||||
test:client:
|
||||
stage: test
|
||||
<<: *node_cache
|
||||
script:
|
||||
- npm ci --prefer-offline
|
||||
- cd client && npm ci --prefer-offline
|
||||
- npm run test:client
|
||||
cache:
|
||||
key:
|
||||
files:
|
||||
- client/package-lock.json
|
||||
paths:
|
||||
- client/node_modules/
|
||||
policy: pull-push
|
||||
rules:
|
||||
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
|
||||
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
|
||||
|
||||
test:mcp:
|
||||
stage: test
|
||||
<<: *node_cache
|
||||
script:
|
||||
- npm run mcp:install
|
||||
- npm run mcp:typecheck
|
||||
- npm run mcp:test
|
||||
rules:
|
||||
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
|
||||
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
|
||||
|
||||
# ── Build stage ─────────────────────────────────────────────────────────────
|
||||
.kaniko_build: &kaniko_build
|
||||
stage: build
|
||||
image:
|
||||
name: gcr.io/kaniko-project/executor:v1.22.0-debug
|
||||
entrypoint: [""]
|
||||
before_script:
|
||||
- mkdir -p /kaniko/.docker
|
||||
- echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64)\"}}}" > /kaniko/.docker/config.json
|
||||
|
||||
build:app:
|
||||
<<: *kaniko_build
|
||||
script:
|
||||
- >-
|
||||
/kaniko/executor
|
||||
--context "${CI_PROJECT_DIR}"
|
||||
--dockerfile "${CI_PROJECT_DIR}/Dockerfile"
|
||||
--destination "${IMAGE_NAME}:${CI_COMMIT_SHORT_SHA}"
|
||||
--destination "${IMAGE_NAME}:${CI_COMMIT_REF_SLUG}"
|
||||
--destination "${IMAGE_NAME}:latest"
|
||||
${KANIKO_CACHE_ARGS}
|
||||
--label "org.opencontainers.image.revision=${CI_COMMIT_SHA}"
|
||||
--label "org.opencontainers.image.created=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||
--label "org.opencontainers.image.source=${CI_PROJECT_URL}"
|
||||
rules:
|
||||
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
|
||||
- if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/
|
||||
|
||||
build:mcp:
|
||||
<<: *kaniko_build
|
||||
script:
|
||||
- >-
|
||||
/kaniko/executor
|
||||
--context "${CI_PROJECT_DIR}"
|
||||
--dockerfile "${CI_PROJECT_DIR}/mcp/Dockerfile"
|
||||
--destination "${MCP_IMAGE_NAME}:${CI_COMMIT_SHORT_SHA}"
|
||||
--destination "${MCP_IMAGE_NAME}:${CI_COMMIT_REF_SLUG}"
|
||||
--destination "${MCP_IMAGE_NAME}:latest"
|
||||
${KANIKO_CACHE_ARGS}
|
||||
rules:
|
||||
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
|
||||
- if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/
|
||||
|
||||
# Security scan
|
||||
security:scan:
|
||||
stage: build
|
||||
needs: ["build:app", "build:mcp"]
|
||||
image:
|
||||
name: aquasec/trivy:latest
|
||||
entrypoint: [""]
|
||||
script:
|
||||
- trivy image --exit-code 1 --severity HIGH,CRITICAL --format table "${IMAGE_NAME}:${CI_COMMIT_SHORT_SHA}"
|
||||
- trivy image --exit-code 1 --severity HIGH,CRITICAL --format table "${MCP_IMAGE_NAME}:${CI_COMMIT_SHORT_SHA}"
|
||||
- trivy image --exit-code 1 --severity CRITICAL --format json --output trivy-app-report.json "${IMAGE_NAME}:${CI_COMMIT_SHORT_SHA}"
|
||||
- trivy image --exit-code 1 --severity CRITICAL --format json --output trivy-mcp-report.json "${MCP_IMAGE_NAME}:${CI_COMMIT_SHORT_SHA}"
|
||||
artifacts:
|
||||
paths:
|
||||
- trivy-app-report.json
|
||||
- trivy-mcp-report.json
|
||||
expire_in: 30 days
|
||||
allow_failure: false
|
||||
rules:
|
||||
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
|
||||
- if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/
|
||||
|
||||
# ── Deploy Staging ──────────────────────────────────────────────────────────
|
||||
deploy:staging:
|
||||
stage: deploy-staging
|
||||
image:
|
||||
name: alpine/helm:3.14.0
|
||||
entrypoint: [""]
|
||||
needs:
|
||||
- build:app
|
||||
- build:mcp
|
||||
environment:
|
||||
name: staging
|
||||
url: https://staging.agent-monitor.example.com
|
||||
on_stop: stop:staging
|
||||
before_script:
|
||||
- apk add --no-cache curl aws-cli kubectl
|
||||
- aws eks update-kubeconfig --region "${AWS_REGION:-us-west-2}" --name "${EKS_CLUSTER_STAGING:-agent-monitor-staging}"
|
||||
script:
|
||||
- |
|
||||
helm upgrade --install ${APP_NAME} ${HELM_CHART_PATH} \
|
||||
--namespace agent-monitor-staging \
|
||||
--create-namespace \
|
||||
--set image.repository=${IMAGE_NAME} \
|
||||
--set image.tag=${CI_COMMIT_SHORT_SHA} \
|
||||
--set mcp.image.repository=${MCP_IMAGE_NAME} \
|
||||
--set mcp.image.tag=${CI_COMMIT_SHORT_SHA} \
|
||||
--set environment=staging \
|
||||
--values ${HELM_CHART_PATH}/values-staging.yaml \
|
||||
--wait \
|
||||
--atomic \
|
||||
--timeout 600s
|
||||
- |
|
||||
echo "Running health check..."
|
||||
kubectl wait --for=condition=ready pod \
|
||||
-l app.kubernetes.io/name=${APP_NAME} \
|
||||
-n agent-monitor-staging \
|
||||
--timeout=300s
|
||||
echo "✔ Staging deployment successful"
|
||||
rules:
|
||||
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
|
||||
- if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/
|
||||
|
||||
stop:staging:
|
||||
stage: deploy-staging
|
||||
image:
|
||||
name: alpine/helm:3.14.0
|
||||
entrypoint: [""]
|
||||
environment:
|
||||
name: staging
|
||||
action: stop
|
||||
before_script:
|
||||
- apk add --no-cache aws-cli kubectl
|
||||
- aws eks update-kubeconfig --region "${AWS_REGION:-us-west-2}" --name "${EKS_CLUSTER_STAGING:-agent-monitor-staging}"
|
||||
script:
|
||||
- helm uninstall ${APP_NAME} -n agent-monitor-staging --wait || true
|
||||
when: manual
|
||||
rules:
|
||||
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
|
||||
|
||||
# ── Deploy Production ───────────────────────────────────────────────────────
|
||||
deploy:production:
|
||||
stage: deploy-production
|
||||
image:
|
||||
name: alpine/helm:3.14.0
|
||||
entrypoint: [""]
|
||||
needs:
|
||||
- deploy:staging
|
||||
environment:
|
||||
name: production
|
||||
url: https://agent-monitor.example.com
|
||||
before_script:
|
||||
- apk add --no-cache curl aws-cli kubectl
|
||||
- aws eks update-kubeconfig --region "${AWS_REGION:-us-west-2}" --name "${EKS_CLUSTER_PRODUCTION:-agent-monitor-production}"
|
||||
script:
|
||||
# Pre-deploy backup
|
||||
- |
|
||||
POD=$(kubectl get pods -n agent-monitor-production \
|
||||
-l app.kubernetes.io/name=${APP_NAME} \
|
||||
--field-selector=status.phase=Running \
|
||||
-o jsonpath='{.items[0].metadata.name}' 2>/dev/null || echo "")
|
||||
if [ -n "$POD" ]; then
|
||||
echo "Creating pre-deploy backup..."
|
||||
kubectl exec "$POD" -n agent-monitor-production -- \
|
||||
sh -c "cp /app/data/dashboard.db /tmp/pre-deploy-backup.db" 2>/dev/null || true
|
||||
fi
|
||||
# Deploy
|
||||
- |
|
||||
helm upgrade --install ${APP_NAME} ${HELM_CHART_PATH} \
|
||||
--namespace agent-monitor-production \
|
||||
--create-namespace \
|
||||
--set image.repository=${IMAGE_NAME} \
|
||||
--set image.tag=${CI_COMMIT_SHORT_SHA} \
|
||||
--set mcp.image.repository=${MCP_IMAGE_NAME} \
|
||||
--set mcp.image.tag=${CI_COMMIT_SHORT_SHA} \
|
||||
--set environment=production \
|
||||
--values ${HELM_CHART_PATH}/values-production.yaml \
|
||||
--wait \
|
||||
--atomic \
|
||||
--timeout 600s
|
||||
# Health check
|
||||
- |
|
||||
echo "Running production health check..."
|
||||
kubectl wait --for=condition=ready pod \
|
||||
-l app.kubernetes.io/name=${APP_NAME} \
|
||||
-n agent-monitor-production \
|
||||
--timeout=300s
|
||||
echo "✔ Production deployment successful"
|
||||
when: manual
|
||||
allow_failure: false
|
||||
rules:
|
||||
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
|
||||
- if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/
|
||||
|
||||
# ── Rollback ────────────────────────────────────────────────────────────────
|
||||
rollback:staging:
|
||||
stage: rollback
|
||||
image:
|
||||
name: alpine/helm:3.14.0
|
||||
entrypoint: [""]
|
||||
environment:
|
||||
name: staging
|
||||
before_script:
|
||||
- apk add --no-cache aws-cli kubectl
|
||||
- aws eks update-kubeconfig --region "${AWS_REGION:-us-west-2}" --name "${EKS_CLUSTER_STAGING:-agent-monitor-staging}"
|
||||
script:
|
||||
- echo "Rolling back staging..."
|
||||
- helm rollback ${APP_NAME} ${ROLLBACK_REVISION:-0} -n agent-monitor-staging --wait --timeout 300s
|
||||
- |
|
||||
kubectl wait --for=condition=ready pod \
|
||||
-l app.kubernetes.io/name=${APP_NAME} \
|
||||
-n agent-monitor-staging \
|
||||
--timeout=300s
|
||||
- echo "✔ Staging rollback complete"
|
||||
when: manual
|
||||
rules:
|
||||
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
|
||||
|
||||
rollback:production:
|
||||
stage: rollback
|
||||
image:
|
||||
name: alpine/helm:3.14.0
|
||||
entrypoint: [""]
|
||||
environment:
|
||||
name: production
|
||||
before_script:
|
||||
- apk add --no-cache aws-cli kubectl
|
||||
- aws eks update-kubeconfig --region "${AWS_REGION:-us-west-2}" --name "${EKS_CLUSTER_PRODUCTION:-agent-monitor-production}"
|
||||
script:
|
||||
- echo "⚠ Rolling back PRODUCTION..."
|
||||
- helm history ${APP_NAME} -n agent-monitor-production --max 5
|
||||
- helm rollback ${APP_NAME} ${ROLLBACK_REVISION:-0} -n agent-monitor-production --wait --timeout 300s
|
||||
- |
|
||||
kubectl wait --for=condition=ready pod \
|
||||
-l app.kubernetes.io/name=${APP_NAME} \
|
||||
-n agent-monitor-production \
|
||||
--timeout=300s
|
||||
- echo "✔ Production rollback complete"
|
||||
when: manual
|
||||
allow_failure: false
|
||||
rules:
|
||||
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
|
||||
@@ -0,0 +1,18 @@
|
||||
# Patterns to ignore when building packages.
|
||||
.DS_Store
|
||||
.git/
|
||||
.gitignore
|
||||
.bzr/
|
||||
.bzrignore
|
||||
.hg/
|
||||
.hgignore
|
||||
.svn/
|
||||
*.swp
|
||||
*.bak
|
||||
*.tmp
|
||||
*.orig
|
||||
*~
|
||||
.project
|
||||
.idea/
|
||||
*.tmproj
|
||||
.vscode/
|
||||
@@ -0,0 +1,28 @@
|
||||
apiVersion: v2
|
||||
name: agent-monitor
|
||||
description: Claude Code Agent Monitor - Real-time dashboard for tracking Claude Code agent activity
|
||||
type: application
|
||||
version: 1.0.0
|
||||
appVersion: "1.0.0"
|
||||
|
||||
home: https://git.smartgift.io.vn/Smartgift-AI/Claude-Code-Monitor
|
||||
sources:
|
||||
- https://git.smartgift.io.vn/Smartgift-AI/Claude-Code-Monitor
|
||||
|
||||
keywords:
|
||||
- claude
|
||||
- agent
|
||||
- monitor
|
||||
- dashboard
|
||||
- ai
|
||||
- devtools
|
||||
- observability
|
||||
|
||||
maintainers:
|
||||
- name: David Nguyen
|
||||
url: https://git.smartgift.io.vn/Smartgift-AI
|
||||
|
||||
icon: https://git.smartgift.io.vn/Smartgift-AI/Claude-Code-Monitor/raw/branch/master/favicon.svg
|
||||
|
||||
annotations:
|
||||
artifacthub.io/category: monitoring-logging
|
||||
@@ -0,0 +1,79 @@
|
||||
╔══════════════════════════════════════════════════════════════════════╗
|
||||
║ Claude Code Agent Monitor - Deployment Notes ║
|
||||
╚══════════════════════════════════════════════════════════════════════╝
|
||||
|
||||
{{- $fullName := include "agent-monitor.fullname" . -}}
|
||||
|
||||
🎉 {{ $fullName }} has been deployed to namespace "{{ .Release.Namespace }}"!
|
||||
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
📡 Accessing the Dashboard:
|
||||
{{- if .Values.ingress.enabled }}
|
||||
{{- range $host := .Values.ingress.hosts }}
|
||||
{{- range .paths }}
|
||||
|
||||
URL: http{{ if $.Values.ingress.tls }}s{{ end }}://{{ $host.host }}{{ .path }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- else if contains "NodePort" .Values.service.type }}
|
||||
|
||||
export NODE_PORT=$(kubectl get -n {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ $fullName }})
|
||||
export NODE_IP=$(kubectl get nodes -o jsonpath="{.items[0].status.addresses[0].address}")
|
||||
echo "Dashboard URL: http://$NODE_IP:$NODE_PORT"
|
||||
|
||||
{{- else if contains "LoadBalancer" .Values.service.type }}
|
||||
|
||||
NOTE: It may take a few minutes for the LoadBalancer IP to be available.
|
||||
Watch status with:
|
||||
kubectl get -n {{ .Release.Namespace }} svc {{ $fullName }} -w
|
||||
|
||||
export SERVICE_IP=$(kubectl get svc -n {{ .Release.Namespace }} {{ $fullName }} -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
|
||||
echo "Dashboard URL: http://$SERVICE_IP:{{ .Values.service.port }}"
|
||||
|
||||
{{- else }}
|
||||
|
||||
Port-forward to access locally:
|
||||
kubectl port-forward -n {{ .Release.Namespace }} svc/{{ $fullName }} 4820:{{ .Values.service.port }}
|
||||
|
||||
Then open: http://localhost:4820
|
||||
|
||||
{{- end }}
|
||||
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
|
||||
🔍 Health Check:
|
||||
kubectl exec -n {{ .Release.Namespace }} deploy/{{ $fullName }} -- wget -qO- http://localhost:{{ .Values.service.targetPort }}/api/health
|
||||
|
||||
🧪 Run Tests:
|
||||
helm test {{ .Release.Name }} -n {{ .Release.Namespace }}
|
||||
|
||||
📊 View Logs:
|
||||
kubectl logs -n {{ .Release.Namespace }} -l app.kubernetes.io/name={{ include "agent-monitor.name" . }} -f
|
||||
|
||||
📈 Check Pods:
|
||||
kubectl get pods -n {{ .Release.Namespace }} -l app.kubernetes.io/name={{ include "agent-monitor.name" . }}
|
||||
|
||||
{{- if .Values.persistence.enabled }}
|
||||
|
||||
💾 Persistent Storage:
|
||||
SQLite data is stored on PVC: {{ $fullName }}-data ({{ .Values.persistence.size }})
|
||||
{{- end }}
|
||||
|
||||
{{- if .Values.mcp.enabled }}
|
||||
|
||||
🔌 MCP Sidecar:
|
||||
MCP server is running on port {{ .Values.mcp.port }}
|
||||
Access via: kubectl port-forward -n {{ .Release.Namespace }} svc/{{ $fullName }} {{ .Values.mcp.port }}:{{ .Values.mcp.port }}
|
||||
{{- end }}
|
||||
|
||||
{{- if .Values.autoscaling.enabled }}
|
||||
|
||||
📐 Autoscaling:
|
||||
Min replicas: {{ .Values.autoscaling.minReplicas }}
|
||||
Max replicas: {{ .Values.autoscaling.maxReplicas }}
|
||||
CPU target: {{ .Values.autoscaling.targetCPUUtilizationPercentage }}%
|
||||
Memory target: {{ .Values.autoscaling.targetMemoryUtilizationPercentage }}%
|
||||
{{- end }}
|
||||
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
@@ -0,0 +1,88 @@
|
||||
{{/*
|
||||
Expand the name of the chart.
|
||||
*/}}
|
||||
{{- define "agent-monitor.name" -}}
|
||||
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Create a default fully qualified app name.
|
||||
We truncate at 63 chars because some Kubernetes name fields are limited to this
|
||||
(by the DNS naming spec). If release name contains the chart name it will be used
|
||||
as a full name.
|
||||
*/}}
|
||||
{{- define "agent-monitor.fullname" -}}
|
||||
{{- if .Values.fullnameOverride }}
|
||||
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
|
||||
{{- else }}
|
||||
{{- $name := default .Chart.Name .Values.nameOverride }}
|
||||
{{- if contains $name .Release.Name }}
|
||||
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
|
||||
{{- else }}
|
||||
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Create chart name and version as used by the chart label.
|
||||
*/}}
|
||||
{{- define "agent-monitor.chart" -}}
|
||||
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Common labels
|
||||
*/}}
|
||||
{{- define "agent-monitor.labels" -}}
|
||||
helm.sh/chart: {{ include "agent-monitor.chart" . }}
|
||||
{{ include "agent-monitor.selectorLabels" . }}
|
||||
{{- if .Chart.AppVersion }}
|
||||
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
|
||||
{{- end }}
|
||||
app.kubernetes.io/managed-by: {{ .Release.Service }}
|
||||
app.kubernetes.io/part-of: claude-code-agent-monitor
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Selector labels
|
||||
*/}}
|
||||
{{- define "agent-monitor.selectorLabels" -}}
|
||||
app.kubernetes.io/name: {{ include "agent-monitor.name" . }}
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Create the name of the service account to use
|
||||
*/}}
|
||||
{{- define "agent-monitor.serviceAccountName" -}}
|
||||
{{- if .Values.serviceAccount.create }}
|
||||
{{- default (include "agent-monitor.fullname" .) .Values.serviceAccount.name }}
|
||||
{{- else }}
|
||||
{{- default "default" .Values.serviceAccount.name }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Return the container image string
|
||||
*/}}
|
||||
{{- define "agent-monitor.image" -}}
|
||||
{{- $tag := default .Chart.AppVersion .Values.image.tag -}}
|
||||
{{- if .Values.image.registry -}}
|
||||
{{- printf "%s/%s:%s" .Values.image.registry .Values.image.repository $tag -}}
|
||||
{{- else -}}
|
||||
{{- printf "%s:%s" .Values.image.repository $tag -}}
|
||||
{{- end -}}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Return the MCP sidecar container image string
|
||||
*/}}
|
||||
{{- define "agent-monitor.mcpImage" -}}
|
||||
{{- $tag := default .Chart.AppVersion .Values.mcp.image.tag -}}
|
||||
{{- if .Values.mcp.image.registry -}}
|
||||
{{- printf "%s/%s:%s" .Values.mcp.image.registry .Values.mcp.image.repository $tag -}}
|
||||
{{- else -}}
|
||||
{{- printf "%s:%s" .Values.mcp.image.repository $tag -}}
|
||||
{{- end -}}
|
||||
{{- end }}
|
||||
@@ -0,0 +1,13 @@
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: {{ include "agent-monitor.fullname" . }}-config
|
||||
labels:
|
||||
{{- include "agent-monitor.labels" . | nindent 4 }}
|
||||
data:
|
||||
{{- range $key, $value := .Values.env }}
|
||||
{{ $key }}: {{ $value | quote }}
|
||||
{{- end }}
|
||||
{{- range $key, $value := .Values.extraEnv }}
|
||||
{{ $key }}: {{ $value | quote }}
|
||||
{{- end }}
|
||||
@@ -0,0 +1,159 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: {{ include "agent-monitor.fullname" . }}
|
||||
labels:
|
||||
{{- include "agent-monitor.labels" . | nindent 4 }}
|
||||
spec:
|
||||
{{- if not .Values.autoscaling.enabled }}
|
||||
replicas: {{ .Values.replicaCount }}
|
||||
{{- end }}
|
||||
revisionHistoryLimit: {{ .Values.deployment.revisionHistoryLimit | default 5 }}
|
||||
{{- with .Values.deployment.strategy }}
|
||||
strategy:
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
selector:
|
||||
matchLabels:
|
||||
{{- include "agent-monitor.selectorLabels" . | nindent 6 }}
|
||||
template:
|
||||
metadata:
|
||||
annotations:
|
||||
checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}
|
||||
{{- with .Values.podAnnotations }}
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
labels:
|
||||
{{- include "agent-monitor.labels" . | nindent 8 }}
|
||||
{{- with .Values.podLabels }}
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
{{- with .Values.imagePullSecrets }}
|
||||
imagePullSecrets:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
serviceAccountName: {{ include "agent-monitor.serviceAccountName" . }}
|
||||
automountServiceAccountToken: false
|
||||
{{- with .Values.podSecurityContext }}
|
||||
securityContext:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
containers:
|
||||
# ── Main application container ────────────────────────────────────
|
||||
- name: {{ .Chart.Name }}
|
||||
{{- with .Values.securityContext }}
|
||||
securityContext:
|
||||
{{- toYaml . | nindent 12 }}
|
||||
{{- end }}
|
||||
image: {{ include "agent-monitor.image" . }}
|
||||
imagePullPolicy: {{ .Values.image.pullPolicy }}
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: {{ .Values.service.targetPort }}
|
||||
protocol: TCP
|
||||
{{- with .Values.livenessProbe }}
|
||||
livenessProbe:
|
||||
{{- toYaml . | nindent 12 }}
|
||||
{{- end }}
|
||||
{{- with .Values.readinessProbe }}
|
||||
readinessProbe:
|
||||
{{- toYaml . | nindent 12 }}
|
||||
{{- end }}
|
||||
{{- with .Values.startupProbe }}
|
||||
startupProbe:
|
||||
{{- toYaml . | nindent 12 }}
|
||||
{{- end }}
|
||||
envFrom:
|
||||
- configMapRef:
|
||||
name: {{ include "agent-monitor.fullname" . }}-config
|
||||
{{- with .Values.extraEnvFrom }}
|
||||
{{- toYaml . | nindent 12 }}
|
||||
{{- end }}
|
||||
{{- with .Values.resources }}
|
||||
resources:
|
||||
{{- toYaml . | nindent 12 }}
|
||||
{{- end }}
|
||||
lifecycle:
|
||||
preStop:
|
||||
exec:
|
||||
# Allow in-flight requests (including WebSockets) to drain before SIGTERM
|
||||
command: ["sh", "-c", "sleep 5"]
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /app/data
|
||||
- name: tmp
|
||||
mountPath: /tmp
|
||||
{{- if .Values.mcp.enabled }}
|
||||
# ── MCP sidecar container ─────────────────────────────────────────
|
||||
- name: mcp
|
||||
{{- with .Values.mcp.securityContext }}
|
||||
securityContext:
|
||||
{{- toYaml . | nindent 12 }}
|
||||
{{- end }}
|
||||
image: {{ include "agent-monitor.mcpImage" . }}
|
||||
imagePullPolicy: {{ .Values.mcp.image.pullPolicy }}
|
||||
ports:
|
||||
- name: mcp
|
||||
containerPort: {{ .Values.mcp.port }}
|
||||
protocol: TCP
|
||||
env:
|
||||
{{- range $key, $value := .Values.mcp.env }}
|
||||
- name: {{ $key }}
|
||||
value: {{ $value | quote }}
|
||||
{{- end }}
|
||||
startupProbe:
|
||||
tcpSocket:
|
||||
port: mcp
|
||||
failureThreshold: 30
|
||||
periodSeconds: 2
|
||||
livenessProbe:
|
||||
tcpSocket:
|
||||
port: mcp
|
||||
initialDelaySeconds: 15
|
||||
periodSeconds: 20
|
||||
timeoutSeconds: 3
|
||||
failureThreshold: 3
|
||||
readinessProbe:
|
||||
tcpSocket:
|
||||
port: mcp
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 10
|
||||
timeoutSeconds: 3
|
||||
failureThreshold: 3
|
||||
{{- with .Values.mcp.resources }}
|
||||
resources:
|
||||
{{- toYaml . | nindent 12 }}
|
||||
{{- end }}
|
||||
volumeMounts:
|
||||
- name: tmp
|
||||
mountPath: /tmp
|
||||
{{- end }}
|
||||
volumes:
|
||||
- name: data
|
||||
{{- if .Values.persistence.enabled }}
|
||||
persistentVolumeClaim:
|
||||
claimName: {{ .Values.persistence.existingClaim | default (printf "%s-data" (include "agent-monitor.fullname" .)) }}
|
||||
{{- else }}
|
||||
emptyDir: {}
|
||||
{{- end }}
|
||||
- name: tmp
|
||||
emptyDir:
|
||||
sizeLimit: 100Mi
|
||||
terminationGracePeriodSeconds: 30
|
||||
{{- with .Values.nodeSelector }}
|
||||
nodeSelector:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- with .Values.affinity }}
|
||||
affinity:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- with .Values.tolerations }}
|
||||
tolerations:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- with .Values.topologySpreadConstraints }}
|
||||
topologySpreadConstraints:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
@@ -0,0 +1,53 @@
|
||||
{{- if .Values.autoscaling.enabled -}}
|
||||
apiVersion: autoscaling/v2
|
||||
kind: HorizontalPodAutoscaler
|
||||
metadata:
|
||||
name: {{ include "agent-monitor.fullname" . }}
|
||||
labels:
|
||||
{{- include "agent-monitor.labels" . | nindent 4 }}
|
||||
spec:
|
||||
scaleTargetRef:
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
name: {{ include "agent-monitor.fullname" . }}
|
||||
minReplicas: {{ .Values.autoscaling.minReplicas }}
|
||||
maxReplicas: {{ .Values.autoscaling.maxReplicas }}
|
||||
metrics:
|
||||
{{- if .Values.autoscaling.targetCPUUtilizationPercentage }}
|
||||
- type: Resource
|
||||
resource:
|
||||
name: cpu
|
||||
target:
|
||||
type: Utilization
|
||||
averageUtilization: {{ .Values.autoscaling.targetCPUUtilizationPercentage }}
|
||||
{{- end }}
|
||||
{{- if .Values.autoscaling.targetMemoryUtilizationPercentage }}
|
||||
- type: Resource
|
||||
resource:
|
||||
name: memory
|
||||
target:
|
||||
type: Utilization
|
||||
averageUtilization: {{ .Values.autoscaling.targetMemoryUtilizationPercentage }}
|
||||
{{- end }}
|
||||
behavior:
|
||||
scaleDown:
|
||||
stabilizationWindowSeconds: {{ .Values.autoscaling.scaleDownStabilizationWindowSeconds | default 300 }}
|
||||
policies:
|
||||
- type: Pods
|
||||
value: 1
|
||||
periodSeconds: 60
|
||||
- type: Percent
|
||||
value: 10
|
||||
periodSeconds: 60
|
||||
selectPolicy: Min
|
||||
scaleUp:
|
||||
stabilizationWindowSeconds: 30
|
||||
policies:
|
||||
- type: Pods
|
||||
value: 2
|
||||
periodSeconds: 60
|
||||
- type: Percent
|
||||
value: 50
|
||||
periodSeconds: 60
|
||||
selectPolicy: Max
|
||||
{{- end }}
|
||||
@@ -0,0 +1,41 @@
|
||||
{{- if .Values.ingress.enabled -}}
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: {{ include "agent-monitor.fullname" . }}
|
||||
labels:
|
||||
{{- include "agent-monitor.labels" . | nindent 4 }}
|
||||
{{- with .Values.ingress.annotations }}
|
||||
annotations:
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
{{- if .Values.ingress.className }}
|
||||
ingressClassName: {{ .Values.ingress.className }}
|
||||
{{- end }}
|
||||
{{- if .Values.ingress.tls }}
|
||||
tls:
|
||||
{{- range .Values.ingress.tls }}
|
||||
- hosts:
|
||||
{{- range .hosts }}
|
||||
- {{ . | quote }}
|
||||
{{- end }}
|
||||
secretName: {{ .secretName }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
rules:
|
||||
{{- range .Values.ingress.hosts }}
|
||||
- host: {{ .host | quote }}
|
||||
http:
|
||||
paths:
|
||||
{{- range .paths }}
|
||||
- path: {{ .path }}
|
||||
pathType: {{ .pathType }}
|
||||
backend:
|
||||
service:
|
||||
name: {{ include "agent-monitor.fullname" $ }}
|
||||
port:
|
||||
name: http
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
@@ -0,0 +1,46 @@
|
||||
{{- if .Values.networkPolicy.enabled -}}
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: {{ include "agent-monitor.fullname" . }}
|
||||
labels:
|
||||
{{- include "agent-monitor.labels" . | nindent 4 }}
|
||||
spec:
|
||||
podSelector:
|
||||
matchLabels:
|
||||
{{- include "agent-monitor.selectorLabels" . | nindent 6 }}
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
ingress:
|
||||
# Allow HTTP traffic to the application port from any pod (ingress controllers, etc.)
|
||||
- ports:
|
||||
- port: {{ .Values.service.targetPort }}
|
||||
protocol: TCP
|
||||
{{- if .Values.mcp.enabled }}
|
||||
# Allow MCP traffic when sidecar is enabled
|
||||
- ports:
|
||||
- port: {{ .Values.mcp.port }}
|
||||
protocol: TCP
|
||||
{{- end }}
|
||||
{{- with .Values.networkPolicy.additionalIngressRules }}
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
egress:
|
||||
# Allow DNS resolution
|
||||
- ports:
|
||||
- port: 53
|
||||
protocol: UDP
|
||||
- port: 53
|
||||
protocol: TCP
|
||||
# Allow outbound HTTPS (for external API calls)
|
||||
- ports:
|
||||
- port: 443
|
||||
protocol: TCP
|
||||
# Allow internal communication within the cluster
|
||||
- to:
|
||||
- podSelector: {}
|
||||
{{- with .Values.networkPolicy.additionalEgressRules }}
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
@@ -0,0 +1,18 @@
|
||||
{{- if .Values.podDisruptionBudget.enabled -}}
|
||||
apiVersion: policy/v1
|
||||
kind: PodDisruptionBudget
|
||||
metadata:
|
||||
name: {{ include "agent-monitor.fullname" . }}
|
||||
labels:
|
||||
{{- include "agent-monitor.labels" . | nindent 4 }}
|
||||
spec:
|
||||
{{- if .Values.podDisruptionBudget.minAvailable }}
|
||||
minAvailable: {{ .Values.podDisruptionBudget.minAvailable }}
|
||||
{{- end }}
|
||||
{{- if .Values.podDisruptionBudget.maxUnavailable }}
|
||||
maxUnavailable: {{ .Values.podDisruptionBudget.maxUnavailable }}
|
||||
{{- end }}
|
||||
selector:
|
||||
matchLabels:
|
||||
{{- include "agent-monitor.selectorLabels" . | nindent 6 }}
|
||||
{{- end }}
|
||||
@@ -0,0 +1,23 @@
|
||||
{{- if and .Values.persistence.enabled (not .Values.persistence.existingClaim) -}}
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: {{ include "agent-monitor.fullname" . }}-data
|
||||
labels:
|
||||
{{- include "agent-monitor.labels" . | nindent 4 }}
|
||||
{{- with .Values.persistence.annotations }}
|
||||
annotations:
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
accessModes:
|
||||
{{- range .Values.persistence.accessModes }}
|
||||
- {{ . }}
|
||||
{{- end }}
|
||||
{{- if .Values.persistence.storageClass }}
|
||||
storageClassName: {{ .Values.persistence.storageClass | quote }}
|
||||
{{- end }}
|
||||
resources:
|
||||
requests:
|
||||
storage: {{ .Values.persistence.size }}
|
||||
{{- end }}
|
||||
@@ -0,0 +1,33 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: {{ include "agent-monitor.fullname" . }}
|
||||
labels:
|
||||
{{- include "agent-monitor.labels" . | nindent 4 }}
|
||||
{{- with .Values.service.annotations }}
|
||||
annotations:
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
type: {{ .Values.service.type }}
|
||||
# Sticky sessions for WebSocket connections
|
||||
sessionAffinity: ClientIP
|
||||
sessionAffinityConfig:
|
||||
clientIP:
|
||||
timeoutSeconds: 10800
|
||||
ports:
|
||||
- name: http
|
||||
port: {{ .Values.service.port }}
|
||||
targetPort: http
|
||||
protocol: TCP
|
||||
{{- if and (eq .Values.service.type "NodePort") .Values.service.nodePort }}
|
||||
nodePort: {{ .Values.service.nodePort }}
|
||||
{{- end }}
|
||||
{{- if .Values.mcp.enabled }}
|
||||
- name: mcp
|
||||
port: {{ .Values.mcp.port }}
|
||||
targetPort: mcp
|
||||
protocol: TCP
|
||||
{{- end }}
|
||||
selector:
|
||||
{{- include "agent-monitor.selectorLabels" . | nindent 4 }}
|
||||
@@ -0,0 +1,13 @@
|
||||
{{- if .Values.serviceAccount.create -}}
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: {{ include "agent-monitor.serviceAccountName" . }}
|
||||
labels:
|
||||
{{- include "agent-monitor.labels" . | nindent 4 }}
|
||||
{{- with .Values.serviceAccount.annotations }}
|
||||
annotations:
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
automountServiceAccountToken: {{ .Values.serviceAccount.automountServiceAccountToken | default false }}
|
||||
{{- end }}
|
||||
@@ -0,0 +1,29 @@
|
||||
{{- if and .Values.monitoring.enabled .Values.monitoring.serviceMonitor.enabled -}}
|
||||
apiVersion: monitoring.coreos.com/v1
|
||||
kind: ServiceMonitor
|
||||
metadata:
|
||||
name: {{ include "agent-monitor.fullname" . }}
|
||||
{{- if .Values.monitoring.serviceMonitor.namespace }}
|
||||
namespace: {{ .Values.monitoring.serviceMonitor.namespace }}
|
||||
{{- end }}
|
||||
labels:
|
||||
{{- include "agent-monitor.labels" . | nindent 4 }}
|
||||
{{- with .Values.monitoring.serviceMonitor.labels }}
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
{{- include "agent-monitor.selectorLabels" . | nindent 6 }}
|
||||
{{- if .Values.monitoring.serviceMonitor.namespace }}
|
||||
namespaceSelector:
|
||||
matchNames:
|
||||
- {{ .Release.Namespace }}
|
||||
{{- end }}
|
||||
endpoints:
|
||||
- port: http
|
||||
path: {{ .Values.monitoring.serviceMonitor.path | default "/api/metrics" }}
|
||||
interval: {{ .Values.monitoring.serviceMonitor.interval | default "30s" }}
|
||||
scrapeTimeout: {{ .Values.monitoring.serviceMonitor.scrapeTimeout | default "10s" }}
|
||||
honorLabels: {{ .Values.monitoring.serviceMonitor.honorLabels | default false }}
|
||||
{{- end }}
|
||||
@@ -0,0 +1,34 @@
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: "{{ include "agent-monitor.fullname" . }}-test-connection"
|
||||
labels:
|
||||
{{- include "agent-monitor.labels" . | nindent 4 }}
|
||||
annotations:
|
||||
"helm.sh/hook": test
|
||||
"helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
|
||||
spec:
|
||||
restartPolicy: Never
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 65534
|
||||
runAsGroup: 65534
|
||||
seccompProfile:
|
||||
type: RuntimeDefault
|
||||
containers:
|
||||
- name: wget
|
||||
image: busybox:1.36
|
||||
command: ['wget']
|
||||
args:
|
||||
- '--timeout=10'
|
||||
- '--tries=3'
|
||||
- '-qO-'
|
||||
- 'http://{{ include "agent-monitor.fullname" . }}:{{ .Values.service.port }}/api/health'
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 65534
|
||||
readOnlyRootFilesystem: true
|
||||
allowPrivilegeEscalation: false
|
||||
capabilities:
|
||||
drop:
|
||||
- ALL
|
||||
@@ -0,0 +1,50 @@
|
||||
# =============================================================================
|
||||
# Development Environment Overrides
|
||||
# =============================================================================
|
||||
# Usage: helm install agent-monitor ./agent-monitor -f values-dev.yaml
|
||||
|
||||
replicaCount: 1
|
||||
|
||||
image:
|
||||
pullPolicy: Always
|
||||
|
||||
env:
|
||||
NODE_ENV: development
|
||||
DASHBOARD_PORT: "4820"
|
||||
LOG_LEVEL: debug
|
||||
|
||||
resources:
|
||||
requests:
|
||||
cpu: 50m
|
||||
memory: 64Mi
|
||||
limits:
|
||||
cpu: 250m
|
||||
memory: 256Mi
|
||||
|
||||
autoscaling:
|
||||
enabled: false
|
||||
|
||||
persistence:
|
||||
enabled: true
|
||||
size: 1Gi
|
||||
|
||||
podDisruptionBudget:
|
||||
enabled: false
|
||||
|
||||
networkPolicy:
|
||||
enabled: false
|
||||
|
||||
monitoring:
|
||||
enabled: false
|
||||
|
||||
ingress:
|
||||
enabled: false
|
||||
|
||||
# Relax security for dev debugging
|
||||
securityContext:
|
||||
readOnlyRootFilesystem: false
|
||||
runAsNonRoot: true
|
||||
allowPrivilegeEscalation: false
|
||||
capabilities:
|
||||
drop:
|
||||
- ALL
|
||||
@@ -0,0 +1,117 @@
|
||||
# =============================================================================
|
||||
# Production Environment Overrides
|
||||
# =============================================================================
|
||||
# Usage: helm install agent-monitor ./agent-monitor -f values-production.yaml
|
||||
|
||||
replicaCount: 3
|
||||
|
||||
env:
|
||||
NODE_ENV: production
|
||||
DASHBOARD_PORT: "4820"
|
||||
LOG_LEVEL: warn
|
||||
|
||||
resources:
|
||||
requests:
|
||||
cpu: 250m
|
||||
memory: 256Mi
|
||||
limits:
|
||||
cpu: "1"
|
||||
memory: "1Gi"
|
||||
|
||||
autoscaling:
|
||||
enabled: true
|
||||
minReplicas: 3
|
||||
maxReplicas: 20
|
||||
targetCPUUtilizationPercentage: 70
|
||||
targetMemoryUtilizationPercentage: 80
|
||||
scaleDownStabilizationWindowSeconds: 600
|
||||
|
||||
deployment:
|
||||
strategy:
|
||||
type: RollingUpdate
|
||||
rollingUpdate:
|
||||
maxSurge: 1
|
||||
maxUnavailable: 0
|
||||
|
||||
persistence:
|
||||
enabled: true
|
||||
size: 20Gi
|
||||
storageClass: gp3
|
||||
|
||||
podDisruptionBudget:
|
||||
enabled: true
|
||||
minAvailable: 2
|
||||
|
||||
networkPolicy:
|
||||
enabled: true
|
||||
|
||||
# -- Spread pods across nodes (required) and zones (preferred) for high availability
|
||||
affinity:
|
||||
podAntiAffinity:
|
||||
requiredDuringSchedulingIgnoredDuringExecution:
|
||||
- labelSelector:
|
||||
matchExpressions:
|
||||
- key: app.kubernetes.io/name
|
||||
operator: In
|
||||
values:
|
||||
- agent-monitor
|
||||
topologyKey: kubernetes.io/hostname
|
||||
preferredDuringSchedulingIgnoredDuringExecution:
|
||||
- weight: 50
|
||||
podAffinityTerm:
|
||||
labelSelector:
|
||||
matchExpressions:
|
||||
- key: app.kubernetes.io/name
|
||||
operator: In
|
||||
values:
|
||||
- agent-monitor
|
||||
topologyKey: topology.kubernetes.io/zone
|
||||
|
||||
topologySpreadConstraints:
|
||||
- maxSkew: 1
|
||||
topologyKey: topology.kubernetes.io/zone
|
||||
whenUnsatisfiable: ScheduleAnyway
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
|
||||
ingress:
|
||||
enabled: true
|
||||
className: nginx
|
||||
annotations:
|
||||
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
|
||||
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
|
||||
nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
|
||||
nginx.ingress.kubernetes.io/upstream-hash-by: "$remote_addr"
|
||||
nginx.ingress.kubernetes.io/configuration-snippet: |
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
nginx.ingress.kubernetes.io/ssl-redirect: "true"
|
||||
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
|
||||
cert-manager.io/cluster-issuer: letsencrypt-prod
|
||||
hosts:
|
||||
- host: agent-monitor.example.com
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
tls:
|
||||
- secretName: agent-monitor-production-tls
|
||||
hosts:
|
||||
- agent-monitor.example.com
|
||||
|
||||
monitoring:
|
||||
enabled: true
|
||||
serviceMonitor:
|
||||
enabled: true
|
||||
interval: 15s
|
||||
scrapeTimeout: 10s
|
||||
|
||||
mcp:
|
||||
enabled: true
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 128Mi
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
@@ -0,0 +1,65 @@
|
||||
# =============================================================================
|
||||
# Staging Environment Overrides
|
||||
# =============================================================================
|
||||
# Usage: helm install agent-monitor ./agent-monitor -f values-staging.yaml
|
||||
|
||||
replicaCount: 2
|
||||
|
||||
env:
|
||||
NODE_ENV: production
|
||||
DASHBOARD_PORT: "4820"
|
||||
LOG_LEVEL: info
|
||||
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 128Mi
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
|
||||
autoscaling:
|
||||
enabled: true
|
||||
minReplicas: 2
|
||||
maxReplicas: 5
|
||||
targetCPUUtilizationPercentage: 75
|
||||
targetMemoryUtilizationPercentage: 80
|
||||
scaleDownStabilizationWindowSeconds: 180
|
||||
|
||||
persistence:
|
||||
enabled: true
|
||||
size: 5Gi
|
||||
|
||||
podDisruptionBudget:
|
||||
enabled: true
|
||||
minAvailable: 1
|
||||
|
||||
networkPolicy:
|
||||
enabled: true
|
||||
|
||||
ingress:
|
||||
enabled: true
|
||||
className: nginx
|
||||
annotations:
|
||||
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
|
||||
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
|
||||
nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
|
||||
nginx.ingress.kubernetes.io/upstream-hash-by: "$remote_addr"
|
||||
nginx.ingress.kubernetes.io/configuration-snippet: |
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
hosts:
|
||||
- host: agent-monitor.staging.internal
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
tls:
|
||||
- secretName: agent-monitor-staging-tls
|
||||
hosts:
|
||||
- agent-monitor.staging.internal
|
||||
|
||||
monitoring:
|
||||
enabled: true
|
||||
serviceMonitor:
|
||||
enabled: true
|
||||
interval: 30s
|
||||
@@ -0,0 +1,312 @@
|
||||
# =============================================================================
|
||||
# Claude Code Agent Monitor - Default Helm Values
|
||||
# =============================================================================
|
||||
# Override these values per environment using values-dev.yaml, values-staging.yaml,
|
||||
# or values-production.yaml.
|
||||
|
||||
# -- Number of pod replicas
|
||||
replicaCount: 2
|
||||
|
||||
# -- Container image configuration
|
||||
image:
|
||||
registry: ghcr.io
|
||||
repository: smartgift/claude-code-monitor
|
||||
tag: "" # Defaults to .Chart.AppVersion if empty
|
||||
pullPolicy: IfNotPresent
|
||||
|
||||
# -- Image pull secrets for private registries
|
||||
imagePullSecrets: []
|
||||
|
||||
# -- Override the release name
|
||||
nameOverride: ""
|
||||
# -- Override the full release name
|
||||
fullnameOverride: ""
|
||||
|
||||
# =============================================================================
|
||||
# Service Account
|
||||
# =============================================================================
|
||||
serviceAccount:
|
||||
# -- Whether to create a ServiceAccount
|
||||
create: true
|
||||
# -- Annotations to add to the ServiceAccount
|
||||
annotations: {}
|
||||
# -- The name of the ServiceAccount (auto-generated if empty)
|
||||
name: ""
|
||||
# -- Automount API credentials
|
||||
automountServiceAccountToken: false
|
||||
|
||||
# =============================================================================
|
||||
# Pod Configuration
|
||||
# =============================================================================
|
||||
|
||||
# -- Annotations to add to pods
|
||||
podAnnotations: {}
|
||||
|
||||
# -- Labels to add to pods
|
||||
podLabels: {}
|
||||
|
||||
# -- Pod-level security context
|
||||
podSecurityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 1000
|
||||
runAsGroup: 1000
|
||||
fsGroup: 1000
|
||||
fsGroupChangePolicy: OnRootMismatch
|
||||
seccompProfile:
|
||||
type: RuntimeDefault
|
||||
|
||||
# -- Container-level security context
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
readOnlyRootFilesystem: true
|
||||
allowPrivilegeEscalation: false
|
||||
capabilities:
|
||||
drop:
|
||||
- ALL
|
||||
|
||||
# =============================================================================
|
||||
# Deployment Strategy
|
||||
# =============================================================================
|
||||
deployment:
|
||||
strategy:
|
||||
type: RollingUpdate
|
||||
rollingUpdate:
|
||||
maxSurge: 1
|
||||
maxUnavailable: 0
|
||||
|
||||
# =============================================================================
|
||||
# Service
|
||||
# =============================================================================
|
||||
service:
|
||||
# -- Service type (ClusterIP, NodePort, LoadBalancer)
|
||||
type: ClusterIP
|
||||
# -- Service port (external)
|
||||
port: 80
|
||||
# -- Container port the application listens on
|
||||
targetPort: 4820
|
||||
# -- Node port (only used when type is NodePort)
|
||||
nodePort: ""
|
||||
# -- Additional service annotations
|
||||
annotations: {}
|
||||
|
||||
# =============================================================================
|
||||
# Ingress
|
||||
# =============================================================================
|
||||
ingress:
|
||||
# -- Enable ingress resource
|
||||
enabled: false
|
||||
# -- Ingress class name (e.g. nginx, traefik, alb)
|
||||
className: ""
|
||||
# -- Ingress annotations
|
||||
annotations: {}
|
||||
# nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
|
||||
# nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
|
||||
# nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
|
||||
# nginx.ingress.kubernetes.io/proxy-set-headers: "Upgrade=$http_upgrade,Connection=upgrade"
|
||||
# -- Ingress host definitions
|
||||
hosts:
|
||||
- host: agent-monitor.local
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
# -- TLS configuration
|
||||
tls: []
|
||||
# - secretName: agent-monitor-tls
|
||||
# hosts:
|
||||
# - agent-monitor.local
|
||||
|
||||
# =============================================================================
|
||||
# Resource Limits
|
||||
# =============================================================================
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 128Mi
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
|
||||
# =============================================================================
|
||||
# Autoscaling (HPA)
|
||||
# =============================================================================
|
||||
autoscaling:
|
||||
# -- Enable Horizontal Pod Autoscaler
|
||||
enabled: true
|
||||
# -- Minimum number of replicas
|
||||
minReplicas: 2
|
||||
# -- Maximum number of replicas
|
||||
maxReplicas: 10
|
||||
# -- Target CPU utilization percentage
|
||||
targetCPUUtilizationPercentage: 70
|
||||
# -- Target memory utilization percentage
|
||||
targetMemoryUtilizationPercentage: 80
|
||||
# -- Scale-down stabilization window (seconds)
|
||||
scaleDownStabilizationWindowSeconds: 300
|
||||
|
||||
# =============================================================================
|
||||
# Persistence (SQLite database)
|
||||
# =============================================================================
|
||||
persistence:
|
||||
# -- Enable persistent storage for SQLite data
|
||||
enabled: true
|
||||
# -- Storage class name (empty string uses default)
|
||||
storageClass: ""
|
||||
# -- Access modes for the PVC
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
# -- Size of the persistent volume
|
||||
size: 10Gi
|
||||
# -- Annotations for the PVC
|
||||
annotations: {}
|
||||
# -- Use an existing PVC instead of creating one
|
||||
existingClaim: ""
|
||||
|
||||
# =============================================================================
|
||||
# Scheduling
|
||||
# =============================================================================
|
||||
|
||||
# -- Node selector for pod placement
|
||||
nodeSelector: {}
|
||||
|
||||
# -- Tolerations for pod placement
|
||||
tolerations: []
|
||||
|
||||
# -- Affinity rules for pod placement
|
||||
affinity: {}
|
||||
|
||||
# -- Topology spread constraints
|
||||
topologySpreadConstraints: []
|
||||
|
||||
# =============================================================================
|
||||
# Environment Variables
|
||||
# =============================================================================
|
||||
env:
|
||||
# -- Node.js environment
|
||||
NODE_ENV: production
|
||||
# -- Port the dashboard server listens on (must match service.targetPort)
|
||||
DASHBOARD_PORT: "4820"
|
||||
# -- Log level (debug, info, warn, error)
|
||||
LOG_LEVEL: info
|
||||
|
||||
# -- Additional environment variables as key-value pairs
|
||||
extraEnv: {}
|
||||
# MY_CUSTOM_VAR: my-value
|
||||
|
||||
# -- Extra environment variables from secrets or configmaps
|
||||
extraEnvFrom: []
|
||||
# - secretRef:
|
||||
# name: my-secret
|
||||
# - configMapRef:
|
||||
# name: my-configmap
|
||||
|
||||
# =============================================================================
|
||||
# Health Probes
|
||||
# =============================================================================
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /api/health
|
||||
port: http
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 15
|
||||
timeoutSeconds: 5
|
||||
failureThreshold: 3
|
||||
successThreshold: 1
|
||||
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /api/health
|
||||
port: http
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 10
|
||||
timeoutSeconds: 3
|
||||
failureThreshold: 3
|
||||
successThreshold: 1
|
||||
|
||||
startupProbe:
|
||||
httpGet:
|
||||
path: /api/health
|
||||
port: http
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 5
|
||||
timeoutSeconds: 3
|
||||
failureThreshold: 30
|
||||
successThreshold: 1
|
||||
|
||||
# =============================================================================
|
||||
# Pod Disruption Budget
|
||||
# =============================================================================
|
||||
podDisruptionBudget:
|
||||
# -- Enable PodDisruptionBudget
|
||||
enabled: true
|
||||
# -- Minimum available pods during voluntary disruptions
|
||||
minAvailable: 1
|
||||
# -- Maximum unavailable pods (alternative to minAvailable)
|
||||
# maxUnavailable: 1
|
||||
|
||||
# =============================================================================
|
||||
# Network Policy
|
||||
# =============================================================================
|
||||
networkPolicy:
|
||||
# -- Enable NetworkPolicy
|
||||
enabled: true
|
||||
# -- Additional ingress rules
|
||||
additionalIngressRules: []
|
||||
# -- Additional egress rules
|
||||
additionalEgressRules: []
|
||||
|
||||
# =============================================================================
|
||||
# MCP Sidecar (Model Context Protocol server)
|
||||
# =============================================================================
|
||||
mcp:
|
||||
# -- Enable MCP sidecar container
|
||||
enabled: false
|
||||
image:
|
||||
registry: ghcr.io
|
||||
repository: smartgift/claude-code-monitor-mcp
|
||||
tag: "" # Defaults to .Chart.AppVersion if empty
|
||||
pullPolicy: IfNotPresent
|
||||
# -- MCP HTTP transport port
|
||||
port: 8819
|
||||
# -- Resource limits for MCP sidecar
|
||||
resources:
|
||||
requests:
|
||||
cpu: 50m
|
||||
memory: 64Mi
|
||||
limits:
|
||||
cpu: 200m
|
||||
memory: 256Mi
|
||||
# -- MCP environment variables
|
||||
env:
|
||||
MCP_TRANSPORT: http
|
||||
MCP_PORT: "8819"
|
||||
DASHBOARD_URL: "http://localhost:4820"
|
||||
# -- Container security context for MCP
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
readOnlyRootFilesystem: true
|
||||
allowPrivilegeEscalation: false
|
||||
capabilities:
|
||||
drop:
|
||||
- ALL
|
||||
|
||||
# =============================================================================
|
||||
# Monitoring
|
||||
# =============================================================================
|
||||
monitoring:
|
||||
# -- Enable Prometheus monitoring
|
||||
enabled: false
|
||||
serviceMonitor:
|
||||
# -- Enable ServiceMonitor resource (requires Prometheus Operator)
|
||||
enabled: false
|
||||
# -- Namespace for the ServiceMonitor (defaults to release namespace)
|
||||
namespace: ""
|
||||
# -- Additional labels for the ServiceMonitor
|
||||
labels: {}
|
||||
# -- Scrape interval
|
||||
interval: 30s
|
||||
# -- Scrape timeout
|
||||
scrapeTimeout: 10s
|
||||
# -- Metric path
|
||||
path: /api/metrics
|
||||
# -- Honor labels from the target
|
||||
honorLabels: false
|
||||
@@ -0,0 +1,58 @@
|
||||
# Kubernetes Manifests
|
||||
|
||||
Production-ready Kubernetes resources using Kustomize for environment management, with optional blue-green and canary deployment strategies.
|
||||
|
||||
## Structure
|
||||
|
||||
```
|
||||
kubernetes/
|
||||
├── base/ # Shared base (all environments inherit from this)
|
||||
│ ├── kustomization.yaml
|
||||
│ ├── namespace.yaml # agent-monitor namespace with Pod Security Standards
|
||||
│ ├── configmap.yaml # Environment configuration
|
||||
│ ├── serviceaccount.yaml # Minimal-privilege service account
|
||||
│ ├── deployment.yaml # Main deployment (2 replicas, 3 health probes)
|
||||
│ ├── service.yaml # ClusterIP with WebSocket sticky sessions
|
||||
│ ├── ingress.yaml # NGINX ingress with TLS + WebSocket headers
|
||||
│ ├── pvc.yaml # 10Gi persistent volume for SQLite
|
||||
│ ├── hpa.yaml # Horizontal Pod Autoscaler (2–10 pods)
|
||||
│ ├── pdb.yaml # Pod Disruption Budget (minAvailable: 1)
|
||||
│ └── networkpolicy.yaml # Ingress restricted to NGINX controller
|
||||
├── overlays/
|
||||
│ ├── dev/ # 1 replica, no HPA, minimal resources
|
||||
│ ├── staging/ # 2 replicas, standard resources
|
||||
│ └── production/ # 3 replicas, HPA 3–20, strict anti-affinity
|
||||
├── strategies/
|
||||
│ ├── blue-green/ # Dual-slot deployment with service switching
|
||||
│ └── canary/ # Progressive rollout with Argo Rollouts analysis
|
||||
└── components/
|
||||
├── mcp-sidecar/ # Adds MCP server container to pods
|
||||
└── monitoring/ # Adds Prometheus ServiceMonitor
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
# Apply an environment
|
||||
kubectl apply -k overlays/dev/
|
||||
kubectl apply -k overlays/staging/
|
||||
kubectl apply -k overlays/production/
|
||||
|
||||
# Add MCP sidecar (edit overlay kustomization.yaml):
|
||||
# components:
|
||||
# - ../../components/mcp-sidecar
|
||||
|
||||
# Blue-green switch
|
||||
kubectl patch svc agent-monitor -n agent-monitor \
|
||||
-p '{"spec":{"selector":{"slot":"green"}}}'
|
||||
```
|
||||
|
||||
## Security
|
||||
|
||||
All manifests enforce:
|
||||
- `runAsNonRoot: true`
|
||||
- `readOnlyRootFilesystem: true`
|
||||
- `drop: [ALL]` capabilities
|
||||
- `seccompProfile: RuntimeDefault`
|
||||
- No service account token auto-mount
|
||||
- NetworkPolicy restricting ingress sources
|
||||
@@ -0,0 +1,15 @@
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: agent-monitor-config
|
||||
namespace: agent-monitor
|
||||
labels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
app.kubernetes.io/version: "1.0.0"
|
||||
app.kubernetes.io/component: config
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
data:
|
||||
NODE_ENV: "production"
|
||||
DASHBOARD_PORT: "4820"
|
||||
LOG_LEVEL: "info"
|
||||
@@ -0,0 +1,147 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: agent-monitor
|
||||
namespace: agent-monitor
|
||||
labels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
app.kubernetes.io/version: "1.0.0"
|
||||
app.kubernetes.io/component: server
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
spec:
|
||||
replicas: 2
|
||||
revisionHistoryLimit: 5
|
||||
strategy:
|
||||
type: RollingUpdate
|
||||
rollingUpdate:
|
||||
maxSurge: 1
|
||||
maxUnavailable: 0
|
||||
selector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
app.kubernetes.io/version: "1.0.0"
|
||||
app.kubernetes.io/component: server
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
annotations:
|
||||
# Force rollout on configmap changes via kustomize hash
|
||||
checksum/config: "placeholder"
|
||||
spec:
|
||||
serviceAccountName: agent-monitor
|
||||
automountServiceAccountToken: false
|
||||
terminationGracePeriodSeconds: 30
|
||||
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 1000
|
||||
runAsGroup: 1000
|
||||
fsGroup: 1000
|
||||
fsGroupChangePolicy: OnRootMismatch
|
||||
seccompProfile:
|
||||
type: RuntimeDefault
|
||||
|
||||
affinity:
|
||||
podAntiAffinity:
|
||||
preferredDuringSchedulingIgnoredDuringExecution:
|
||||
- weight: 100
|
||||
podAffinityTerm:
|
||||
labelSelector:
|
||||
matchExpressions:
|
||||
- key: app.kubernetes.io/name
|
||||
operator: In
|
||||
values:
|
||||
- agent-monitor
|
||||
topologyKey: kubernetes.io/hostname
|
||||
|
||||
topologySpreadConstraints:
|
||||
- maxSkew: 1
|
||||
topologyKey: kubernetes.io/hostname
|
||||
whenUnsatisfiable: ScheduleAnyway
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
|
||||
containers:
|
||||
- name: agent-monitor
|
||||
image: ${IMAGE_REGISTRY}/agent-monitor:${IMAGE_TAG}
|
||||
imagePullPolicy: IfNotPresent
|
||||
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: 4820
|
||||
protocol: TCP
|
||||
|
||||
envFrom:
|
||||
- configMapRef:
|
||||
name: agent-monitor-config
|
||||
|
||||
resources:
|
||||
requests:
|
||||
memory: "128Mi"
|
||||
cpu: "100m"
|
||||
limits:
|
||||
memory: "512Mi"
|
||||
cpu: "500m"
|
||||
|
||||
startupProbe:
|
||||
httpGet:
|
||||
path: /api/health
|
||||
port: http
|
||||
failureThreshold: 30
|
||||
periodSeconds: 2
|
||||
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /api/health
|
||||
port: http
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 5
|
||||
timeoutSeconds: 3
|
||||
successThreshold: 1
|
||||
failureThreshold: 3
|
||||
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /api/health
|
||||
port: http
|
||||
initialDelaySeconds: 15
|
||||
periodSeconds: 15
|
||||
timeoutSeconds: 5
|
||||
successThreshold: 1
|
||||
failureThreshold: 3
|
||||
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 1000
|
||||
runAsGroup: 1000
|
||||
readOnlyRootFilesystem: true
|
||||
allowPrivilegeEscalation: false
|
||||
capabilities:
|
||||
drop:
|
||||
- ALL
|
||||
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /app/data
|
||||
- name: tmp
|
||||
mountPath: /tmp
|
||||
|
||||
lifecycle:
|
||||
preStop:
|
||||
exec:
|
||||
# Allow in-flight requests to drain before SIGTERM
|
||||
command: ["sh", "-c", "sleep 5"]
|
||||
|
||||
volumes:
|
||||
- name: data
|
||||
persistentVolumeClaim:
|
||||
claimName: agent-monitor-data
|
||||
- name: tmp
|
||||
emptyDir:
|
||||
sizeLimit: 100Mi
|
||||
@@ -0,0 +1,49 @@
|
||||
apiVersion: autoscaling/v2
|
||||
kind: HorizontalPodAutoscaler
|
||||
metadata:
|
||||
name: agent-monitor
|
||||
namespace: agent-monitor
|
||||
labels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
app.kubernetes.io/version: "1.0.0"
|
||||
app.kubernetes.io/component: autoscaling
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
spec:
|
||||
scaleTargetRef:
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
name: agent-monitor
|
||||
minReplicas: 2
|
||||
maxReplicas: 10
|
||||
metrics:
|
||||
- type: Resource
|
||||
resource:
|
||||
name: cpu
|
||||
target:
|
||||
type: Utilization
|
||||
averageUtilization: 70
|
||||
- type: Resource
|
||||
resource:
|
||||
name: memory
|
||||
target:
|
||||
type: Utilization
|
||||
averageUtilization: 80
|
||||
behavior:
|
||||
scaleDown:
|
||||
stabilizationWindowSeconds: 300
|
||||
policies:
|
||||
- type: Pods
|
||||
value: 1
|
||||
periodSeconds: 60
|
||||
selectPolicy: Min
|
||||
scaleUp:
|
||||
stabilizationWindowSeconds: 30
|
||||
policies:
|
||||
- type: Pods
|
||||
value: 2
|
||||
periodSeconds: 60
|
||||
- type: Percent
|
||||
value: 50
|
||||
periodSeconds: 60
|
||||
selectPolicy: Max
|
||||
@@ -0,0 +1,51 @@
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: agent-monitor
|
||||
namespace: agent-monitor
|
||||
labels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
app.kubernetes.io/version: "1.0.0"
|
||||
app.kubernetes.io/component: ingress
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
annotations:
|
||||
# NGINX Ingress Controller annotations
|
||||
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
|
||||
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
|
||||
# WebSocket upgrade support
|
||||
nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
|
||||
nginx.ingress.kubernetes.io/upstream-hash-by: "$remote_addr"
|
||||
nginx.ingress.kubernetes.io/configuration-snippet: |
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
# Sticky sessions for WebSocket
|
||||
nginx.ingress.kubernetes.io/affinity: cookie
|
||||
nginx.ingress.kubernetes.io/affinity-mode: persistent
|
||||
nginx.ingress.kubernetes.io/session-cookie-name: agent-monitor-affinity
|
||||
nginx.ingress.kubernetes.io/session-cookie-max-age: "10800"
|
||||
nginx.ingress.kubernetes.io/session-cookie-samesite: Strict
|
||||
nginx.ingress.kubernetes.io/session-cookie-secure: "true"
|
||||
# Security headers
|
||||
nginx.ingress.kubernetes.io/ssl-redirect: "true"
|
||||
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
|
||||
nginx.ingress.kubernetes.io/hsts: "true"
|
||||
nginx.ingress.kubernetes.io/hsts-max-age: "31536000"
|
||||
nginx.ingress.kubernetes.io/hsts-include-subdomains: "true"
|
||||
spec:
|
||||
ingressClassName: nginx
|
||||
tls:
|
||||
- hosts:
|
||||
- agent-monitor.example.com
|
||||
secretName: agent-monitor-tls
|
||||
rules:
|
||||
- host: agent-monitor.example.com
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: agent-monitor
|
||||
port:
|
||||
name: http
|
||||
@@ -0,0 +1,23 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
metadata:
|
||||
name: agent-monitor-base
|
||||
|
||||
commonLabels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
app.kubernetes.io/version: "1.0.0"
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
|
||||
resources:
|
||||
- namespace.yaml
|
||||
- configmap.yaml
|
||||
- serviceaccount.yaml
|
||||
- pvc.yaml
|
||||
- deployment.yaml
|
||||
- service.yaml
|
||||
- ingress.yaml
|
||||
- hpa.yaml
|
||||
- pdb.yaml
|
||||
- networkpolicy.yaml
|
||||
@@ -0,0 +1,17 @@
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: agent-monitor
|
||||
labels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
app.kubernetes.io/version: "1.0.0"
|
||||
app.kubernetes.io/component: namespace
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
# Enable Pod Security Standards (restricted)
|
||||
pod-security.kubernetes.io/enforce: restricted
|
||||
pod-security.kubernetes.io/enforce-version: latest
|
||||
pod-security.kubernetes.io/audit: restricted
|
||||
pod-security.kubernetes.io/audit-version: latest
|
||||
pod-security.kubernetes.io/warn: restricted
|
||||
pod-security.kubernetes.io/warn-version: latest
|
||||
@@ -0,0 +1,53 @@
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: agent-monitor
|
||||
namespace: agent-monitor
|
||||
labels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
app.kubernetes.io/version: "1.0.0"
|
||||
app.kubernetes.io/component: network
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
spec:
|
||||
podSelector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
ingress:
|
||||
# Allow traffic from ingress controller
|
||||
- from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: ingress-nginx
|
||||
- podSelector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: ingress-nginx
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 4820
|
||||
# Allow intra-namespace traffic (pod-to-pod)
|
||||
- from:
|
||||
- podSelector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 4820
|
||||
egress:
|
||||
# Allow DNS resolution (required for service discovery)
|
||||
- ports:
|
||||
- port: 53
|
||||
protocol: UDP
|
||||
- port: 53
|
||||
protocol: TCP
|
||||
# Allow outbound HTTPS (for external API calls if needed)
|
||||
- ports:
|
||||
- port: 443
|
||||
protocol: TCP
|
||||
# Allow internal communication within the namespace (pod-to-pod)
|
||||
- to:
|
||||
- podSelector: {}
|
||||
@@ -0,0 +1,17 @@
|
||||
apiVersion: policy/v1
|
||||
kind: PodDisruptionBudget
|
||||
metadata:
|
||||
name: agent-monitor
|
||||
namespace: agent-monitor
|
||||
labels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
app.kubernetes.io/version: "1.0.0"
|
||||
app.kubernetes.io/component: availability
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
spec:
|
||||
minAvailable: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
@@ -0,0 +1,17 @@
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: agent-monitor-data
|
||||
namespace: agent-monitor
|
||||
labels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
app.kubernetes.io/version: "1.0.0"
|
||||
app.kubernetes.io/component: storage
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 10Gi
|
||||
@@ -0,0 +1,29 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: agent-monitor
|
||||
namespace: agent-monitor
|
||||
labels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
app.kubernetes.io/version: "1.0.0"
|
||||
app.kubernetes.io/component: server
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
annotations:
|
||||
# Document the WebSocket requirement
|
||||
service.kubernetes.io/topology-mode: Auto
|
||||
spec:
|
||||
type: ClusterIP
|
||||
# Sticky sessions for WebSocket support
|
||||
sessionAffinity: ClientIP
|
||||
sessionAffinityConfig:
|
||||
clientIP:
|
||||
timeoutSeconds: 10800
|
||||
selector:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
ports:
|
||||
- name: http
|
||||
port: 80
|
||||
targetPort: http
|
||||
protocol: TCP
|
||||
@@ -0,0 +1,12 @@
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: agent-monitor
|
||||
namespace: agent-monitor
|
||||
labels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
app.kubernetes.io/version: "1.0.0"
|
||||
app.kubernetes.io/component: serviceaccount
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
automountServiceAccountToken: false
|
||||
@@ -0,0 +1,69 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: agent-monitor
|
||||
namespace: agent-monitor
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: mcp-sidecar
|
||||
image: ${IMAGE_REGISTRY}/agent-monitor-mcp:${IMAGE_TAG}
|
||||
imagePullPolicy: IfNotPresent
|
||||
|
||||
ports:
|
||||
- name: mcp
|
||||
containerPort: 8819
|
||||
protocol: TCP
|
||||
|
||||
env:
|
||||
- name: MCP_TRANSPORT
|
||||
value: "http"
|
||||
- name: MCP_DASHBOARD_BASE_URL
|
||||
value: "http://localhost:4820"
|
||||
- name: MCP_PORT
|
||||
value: "8819"
|
||||
|
||||
resources:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "50m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "250m"
|
||||
|
||||
startupProbe:
|
||||
tcpSocket:
|
||||
port: mcp
|
||||
failureThreshold: 30
|
||||
periodSeconds: 2
|
||||
|
||||
readinessProbe:
|
||||
tcpSocket:
|
||||
port: mcp
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 10
|
||||
timeoutSeconds: 3
|
||||
failureThreshold: 3
|
||||
|
||||
livenessProbe:
|
||||
tcpSocket:
|
||||
port: mcp
|
||||
initialDelaySeconds: 10
|
||||
periodSeconds: 30
|
||||
timeoutSeconds: 5
|
||||
failureThreshold: 3
|
||||
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 1000
|
||||
runAsGroup: 1000
|
||||
readOnlyRootFilesystem: true
|
||||
allowPrivilegeEscalation: false
|
||||
capabilities:
|
||||
drop:
|
||||
- ALL
|
||||
|
||||
volumeMounts:
|
||||
- name: tmp
|
||||
mountPath: /tmp
|
||||
@@ -0,0 +1,11 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||
kind: Component
|
||||
|
||||
metadata:
|
||||
name: mcp-sidecar
|
||||
|
||||
patches:
|
||||
- path: deployment-patch.yaml
|
||||
target:
|
||||
kind: Deployment
|
||||
name: agent-monitor
|
||||
@@ -0,0 +1,8 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||
kind: Component
|
||||
|
||||
metadata:
|
||||
name: monitoring
|
||||
|
||||
resources:
|
||||
- servicemonitor.yaml
|
||||
@@ -0,0 +1,31 @@
|
||||
apiVersion: monitoring.coreos.com/v1
|
||||
kind: ServiceMonitor
|
||||
metadata:
|
||||
name: agent-monitor
|
||||
namespace: agent-monitor
|
||||
labels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
app.kubernetes.io/version: "1.0.0"
|
||||
app.kubernetes.io/component: monitoring
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
# Common label for Prometheus Operator discovery
|
||||
release: prometheus
|
||||
spec:
|
||||
selector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
namespaceSelector:
|
||||
matchNames:
|
||||
- agent-monitor
|
||||
endpoints:
|
||||
- port: http
|
||||
path: /api/health
|
||||
interval: 30s
|
||||
scrapeTimeout: 10s
|
||||
honorLabels: true
|
||||
metricRelabelings:
|
||||
- sourceLabels: [__name__]
|
||||
regex: "(http_requests_total|http_request_duration_.*|nodejs_.*|process_.*)"
|
||||
action: keep
|
||||
@@ -0,0 +1,36 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
metadata:
|
||||
name: agent-monitor-dev
|
||||
|
||||
namespace: agent-monitor
|
||||
|
||||
resources:
|
||||
- ../../base
|
||||
|
||||
images:
|
||||
- name: ${IMAGE_REGISTRY}/agent-monitor
|
||||
newName: agent-monitor
|
||||
newTag: dev
|
||||
|
||||
configMapGenerator:
|
||||
- name: agent-monitor-config
|
||||
behavior: merge
|
||||
literals:
|
||||
- NODE_ENV=development
|
||||
- LOG_LEVEL=debug
|
||||
|
||||
patches:
|
||||
# Disable HPA in dev (single replica, no autoscaling needed)
|
||||
- target:
|
||||
kind: HorizontalPodAutoscaler
|
||||
name: agent-monitor
|
||||
patch: |
|
||||
$patch: delete
|
||||
apiVersion: autoscaling/v2
|
||||
kind: HorizontalPodAutoscaler
|
||||
metadata:
|
||||
name: agent-monitor
|
||||
# Deployment overrides for dev
|
||||
- path: patches/deployment-patch.yaml
|
||||
@@ -0,0 +1,22 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: agent-monitor
|
||||
namespace: agent-monitor
|
||||
spec:
|
||||
# Single replica for dev
|
||||
replicas: 1
|
||||
template:
|
||||
spec:
|
||||
# Remove anti-affinity in dev (single node is fine)
|
||||
affinity: null
|
||||
topologySpreadConstraints: []
|
||||
containers:
|
||||
- name: agent-monitor
|
||||
resources:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "50m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "250m"
|
||||
@@ -0,0 +1,26 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
metadata:
|
||||
name: agent-monitor-production
|
||||
|
||||
namespace: agent-monitor
|
||||
|
||||
resources:
|
||||
- ../../base
|
||||
|
||||
patches:
|
||||
- path: patches/deployment-patch.yaml
|
||||
- path: patches/hpa-patch.yaml
|
||||
|
||||
images:
|
||||
- name: ${IMAGE_REGISTRY}/agent-monitor
|
||||
newName: agent-monitor
|
||||
newTag: latest
|
||||
|
||||
configMapGenerator:
|
||||
- name: agent-monitor-config
|
||||
behavior: merge
|
||||
literals:
|
||||
- NODE_ENV=production
|
||||
- LOG_LEVEL=warn
|
||||
@@ -0,0 +1,42 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: agent-monitor
|
||||
namespace: agent-monitor
|
||||
spec:
|
||||
replicas: 3
|
||||
template:
|
||||
spec:
|
||||
# Production: require spreading across nodes
|
||||
affinity:
|
||||
podAntiAffinity:
|
||||
requiredDuringSchedulingIgnoredDuringExecution:
|
||||
- labelSelector:
|
||||
matchExpressions:
|
||||
- key: app.kubernetes.io/name
|
||||
operator: In
|
||||
values:
|
||||
- agent-monitor
|
||||
topologyKey: kubernetes.io/hostname
|
||||
topologySpreadConstraints:
|
||||
- maxSkew: 1
|
||||
topologyKey: kubernetes.io/hostname
|
||||
whenUnsatisfiable: DoNotSchedule
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
- maxSkew: 1
|
||||
topologyKey: topology.kubernetes.io/zone
|
||||
whenUnsatisfiable: ScheduleAnyway
|
||||
labelSelector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
containers:
|
||||
- name: agent-monitor
|
||||
resources:
|
||||
requests:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
limits:
|
||||
memory: "1Gi"
|
||||
cpu: "1000m"
|
||||
@@ -0,0 +1,26 @@
|
||||
apiVersion: autoscaling/v2
|
||||
kind: HorizontalPodAutoscaler
|
||||
metadata:
|
||||
name: agent-monitor
|
||||
namespace: agent-monitor
|
||||
spec:
|
||||
minReplicas: 3
|
||||
maxReplicas: 20
|
||||
behavior:
|
||||
scaleDown:
|
||||
stabilizationWindowSeconds: 600
|
||||
policies:
|
||||
- type: Pods
|
||||
value: 1
|
||||
periodSeconds: 120
|
||||
selectPolicy: Min
|
||||
scaleUp:
|
||||
stabilizationWindowSeconds: 60
|
||||
policies:
|
||||
- type: Pods
|
||||
value: 4
|
||||
periodSeconds: 60
|
||||
- type: Percent
|
||||
value: 100
|
||||
periodSeconds: 60
|
||||
selectPolicy: Max
|
||||
@@ -0,0 +1,25 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
metadata:
|
||||
name: agent-monitor-staging
|
||||
|
||||
namespace: agent-monitor
|
||||
|
||||
resources:
|
||||
- ../../base
|
||||
|
||||
patches:
|
||||
- path: patches/deployment-patch.yaml
|
||||
|
||||
images:
|
||||
- name: ${IMAGE_REGISTRY}/agent-monitor
|
||||
newName: agent-monitor
|
||||
newTag: staging
|
||||
|
||||
configMapGenerator:
|
||||
- name: agent-monitor-config
|
||||
behavior: merge
|
||||
literals:
|
||||
- NODE_ENV=staging
|
||||
- LOG_LEVEL=info
|
||||
@@ -0,0 +1,18 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: agent-monitor
|
||||
namespace: agent-monitor
|
||||
spec:
|
||||
replicas: 2
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: agent-monitor
|
||||
resources:
|
||||
requests:
|
||||
memory: "128Mi"
|
||||
cpu: "100m"
|
||||
limits:
|
||||
memory: "512Mi"
|
||||
cpu: "500m"
|
||||
@@ -0,0 +1,136 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: agent-monitor-blue
|
||||
namespace: agent-monitor
|
||||
labels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
app.kubernetes.io/version: "1.0.0"
|
||||
app.kubernetes.io/component: server
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
slot: blue
|
||||
spec:
|
||||
replicas: 3
|
||||
revisionHistoryLimit: 5
|
||||
strategy:
|
||||
type: RollingUpdate
|
||||
rollingUpdate:
|
||||
maxSurge: 1
|
||||
maxUnavailable: 0
|
||||
selector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
slot: blue
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
app.kubernetes.io/version: "1.0.0"
|
||||
app.kubernetes.io/component: server
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
slot: blue
|
||||
spec:
|
||||
serviceAccountName: agent-monitor
|
||||
automountServiceAccountToken: false
|
||||
terminationGracePeriodSeconds: 30
|
||||
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 1000
|
||||
runAsGroup: 1000
|
||||
fsGroup: 1000
|
||||
fsGroupChangePolicy: OnRootMismatch
|
||||
seccompProfile:
|
||||
type: RuntimeDefault
|
||||
|
||||
affinity:
|
||||
podAntiAffinity:
|
||||
requiredDuringSchedulingIgnoredDuringExecution:
|
||||
- labelSelector:
|
||||
matchExpressions:
|
||||
- key: app.kubernetes.io/name
|
||||
operator: In
|
||||
values:
|
||||
- agent-monitor
|
||||
topologyKey: kubernetes.io/hostname
|
||||
|
||||
containers:
|
||||
- name: agent-monitor
|
||||
image: ${IMAGE_REGISTRY}/agent-monitor:blue
|
||||
imagePullPolicy: IfNotPresent
|
||||
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: 4820
|
||||
protocol: TCP
|
||||
|
||||
envFrom:
|
||||
- configMapRef:
|
||||
name: agent-monitor-config
|
||||
|
||||
resources:
|
||||
requests:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
limits:
|
||||
memory: "1Gi"
|
||||
cpu: "1000m"
|
||||
|
||||
startupProbe:
|
||||
httpGet:
|
||||
path: /api/health
|
||||
port: http
|
||||
failureThreshold: 30
|
||||
periodSeconds: 2
|
||||
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /api/health
|
||||
port: http
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 5
|
||||
timeoutSeconds: 3
|
||||
successThreshold: 1
|
||||
failureThreshold: 3
|
||||
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /api/health
|
||||
port: http
|
||||
initialDelaySeconds: 15
|
||||
periodSeconds: 15
|
||||
timeoutSeconds: 5
|
||||
successThreshold: 1
|
||||
failureThreshold: 3
|
||||
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 1000
|
||||
runAsGroup: 1000
|
||||
readOnlyRootFilesystem: true
|
||||
allowPrivilegeEscalation: false
|
||||
capabilities:
|
||||
drop:
|
||||
- ALL
|
||||
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /app/data
|
||||
- name: tmp
|
||||
mountPath: /tmp
|
||||
|
||||
lifecycle:
|
||||
preStop:
|
||||
exec:
|
||||
command: ["sh", "-c", "sleep 5"]
|
||||
|
||||
volumes:
|
||||
- name: data
|
||||
persistentVolumeClaim:
|
||||
claimName: agent-monitor-data
|
||||
- name: tmp
|
||||
emptyDir:
|
||||
sizeLimit: 100Mi
|
||||
@@ -0,0 +1,136 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: agent-monitor-green
|
||||
namespace: agent-monitor
|
||||
labels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
app.kubernetes.io/version: "1.0.0"
|
||||
app.kubernetes.io/component: server
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
slot: green
|
||||
spec:
|
||||
replicas: 3
|
||||
revisionHistoryLimit: 5
|
||||
strategy:
|
||||
type: RollingUpdate
|
||||
rollingUpdate:
|
||||
maxSurge: 1
|
||||
maxUnavailable: 0
|
||||
selector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
slot: green
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
app.kubernetes.io/version: "1.0.0"
|
||||
app.kubernetes.io/component: server
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
slot: green
|
||||
spec:
|
||||
serviceAccountName: agent-monitor
|
||||
automountServiceAccountToken: false
|
||||
terminationGracePeriodSeconds: 30
|
||||
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 1000
|
||||
runAsGroup: 1000
|
||||
fsGroup: 1000
|
||||
fsGroupChangePolicy: OnRootMismatch
|
||||
seccompProfile:
|
||||
type: RuntimeDefault
|
||||
|
||||
affinity:
|
||||
podAntiAffinity:
|
||||
requiredDuringSchedulingIgnoredDuringExecution:
|
||||
- labelSelector:
|
||||
matchExpressions:
|
||||
- key: app.kubernetes.io/name
|
||||
operator: In
|
||||
values:
|
||||
- agent-monitor
|
||||
topologyKey: kubernetes.io/hostname
|
||||
|
||||
containers:
|
||||
- name: agent-monitor
|
||||
image: ${IMAGE_REGISTRY}/agent-monitor:green
|
||||
imagePullPolicy: IfNotPresent
|
||||
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: 4820
|
||||
protocol: TCP
|
||||
|
||||
envFrom:
|
||||
- configMapRef:
|
||||
name: agent-monitor-config
|
||||
|
||||
resources:
|
||||
requests:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
limits:
|
||||
memory: "1Gi"
|
||||
cpu: "1000m"
|
||||
|
||||
startupProbe:
|
||||
httpGet:
|
||||
path: /api/health
|
||||
port: http
|
||||
failureThreshold: 30
|
||||
periodSeconds: 2
|
||||
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /api/health
|
||||
port: http
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 5
|
||||
timeoutSeconds: 3
|
||||
successThreshold: 1
|
||||
failureThreshold: 3
|
||||
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /api/health
|
||||
port: http
|
||||
initialDelaySeconds: 15
|
||||
periodSeconds: 15
|
||||
timeoutSeconds: 5
|
||||
successThreshold: 1
|
||||
failureThreshold: 3
|
||||
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 1000
|
||||
runAsGroup: 1000
|
||||
readOnlyRootFilesystem: true
|
||||
allowPrivilegeEscalation: false
|
||||
capabilities:
|
||||
drop:
|
||||
- ALL
|
||||
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /app/data
|
||||
- name: tmp
|
||||
mountPath: /tmp
|
||||
|
||||
lifecycle:
|
||||
preStop:
|
||||
exec:
|
||||
command: ["sh", "-c", "sleep 5"]
|
||||
|
||||
volumes:
|
||||
- name: data
|
||||
persistentVolumeClaim:
|
||||
claimName: agent-monitor-data
|
||||
- name: tmp
|
||||
emptyDir:
|
||||
sizeLimit: 100Mi
|
||||
@@ -0,0 +1,32 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: agent-monitor
|
||||
namespace: agent-monitor
|
||||
labels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
app.kubernetes.io/version: "1.0.0"
|
||||
app.kubernetes.io/component: server
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
annotations:
|
||||
# Document which slot is currently active
|
||||
# To switch traffic: kubectl patch svc agent-monitor -n agent-monitor \
|
||||
# -p '{"spec":{"selector":{"slot":"green"}}}'
|
||||
agent-monitor.io/active-slot: blue
|
||||
spec:
|
||||
type: ClusterIP
|
||||
sessionAffinity: ClientIP
|
||||
sessionAffinityConfig:
|
||||
clientIP:
|
||||
timeoutSeconds: 10800
|
||||
selector:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
# Toggle this value between "blue" and "green" to switch traffic
|
||||
slot: blue
|
||||
ports:
|
||||
- name: http
|
||||
port: 80
|
||||
targetPort: http
|
||||
protocol: TCP
|
||||
@@ -0,0 +1,97 @@
|
||||
apiVersion: argoproj.io/v1alpha1
|
||||
kind: AnalysisTemplate
|
||||
metadata:
|
||||
name: agent-monitor-canary-analysis
|
||||
namespace: agent-monitor
|
||||
labels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor
|
||||
app.kubernetes.io/version: "1.0.0"
|
||||
app.kubernetes.io/component: canary-analysis
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
spec:
|
||||
args:
|
||||
- name: service-name
|
||||
value: agent-monitor-canary
|
||||
- name: namespace
|
||||
value: agent-monitor
|
||||
metrics:
|
||||
# Success rate must be above 99%
|
||||
- name: success-rate
|
||||
interval: 60s
|
||||
count: 5
|
||||
successCondition: result[0] >= 0.99
|
||||
failureLimit: 2
|
||||
provider:
|
||||
prometheus:
|
||||
address: http://prometheus.monitoring.svc.cluster.local:9090
|
||||
query: |
|
||||
sum(
|
||||
rate(
|
||||
http_requests_total{
|
||||
namespace="{{args.namespace}}",
|
||||
service="{{args.service-name}}",
|
||||
code!~"5.."
|
||||
}[2m]
|
||||
)
|
||||
)
|
||||
/
|
||||
sum(
|
||||
rate(
|
||||
http_requests_total{
|
||||
namespace="{{args.namespace}}",
|
||||
service="{{args.service-name}}"
|
||||
}[2m]
|
||||
)
|
||||
)
|
||||
|
||||
# P99 latency must be under 500ms
|
||||
- name: p99-latency
|
||||
interval: 60s
|
||||
count: 5
|
||||
successCondition: result[0] < 500
|
||||
failureLimit: 2
|
||||
provider:
|
||||
prometheus:
|
||||
address: http://prometheus.monitoring.svc.cluster.local:9090
|
||||
query: |
|
||||
histogram_quantile(
|
||||
0.99,
|
||||
sum(
|
||||
rate(
|
||||
http_request_duration_milliseconds_bucket{
|
||||
namespace="{{args.namespace}}",
|
||||
service="{{args.service-name}}"
|
||||
}[2m]
|
||||
)
|
||||
) by (le)
|
||||
)
|
||||
|
||||
# Error rate must stay below 1%
|
||||
- name: error-rate
|
||||
interval: 60s
|
||||
count: 5
|
||||
successCondition: result[0] <= 0.01
|
||||
failureLimit: 2
|
||||
provider:
|
||||
prometheus:
|
||||
address: http://prometheus.monitoring.svc.cluster.local:9090
|
||||
query: |
|
||||
sum(
|
||||
rate(
|
||||
http_requests_total{
|
||||
namespace="{{args.namespace}}",
|
||||
service="{{args.service-name}}",
|
||||
code=~"5.."
|
||||
}[2m]
|
||||
)
|
||||
)
|
||||
/
|
||||
sum(
|
||||
rate(
|
||||
http_requests_total{
|
||||
namespace="{{args.namespace}}",
|
||||
service="{{args.service-name}}"
|
||||
}[2m]
|
||||
)
|
||||
)
|
||||
@@ -0,0 +1,124 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: agent-monitor-canary
|
||||
namespace: agent-monitor
|
||||
labels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor-canary
|
||||
app.kubernetes.io/version: "1.0.0"
|
||||
app.kubernetes.io/component: server
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
track: canary
|
||||
spec:
|
||||
replicas: 1
|
||||
revisionHistoryLimit: 5
|
||||
selector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor-canary
|
||||
track: canary
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/instance: agent-monitor-canary
|
||||
app.kubernetes.io/version: "1.0.0"
|
||||
app.kubernetes.io/component: server
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
track: canary
|
||||
annotations:
|
||||
prometheus.io/scrape: "true"
|
||||
prometheus.io/port: "4820"
|
||||
prometheus.io/path: "/api/health"
|
||||
spec:
|
||||
serviceAccountName: agent-monitor
|
||||
automountServiceAccountToken: false
|
||||
terminationGracePeriodSeconds: 30
|
||||
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 1000
|
||||
runAsGroup: 1000
|
||||
fsGroup: 1000
|
||||
fsGroupChangePolicy: OnRootMismatch
|
||||
seccompProfile:
|
||||
type: RuntimeDefault
|
||||
|
||||
containers:
|
||||
- name: agent-monitor
|
||||
image: ${IMAGE_REGISTRY}/agent-monitor:canary
|
||||
imagePullPolicy: Always
|
||||
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: 4820
|
||||
protocol: TCP
|
||||
|
||||
envFrom:
|
||||
- configMapRef:
|
||||
name: agent-monitor-config
|
||||
|
||||
resources:
|
||||
requests:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
limits:
|
||||
memory: "1Gi"
|
||||
cpu: "1000m"
|
||||
|
||||
startupProbe:
|
||||
httpGet:
|
||||
path: /api/health
|
||||
port: http
|
||||
failureThreshold: 30
|
||||
periodSeconds: 2
|
||||
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /api/health
|
||||
port: http
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 5
|
||||
timeoutSeconds: 3
|
||||
successThreshold: 1
|
||||
failureThreshold: 3
|
||||
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /api/health
|
||||
port: http
|
||||
initialDelaySeconds: 15
|
||||
periodSeconds: 15
|
||||
timeoutSeconds: 5
|
||||
successThreshold: 1
|
||||
failureThreshold: 3
|
||||
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 1000
|
||||
runAsGroup: 1000
|
||||
readOnlyRootFilesystem: true
|
||||
allowPrivilegeEscalation: false
|
||||
capabilities:
|
||||
drop:
|
||||
- ALL
|
||||
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /app/data
|
||||
- name: tmp
|
||||
mountPath: /tmp
|
||||
|
||||
lifecycle:
|
||||
preStop:
|
||||
exec:
|
||||
command: ["sh", "-c", "sleep 5"]
|
||||
|
||||
volumes:
|
||||
- name: data
|
||||
persistentVolumeClaim:
|
||||
claimName: agent-monitor-data
|
||||
- name: tmp
|
||||
emptyDir:
|
||||
sizeLimit: 100Mi
|
||||
@@ -0,0 +1,241 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Alertmanager configuration for Claude Code Agent Monitor
|
||||
#
|
||||
# Replace placeholder values (marked with <PLACEHOLDER>) before deploying.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
global:
|
||||
# SMTP defaults for email notifications
|
||||
smtp_smarthost: "<SMTP_HOST>:587"
|
||||
smtp_from: "alertmanager@example.com"
|
||||
smtp_auth_username: "<SMTP_USERNAME>"
|
||||
smtp_auth_password: "<SMTP_PASSWORD>"
|
||||
smtp_require_tls: true
|
||||
|
||||
# Slack API URL (override per-receiver if needed)
|
||||
slack_api_url: "<SLACK_WEBHOOK_URL>"
|
||||
|
||||
# PagerDuty URL
|
||||
pagerduty_url: "https://events.pagerduty.com/v2/enqueue"
|
||||
|
||||
# Global resolve timeout
|
||||
resolve_timeout: 5m
|
||||
|
||||
# ── Templates ───────────────────────────────────────────────────────────────
|
||||
templates:
|
||||
- "/etc/alertmanager/templates/*.tmpl"
|
||||
|
||||
# ── Inhibition rules ───────────────────────────────────────────────────────
|
||||
# Prevent lower-severity alerts from firing when a higher-severity alert
|
||||
# for the same service/alertname is already active.
|
||||
inhibit_rules:
|
||||
# If a critical alert is firing, suppress warning alerts for the same service
|
||||
- source_matchers:
|
||||
- severity = critical
|
||||
target_matchers:
|
||||
- severity = warning
|
||||
equal:
|
||||
- alertname
|
||||
- service
|
||||
- namespace
|
||||
|
||||
# If a critical alert is firing, suppress info alerts for the same service
|
||||
- source_matchers:
|
||||
- severity = critical
|
||||
target_matchers:
|
||||
- severity = info
|
||||
equal:
|
||||
- service
|
||||
- namespace
|
||||
|
||||
# If a warning alert is firing, suppress info alerts for the same service
|
||||
- source_matchers:
|
||||
- severity = warning
|
||||
target_matchers:
|
||||
- severity = info
|
||||
equal:
|
||||
- service
|
||||
- namespace
|
||||
|
||||
# If AgentMonitorDown is firing, suppress all other agent-monitor alerts
|
||||
- source_matchers:
|
||||
- alertname = AgentMonitorDown
|
||||
target_matchers:
|
||||
- service = agent-monitor
|
||||
equal:
|
||||
- namespace
|
||||
|
||||
# ── Route tree ──────────────────────────────────────────────────────────────
|
||||
route:
|
||||
# Default receiver for unmatched alerts
|
||||
receiver: slack-default
|
||||
|
||||
# Group alerts by these labels
|
||||
group_by:
|
||||
- alertname
|
||||
- environment
|
||||
- namespace
|
||||
|
||||
# Wait before sending initial notification (allows grouping)
|
||||
group_wait: 30s
|
||||
|
||||
# Wait before sending updates to an existing group
|
||||
group_interval: 5m
|
||||
|
||||
# Wait before re-sending a resolved notification
|
||||
repeat_interval: 4h
|
||||
|
||||
# Child routes (evaluated top-to-bottom, first match wins)
|
||||
routes:
|
||||
# ── Critical → PagerDuty + Slack ──────────────────────────────────────
|
||||
- matchers:
|
||||
- severity = critical
|
||||
receiver: pagerduty-critical
|
||||
group_wait: 10s
|
||||
repeat_interval: 1h
|
||||
continue: true # Also notify Slack
|
||||
|
||||
- matchers:
|
||||
- severity = critical
|
||||
receiver: slack-critical
|
||||
group_wait: 10s
|
||||
repeat_interval: 1h
|
||||
|
||||
# ── Warning → Slack ───────────────────────────────────────────────────
|
||||
- matchers:
|
||||
- severity = warning
|
||||
receiver: slack-warning
|
||||
group_wait: 30s
|
||||
repeat_interval: 4h
|
||||
|
||||
# ── Info → Email ──────────────────────────────────────────────────────
|
||||
- matchers:
|
||||
- severity = info
|
||||
receiver: email-info
|
||||
group_wait: 1m
|
||||
repeat_interval: 12h
|
||||
|
||||
# ── Watchdog (deadman's switch) ───────────────────────────────────────
|
||||
- matchers:
|
||||
- alertname = Watchdog
|
||||
receiver: "null"
|
||||
repeat_interval: 5m
|
||||
|
||||
# ── Receivers ───────────────────────────────────────────────────────────────
|
||||
receivers:
|
||||
# ── Null receiver (discard alerts) ────────────────────────────────────────
|
||||
- name: "null"
|
||||
|
||||
# ── Slack: Default channel ────────────────────────────────────────────────
|
||||
- name: slack-default
|
||||
slack_configs:
|
||||
- channel: "#agent-monitor-alerts"
|
||||
send_resolved: true
|
||||
username: "AlertManager"
|
||||
icon_emoji: ":bell:"
|
||||
title: >-
|
||||
[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}]
|
||||
{{ .CommonLabels.alertname }}
|
||||
text: >-
|
||||
{{ range .Alerts }}
|
||||
*Alert:* {{ .Labels.alertname }} - `{{ .Labels.severity }}`
|
||||
*Environment:* {{ .Labels.namespace }}
|
||||
*Description:* {{ .Annotations.description }}
|
||||
*Details:*
|
||||
{{ range .Labels.SortedPairs }} • *{{ .Name }}:* `{{ .Value }}`
|
||||
{{ end }}
|
||||
{{ end }}
|
||||
actions:
|
||||
- type: button
|
||||
text: "Dashboard :grafana:"
|
||||
url: "{{ (index .Alerts 0).Annotations.dashboard_url }}"
|
||||
- type: button
|
||||
text: "Runbook :book:"
|
||||
url: "{{ (index .Alerts 0).Annotations.runbook_url }}"
|
||||
|
||||
# ── Slack: Critical alerts ────────────────────────────────────────────────
|
||||
- name: slack-critical
|
||||
slack_configs:
|
||||
- channel: "#agent-monitor-critical"
|
||||
send_resolved: true
|
||||
username: "AlertManager"
|
||||
icon_emoji: ":rotating_light:"
|
||||
color: >-
|
||||
{{ if eq .Status "firing" }}danger{{ else }}good{{ end }}
|
||||
title: >-
|
||||
:rotating_light: [{{ .Status | toUpper }}]
|
||||
{{ .CommonLabels.alertname }}
|
||||
text: >-
|
||||
{{ range .Alerts }}
|
||||
*CRITICAL Alert:* {{ .Labels.alertname }}
|
||||
*Environment:* {{ .Labels.namespace }}
|
||||
*Description:* {{ .Annotations.description }}
|
||||
*Summary:* {{ .Annotations.summary }}
|
||||
*Started:* {{ .StartsAt.Format "2006-01-02 15:04:05 UTC" }}
|
||||
{{ end }}
|
||||
actions:
|
||||
- type: button
|
||||
text: "Dashboard"
|
||||
url: "{{ (index .Alerts 0).Annotations.dashboard_url }}"
|
||||
- type: button
|
||||
text: "Runbook"
|
||||
url: "{{ (index .Alerts 0).Annotations.runbook_url }}"
|
||||
|
||||
# ── Slack: Warning alerts ─────────────────────────────────────────────────
|
||||
- name: slack-warning
|
||||
slack_configs:
|
||||
- channel: "#agent-monitor-alerts"
|
||||
send_resolved: true
|
||||
username: "AlertManager"
|
||||
icon_emoji: ":warning:"
|
||||
color: >-
|
||||
{{ if eq .Status "firing" }}warning{{ else }}good{{ end }}
|
||||
title: >-
|
||||
:warning: [{{ .Status | toUpper }}]
|
||||
{{ .CommonLabels.alertname }}
|
||||
text: >-
|
||||
{{ range .Alerts }}
|
||||
*Warning:* {{ .Labels.alertname }}
|
||||
*Environment:* {{ .Labels.namespace }}
|
||||
*Description:* {{ .Annotations.description }}
|
||||
{{ end }}
|
||||
|
||||
# ── PagerDuty: Critical alerts ───────────────────────────────────────────
|
||||
- name: pagerduty-critical
|
||||
pagerduty_configs:
|
||||
- routing_key: "<PAGERDUTY_ROUTING_KEY>"
|
||||
severity: >-
|
||||
{{ if eq .CommonLabels.severity "critical" }}critical{{ else }}warning{{ end }}
|
||||
description: >-
|
||||
{{ .CommonAnnotations.summary }}
|
||||
details:
|
||||
environment: "{{ .CommonLabels.namespace }}"
|
||||
alertname: "{{ .CommonLabels.alertname }}"
|
||||
service: "{{ .CommonLabels.service }}"
|
||||
description: "{{ .CommonAnnotations.description }}"
|
||||
num_firing: "{{ .Alerts.Firing | len }}"
|
||||
|
||||
# ── Email: Info-level alerts ──────────────────────────────────────────────
|
||||
- name: email-info
|
||||
email_configs:
|
||||
- to: "<ALERT_EMAIL_RECIPIENTS>"
|
||||
send_resolved: true
|
||||
headers:
|
||||
Subject: >-
|
||||
[Agent Monitor {{ .Status | toUpper }}]
|
||||
{{ .CommonLabels.alertname }}
|
||||
({{ .CommonLabels.namespace }})
|
||||
html: |
|
||||
<h2>{{ .CommonLabels.alertname }}</h2>
|
||||
<p><b>Status:</b> {{ .Status }}</p>
|
||||
<p><b>Environment:</b> {{ .CommonLabels.namespace }}</p>
|
||||
<table border="1" cellpadding="5">
|
||||
<tr><th>Alert</th><th>Severity</th><th>Description</th><th>Started</th></tr>
|
||||
{{ range .Alerts }}
|
||||
<tr>
|
||||
<td>{{ .Labels.alertname }}</td>
|
||||
<td>{{ .Labels.severity }}</td>
|
||||
<td>{{ .Annotations.description }}</td>
|
||||
<td>{{ .StartsAt.Format "2006-01-02 15:04:05 UTC" }}</td>
|
||||
</tr>
|
||||
{{ end }}
|
||||
</table>
|
||||
@@ -0,0 +1,140 @@
|
||||
# Coralogix Integration
|
||||
|
||||
Full-stack observability for Claude Code Agent Monitor via [Coralogix](https://coralogix.com) — logs, metrics, traces, and SLO tracking through a single platform.
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "Kubernetes Cluster"
|
||||
APP["Agent Monitor Pods"]
|
||||
MCP["MCP Sidecar"]
|
||||
OTEL["OTel Collector<br/>(DaemonSet)"]
|
||||
end
|
||||
|
||||
APP -->|"metrics + logs"| OTEL
|
||||
MCP -->|"metrics + logs"| OTEL
|
||||
|
||||
OTEL -->|"OTLP (gRPC)"| CX["Coralogix Platform"]
|
||||
|
||||
subgraph "Coralogix"
|
||||
CX --> LOGS["Log Analytics<br/>DataPrime Queries"]
|
||||
CX --> MET["Metrics<br/>PromQL + Recording Rules"]
|
||||
CX --> TRACE["Distributed Tracing"]
|
||||
CX --> ALERT["Alert Engine"]
|
||||
CX --> DASH["Custom Dashboards"]
|
||||
CX --> SLO["SLO Management"]
|
||||
end
|
||||
|
||||
ALERT -->|"Critical"| PD["PagerDuty"]
|
||||
ALERT -->|"Warning"| SLACK["Slack"]
|
||||
|
||||
style OTEL fill:#4f46e5,color:#fff
|
||||
style CX fill:#1a1a2e,color:#fff
|
||||
style LOGS fill:#7c3aed,color:#fff
|
||||
style MET fill:#e6522c,color:#fff
|
||||
style TRACE fill:#059669,color:#fff
|
||||
style ALERT fill:#dc2626,color:#fff
|
||||
style DASH fill:#f46800,color:#fff
|
||||
style SLO fill:#0ea5e9,color:#fff
|
||||
```
|
||||
|
||||
## Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `values.yaml` | Helm values for Coralogix OpenTelemetry Collector |
|
||||
| `alerts.yaml` | Alert definitions (mirrors Prometheus/Alertmanager rules) |
|
||||
| `dashboards.yaml` | Custom dashboard with 6 rows, 18 panels, SLO tracking |
|
||||
| `coralogix-terraform.tf` | Terraform-managed alerts, parsing rules, recording rules |
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Add the Helm Repository
|
||||
|
||||
```bash
|
||||
helm repo add coralogix https://cgx.jfrog.io/artifactory/coralogix-charts-virtual
|
||||
helm repo update
|
||||
```
|
||||
|
||||
### 2. Create the API Key Secret
|
||||
|
||||
```bash
|
||||
kubectl create secret generic coralogix-keys \
|
||||
--namespace agent-monitor \
|
||||
--from-literal=PRIVATE_KEY=<YOUR_CORALOGIX_SEND_YOUR_DATA_KEY>
|
||||
```
|
||||
|
||||
### 3. Deploy the OTel Collector
|
||||
|
||||
```bash
|
||||
helm install coralogix-otel coralogix/opentelemetry \
|
||||
--namespace agent-monitor \
|
||||
-f deployments/monitoring/coralogix/values.yaml
|
||||
```
|
||||
|
||||
### 4. Import the Dashboard
|
||||
|
||||
Upload `dashboards.yaml` via the Coralogix UI:
|
||||
|
||||
**Dashboards → Custom Dashboards → Import**
|
||||
|
||||
### 5. (Optional) Terraform-managed Alerts
|
||||
|
||||
```bash
|
||||
cd deployments/monitoring/coralogix
|
||||
export CORALOGIX_API_KEY="<your-key>"
|
||||
export CORALOGIX_ENV="coralogix.com"
|
||||
terraform init
|
||||
terraform apply
|
||||
```
|
||||
|
||||
## What Gets Collected
|
||||
|
||||
| Signal | Source | Destination |
|
||||
|--------|--------|-------------|
|
||||
| **Logs** | Pod stdout/stderr (JSON structured) | Coralogix Log Analytics |
|
||||
| **Metrics** | Prometheus scrape (`/api/health`) | Coralogix Metrics |
|
||||
| **K8s Metrics** | kubelet, cAdvisor, host metrics | Coralogix Metrics |
|
||||
| **Traces** | OTLP from application (if instrumented) | Coralogix Tracing |
|
||||
|
||||
## Alert Parity
|
||||
|
||||
All 10 Prometheus/Alertmanager rules are replicated in Coralogix:
|
||||
|
||||
| Alert | Severity | Prometheus | Coralogix |
|
||||
|-------|----------|:----------:|:---------:|
|
||||
| Instance Down | Critical | ✓ | ✓ |
|
||||
| High Error Rate | Critical | ✓ | ✓ |
|
||||
| Pod Restart Loop | Critical | ✓ | ✓ |
|
||||
| PV Nearly Full | Critical | ✓ | ✓ |
|
||||
| High Latency | Warning | ✓ | ✓ |
|
||||
| WebSocket Spike | Warning | ✓ | ✓ |
|
||||
| High Memory | Warning | ✓ | ✓ |
|
||||
| High CPU | Warning | ✓ | ✓ |
|
||||
| HPA Maxed Out | Warning | ✓ | ✓ |
|
||||
| Slow DB Queries | Warning | ✓ | ✓ |
|
||||
|
||||
## Dashboard Panels
|
||||
|
||||
The custom dashboard provides 18 panels across 6 rows:
|
||||
|
||||
1. **Overview** — Active sessions, request rate, WebSocket connections
|
||||
2. **HTTP Performance** — Latency distribution, error rate, status codes
|
||||
3. **Application Logs** — Error log stream (DataPrime), log volume by severity, hook throughput
|
||||
4. **Infrastructure** — CPU, memory, pod status
|
||||
5. **Database & Storage** — SQLite query duration, PV usage, network I/O
|
||||
6. **SLO Tracking** — Availability SLO (99.9%), latency SLO (P95 < 500ms), error budget burn
|
||||
|
||||
## Coralogix Regions
|
||||
|
||||
Set `global.domain` in `values.yaml` to match your Coralogix region:
|
||||
|
||||
| Region | Domain |
|
||||
|--------|--------|
|
||||
| US1 | `coralogix.us` |
|
||||
| US2 | `cx498.coralogix.com` |
|
||||
| EU1 | `coralogix.com` |
|
||||
| EU2 | `eu2.coralogix.com` |
|
||||
| AP1 (India) | `coralogix.in` |
|
||||
| AP2 (Singapore) | `coralogix.sg` |
|
||||
@@ -0,0 +1,194 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Coralogix Alert Definitions for Claude Code Agent Monitor
|
||||
#
|
||||
# These alerts can be imported via the Coralogix Alerts API or Terraform
|
||||
# provider (coralogix/coralogix). They mirror the Prometheus/Alertmanager
|
||||
# rules in ../prometheus/rules/ for consistency across observability stacks.
|
||||
#
|
||||
# API import:
|
||||
# curl -X POST "https://api.coralogix.com/api/v1/external/alerts" \
|
||||
# -H "Authorization: Bearer $CORALOGIX_API_KEY" \
|
||||
# -H "Content-Type: application/json" \
|
||||
# -d @deployments/monitoring/coralogix/alerts.yaml
|
||||
#
|
||||
# Terraform:
|
||||
# See the coralogix_alert resources in coralogix-terraform.tf
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
alerts:
|
||||
# ── Critical ───────────────────────────────────────────────────────────────
|
||||
|
||||
- name: "Agent Monitor Down"
|
||||
description: "No metrics received from agent-monitor pods for > 2 minutes"
|
||||
severity: critical
|
||||
type: metric
|
||||
condition:
|
||||
metric_name: "up"
|
||||
filter:
|
||||
job: "agent-monitor"
|
||||
threshold: 1
|
||||
comparison: less_than
|
||||
for_duration: "2m"
|
||||
of_last: "5m"
|
||||
notifications:
|
||||
- integration: pagerduty
|
||||
- integration: slack
|
||||
channel: "#agent-monitor-critical"
|
||||
labels:
|
||||
service: agent-monitor
|
||||
team: platform
|
||||
|
||||
- name: "High Error Rate"
|
||||
description: "5xx error rate exceeds 5% of total requests for 5 minutes"
|
||||
severity: critical
|
||||
type: ratio
|
||||
condition:
|
||||
numerator:
|
||||
query: 'http_requests_total{job="agent-monitor", status=~"5.."}'
|
||||
denominator:
|
||||
query: 'http_requests_total{job="agent-monitor"}'
|
||||
threshold: 0.05
|
||||
comparison: greater_than
|
||||
for_duration: "5m"
|
||||
notifications:
|
||||
- integration: pagerduty
|
||||
- integration: slack
|
||||
channel: "#agent-monitor-critical"
|
||||
labels:
|
||||
service: agent-monitor
|
||||
|
||||
- name: "Pod Restart Loop"
|
||||
description: "Agent Monitor pod has restarted > 5 times in 15 minutes"
|
||||
severity: critical
|
||||
type: metric
|
||||
condition:
|
||||
query: 'increase(kube_pod_container_status_restarts_total{namespace=~"agent-monitor.*", container="agent-monitor"}[15m])'
|
||||
threshold: 5
|
||||
comparison: greater_than
|
||||
for_duration: "1m"
|
||||
notifications:
|
||||
- integration: pagerduty
|
||||
- integration: slack
|
||||
channel: "#agent-monitor-critical"
|
||||
labels:
|
||||
service: agent-monitor
|
||||
|
||||
- name: "Persistent Volume Nearly Full"
|
||||
description: "SQLite persistent volume is > 90% full"
|
||||
severity: critical
|
||||
type: metric
|
||||
condition:
|
||||
query: '(kubelet_volume_stats_used_bytes{namespace=~"agent-monitor.*"} / kubelet_volume_stats_capacity_bytes{namespace=~"agent-monitor.*"}) * 100'
|
||||
threshold: 90
|
||||
comparison: greater_than
|
||||
for_duration: "5m"
|
||||
notifications:
|
||||
- integration: pagerduty
|
||||
- integration: slack
|
||||
channel: "#agent-monitor-critical"
|
||||
labels:
|
||||
service: agent-monitor
|
||||
|
||||
# ── Warning ────────────────────────────────────────────────────────────────
|
||||
|
||||
- name: "High Latency"
|
||||
description: "P95 request latency exceeds 2 seconds for 5 minutes"
|
||||
severity: warning
|
||||
type: metric
|
||||
condition:
|
||||
query: 'histogram_quantile(0.95, rate(http_request_duration_seconds_bucket{job="agent-monitor"}[5m]))'
|
||||
threshold: 2
|
||||
comparison: greater_than
|
||||
for_duration: "5m"
|
||||
notifications:
|
||||
- integration: slack
|
||||
channel: "#agent-monitor-alerts"
|
||||
labels:
|
||||
service: agent-monitor
|
||||
|
||||
- name: "WebSocket Connection Spike"
|
||||
description: "Active WebSocket connections exceed 1000"
|
||||
severity: warning
|
||||
type: metric
|
||||
condition:
|
||||
metric_name: "websocket_connections_active"
|
||||
filter:
|
||||
job: "agent-monitor"
|
||||
threshold: 1000
|
||||
comparison: greater_than
|
||||
for_duration: "2m"
|
||||
notifications:
|
||||
- integration: slack
|
||||
channel: "#agent-monitor-alerts"
|
||||
labels:
|
||||
service: agent-monitor
|
||||
|
||||
- name: "High Memory Usage"
|
||||
description: "Container memory usage exceeds 85% of limit"
|
||||
severity: warning
|
||||
type: metric
|
||||
condition:
|
||||
query: '(container_memory_working_set_bytes{namespace=~"agent-monitor.*", container="agent-monitor"} / container_spec_memory_limit_bytes{namespace=~"agent-monitor.*", container="agent-monitor"}) * 100'
|
||||
threshold: 85
|
||||
comparison: greater_than
|
||||
for_duration: "5m"
|
||||
notifications:
|
||||
- integration: slack
|
||||
channel: "#agent-monitor-alerts"
|
||||
labels:
|
||||
service: agent-monitor
|
||||
|
||||
- name: "High CPU Usage"
|
||||
description: "Container CPU usage exceeds 80% for 10 minutes"
|
||||
severity: warning
|
||||
type: metric
|
||||
condition:
|
||||
query: '(rate(container_cpu_usage_seconds_total{namespace=~"agent-monitor.*", container="agent-monitor"}[5m]) / container_spec_cpu_quota{namespace=~"agent-monitor.*", container="agent-monitor"} * 100000)'
|
||||
threshold: 80
|
||||
comparison: greater_than
|
||||
for_duration: "10m"
|
||||
notifications:
|
||||
- integration: slack
|
||||
channel: "#agent-monitor-alerts"
|
||||
labels:
|
||||
service: agent-monitor
|
||||
|
||||
- name: "HPA Maxed Out"
|
||||
description: "HPA replicas at max for 15 minutes — may need capacity increase"
|
||||
severity: warning
|
||||
type: metric
|
||||
condition:
|
||||
query: 'kube_horizontalpodautoscaler_status_current_replicas{namespace=~"agent-monitor.*"} == kube_horizontalpodautoscaler_spec_max_replicas{namespace=~"agent-monitor.*"}'
|
||||
threshold: 1
|
||||
comparison: greater_than_or_equal
|
||||
for_duration: "15m"
|
||||
notifications:
|
||||
- integration: slack
|
||||
channel: "#agent-monitor-alerts"
|
||||
labels:
|
||||
service: agent-monitor
|
||||
|
||||
- name: "Slow Database Queries"
|
||||
description: "SQLite query duration exceeds 1 second"
|
||||
severity: warning
|
||||
type: metric
|
||||
condition:
|
||||
metric_name: "sqlite_query_duration_seconds"
|
||||
filter:
|
||||
job: "agent-monitor"
|
||||
threshold: 1
|
||||
comparison: greater_than
|
||||
for_duration: "5m"
|
||||
notifications:
|
||||
- integration: slack
|
||||
channel: "#agent-monitor-alerts"
|
||||
labels:
|
||||
service: agent-monitor
|
||||
|
||||
# ── Notification integrations ────────────────────────────────────────────────
|
||||
# Configure these in Coralogix UI: Settings → Integrations → Outbound Webhooks
|
||||
#
|
||||
# Required integrations:
|
||||
# - pagerduty: PagerDuty Events API v2 routing key
|
||||
# - slack: Slack webhook for #agent-monitor-critical and #agent-monitor-alerts
|
||||
# - email: (optional) Email notification group
|
||||
@@ -0,0 +1,353 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Coralogix Terraform Integration for Claude Code Agent Monitor
|
||||
#
|
||||
# Provisions Coralogix resources via the official Terraform provider:
|
||||
# - Alert rules (mirroring Prometheus/Alertmanager rules)
|
||||
# - Log parsing rules for structured JSON ingestion
|
||||
# - Recording rules for pre-aggregated SLO metrics
|
||||
# - Dashboard provisioning
|
||||
#
|
||||
# Usage:
|
||||
# export CORALOGIX_API_KEY="<your-send-your-data-key>"
|
||||
# export CORALOGIX_ENV="<your-coralogix-domain>" # e.g. coralogix.com
|
||||
# terraform init
|
||||
# terraform plan
|
||||
# terraform apply
|
||||
#
|
||||
# Requires: hashicorp/terraform >= 1.5, coralogix/coralogix >= 1.10
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
terraform {
|
||||
required_version = ">= 1.5"
|
||||
required_providers {
|
||||
coralogix = {
|
||||
source = "coralogix/coralogix"
|
||||
version = "~> 1.10"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
provider "coralogix" {
|
||||
# API key and environment are sourced from:
|
||||
# CORALOGIX_API_KEY – Send-Your-Data API key
|
||||
# CORALOGIX_ENV – Domain (e.g. coralogix.com, eu2.coralogix.com)
|
||||
}
|
||||
|
||||
# ── Variables ────────────────────────────────────────────────────────────────
|
||||
|
||||
variable "environment" {
|
||||
description = "Deployment environment (dev, staging, production)"
|
||||
type = string
|
||||
default = "production"
|
||||
validation {
|
||||
condition = contains(["dev", "staging", "production"], var.environment)
|
||||
error_message = "environment must be one of: dev, staging, production."
|
||||
}
|
||||
}
|
||||
|
||||
variable "notification_group_id" {
|
||||
description = "Coralogix notification group ID for alert routing"
|
||||
type = string
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "pagerduty_webhook_id" {
|
||||
description = "Coralogix outbound webhook ID for PagerDuty integration"
|
||||
type = string
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "slack_webhook_id" {
|
||||
description = "Coralogix outbound webhook ID for Slack integration"
|
||||
type = string
|
||||
default = ""
|
||||
}
|
||||
|
||||
locals {
|
||||
app_name = "agent-monitor"
|
||||
subsystem = "kubernetes"
|
||||
alert_prefix = "[Agent Monitor]"
|
||||
}
|
||||
|
||||
# ── Parsing Rules ────────────────────────────────────────────────────────────
|
||||
# Structured JSON log parsing for agent-monitor application logs
|
||||
|
||||
resource "coralogix_rules_group" "agent_monitor_parsing" {
|
||||
name = "${local.alert_prefix} Log Parsing"
|
||||
description = "Parse structured JSON logs from Agent Monitor pods"
|
||||
enabled = true
|
||||
order = 1
|
||||
|
||||
rule_subgroups {
|
||||
rules {
|
||||
name = "JSON Extract"
|
||||
description = "Extract structured fields from JSON application logs"
|
||||
source_field = "text"
|
||||
enabled = true
|
||||
|
||||
parse_json_field {
|
||||
destination_field = "json"
|
||||
keep_source_field = false
|
||||
keep_destination_field = true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
rule_subgroups {
|
||||
rules {
|
||||
name = "Severity Mapping"
|
||||
description = "Map log level field to Coralogix severity"
|
||||
source_field = "json.level"
|
||||
enabled = true
|
||||
|
||||
extract {
|
||||
regexp = "(?P<severity>debug|info|warn|error|fatal)"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# ── Recording Rules ──────────────────────────────────────────────────────────
|
||||
# Pre-aggregate SLO metrics for efficient dashboard queries
|
||||
|
||||
resource "coralogix_recording_rule_group_set" "slo_metrics" {
|
||||
name = "${local.alert_prefix} SLO Recording Rules"
|
||||
|
||||
groups {
|
||||
name = "agent_monitor_slo"
|
||||
interval = 60 # seconds
|
||||
|
||||
rules {
|
||||
record = "agent_monitor:http_availability:ratio_rate5m"
|
||||
expr = <<-EOT
|
||||
1 - (
|
||||
sum(rate(http_requests_total{job="agent-monitor", status=~"5.."}[5m]))
|
||||
/
|
||||
sum(rate(http_requests_total{job="agent-monitor"}[5m]))
|
||||
)
|
||||
EOT
|
||||
labels = {
|
||||
service = local.app_name
|
||||
environment = var.environment
|
||||
}
|
||||
}
|
||||
|
||||
rules {
|
||||
record = "agent_monitor:http_latency_p95:seconds_rate5m"
|
||||
expr = <<-EOT
|
||||
histogram_quantile(0.95,
|
||||
sum(rate(http_request_duration_seconds_bucket{job="agent-monitor"}[5m])) by (le)
|
||||
)
|
||||
EOT
|
||||
labels = {
|
||||
service = local.app_name
|
||||
environment = var.environment
|
||||
}
|
||||
}
|
||||
|
||||
rules {
|
||||
record = "agent_monitor:websocket_connections:total"
|
||||
expr = <<-EOT
|
||||
sum(websocket_connections_active{job="agent-monitor"})
|
||||
EOT
|
||||
labels = {
|
||||
service = local.app_name
|
||||
environment = var.environment
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# ── Alert Rules ──────────────────────────────────────────────────────────────
|
||||
|
||||
resource "coralogix_alert" "instance_down" {
|
||||
name = "${local.alert_prefix} Instance Down"
|
||||
description = "No metrics received from agent-monitor pods for > 2 minutes"
|
||||
severity = "Critical"
|
||||
enabled = true
|
||||
|
||||
metric {
|
||||
promql {
|
||||
text = "up{job=\"agent-monitor\"} == 0"
|
||||
condition = "more_than"
|
||||
threshold = 0
|
||||
}
|
||||
duration = "2m"
|
||||
}
|
||||
|
||||
notifications_group {
|
||||
dynamic "notification" {
|
||||
for_each = var.pagerduty_webhook_id != "" ? [1] : []
|
||||
content {
|
||||
integration_id = var.pagerduty_webhook_id
|
||||
}
|
||||
}
|
||||
dynamic "notification" {
|
||||
for_each = var.slack_webhook_id != "" ? [1] : []
|
||||
content {
|
||||
integration_id = var.slack_webhook_id
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
labels = {
|
||||
service = local.app_name
|
||||
environment = var.environment
|
||||
team = "platform"
|
||||
}
|
||||
}
|
||||
|
||||
resource "coralogix_alert" "high_error_rate" {
|
||||
name = "${local.alert_prefix} High Error Rate"
|
||||
description = "5xx error rate exceeds 5% of total requests for 5 minutes"
|
||||
severity = "Critical"
|
||||
enabled = true
|
||||
|
||||
metric {
|
||||
promql {
|
||||
text = <<-EOT
|
||||
(
|
||||
sum(rate(http_requests_total{job="agent-monitor", status=~"5.."}[5m]))
|
||||
/
|
||||
sum(rate(http_requests_total{job="agent-monitor"}[5m]))
|
||||
) * 100 > 5
|
||||
EOT
|
||||
condition = "more_than"
|
||||
threshold = 5
|
||||
}
|
||||
duration = "5m"
|
||||
}
|
||||
|
||||
notifications_group {
|
||||
dynamic "notification" {
|
||||
for_each = var.pagerduty_webhook_id != "" ? [1] : []
|
||||
content {
|
||||
integration_id = var.pagerduty_webhook_id
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
labels = {
|
||||
service = local.app_name
|
||||
environment = var.environment
|
||||
}
|
||||
}
|
||||
|
||||
resource "coralogix_alert" "high_latency" {
|
||||
name = "${local.alert_prefix} High Latency"
|
||||
description = "P95 request latency exceeds 2 seconds for 5 minutes"
|
||||
severity = "Warning"
|
||||
enabled = true
|
||||
|
||||
metric {
|
||||
promql {
|
||||
text = "histogram_quantile(0.95, sum(rate(http_request_duration_seconds_bucket{job=\"agent-monitor\"}[5m])) by (le)) > 2"
|
||||
condition = "more_than"
|
||||
threshold = 2
|
||||
}
|
||||
duration = "5m"
|
||||
}
|
||||
|
||||
notifications_group {
|
||||
dynamic "notification" {
|
||||
for_each = var.slack_webhook_id != "" ? [1] : []
|
||||
content {
|
||||
integration_id = var.slack_webhook_id
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
labels = {
|
||||
service = local.app_name
|
||||
environment = var.environment
|
||||
}
|
||||
}
|
||||
|
||||
resource "coralogix_alert" "high_memory" {
|
||||
name = "${local.alert_prefix} High Memory Usage"
|
||||
description = "Container memory usage exceeds 85% of limit"
|
||||
severity = "Warning"
|
||||
enabled = true
|
||||
|
||||
metric {
|
||||
promql {
|
||||
text = <<-EOT
|
||||
(
|
||||
container_memory_working_set_bytes{namespace=~"agent-monitor.*", container="agent-monitor"}
|
||||
/
|
||||
container_spec_memory_limit_bytes{namespace=~"agent-monitor.*", container="agent-monitor"}
|
||||
) * 100 > 85
|
||||
EOT
|
||||
condition = "more_than"
|
||||
threshold = 85
|
||||
}
|
||||
duration = "5m"
|
||||
}
|
||||
|
||||
notifications_group {
|
||||
dynamic "notification" {
|
||||
for_each = var.slack_webhook_id != "" ? [1] : []
|
||||
content {
|
||||
integration_id = var.slack_webhook_id
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
labels = {
|
||||
service = local.app_name
|
||||
environment = var.environment
|
||||
}
|
||||
}
|
||||
|
||||
resource "coralogix_alert" "pod_restart_loop" {
|
||||
name = "${local.alert_prefix} Pod Restart Loop"
|
||||
description = "Agent Monitor pod has restarted > 5 times in 15 minutes"
|
||||
severity = "Critical"
|
||||
enabled = true
|
||||
|
||||
metric {
|
||||
promql {
|
||||
text = "increase(kube_pod_container_status_restarts_total{namespace=~\"agent-monitor.*\", container=\"agent-monitor\"}[15m]) > 5"
|
||||
condition = "more_than"
|
||||
threshold = 5
|
||||
}
|
||||
duration = "1m"
|
||||
}
|
||||
|
||||
notifications_group {
|
||||
dynamic "notification" {
|
||||
for_each = var.pagerduty_webhook_id != "" ? [1] : []
|
||||
content {
|
||||
integration_id = var.pagerduty_webhook_id
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
labels = {
|
||||
service = local.app_name
|
||||
environment = var.environment
|
||||
}
|
||||
}
|
||||
|
||||
# ── Outputs ──────────────────────────────────────────────────────────────────
|
||||
|
||||
output "parsing_rule_group_id" {
|
||||
description = "ID of the Coralogix parsing rule group"
|
||||
value = coralogix_rules_group.agent_monitor_parsing.id
|
||||
}
|
||||
|
||||
output "recording_rule_set_id" {
|
||||
description = "ID of the Coralogix recording rule group set"
|
||||
value = coralogix_recording_rule_group_set.slo_metrics.id
|
||||
}
|
||||
|
||||
output "alert_ids" {
|
||||
description = "IDs of all provisioned Coralogix alerts"
|
||||
value = {
|
||||
instance_down = coralogix_alert.instance_down.id
|
||||
high_error_rate = coralogix_alert.high_error_rate.id
|
||||
high_latency = coralogix_alert.high_latency.id
|
||||
high_memory = coralogix_alert.high_memory.id
|
||||
pod_restart = coralogix_alert.pod_restart_loop.id
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,245 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Coralogix Custom Dashboard – Agent Monitor
|
||||
#
|
||||
# Import via Coralogix UI: Dashboards → Custom Dashboards → Import
|
||||
# Or via API:
|
||||
# curl -X POST "https://api.coralogix.com/api/v1/external/grafana/api/dashboards/db" \
|
||||
# -H "Authorization: Bearer $CORALOGIX_API_KEY" \
|
||||
# -H "Content-Type: application/json" \
|
||||
# -d @dashboards.yaml
|
||||
#
|
||||
# This dashboard mirrors the Grafana dashboard (../grafana/dashboards/) while
|
||||
# leveraging Coralogix-native features: DataPrime queries, log correlation,
|
||||
# distributed tracing waterfall, and Apdex scoring.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
dashboard:
|
||||
name: "Agent Monitor – Operations"
|
||||
description: "Claude Code Agent Monitor: real-time operations, SLOs, and infrastructure health"
|
||||
folder: "Agent Monitor"
|
||||
tags:
|
||||
- agent-monitor
|
||||
- operations
|
||||
- sre
|
||||
|
||||
# ── Row 1: Overview ──────────────────────────────────────────────────────
|
||||
rows:
|
||||
- name: "Overview"
|
||||
panels:
|
||||
- title: "Active Sessions"
|
||||
type: line-chart
|
||||
query:
|
||||
type: metrics
|
||||
promql: 'agent_monitor_active_sessions'
|
||||
legend: "{{namespace}}"
|
||||
span: 4
|
||||
|
||||
- title: "Request Rate (req/s)"
|
||||
type: line-chart
|
||||
query:
|
||||
type: metrics
|
||||
promql: 'sum(rate(http_requests_total{job="agent-monitor"}[5m]))'
|
||||
legend: "Requests/sec"
|
||||
span: 4
|
||||
|
||||
- title: "WebSocket Connections"
|
||||
type: line-chart
|
||||
query:
|
||||
type: metrics
|
||||
promql: 'websocket_connections_active{job="agent-monitor"}'
|
||||
legend: "{{pod}}"
|
||||
span: 4
|
||||
|
||||
# ── Row 2: HTTP Performance ────────────────────────────────────────────
|
||||
- name: "HTTP Performance"
|
||||
panels:
|
||||
- title: "Latency Distribution (P50 / P95 / P99)"
|
||||
type: line-chart
|
||||
query:
|
||||
type: metrics
|
||||
promql: |
|
||||
histogram_quantile(0.50, sum(rate(http_request_duration_seconds_bucket{job="agent-monitor"}[5m])) by (le))
|
||||
histogram_quantile(0.95, sum(rate(http_request_duration_seconds_bucket{job="agent-monitor"}[5m])) by (le))
|
||||
histogram_quantile(0.99, sum(rate(http_request_duration_seconds_bucket{job="agent-monitor"}[5m])) by (le))
|
||||
span: 6
|
||||
|
||||
- title: "Error Rate (%)"
|
||||
type: line-chart
|
||||
query:
|
||||
type: metrics
|
||||
promql: |
|
||||
sum(rate(http_requests_total{job="agent-monitor", status=~"5.."}[5m]))
|
||||
/
|
||||
sum(rate(http_requests_total{job="agent-monitor"}[5m])) * 100
|
||||
legend: "5xx %"
|
||||
thresholds:
|
||||
- value: 1
|
||||
color: yellow
|
||||
- value: 5
|
||||
color: red
|
||||
span: 3
|
||||
|
||||
- title: "Status Code Distribution"
|
||||
type: bar-chart
|
||||
query:
|
||||
type: metrics
|
||||
promql: 'sum by (status) (increase(http_requests_total{job="agent-monitor"}[1h]))'
|
||||
span: 3
|
||||
|
||||
# ── Row 3: Logs (DataPrime) ────────────────────────────────────────────
|
||||
- name: "Application Logs"
|
||||
panels:
|
||||
- title: "Error Logs"
|
||||
type: dataprime
|
||||
query: |
|
||||
source logs
|
||||
| filter $d.cx.application.name == 'agent-monitor'
|
||||
| filter $d.severity == 'ERROR' || $d.severity == 'FATAL'
|
||||
| select $m.timestamp, $d.k8s.pod.name, $d.message
|
||||
| order by $m.timestamp desc
|
||||
| limit 100
|
||||
span: 6
|
||||
|
||||
- title: "Log Volume by Severity"
|
||||
type: bar-chart
|
||||
query:
|
||||
type: dataprime
|
||||
expression: |
|
||||
source logs
|
||||
| filter $d.cx.application.name == 'agent-monitor'
|
||||
| count_group_by $d.severity as count
|
||||
span: 3
|
||||
|
||||
- title: "Hook Event Throughput"
|
||||
type: line-chart
|
||||
query:
|
||||
type: dataprime
|
||||
expression: |
|
||||
source logs
|
||||
| filter $d.cx.application.name == 'agent-monitor'
|
||||
| filter $d.message matches 'hook.*event'
|
||||
| count_per_time 1m as throughput
|
||||
span: 3
|
||||
|
||||
# ── Row 4: Infrastructure ──────────────────────────────────────────────
|
||||
- name: "Infrastructure"
|
||||
panels:
|
||||
- title: "CPU Usage (%)"
|
||||
type: line-chart
|
||||
query:
|
||||
type: metrics
|
||||
promql: |
|
||||
rate(container_cpu_usage_seconds_total{
|
||||
namespace=~"agent-monitor.*",
|
||||
container="agent-monitor"
|
||||
}[5m]) * 100
|
||||
legend: "{{pod}}"
|
||||
span: 4
|
||||
|
||||
- title: "Memory Usage (MiB)"
|
||||
type: line-chart
|
||||
query:
|
||||
type: metrics
|
||||
promql: |
|
||||
container_memory_working_set_bytes{
|
||||
namespace=~"agent-monitor.*",
|
||||
container="agent-monitor"
|
||||
} / 1024 / 1024
|
||||
legend: "{{pod}}"
|
||||
span: 4
|
||||
|
||||
- title: "Pod Status"
|
||||
type: gauge
|
||||
query:
|
||||
type: metrics
|
||||
promql: |
|
||||
count by (phase) (
|
||||
kube_pod_status_phase{namespace=~"agent-monitor.*"}
|
||||
)
|
||||
span: 4
|
||||
|
||||
# ── Row 5: Database & Storage ──────────────────────────────────────────
|
||||
- name: "Database & Storage"
|
||||
panels:
|
||||
- title: "SQLite Query Duration (ms)"
|
||||
type: line-chart
|
||||
query:
|
||||
type: metrics
|
||||
promql: 'sqlite_query_duration_seconds{job="agent-monitor"} * 1000'
|
||||
span: 4
|
||||
|
||||
- title: "PV Usage (%)"
|
||||
type: gauge
|
||||
query:
|
||||
type: metrics
|
||||
promql: |
|
||||
(kubelet_volume_stats_used_bytes{namespace=~"agent-monitor.*"}
|
||||
/ kubelet_volume_stats_capacity_bytes{namespace=~"agent-monitor.*"}) * 100
|
||||
thresholds:
|
||||
- value: 70
|
||||
color: yellow
|
||||
- value: 90
|
||||
color: red
|
||||
span: 4
|
||||
|
||||
- title: "Network I/O (bytes/s)"
|
||||
type: line-chart
|
||||
query:
|
||||
type: metrics
|
||||
promql: |
|
||||
sum by (pod) (rate(container_network_receive_bytes_total{namespace=~"agent-monitor.*"}[5m]))
|
||||
sum by (pod) (rate(container_network_transmit_bytes_total{namespace=~"agent-monitor.*"}[5m]))
|
||||
span: 4
|
||||
|
||||
# ── Row 6: SLO Tracking ────────────────────────────────────────────────
|
||||
- name: "SLO Tracking"
|
||||
panels:
|
||||
- title: "Availability SLO (99.9% target)"
|
||||
type: gauge
|
||||
query:
|
||||
type: metrics
|
||||
promql: |
|
||||
(1 - sum(rate(http_requests_total{job="agent-monitor", status=~"5.."}[30d]))
|
||||
/ sum(rate(http_requests_total{job="agent-monitor"}[30d]))) * 100
|
||||
thresholds:
|
||||
- value: 99.9
|
||||
color: green
|
||||
- value: 99.5
|
||||
color: yellow
|
||||
- value: 99.0
|
||||
color: red
|
||||
span: 4
|
||||
|
||||
- title: "Latency SLO (P95 < 500ms)"
|
||||
type: gauge
|
||||
query:
|
||||
type: metrics
|
||||
promql: |
|
||||
histogram_quantile(0.95,
|
||||
sum(rate(http_request_duration_seconds_bucket{job="agent-monitor"}[30d])) by (le)
|
||||
) * 1000
|
||||
thresholds:
|
||||
- value: 300
|
||||
color: green
|
||||
- value: 500
|
||||
color: yellow
|
||||
- value: 1000
|
||||
color: red
|
||||
span: 4
|
||||
|
||||
- title: "Error Budget Remaining"
|
||||
type: gauge
|
||||
query:
|
||||
type: metrics
|
||||
promql: |
|
||||
(0.001 - sum(rate(http_requests_total{job="agent-monitor", status=~"5.."}[30d]))
|
||||
/ sum(rate(http_requests_total{job="agent-monitor"}[30d])))
|
||||
/ 0.001 * 100
|
||||
thresholds:
|
||||
- value: 50
|
||||
color: green
|
||||
- value: 25
|
||||
color: yellow
|
||||
- value: 0
|
||||
color: red
|
||||
span: 4
|
||||
@@ -0,0 +1,180 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Coralogix OpenTelemetry Collector – Helm Values
|
||||
#
|
||||
# Deploys the Coralogix OTel collector as a DaemonSet + Gateway for shipping
|
||||
# logs, metrics, and traces from the Agent Monitor cluster.
|
||||
#
|
||||
# Prerequisites:
|
||||
# 1. Add Coralogix Helm repo:
|
||||
# helm repo add coralogix https://cgx.jfrog.io/artifactory/coralogix-charts-virtual
|
||||
# helm repo update
|
||||
# 2. Create the API key secret:
|
||||
# kubectl create secret generic coralogix-keys \
|
||||
# --namespace agent-monitor \
|
||||
# --from-literal=PRIVATE_KEY=<YOUR_CORALOGIX_PRIVATE_KEY>
|
||||
#
|
||||
# Install:
|
||||
# helm install coralogix-otel coralogix/opentelemetry \
|
||||
# --namespace agent-monitor \
|
||||
# -f deployments/monitoring/coralogix/values.yaml
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
global:
|
||||
# Coralogix domain – set to your region's endpoint
|
||||
# Options: coralogix.com | eu2.coralogix.com | coralogix.in | coralogix.us |
|
||||
# cx498.coralogix.com | coralogix.eu | coralogix.sg
|
||||
domain: "coralogix.com"
|
||||
|
||||
# Reference the API key from the pre-created K8s secret
|
||||
clusterName: "agent-monitor"
|
||||
|
||||
# ── Secret reference ─────────────────────────────────────────────────────────
|
||||
secret:
|
||||
enabled: true
|
||||
name: "coralogix-keys"
|
||||
# Key in the secret containing the Coralogix Send-Your-Data API key
|
||||
privateKeySecretRef:
|
||||
key: "PRIVATE_KEY"
|
||||
|
||||
# ── Collector – DaemonSet mode (node-level collection) ───────────────────────
|
||||
opentelemetry-collector:
|
||||
mode: daemonset
|
||||
|
||||
presets:
|
||||
# Collect Kubernetes pod/container logs
|
||||
logsCollection:
|
||||
enabled: true
|
||||
includeCollectorLogs: false
|
||||
|
||||
# Enrich telemetry with Kubernetes metadata
|
||||
kubernetesAttributes:
|
||||
enabled: true
|
||||
extractAllPodLabels: true
|
||||
extractAllPodAnnotations: false
|
||||
|
||||
# Collect host-level metrics (CPU, memory, disk, network)
|
||||
hostMetrics:
|
||||
enabled: true
|
||||
|
||||
# Collect kubelet/cAdvisor metrics
|
||||
kubeletMetrics:
|
||||
enabled: true
|
||||
|
||||
config:
|
||||
receivers:
|
||||
# Scrape Prometheus metrics from agent-monitor pods
|
||||
prometheus:
|
||||
config:
|
||||
scrape_configs:
|
||||
- job_name: "agent-monitor"
|
||||
scrape_interval: 15s
|
||||
kubernetes_sd_configs:
|
||||
- role: pod
|
||||
namespaces:
|
||||
names:
|
||||
- agent-monitor
|
||||
- agent-monitor-staging
|
||||
- agent-monitor-production
|
||||
relabel_configs:
|
||||
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
|
||||
action: keep
|
||||
regex: "true"
|
||||
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
|
||||
action: replace
|
||||
target_label: __metrics_path__
|
||||
regex: (.+)
|
||||
- source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
|
||||
action: replace
|
||||
regex: ([^:]+)(?::\d+)?;(\d+)
|
||||
replacement: $$1:$$2
|
||||
target_label: __address__
|
||||
|
||||
# Receive OTLP from in-cluster services (gRPC + HTTP)
|
||||
otlp:
|
||||
protocols:
|
||||
grpc:
|
||||
endpoint: "0.0.0.0:4317"
|
||||
http:
|
||||
endpoint: "0.0.0.0:4318"
|
||||
|
||||
processors:
|
||||
# Batch telemetry for efficient export
|
||||
batch:
|
||||
send_batch_size: 1024
|
||||
send_batch_max_size: 2048
|
||||
timeout: 5s
|
||||
|
||||
# Enrich with resource attributes
|
||||
resource:
|
||||
attributes:
|
||||
- key: cx.application.name
|
||||
value: "agent-monitor"
|
||||
action: upsert
|
||||
- key: cx.subsystem.name
|
||||
from_attribute: k8s.container.name
|
||||
action: upsert
|
||||
- key: k8s.cluster.name
|
||||
value: "agent-monitor"
|
||||
action: upsert
|
||||
|
||||
# Memory limiter to prevent OOM
|
||||
memory_limiter:
|
||||
check_interval: 5s
|
||||
limit_percentage: 80
|
||||
spike_limit_percentage: 25
|
||||
|
||||
# Filter out noisy internal logs
|
||||
filter/drop-internal:
|
||||
logs:
|
||||
exclude:
|
||||
match_type: regexp
|
||||
bodies:
|
||||
- ".*kube-probe.*"
|
||||
- ".*healthz.*"
|
||||
|
||||
exporters:
|
||||
coralogix:
|
||||
domain: "${CORALOGIX_DOMAIN}"
|
||||
private_key: "${PRIVATE_KEY}"
|
||||
application_name: "agent-monitor"
|
||||
subsystem_name: "kubernetes"
|
||||
timeout: 30s
|
||||
|
||||
service:
|
||||
pipelines:
|
||||
logs:
|
||||
receivers: [otlp]
|
||||
processors: [memory_limiter, resource, filter/drop-internal, batch]
|
||||
exporters: [coralogix]
|
||||
metrics:
|
||||
receivers: [otlp, prometheus]
|
||||
processors: [memory_limiter, resource, batch]
|
||||
exporters: [coralogix]
|
||||
traces:
|
||||
receivers: [otlp]
|
||||
processors: [memory_limiter, resource, batch]
|
||||
exporters: [coralogix]
|
||||
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 256Mi
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
|
||||
tolerations:
|
||||
- operator: Exists
|
||||
effect: NoSchedule
|
||||
|
||||
# ── Gateway mode (optional – for centralized export) ─────────────────────────
|
||||
opentelemetry-gateway:
|
||||
enabled: false
|
||||
replicaCount: 2
|
||||
resources:
|
||||
requests:
|
||||
cpu: 250m
|
||||
memory: 512Mi
|
||||
limits:
|
||||
cpu: "1"
|
||||
memory: "1Gi"
|
||||
@@ -0,0 +1,707 @@
|
||||
{
|
||||
"annotations": {
|
||||
"list": [
|
||||
{
|
||||
"builtIn": 1,
|
||||
"datasource": { "type": "grafana", "uid": "-- Grafana --" },
|
||||
"enable": true,
|
||||
"hide": true,
|
||||
"iconColor": "rgba(0, 211, 255, 1)",
|
||||
"name": "Annotations & Alerts",
|
||||
"type": "dashboard"
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
||||
"enable": true,
|
||||
"expr": "changes(kube_deployment_status_observed_generation{namespace=~\"$namespace\", deployment=~\"agent-monitor.*\"}[2m]) > 0",
|
||||
"iconColor": "#6ED0E0",
|
||||
"name": "Deployments",
|
||||
"titleFormat": "Deployment updated"
|
||||
}
|
||||
]
|
||||
},
|
||||
"description": "Comprehensive monitoring dashboard for Claude Code Agent Monitor",
|
||||
"editable": true,
|
||||
"fiscalYearStartMonth": 0,
|
||||
"graphTooltip": 1,
|
||||
"id": null,
|
||||
"links": [],
|
||||
"panels": [
|
||||
{
|
||||
"collapsed": false,
|
||||
"gridPos": { "h": 1, "w": 24, "x": 0, "y": 0 },
|
||||
"id": 100,
|
||||
"title": "Overview",
|
||||
"type": "row"
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "thresholds" },
|
||||
"mappings": [
|
||||
{
|
||||
"options": {
|
||||
"0": { "color": "red", "text": "DOWN" },
|
||||
"1": { "color": "green", "text": "UP" }
|
||||
},
|
||||
"type": "value"
|
||||
}
|
||||
],
|
||||
"thresholds": {
|
||||
"steps": [
|
||||
{ "color": "red", "value": null },
|
||||
{ "color": "green", "value": 1 }
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"gridPos": { "h": 4, "w": 4, "x": 0, "y": 1 },
|
||||
"id": 1,
|
||||
"options": {
|
||||
"colorMode": "background",
|
||||
"graphMode": "none",
|
||||
"justifyMode": "center",
|
||||
"textMode": "value",
|
||||
"reduceOptions": { "calcs": ["lastNotNull"] }
|
||||
},
|
||||
"title": "Uptime Status",
|
||||
"type": "stat",
|
||||
"targets": [
|
||||
{
|
||||
"expr": "up{job=\"agent-monitor\", namespace=~\"$namespace\"}",
|
||||
"legendFormat": "{{instance}}",
|
||||
"refId": "A"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "thresholds" },
|
||||
"unit": "dtdurations",
|
||||
"thresholds": { "steps": [{ "color": "green", "value": null }] }
|
||||
}
|
||||
},
|
||||
"gridPos": { "h": 4, "w": 4, "x": 4, "y": 1 },
|
||||
"id": 2,
|
||||
"options": {
|
||||
"colorMode": "value",
|
||||
"graphMode": "none",
|
||||
"justifyMode": "center",
|
||||
"textMode": "value",
|
||||
"reduceOptions": { "calcs": ["lastNotNull"] }
|
||||
},
|
||||
"title": "Uptime Duration",
|
||||
"type": "stat",
|
||||
"targets": [
|
||||
{
|
||||
"expr": "time() - process_start_time_seconds{job=\"agent-monitor\", namespace=~\"$namespace\"}",
|
||||
"legendFormat": "",
|
||||
"refId": "A"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "thresholds" },
|
||||
"unit": "short",
|
||||
"thresholds": {
|
||||
"steps": [
|
||||
{ "color": "green", "value": null },
|
||||
{ "color": "yellow", "value": 2 },
|
||||
{ "color": "red", "value": 0 }
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"gridPos": { "h": 4, "w": 4, "x": 8, "y": 1 },
|
||||
"id": 3,
|
||||
"options": {
|
||||
"colorMode": "value",
|
||||
"graphMode": "area",
|
||||
"justifyMode": "center",
|
||||
"textMode": "value",
|
||||
"reduceOptions": { "calcs": ["lastNotNull"] }
|
||||
},
|
||||
"title": "Pod Count",
|
||||
"type": "stat",
|
||||
"targets": [
|
||||
{
|
||||
"expr": "count(kube_pod_status_ready{namespace=~\"$namespace\", condition=\"true\"} == 1)",
|
||||
"legendFormat": "Ready",
|
||||
"refId": "A"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "thresholds" },
|
||||
"unit": "short",
|
||||
"thresholds": {
|
||||
"steps": [
|
||||
{ "color": "green", "value": null },
|
||||
{ "color": "yellow", "value": 1 },
|
||||
{ "color": "red", "value": 3 }
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"gridPos": { "h": 4, "w": 4, "x": 12, "y": 1 },
|
||||
"id": 4,
|
||||
"options": {
|
||||
"colorMode": "value",
|
||||
"graphMode": "area",
|
||||
"justifyMode": "center",
|
||||
"textMode": "value",
|
||||
"reduceOptions": { "calcs": ["lastNotNull"] }
|
||||
},
|
||||
"title": "Pod Restarts (1h)",
|
||||
"type": "stat",
|
||||
"targets": [
|
||||
{
|
||||
"expr": "sum(increase(kube_pod_container_status_restarts_total{namespace=~\"$namespace\", container=\"agent-monitor\"}[1h]))",
|
||||
"legendFormat": "",
|
||||
"refId": "A"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "thresholds" },
|
||||
"unit": "short",
|
||||
"thresholds": { "steps": [{ "color": "blue", "value": null }] }
|
||||
}
|
||||
},
|
||||
"gridPos": { "h": 4, "w": 4, "x": 16, "y": 1 },
|
||||
"id": 5,
|
||||
"options": {
|
||||
"colorMode": "value",
|
||||
"graphMode": "area",
|
||||
"justifyMode": "center",
|
||||
"textMode": "value",
|
||||
"reduceOptions": { "calcs": ["lastNotNull"] }
|
||||
},
|
||||
"title": "Active WebSocket Connections",
|
||||
"type": "stat",
|
||||
"targets": [
|
||||
{
|
||||
"expr": "sum(websocket_connections_active{job=\"agent-monitor\", namespace=~\"$namespace\"})",
|
||||
"legendFormat": "",
|
||||
"refId": "A"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "thresholds" },
|
||||
"unit": "decbytes",
|
||||
"thresholds": {
|
||||
"steps": [
|
||||
{ "color": "green", "value": null },
|
||||
{ "color": "yellow", "value": 5368709120 },
|
||||
{ "color": "red", "value": 8589934592 }
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"gridPos": { "h": 4, "w": 4, "x": 20, "y": 1 },
|
||||
"id": 6,
|
||||
"options": {
|
||||
"colorMode": "value",
|
||||
"graphMode": "area",
|
||||
"justifyMode": "center",
|
||||
"textMode": "value",
|
||||
"reduceOptions": { "calcs": ["lastNotNull"] }
|
||||
},
|
||||
"title": "SQLite DB Size",
|
||||
"type": "stat",
|
||||
"targets": [
|
||||
{
|
||||
"expr": "kubelet_volume_stats_used_bytes{namespace=~\"$namespace\", persistentvolumeclaim=~\"agent-monitor.*\"}",
|
||||
"legendFormat": "",
|
||||
"refId": "A"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"collapsed": false,
|
||||
"gridPos": { "h": 1, "w": 24, "x": 0, "y": 5 },
|
||||
"id": 101,
|
||||
"title": "HTTP Traffic",
|
||||
"type": "row"
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "palette-classic" },
|
||||
"custom": {
|
||||
"axisLabel": "req/s",
|
||||
"drawStyle": "line",
|
||||
"fillOpacity": 15,
|
||||
"lineWidth": 2,
|
||||
"pointSize": 5,
|
||||
"spanNulls": false,
|
||||
"stacking": { "mode": "none" }
|
||||
},
|
||||
"unit": "reqps"
|
||||
},
|
||||
"overrides": [
|
||||
{
|
||||
"matcher": { "id": "byRegexp", "options": "5.." },
|
||||
"properties": [{ "id": "color", "value": { "fixedColor": "red", "mode": "fixed" } }]
|
||||
},
|
||||
{
|
||||
"matcher": { "id": "byRegexp", "options": "4.." },
|
||||
"properties": [{ "id": "color", "value": { "fixedColor": "yellow", "mode": "fixed" } }]
|
||||
},
|
||||
{
|
||||
"matcher": { "id": "byRegexp", "options": "2.." },
|
||||
"properties": [{ "id": "color", "value": { "fixedColor": "green", "mode": "fixed" } }]
|
||||
}
|
||||
]
|
||||
},
|
||||
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 6 },
|
||||
"id": 10,
|
||||
"options": {
|
||||
"legend": { "calcs": ["mean", "max"], "displayMode": "table", "placement": "bottom" },
|
||||
"tooltip": { "mode": "multi", "sort": "desc" }
|
||||
},
|
||||
"title": "Request Rate (RPS)",
|
||||
"type": "timeseries",
|
||||
"targets": [
|
||||
{
|
||||
"expr": "sum(rate(http_requests_total{job=\"agent-monitor\", namespace=~\"$namespace\"}[5m])) by (status)",
|
||||
"legendFormat": "{{ status }}",
|
||||
"refId": "A"
|
||||
},
|
||||
{
|
||||
"expr": "sum(rate(http_requests_total{job=\"agent-monitor\", namespace=~\"$namespace\"}[5m]))",
|
||||
"legendFormat": "Total",
|
||||
"refId": "B"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "palette-classic" },
|
||||
"custom": {
|
||||
"axisLabel": "%",
|
||||
"drawStyle": "line",
|
||||
"fillOpacity": 20,
|
||||
"lineWidth": 2,
|
||||
"thresholdsStyle": { "mode": "line" }
|
||||
},
|
||||
"max": 100,
|
||||
"min": 0,
|
||||
"unit": "percent",
|
||||
"thresholds": {
|
||||
"steps": [
|
||||
{ "color": "green", "value": null },
|
||||
{ "color": "yellow", "value": 2 },
|
||||
{ "color": "red", "value": 5 }
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 6 },
|
||||
"id": 11,
|
||||
"options": {
|
||||
"legend": { "calcs": ["mean", "max"], "displayMode": "table", "placement": "bottom" },
|
||||
"tooltip": { "mode": "multi" }
|
||||
},
|
||||
"title": "Error Rate (%)",
|
||||
"type": "timeseries",
|
||||
"targets": [
|
||||
{
|
||||
"expr": "100 * sum(rate(http_requests_total{job=\"agent-monitor\", namespace=~\"$namespace\", status=~\"5..\"}[5m])) / sum(rate(http_requests_total{job=\"agent-monitor\", namespace=~\"$namespace\"}[5m]))",
|
||||
"legendFormat": "5xx Error Rate",
|
||||
"refId": "A"
|
||||
},
|
||||
{
|
||||
"expr": "100 * sum(rate(http_requests_total{job=\"agent-monitor\", namespace=~\"$namespace\", status=~\"4..\"}[5m])) / sum(rate(http_requests_total{job=\"agent-monitor\", namespace=~\"$namespace\"}[5m]))",
|
||||
"legendFormat": "4xx Error Rate",
|
||||
"refId": "B"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"collapsed": false,
|
||||
"gridPos": { "h": 1, "w": 24, "x": 0, "y": 14 },
|
||||
"id": 102,
|
||||
"title": "Latency",
|
||||
"type": "row"
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "palette-classic" },
|
||||
"custom": {
|
||||
"axisLabel": "seconds",
|
||||
"drawStyle": "line",
|
||||
"fillOpacity": 10,
|
||||
"lineWidth": 2,
|
||||
"spanNulls": false
|
||||
},
|
||||
"unit": "s",
|
||||
"thresholds": {
|
||||
"steps": [
|
||||
{ "color": "green", "value": null },
|
||||
{ "color": "yellow", "value": 0.5 },
|
||||
{ "color": "red", "value": 1 }
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 15 },
|
||||
"id": 20,
|
||||
"options": {
|
||||
"legend": { "calcs": ["mean", "max"], "displayMode": "table", "placement": "bottom" },
|
||||
"tooltip": { "mode": "multi", "sort": "desc" }
|
||||
},
|
||||
"title": "Response Latency (p50 / p90 / p99)",
|
||||
"type": "timeseries",
|
||||
"targets": [
|
||||
{
|
||||
"expr": "histogram_quantile(0.50, sum(rate(http_request_duration_seconds_bucket{job=\"agent-monitor\", namespace=~\"$namespace\"}[5m])) by (le))",
|
||||
"legendFormat": "p50",
|
||||
"refId": "A"
|
||||
},
|
||||
{
|
||||
"expr": "histogram_quantile(0.90, sum(rate(http_request_duration_seconds_bucket{job=\"agent-monitor\", namespace=~\"$namespace\"}[5m])) by (le))",
|
||||
"legendFormat": "p90",
|
||||
"refId": "B"
|
||||
},
|
||||
{
|
||||
"expr": "histogram_quantile(0.99, sum(rate(http_request_duration_seconds_bucket{job=\"agent-monitor\", namespace=~\"$namespace\"}[5m])) by (le))",
|
||||
"legendFormat": "p99",
|
||||
"refId": "C"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "palette-classic" },
|
||||
"custom": {
|
||||
"drawStyle": "bars",
|
||||
"fillOpacity": 80,
|
||||
"lineWidth": 1,
|
||||
"stacking": { "mode": "normal" }
|
||||
},
|
||||
"unit": "reqps"
|
||||
}
|
||||
},
|
||||
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 15 },
|
||||
"id": 21,
|
||||
"options": {
|
||||
"legend": { "calcs": ["mean"], "displayMode": "table", "placement": "bottom" },
|
||||
"tooltip": { "mode": "multi" }
|
||||
},
|
||||
"title": "Request Rate by Endpoint",
|
||||
"type": "timeseries",
|
||||
"targets": [
|
||||
{
|
||||
"expr": "sum(rate(http_requests_total{job=\"agent-monitor\", namespace=~\"$namespace\"}[5m])) by (handler)",
|
||||
"legendFormat": "{{ handler }}",
|
||||
"refId": "A"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"collapsed": false,
|
||||
"gridPos": { "h": 1, "w": 24, "x": 0, "y": 23 },
|
||||
"id": 103,
|
||||
"title": "WebSocket & Connections",
|
||||
"type": "row"
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "palette-classic" },
|
||||
"custom": { "drawStyle": "line", "fillOpacity": 25, "lineWidth": 2, "spanNulls": false },
|
||||
"unit": "short"
|
||||
}
|
||||
},
|
||||
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 24 },
|
||||
"id": 30,
|
||||
"options": {
|
||||
"legend": {
|
||||
"calcs": ["mean", "max", "last"],
|
||||
"displayMode": "table",
|
||||
"placement": "bottom"
|
||||
},
|
||||
"tooltip": { "mode": "multi" }
|
||||
},
|
||||
"title": "Active WebSocket Connections",
|
||||
"type": "timeseries",
|
||||
"targets": [
|
||||
{
|
||||
"expr": "websocket_connections_active{job=\"agent-monitor\", namespace=~\"$namespace\"}",
|
||||
"legendFormat": "{{ pod }}",
|
||||
"refId": "A"
|
||||
},
|
||||
{
|
||||
"expr": "sum(websocket_connections_active{job=\"agent-monitor\", namespace=~\"$namespace\"})",
|
||||
"legendFormat": "Total",
|
||||
"refId": "B"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "palette-classic" },
|
||||
"custom": { "drawStyle": "line", "fillOpacity": 10, "lineWidth": 2 },
|
||||
"unit": "short"
|
||||
}
|
||||
},
|
||||
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 24 },
|
||||
"id": 31,
|
||||
"options": {
|
||||
"legend": { "calcs": ["sum"], "displayMode": "table", "placement": "bottom" },
|
||||
"tooltip": { "mode": "multi" }
|
||||
},
|
||||
"title": "WebSocket Messages (rate/s)",
|
||||
"type": "timeseries",
|
||||
"targets": [
|
||||
{
|
||||
"expr": "sum(rate(websocket_messages_sent_total{job=\"agent-monitor\", namespace=~\"$namespace\"}[5m]))",
|
||||
"legendFormat": "Sent",
|
||||
"refId": "A"
|
||||
},
|
||||
{
|
||||
"expr": "sum(rate(websocket_messages_received_total{job=\"agent-monitor\", namespace=~\"$namespace\"}[5m]))",
|
||||
"legendFormat": "Received",
|
||||
"refId": "B"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"collapsed": false,
|
||||
"gridPos": { "h": 1, "w": 24, "x": 0, "y": 32 },
|
||||
"id": 104,
|
||||
"title": "Resource Usage",
|
||||
"type": "row"
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "palette-classic" },
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"fillOpacity": 20,
|
||||
"lineWidth": 2,
|
||||
"thresholdsStyle": { "mode": "line+area" }
|
||||
},
|
||||
"unit": "bytes",
|
||||
"thresholds": {
|
||||
"steps": [
|
||||
{ "color": "transparent", "value": null },
|
||||
{ "color": "red", "value": 536870912 }
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 33 },
|
||||
"id": 40,
|
||||
"options": {
|
||||
"legend": { "calcs": ["mean", "max"], "displayMode": "table", "placement": "bottom" },
|
||||
"tooltip": { "mode": "multi" }
|
||||
},
|
||||
"title": "Memory Usage",
|
||||
"type": "timeseries",
|
||||
"targets": [
|
||||
{
|
||||
"expr": "container_memory_working_set_bytes{namespace=~\"$namespace\", container=\"agent-monitor\"}",
|
||||
"legendFormat": "{{ pod }} (working set)",
|
||||
"refId": "A"
|
||||
},
|
||||
{
|
||||
"expr": "container_spec_memory_limit_bytes{namespace=~\"$namespace\", container=\"agent-monitor\"}",
|
||||
"legendFormat": "{{ pod }} (limit)",
|
||||
"refId": "B"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "palette-classic" },
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"fillOpacity": 20,
|
||||
"lineWidth": 2,
|
||||
"thresholdsStyle": { "mode": "line" }
|
||||
},
|
||||
"unit": "percentunit",
|
||||
"max": 1,
|
||||
"min": 0,
|
||||
"thresholds": {
|
||||
"steps": [
|
||||
{ "color": "green", "value": null },
|
||||
{ "color": "yellow", "value": 0.7 },
|
||||
{ "color": "red", "value": 0.9 }
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 33 },
|
||||
"id": 41,
|
||||
"options": {
|
||||
"legend": { "calcs": ["mean", "max"], "displayMode": "table", "placement": "bottom" },
|
||||
"tooltip": { "mode": "multi" }
|
||||
},
|
||||
"title": "CPU Usage",
|
||||
"type": "timeseries",
|
||||
"targets": [
|
||||
{
|
||||
"expr": "rate(container_cpu_usage_seconds_total{namespace=~\"$namespace\", container=\"agent-monitor\"}[5m])",
|
||||
"legendFormat": "{{ pod }}",
|
||||
"refId": "A"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"collapsed": false,
|
||||
"gridPos": { "h": 1, "w": 24, "x": 0, "y": 41 },
|
||||
"id": 105,
|
||||
"title": "Storage",
|
||||
"type": "row"
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "palette-classic" },
|
||||
"custom": {
|
||||
"drawStyle": "line",
|
||||
"fillOpacity": 20,
|
||||
"lineWidth": 2,
|
||||
"thresholdsStyle": { "mode": "line+area" }
|
||||
},
|
||||
"unit": "decbytes",
|
||||
"thresholds": {
|
||||
"steps": [
|
||||
{ "color": "transparent", "value": null },
|
||||
{ "color": "yellow", "value": 8589934592 },
|
||||
{ "color": "red", "value": 9663676416 }
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 42 },
|
||||
"id": 50,
|
||||
"options": {
|
||||
"legend": { "calcs": ["last", "max"], "displayMode": "table", "placement": "bottom" },
|
||||
"tooltip": { "mode": "multi" }
|
||||
},
|
||||
"title": "SQLite DB Size (PV Usage)",
|
||||
"type": "timeseries",
|
||||
"targets": [
|
||||
{
|
||||
"expr": "kubelet_volume_stats_used_bytes{namespace=~\"$namespace\", persistentvolumeclaim=~\"agent-monitor.*\"}",
|
||||
"legendFormat": "Used ({{ persistentvolumeclaim }})",
|
||||
"refId": "A"
|
||||
},
|
||||
{
|
||||
"expr": "kubelet_volume_stats_capacity_bytes{namespace=~\"$namespace\", persistentvolumeclaim=~\"agent-monitor.*\"}",
|
||||
"legendFormat": "Capacity ({{ persistentvolumeclaim }})",
|
||||
"refId": "B"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "thresholds" },
|
||||
"max": 100,
|
||||
"min": 0,
|
||||
"unit": "percent",
|
||||
"thresholds": {
|
||||
"steps": [
|
||||
{ "color": "green", "value": null },
|
||||
{ "color": "yellow", "value": 75 },
|
||||
{ "color": "red", "value": 90 }
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 42 },
|
||||
"id": 51,
|
||||
"options": {
|
||||
"orientation": "horizontal",
|
||||
"reduceOptions": { "calcs": ["lastNotNull"] },
|
||||
"showThresholdLabels": false,
|
||||
"showThresholdMarkers": true
|
||||
},
|
||||
"title": "PV Usage (%)",
|
||||
"type": "gauge",
|
||||
"targets": [
|
||||
{
|
||||
"expr": "100 * kubelet_volume_stats_used_bytes{namespace=~\"$namespace\", persistentvolumeclaim=~\"agent-monitor.*\"} / kubelet_volume_stats_capacity_bytes{namespace=~\"$namespace\", persistentvolumeclaim=~\"agent-monitor.*\"}",
|
||||
"legendFormat": "{{ persistentvolumeclaim }}",
|
||||
"refId": "A"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"refresh": "30s",
|
||||
"schemaVersion": 39,
|
||||
"tags": ["agent-monitor", "nodejs", "websocket"],
|
||||
"templating": {
|
||||
"list": [
|
||||
{
|
||||
"current": { "selected": false, "text": "Prometheus", "value": "prometheus" },
|
||||
"hide": 0,
|
||||
"includeAll": false,
|
||||
"label": "Data Source",
|
||||
"name": "datasource",
|
||||
"options": [],
|
||||
"query": "prometheus",
|
||||
"type": "datasource"
|
||||
},
|
||||
{
|
||||
"current": {},
|
||||
"datasource": { "type": "prometheus", "uid": "${datasource}" },
|
||||
"definition": "label_values(up{job=\"agent-monitor\"}, namespace)",
|
||||
"hide": 0,
|
||||
"includeAll": true,
|
||||
"label": "Namespace",
|
||||
"multi": false,
|
||||
"name": "namespace",
|
||||
"query": { "qryType": 1, "query": "label_values(up{job=\"agent-monitor\"}, namespace)" },
|
||||
"refresh": 2,
|
||||
"type": "query"
|
||||
}
|
||||
]
|
||||
},
|
||||
"time": { "from": "now-6h", "to": "now" },
|
||||
"timepicker": {
|
||||
"refresh_intervals": ["10s", "30s", "1m", "5m", "15m"],
|
||||
"time_options": ["5m", "15m", "1h", "6h", "12h", "24h", "2d", "7d", "30d"]
|
||||
},
|
||||
"timezone": "browser",
|
||||
"title": "Claude Code Agent Monitor",
|
||||
"uid": "agent-monitor-main",
|
||||
"version": 1
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Grafana datasource provisioning for Claude Code Agent Monitor
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
apiVersion: 1
|
||||
|
||||
# Prevent users from deleting provisioned data sources in the UI
|
||||
deleteDatasources:
|
||||
- name: Prometheus
|
||||
orgId: 1
|
||||
|
||||
datasources:
|
||||
# ── Primary Prometheus datasource ─────────────────────────────────────────
|
||||
- name: Prometheus
|
||||
type: prometheus
|
||||
uid: prometheus
|
||||
access: proxy
|
||||
url: http://prometheus-server:9090
|
||||
isDefault: true
|
||||
editable: false
|
||||
jsonData:
|
||||
# Scrape interval matches prometheus.yaml global setting
|
||||
timeInterval: "15s"
|
||||
# Query timeout
|
||||
queryTimeout: "30s"
|
||||
# HTTP method for queries
|
||||
httpMethod: POST
|
||||
# Manage alerts via Prometheus
|
||||
manageAlerts: true
|
||||
# Alerting rule settings
|
||||
prometheusType: Prometheus
|
||||
prometheusVersion: ">=2.45.0"
|
||||
# Incremental querying for better performance
|
||||
incrementalQuerying: true
|
||||
incrementalQueryOverlapWindow: "10m"
|
||||
# Exemplar trace support (uncomment if using tracing)
|
||||
# exemplarTraceIdDestinations:
|
||||
# - name: traceId
|
||||
# datasourceUid: tempo
|
||||
version: 1
|
||||
|
||||
# ── Alertmanager datasource ───────────────────────────────────────────────
|
||||
- name: Alertmanager
|
||||
type: alertmanager
|
||||
uid: alertmanager
|
||||
access: proxy
|
||||
url: http://alertmanager:9093
|
||||
editable: false
|
||||
jsonData:
|
||||
implementation: prometheus
|
||||
handleGrafanaManagedAlerts: false
|
||||
version: 1
|
||||
@@ -0,0 +1,226 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Prometheus configuration for Claude Code Agent Monitor
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
global:
|
||||
scrape_interval: 15s
|
||||
scrape_timeout: 10s
|
||||
evaluation_interval: 15s
|
||||
|
||||
external_labels:
|
||||
cluster: "${CLUSTER_NAME:agent-monitor}"
|
||||
environment: "${ENVIRONMENT:production}"
|
||||
|
||||
# ── Rule files ──────────────────────────────────────────────────────────────
|
||||
rule_files:
|
||||
- /etc/prometheus/rules/*.rules.yaml
|
||||
|
||||
# ── Alertmanager ────────────────────────────────────────────────────────────
|
||||
alerting:
|
||||
alertmanagers:
|
||||
- static_configs:
|
||||
- targets:
|
||||
- alertmanager:9093
|
||||
scheme: http
|
||||
timeout: 10s
|
||||
api_version: v2
|
||||
|
||||
# ── Scrape configs ──────────────────────────────────────────────────────────
|
||||
scrape_configs:
|
||||
# ── Agent Monitor application ─────────────────────────────────────────────
|
||||
- job_name: "agent-monitor"
|
||||
metrics_path: /api/health
|
||||
scrape_interval: 15s
|
||||
scrape_timeout: 5s
|
||||
scheme: http
|
||||
|
||||
# Static target for standalone deployments
|
||||
static_configs:
|
||||
- targets:
|
||||
- "agent-monitor:4820"
|
||||
labels:
|
||||
app: agent-monitor
|
||||
component: server
|
||||
|
||||
# Relabeling to add standard labels
|
||||
relabel_configs:
|
||||
- source_labels: [__address__]
|
||||
target_label: instance
|
||||
- target_label: __metrics_path__
|
||||
replacement: /api/health
|
||||
|
||||
metric_relabel_configs:
|
||||
- source_labels: [__name__]
|
||||
regex: "go_.*"
|
||||
action: drop
|
||||
|
||||
# ── Agent Monitor MCP sidecar ─────────────────────────────────────────────
|
||||
- job_name: "agent-monitor-mcp"
|
||||
scrape_interval: 30s
|
||||
static_configs:
|
||||
- targets:
|
||||
- "agent-monitor-mcp:8819"
|
||||
labels:
|
||||
app: agent-monitor
|
||||
component: mcp
|
||||
|
||||
# ── Kubernetes service discovery (pods) ───────────────────────────────────
|
||||
- job_name: "kubernetes-pods"
|
||||
scrape_interval: 15s
|
||||
|
||||
kubernetes_sd_configs:
|
||||
- role: pod
|
||||
namespaces:
|
||||
own_namespace: false
|
||||
names:
|
||||
- agent-monitor
|
||||
- agent-monitor-dev
|
||||
- agent-monitor-staging
|
||||
- agent-monitor-production
|
||||
|
||||
relabel_configs:
|
||||
# Only scrape pods with annotation prometheus.io/scrape=true
|
||||
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
|
||||
action: keep
|
||||
regex: true
|
||||
|
||||
# Use custom metrics path if annotated
|
||||
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
|
||||
action: replace
|
||||
target_label: __metrics_path__
|
||||
regex: (.+)
|
||||
|
||||
# Use custom port if annotated
|
||||
- source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
|
||||
action: replace
|
||||
regex: ([^:]+)(?::\d+)?;(\d+)
|
||||
replacement: $1:$2
|
||||
target_label: __address__
|
||||
|
||||
# Use custom scheme if annotated
|
||||
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scheme]
|
||||
action: replace
|
||||
target_label: __scheme__
|
||||
regex: (.+)
|
||||
|
||||
# Map pod labels to Prometheus labels
|
||||
- action: labelmap
|
||||
regex: __meta_kubernetes_pod_label_(.+)
|
||||
|
||||
# Add namespace label
|
||||
- source_labels: [__meta_kubernetes_namespace]
|
||||
action: replace
|
||||
target_label: namespace
|
||||
|
||||
# Add pod name label
|
||||
- source_labels: [__meta_kubernetes_pod_name]
|
||||
action: replace
|
||||
target_label: pod
|
||||
|
||||
# Add node name label
|
||||
- source_labels: [__meta_kubernetes_pod_node_name]
|
||||
action: replace
|
||||
target_label: node
|
||||
|
||||
# ── Kubernetes service discovery (services) ──────────────────────────────
|
||||
- job_name: "kubernetes-services"
|
||||
scrape_interval: 15s
|
||||
|
||||
kubernetes_sd_configs:
|
||||
- role: service
|
||||
namespaces:
|
||||
names:
|
||||
- agent-monitor
|
||||
- agent-monitor-dev
|
||||
- agent-monitor-staging
|
||||
- agent-monitor-production
|
||||
|
||||
relabel_configs:
|
||||
- source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape]
|
||||
action: keep
|
||||
regex: true
|
||||
|
||||
- source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path]
|
||||
action: replace
|
||||
target_label: __metrics_path__
|
||||
regex: (.+)
|
||||
|
||||
- source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port]
|
||||
action: replace
|
||||
regex: ([^:]+)(?::\d+)?;(\d+)
|
||||
replacement: $1:$2
|
||||
target_label: __address__
|
||||
|
||||
- action: labelmap
|
||||
regex: __meta_kubernetes_service_label_(.+)
|
||||
|
||||
- source_labels: [__meta_kubernetes_namespace]
|
||||
action: replace
|
||||
target_label: namespace
|
||||
|
||||
- source_labels: [__meta_kubernetes_service_name]
|
||||
action: replace
|
||||
target_label: service
|
||||
|
||||
# ── Kubernetes nodes ──────────────────────────────────────────────────────
|
||||
- job_name: "kubernetes-nodes"
|
||||
scrape_interval: 30s
|
||||
scheme: https
|
||||
|
||||
kubernetes_sd_configs:
|
||||
- role: node
|
||||
|
||||
tls_config:
|
||||
ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
|
||||
insecure_skip_verify: false
|
||||
|
||||
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
|
||||
|
||||
relabel_configs:
|
||||
- action: labelmap
|
||||
regex: __meta_kubernetes_node_label_(.+)
|
||||
|
||||
- target_label: __address__
|
||||
replacement: kubernetes.default.svc:443
|
||||
|
||||
- source_labels: [__meta_kubernetes_node_name]
|
||||
regex: (.+)
|
||||
target_label: __metrics_path__
|
||||
replacement: /api/v1/nodes/$1/proxy/metrics
|
||||
|
||||
# ── Kubernetes cadvisor (container metrics) ───────────────────────────────
|
||||
- job_name: "kubernetes-cadvisor"
|
||||
scrape_interval: 15s
|
||||
scheme: https
|
||||
|
||||
kubernetes_sd_configs:
|
||||
- role: node
|
||||
|
||||
tls_config:
|
||||
ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
|
||||
insecure_skip_verify: false
|
||||
|
||||
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
|
||||
|
||||
relabel_configs:
|
||||
- action: labelmap
|
||||
regex: __meta_kubernetes_node_label_(.+)
|
||||
|
||||
- target_label: __address__
|
||||
replacement: kubernetes.default.svc:443
|
||||
|
||||
- source_labels: [__meta_kubernetes_node_name]
|
||||
regex: (.+)
|
||||
target_label: __metrics_path__
|
||||
replacement: /api/v1/nodes/$1/proxy/metrics/cadvisor
|
||||
|
||||
metric_relabel_configs:
|
||||
# Keep only container metrics for agent-monitor pods
|
||||
- source_labels: [container]
|
||||
regex: "agent-monitor.*"
|
||||
action: keep
|
||||
|
||||
# ── Prometheus self-monitoring ────────────────────────────────────────────
|
||||
- job_name: "prometheus"
|
||||
static_configs:
|
||||
- targets:
|
||||
- "localhost:9090"
|
||||
@@ -0,0 +1,310 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Prometheus alerting rules for Claude Code Agent Monitor
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
apiVersion: monitoring.coreos.com/v1
|
||||
kind: PrometheusRule
|
||||
metadata:
|
||||
name: agent-monitor-alerts
|
||||
namespace: agent-monitor
|
||||
labels:
|
||||
app.kubernetes.io/name: agent-monitor
|
||||
app.kubernetes.io/component: monitoring
|
||||
prometheus: kube-prometheus
|
||||
role: alert-rules
|
||||
spec:
|
||||
groups:
|
||||
# ── Availability alerts ─────────────────────────────────────────────────
|
||||
- name: agent-monitor.availability
|
||||
rules:
|
||||
- alert: AgentMonitorDown
|
||||
expr: up{job="agent-monitor"} == 0
|
||||
for: 2m
|
||||
labels:
|
||||
severity: critical
|
||||
service: agent-monitor
|
||||
team: platform
|
||||
annotations:
|
||||
summary: "Agent Monitor is down"
|
||||
description: >-
|
||||
The Agent Monitor instance {{ $labels.instance }} has been
|
||||
unreachable for more than 2 minutes. Immediate investigation
|
||||
required.
|
||||
runbook_url: "https://wiki.example.com/runbooks/agent-monitor-down"
|
||||
dashboard_url: "https://grafana.example.com/d/agent-monitor"
|
||||
|
||||
- alert: AgentMonitorHighAvailabilityDegraded
|
||||
expr: |
|
||||
count(up{job="agent-monitor"} == 1)
|
||||
< count(up{job="agent-monitor"})
|
||||
for: 5m
|
||||
labels:
|
||||
severity: warning
|
||||
service: agent-monitor
|
||||
annotations:
|
||||
summary: "Agent Monitor partial outage"
|
||||
description: >-
|
||||
Not all Agent Monitor replicas are healthy. Only
|
||||
{{ $value }} of expected replicas are up.
|
||||
|
||||
# ── Error rate alerts ───────────────────────────────────────────────────
|
||||
- name: agent-monitor.errors
|
||||
rules:
|
||||
- alert: HighErrorRate
|
||||
expr: |
|
||||
(
|
||||
sum(rate(http_requests_total{job="agent-monitor", status=~"5.."}[5m]))
|
||||
/
|
||||
sum(rate(http_requests_total{job="agent-monitor"}[5m]))
|
||||
) > 0.05
|
||||
for: 5m
|
||||
labels:
|
||||
severity: warning
|
||||
service: agent-monitor
|
||||
annotations:
|
||||
summary: "High HTTP error rate (>5%)"
|
||||
description: >-
|
||||
The Agent Monitor is returning 5xx errors at a rate of
|
||||
{{ $value | humanizePercentage }} over the last 5 minutes.
|
||||
dashboard_url: "https://grafana.example.com/d/agent-monitor?tab=errors"
|
||||
|
||||
- alert: CriticalErrorRate
|
||||
expr: |
|
||||
(
|
||||
sum(rate(http_requests_total{job="agent-monitor", status=~"5.."}[5m]))
|
||||
/
|
||||
sum(rate(http_requests_total{job="agent-monitor"}[5m]))
|
||||
) > 0.25
|
||||
for: 2m
|
||||
labels:
|
||||
severity: critical
|
||||
service: agent-monitor
|
||||
annotations:
|
||||
summary: "Critical HTTP error rate (>25%)"
|
||||
description: >-
|
||||
The Agent Monitor is returning 5xx errors at a rate of
|
||||
{{ $value | humanizePercentage }}. Service may be severely degraded.
|
||||
|
||||
# ── Latency alerts ──────────────────────────────────────────────────────
|
||||
- name: agent-monitor.latency
|
||||
rules:
|
||||
- alert: HighLatency
|
||||
expr: |
|
||||
histogram_quantile(0.99,
|
||||
sum(rate(http_request_duration_seconds_bucket{job="agent-monitor"}[5m])) by (le)
|
||||
) > 1
|
||||
for: 5m
|
||||
labels:
|
||||
severity: warning
|
||||
service: agent-monitor
|
||||
annotations:
|
||||
summary: "High p99 response latency (>1s)"
|
||||
description: >-
|
||||
The 99th percentile response time for Agent Monitor is
|
||||
{{ $value | humanizeDuration }}. Users may experience slow page loads.
|
||||
|
||||
- alert: HighMedianLatency
|
||||
expr: |
|
||||
histogram_quantile(0.50,
|
||||
sum(rate(http_request_duration_seconds_bucket{job="agent-monitor"}[5m])) by (le)
|
||||
) > 0.5
|
||||
for: 5m
|
||||
labels:
|
||||
severity: warning
|
||||
service: agent-monitor
|
||||
annotations:
|
||||
summary: "High median response latency (>500ms)"
|
||||
description: >-
|
||||
The median response time is {{ $value | humanizeDuration }}.
|
||||
This affects the majority of requests.
|
||||
|
||||
# ── Resource alerts ─────────────────────────────────────────────────────
|
||||
- name: agent-monitor.resources
|
||||
rules:
|
||||
- alert: HighMemoryUsage
|
||||
expr: |
|
||||
(
|
||||
container_memory_working_set_bytes{
|
||||
namespace=~"agent-monitor.*",
|
||||
container="agent-monitor"
|
||||
}
|
||||
/
|
||||
container_spec_memory_limit_bytes{
|
||||
namespace=~"agent-monitor.*",
|
||||
container="agent-monitor"
|
||||
}
|
||||
) > 0.90
|
||||
for: 10m
|
||||
labels:
|
||||
severity: warning
|
||||
service: agent-monitor
|
||||
annotations:
|
||||
summary: "High memory usage (>90% of limit)"
|
||||
description: >-
|
||||
Pod {{ $labels.pod }} is using {{ $value | humanizePercentage }}
|
||||
of its memory limit. OOMKill risk is elevated.
|
||||
|
||||
- alert: HighCPUUsage
|
||||
expr: |
|
||||
(
|
||||
rate(container_cpu_usage_seconds_total{
|
||||
namespace=~"agent-monitor.*",
|
||||
container="agent-monitor"
|
||||
}[5m])
|
||||
/
|
||||
container_spec_cpu_quota{
|
||||
namespace=~"agent-monitor.*",
|
||||
container="agent-monitor"
|
||||
}
|
||||
* 100000
|
||||
) > 0.85
|
||||
for: 10m
|
||||
labels:
|
||||
severity: warning
|
||||
service: agent-monitor
|
||||
annotations:
|
||||
summary: "High CPU usage (>85% of limit)"
|
||||
description: >-
|
||||
Pod {{ $labels.pod }} is using {{ $value | humanizePercentage }}
|
||||
of its CPU limit. Consider scaling up.
|
||||
|
||||
- alert: PersistentVolumeNearlyFull
|
||||
expr: |
|
||||
(
|
||||
kubelet_volume_stats_used_bytes{
|
||||
namespace=~"agent-monitor.*",
|
||||
persistentvolumeclaim=~"agent-monitor.*"
|
||||
}
|
||||
/
|
||||
kubelet_volume_stats_capacity_bytes{
|
||||
namespace=~"agent-monitor.*",
|
||||
persistentvolumeclaim=~"agent-monitor.*"
|
||||
}
|
||||
) > 0.85
|
||||
for: 15m
|
||||
labels:
|
||||
severity: warning
|
||||
service: agent-monitor
|
||||
annotations:
|
||||
summary: "Persistent volume nearly full (>85%)"
|
||||
description: >-
|
||||
PVC {{ $labels.persistentvolumeclaim }} in namespace
|
||||
{{ $labels.namespace }} is {{ $value | humanizePercentage }}
|
||||
full. SQLite writes may fail when volume is exhausted.
|
||||
|
||||
- alert: PersistentVolumeCriticallyFull
|
||||
expr: |
|
||||
(
|
||||
kubelet_volume_stats_used_bytes{
|
||||
namespace=~"agent-monitor.*",
|
||||
persistentvolumeclaim=~"agent-monitor.*"
|
||||
}
|
||||
/
|
||||
kubelet_volume_stats_capacity_bytes{
|
||||
namespace=~"agent-monitor.*",
|
||||
persistentvolumeclaim=~"agent-monitor.*"
|
||||
}
|
||||
) > 0.95
|
||||
for: 5m
|
||||
labels:
|
||||
severity: critical
|
||||
service: agent-monitor
|
||||
annotations:
|
||||
summary: "Persistent volume critically full (>95%)"
|
||||
description: >-
|
||||
PVC {{ $labels.persistentvolumeclaim }} is at
|
||||
{{ $value | humanizePercentage }} capacity.
|
||||
Immediate action required to prevent data loss.
|
||||
|
||||
# ── WebSocket alerts ────────────────────────────────────────────────────
|
||||
- name: agent-monitor.websocket
|
||||
rules:
|
||||
- alert: WebSocketConnectionsDrop
|
||||
expr: |
|
||||
(
|
||||
max_over_time(websocket_connections_active{job="agent-monitor"}[10m])
|
||||
- websocket_connections_active{job="agent-monitor"}
|
||||
)
|
||||
/
|
||||
max_over_time(websocket_connections_active{job="agent-monitor"}[10m])
|
||||
> 0.50
|
||||
for: 5m
|
||||
labels:
|
||||
severity: warning
|
||||
service: agent-monitor
|
||||
annotations:
|
||||
summary: "WebSocket connections dropped >50%"
|
||||
description: >-
|
||||
Active WebSocket connections have dropped by more than 50%
|
||||
in the last 5 minutes (from
|
||||
{{ with printf `max_over_time(websocket_connections_active{instance="%s"}[10m])` .Labels.instance | query }}{{ . | first | value }}{{ end }}
|
||||
to {{ $value }}). This may indicate connectivity issues.
|
||||
|
||||
- alert: NoWebSocketConnections
|
||||
expr: |
|
||||
websocket_connections_active{job="agent-monitor"} == 0
|
||||
and on() hour() >= 8 <= 20
|
||||
for: 15m
|
||||
labels:
|
||||
severity: info
|
||||
service: agent-monitor
|
||||
annotations:
|
||||
summary: "No active WebSocket connections during business hours"
|
||||
description: >-
|
||||
There are no active WebSocket connections to Agent Monitor
|
||||
during expected business hours.
|
||||
|
||||
# ── Pod stability alerts ────────────────────────────────────────────────
|
||||
- name: agent-monitor.stability
|
||||
rules:
|
||||
- alert: PodRestarting
|
||||
expr: |
|
||||
increase(
|
||||
kube_pod_container_status_restarts_total{
|
||||
namespace=~"agent-monitor.*",
|
||||
container="agent-monitor"
|
||||
}[15m]
|
||||
) > 3
|
||||
for: 0m
|
||||
labels:
|
||||
severity: critical
|
||||
service: agent-monitor
|
||||
annotations:
|
||||
summary: "Pod restarting frequently (>3 in 15m)"
|
||||
description: >-
|
||||
Pod {{ $labels.pod }} in namespace {{ $labels.namespace }}
|
||||
has restarted {{ $value }} times in the last 15 minutes.
|
||||
Check logs: kubectl logs {{ $labels.pod }} -n {{ $labels.namespace }} --previous
|
||||
|
||||
- alert: PodNotReady
|
||||
expr: |
|
||||
kube_pod_status_ready{
|
||||
namespace=~"agent-monitor.*",
|
||||
condition="true"
|
||||
} == 0
|
||||
for: 5m
|
||||
labels:
|
||||
severity: warning
|
||||
service: agent-monitor
|
||||
annotations:
|
||||
summary: "Pod not ready for >5 minutes"
|
||||
description: >-
|
||||
Pod {{ $labels.pod }} has been in a not-ready state for
|
||||
more than 5 minutes.
|
||||
|
||||
- alert: DeploymentReplicasMismatch
|
||||
expr: |
|
||||
kube_deployment_spec_replicas{namespace=~"agent-monitor.*"}
|
||||
!=
|
||||
kube_deployment_status_ready_replicas{namespace=~"agent-monitor.*"}
|
||||
for: 10m
|
||||
labels:
|
||||
severity: warning
|
||||
service: agent-monitor
|
||||
annotations:
|
||||
summary: "Deployment replicas mismatch"
|
||||
description: >-
|
||||
Deployment {{ $labels.deployment }} has
|
||||
{{ with printf `kube_deployment_status_ready_replicas{deployment="%s",namespace="%s"}` .Labels.deployment .Labels.namespace | query }}{{ . | first | value }}{{ end }}
|
||||
ready replicas but
|
||||
{{ with printf `kube_deployment_spec_replicas{deployment="%s",namespace="%s"}` .Labels.deployment .Labels.namespace | query }}{{ . | first | value }}{{ end }}
|
||||
desired.
|
||||
Executable
+294
@@ -0,0 +1,294 @@
|
||||
#!/usr/bin/env bash
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# blue-green-switch.sh – Switch traffic between blue/green deployment slots
|
||||
#
|
||||
# Usage:
|
||||
# ./blue-green-switch.sh --env production --target green
|
||||
# ./blue-green-switch.sh --env production --target blue --skip-health
|
||||
# ./blue-green-switch.sh --help
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# @author Nguyễn Ngọc Trí Vĩ <vinnt@smartgift.vn>
|
||||
set -euo pipefail
|
||||
|
||||
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
readonly APP_NAME="agent-monitor"
|
||||
readonly APP_PORT=4820
|
||||
|
||||
# ── Colors & logging ───────────────────────────────────────────────────────
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
CYAN='\033[0;36m'
|
||||
BOLD='\033[1m'
|
||||
NC='\033[0m'
|
||||
|
||||
log() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} $*"; }
|
||||
info() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${BLUE}ℹ${NC} $*"; }
|
||||
ok() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${GREEN}✔${NC} $*"; }
|
||||
warn() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${YELLOW}⚠${NC} $*" >&2; }
|
||||
err() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${RED}✖${NC} $*" >&2; }
|
||||
fatal() { err "$@"; exit 1; }
|
||||
|
||||
# ── Defaults ────────────────────────────────────────────────────────────────
|
||||
ENVIRONMENT=""
|
||||
TARGET=""
|
||||
NAMESPACE=""
|
||||
SERVICE_NAME="${APP_NAME}"
|
||||
SKIP_HEALTH_CHECK=false
|
||||
DRY_RUN=false
|
||||
|
||||
# ── Usage ───────────────────────────────────────────────────────────────────
|
||||
usage() {
|
||||
cat <<EOF
|
||||
${BOLD}Usage:${NC}
|
||||
$(basename "$0") --env <environment> --target <blue|green> [options]
|
||||
|
||||
${BOLD}Required:${NC}
|
||||
--env, -e Environment: dev, staging, production
|
||||
--target, -t Target slot: blue, green
|
||||
|
||||
${BOLD}Options:${NC}
|
||||
--namespace, -n Kubernetes namespace (default: agent-monitor-<env>)
|
||||
--service Service name (default: ${APP_NAME})
|
||||
--skip-health Skip health check on target before switching
|
||||
--dry-run Show what would change without applying
|
||||
--help, -h Show this help message
|
||||
|
||||
${BOLD}Examples:${NC}
|
||||
$(basename "$0") --env production --target green
|
||||
$(basename "$0") --env production --target blue # instant rollback
|
||||
|
||||
EOF
|
||||
exit 0
|
||||
}
|
||||
|
||||
# ── Argument parsing ────────────────────────────────────────────────────────
|
||||
parse_args() {
|
||||
[[ $# -eq 0 ]] && usage
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--env|-e) ENVIRONMENT="$2"; shift 2 ;;
|
||||
--target|-t) TARGET="$2"; shift 2 ;;
|
||||
--namespace|-n) NAMESPACE="$2"; shift 2 ;;
|
||||
--service) SERVICE_NAME="$2"; shift 2 ;;
|
||||
--skip-health) SKIP_HEALTH_CHECK=true; shift ;;
|
||||
--dry-run) DRY_RUN=true; shift ;;
|
||||
--help|-h) usage ;;
|
||||
*) fatal "Unknown option: $1" ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -z "$ENVIRONMENT" ]] && fatal "Missing required argument: --env"
|
||||
[[ -z "$TARGET" ]] && fatal "Missing required argument: --target"
|
||||
[[ -z "$NAMESPACE" ]] && NAMESPACE="agent-monitor-${ENVIRONMENT}"
|
||||
|
||||
case "$TARGET" in
|
||||
blue|green) ;;
|
||||
*) fatal "Invalid target: $TARGET. Must be 'blue' or 'green'." ;;
|
||||
esac
|
||||
}
|
||||
|
||||
# ── Detect current active slot ──────────────────────────────────────────────
|
||||
detect_current_slot() {
|
||||
local current
|
||||
current=$(kubectl get svc "${SERVICE_NAME}" -n "${NAMESPACE}" \
|
||||
-o jsonpath='{.spec.selector.slot}' 2>/dev/null || echo "")
|
||||
|
||||
if [[ -z "$current" ]]; then
|
||||
# Try alternative label names
|
||||
current=$(kubectl get svc "${SERVICE_NAME}" -n "${NAMESPACE}" \
|
||||
-o jsonpath='{.spec.selector.deployment-slot}' 2>/dev/null || echo "")
|
||||
fi
|
||||
|
||||
if [[ -z "$current" ]]; then
|
||||
current=$(kubectl get svc "${SERVICE_NAME}" -n "${NAMESPACE}" \
|
||||
-o jsonpath='{.spec.selector.color}' 2>/dev/null || echo "unknown")
|
||||
fi
|
||||
|
||||
echo "$current"
|
||||
}
|
||||
|
||||
# ── Check target slot is healthy ────────────────────────────────────────────
|
||||
check_target_health() {
|
||||
if [[ "$SKIP_HEALTH_CHECK" == true ]]; then
|
||||
info "Skipping target health check (--skip-health)"
|
||||
return 0
|
||||
fi
|
||||
|
||||
info "Checking health of ${BOLD}${TARGET}${NC} slot..."
|
||||
|
||||
# Verify pods exist and are ready
|
||||
local ready_pods
|
||||
ready_pods=$(kubectl get pods -n "${NAMESPACE}" \
|
||||
-l "app.kubernetes.io/name=${APP_NAME},slot=${TARGET}" \
|
||||
--field-selector=status.phase=Running \
|
||||
-o jsonpath='{.items[*].metadata.name}' 2>/dev/null || echo "")
|
||||
|
||||
if [[ -z "$ready_pods" ]]; then
|
||||
# Try with 'color' label
|
||||
ready_pods=$(kubectl get pods -n "${NAMESPACE}" \
|
||||
-l "app.kubernetes.io/name=${APP_NAME},color=${TARGET}" \
|
||||
--field-selector=status.phase=Running \
|
||||
-o jsonpath='{.items[*].metadata.name}' 2>/dev/null || echo "")
|
||||
fi
|
||||
|
||||
if [[ -z "$ready_pods" ]]; then
|
||||
# Try with 'deployment-slot' label
|
||||
ready_pods=$(kubectl get pods -n "${NAMESPACE}" \
|
||||
-l "app.kubernetes.io/name=${APP_NAME},deployment-slot=${TARGET}" \
|
||||
--field-selector=status.phase=Running \
|
||||
-o jsonpath='{.items[*].metadata.name}' 2>/dev/null || echo "")
|
||||
fi
|
||||
|
||||
if [[ -z "$ready_pods" ]]; then
|
||||
fatal "No running pods found for ${TARGET} slot. Deploy first."
|
||||
fi
|
||||
|
||||
info "Found running pods in ${TARGET} slot: ${ready_pods}"
|
||||
|
||||
# Health check via port-forward to first pod
|
||||
local first_pod
|
||||
first_pod=$(echo "$ready_pods" | awk '{print $1}')
|
||||
local local_port=14821
|
||||
|
||||
kubectl port-forward "pod/${first_pod}" "${local_port}:${APP_PORT}" -n "${NAMESPACE}" &
|
||||
local pf_pid=$!
|
||||
sleep 3
|
||||
|
||||
local healthy=false
|
||||
for attempt in 1 2 3 4 5; do
|
||||
if curl -sf --max-time 5 "http://localhost:${local_port}/api/health" | grep -q '"status":"ok"'; then
|
||||
healthy=true
|
||||
break
|
||||
fi
|
||||
info "Health check attempt ${attempt}/5..."
|
||||
sleep 2
|
||||
done
|
||||
|
||||
kill "$pf_pid" 2>/dev/null || true
|
||||
|
||||
if [[ "$healthy" != true ]]; then
|
||||
fatal "${TARGET} slot is NOT healthy. Aborting traffic switch."
|
||||
fi
|
||||
|
||||
ok "${TARGET} slot is healthy"
|
||||
}
|
||||
|
||||
# ── Switch traffic ──────────────────────────────────────────────────────────
|
||||
switch_traffic() {
|
||||
local current_slot
|
||||
current_slot=$(detect_current_slot)
|
||||
|
||||
info "Current active slot: ${BOLD}${current_slot}${NC}"
|
||||
info "Switching to: ${BOLD}${TARGET}${NC}"
|
||||
|
||||
if [[ "$current_slot" == "$TARGET" ]]; then
|
||||
warn "Traffic is already pointing to ${TARGET}. Nothing to do."
|
||||
return 0
|
||||
fi
|
||||
|
||||
if [[ "$ENVIRONMENT" == "production" ]] && [[ "$DRY_RUN" == false ]]; then
|
||||
echo ""
|
||||
warn "Switching ${BOLD}PRODUCTION${NC} traffic from ${current_slot} → ${TARGET}"
|
||||
read -r -p "$(echo -e "${YELLOW}Type 'yes' to confirm:${NC} ")" confirm
|
||||
[[ "$confirm" == "yes" ]] || fatal "Switch cancelled."
|
||||
fi
|
||||
|
||||
# Build the patch – try common label conventions
|
||||
local label_key="slot"
|
||||
local current_labels
|
||||
current_labels=$(kubectl get svc "${SERVICE_NAME}" -n "${NAMESPACE}" -o json 2>/dev/null)
|
||||
|
||||
if echo "$current_labels" | grep -q '"color"'; then
|
||||
label_key="color"
|
||||
elif echo "$current_labels" | grep -q '"deployment-slot"'; then
|
||||
label_key="deployment-slot"
|
||||
fi
|
||||
|
||||
local patch="{\"spec\":{\"selector\":{\"${label_key}\":\"${TARGET}\"}}}"
|
||||
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
info "[DRY-RUN] Would patch service '${SERVICE_NAME}' with:"
|
||||
echo " ${patch}"
|
||||
return
|
||||
fi
|
||||
|
||||
if ! kubectl patch svc "${SERVICE_NAME}" -n "${NAMESPACE}" -p "${patch}"; then
|
||||
fatal "Failed to patch service selector!"
|
||||
fi
|
||||
|
||||
ok "Service '${SERVICE_NAME}' now routing to ${TARGET} slot"
|
||||
|
||||
# Verify the switch
|
||||
local new_slot
|
||||
new_slot=$(detect_current_slot)
|
||||
if [[ "$new_slot" != "$TARGET" ]]; then
|
||||
err "Verification failed! Service selector shows: ${new_slot}"
|
||||
warn "Attempting to revert to ${current_slot}..."
|
||||
kubectl patch svc "${SERVICE_NAME}" -n "${NAMESPACE}" \
|
||||
-p "{\"spec\":{\"selector\":{\"${label_key}\":\"${current_slot}\"}}}" \
|
||||
&& ok "Reverted to ${current_slot}" \
|
||||
|| fatal "Revert failed! Manual intervention needed."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
ok "Verified: traffic now routes to ${TARGET}"
|
||||
}
|
||||
|
||||
# ── Post-switch health check ───────────────────────────────────────────────
|
||||
post_switch_health() {
|
||||
if [[ "$DRY_RUN" == true ]] || [[ "$SKIP_HEALTH_CHECK" == true ]]; then
|
||||
return
|
||||
fi
|
||||
|
||||
info "Running post-switch health check via service..."
|
||||
sleep 5 # Let connections drain
|
||||
|
||||
if [[ -x "${SCRIPT_DIR}/health-check.sh" ]]; then
|
||||
local local_port=14822
|
||||
kubectl port-forward "svc/${SERVICE_NAME}" "${local_port}:${APP_PORT}" -n "${NAMESPACE}" &
|
||||
local pf_pid=$!
|
||||
sleep 3
|
||||
|
||||
if "${SCRIPT_DIR}/health-check.sh" --url "http://localhost:${local_port}" --retries 5 --interval 3; then
|
||||
ok "Post-switch health check passed"
|
||||
else
|
||||
warn "Post-switch health check failed – consider switching back!"
|
||||
fi
|
||||
|
||||
kill "$pf_pid" 2>/dev/null || true
|
||||
fi
|
||||
}
|
||||
|
||||
# ── Main ────────────────────────────────────────────────────────────────────
|
||||
main() {
|
||||
echo ""
|
||||
echo -e "${BOLD}${BLUE}╔══════════════════════════════════════════════════╗${NC}"
|
||||
echo -e "${BOLD}${BLUE}║ Claude Code Agent Monitor – Blue/Green Switch ║${NC}"
|
||||
echo -e "${BOLD}${BLUE}╚══════════════════════════════════════════════════╝${NC}"
|
||||
echo ""
|
||||
|
||||
parse_args "$@"
|
||||
|
||||
info "Configuration:"
|
||||
echo -e " ${BOLD}Environment:${NC} ${ENVIRONMENT}"
|
||||
echo -e " ${BOLD}Target slot:${NC} ${TARGET}"
|
||||
echo -e " ${BOLD}Namespace:${NC} ${NAMESPACE}"
|
||||
echo -e " ${BOLD}Service:${NC} ${SERVICE_NAME}"
|
||||
echo ""
|
||||
|
||||
check_target_health
|
||||
switch_traffic
|
||||
post_switch_health
|
||||
|
||||
echo ""
|
||||
ok "${BOLD}Blue-green switch complete!${NC}"
|
||||
echo -e " ${BOLD}Active slot:${NC} ${TARGET}"
|
||||
echo -e " ${BOLD}Timestamp:${NC} $(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||
echo -e " ${BOLD}Rollback:${NC} $(basename "$0") --env ${ENVIRONMENT} --target $([ "$TARGET" = "blue" ] && echo "green" || echo "blue")"
|
||||
echo ""
|
||||
}
|
||||
|
||||
main "$@"
|
||||
Executable
+298
@@ -0,0 +1,298 @@
|
||||
#!/usr/bin/env bash
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# db-backup.sh – SQLite database backup for Claude Code Agent Monitor
|
||||
#
|
||||
# Usage:
|
||||
# ./db-backup.sh --env production --output ./backups/
|
||||
# ./db-backup.sh --env production --output ./backups/ --upload s3://bucket/path
|
||||
# ./db-backup.sh --help
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# @author Nguyễn Ngọc Trí Vĩ <vinnt@smartgift.vn>
|
||||
set -euo pipefail
|
||||
|
||||
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
readonly APP_NAME="agent-monitor"
|
||||
readonly DB_PATH_IN_CONTAINER="/app/data"
|
||||
|
||||
# ── Colors & logging ───────────────────────────────────────────────────────
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
CYAN='\033[0;36m'
|
||||
BOLD='\033[1m'
|
||||
NC='\033[0m'
|
||||
|
||||
log() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} $*"; }
|
||||
info() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${BLUE}ℹ${NC} $*"; }
|
||||
ok() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${GREEN}✔${NC} $*"; }
|
||||
warn() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${YELLOW}⚠${NC} $*" >&2; }
|
||||
err() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${RED}✖${NC} $*" >&2; }
|
||||
fatal() { err "$@"; exit 1; }
|
||||
|
||||
# ── Defaults ────────────────────────────────────────────────────────────────
|
||||
ENVIRONMENT=""
|
||||
OUTPUT_DIR=""
|
||||
NAMESPACE=""
|
||||
UPLOAD_DEST=""
|
||||
POD_NAME=""
|
||||
DB_FILENAME=""
|
||||
COMPRESS=true
|
||||
|
||||
# ── Usage ───────────────────────────────────────────────────────────────────
|
||||
usage() {
|
||||
cat <<EOF
|
||||
${BOLD}Usage:${NC}
|
||||
$(basename "$0") --env <environment> --output <directory> [options]
|
||||
|
||||
${BOLD}Required:${NC}
|
||||
--env, -e Environment: dev, staging, production
|
||||
--output, -o Local directory for the backup file
|
||||
|
||||
${BOLD}Options:${NC}
|
||||
--namespace, -n Kubernetes namespace (default: agent-monitor-<env>)
|
||||
--pod Specific pod name to copy from (auto-detected if omitted)
|
||||
--upload Upload backup to S3/GCS (e.g., s3://bucket/backups/)
|
||||
--no-compress Skip gzip compression
|
||||
--help, -h Show this help message
|
||||
|
||||
${BOLD}Examples:${NC}
|
||||
$(basename "$0") --env production --output ./backups/
|
||||
$(basename "$0") --env staging --output /tmp/backups --upload s3://my-bucket/db-backups/
|
||||
|
||||
EOF
|
||||
exit 0
|
||||
}
|
||||
|
||||
# ── Argument parsing ────────────────────────────────────────────────────────
|
||||
parse_args() {
|
||||
[[ $# -eq 0 ]] && usage
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--env|-e) ENVIRONMENT="$2"; shift 2 ;;
|
||||
--output|-o) OUTPUT_DIR="$2"; shift 2 ;;
|
||||
--namespace|-n) NAMESPACE="$2"; shift 2 ;;
|
||||
--pod) POD_NAME="$2"; shift 2 ;;
|
||||
--upload) UPLOAD_DEST="$2"; shift 2 ;;
|
||||
--no-compress) COMPRESS=false; shift ;;
|
||||
--help|-h) usage ;;
|
||||
*) fatal "Unknown option: $1" ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -z "$ENVIRONMENT" ]] && fatal "Missing required argument: --env"
|
||||
[[ -z "$OUTPUT_DIR" ]] && fatal "Missing required argument: --output"
|
||||
[[ -z "$NAMESPACE" ]] && NAMESPACE="agent-monitor-${ENVIRONMENT}"
|
||||
}
|
||||
|
||||
# ── Find target pod ────────────────────────────────────────────────────────
|
||||
find_pod() {
|
||||
if [[ -n "$POD_NAME" ]]; then
|
||||
info "Using specified pod: ${POD_NAME}"
|
||||
return
|
||||
fi
|
||||
|
||||
info "Finding running pod in namespace '${NAMESPACE}'..."
|
||||
|
||||
POD_NAME=$(kubectl get pods -n "${NAMESPACE}" \
|
||||
-l "app.kubernetes.io/name=${APP_NAME}" \
|
||||
--field-selector=status.phase=Running \
|
||||
-o jsonpath='{.items[0].metadata.name}' 2>/dev/null || echo "")
|
||||
|
||||
if [[ -z "$POD_NAME" ]]; then
|
||||
fatal "No running pods found for ${APP_NAME} in ${NAMESPACE}"
|
||||
fi
|
||||
|
||||
info "Selected pod: ${POD_NAME}"
|
||||
}
|
||||
|
||||
# ── Create backup ──────────────────────────────────────────────────────────
|
||||
create_backup() {
|
||||
local timestamp
|
||||
timestamp=$(date -u +%Y%m%d_%H%M%S)
|
||||
DB_FILENAME="${APP_NAME}_${ENVIRONMENT}_${timestamp}.db"
|
||||
|
||||
# Create output directory
|
||||
mkdir -p "${OUTPUT_DIR}"
|
||||
|
||||
info "Creating backup..."
|
||||
|
||||
# Use sqlite3 .backup inside the pod for a consistent snapshot
|
||||
# This avoids copying a potentially locked/in-flight database
|
||||
local remote_backup_path="/tmp/${DB_FILENAME}"
|
||||
|
||||
info "Running SQLite backup inside pod (consistent snapshot)..."
|
||||
if kubectl exec "${POD_NAME}" -n "${NAMESPACE}" -- \
|
||||
sh -c "
|
||||
if command -v sqlite3 >/dev/null 2>&1; then
|
||||
sqlite3 '${DB_PATH_IN_CONTAINER}/dashboard.db' '.backup ${remote_backup_path}'
|
||||
else
|
||||
cp '${DB_PATH_IN_CONTAINER}/dashboard.db' '${remote_backup_path}'
|
||||
fi
|
||||
" 2>/dev/null; then
|
||||
ok "In-pod backup created at ${remote_backup_path}"
|
||||
else
|
||||
# Fallback: also copy WAL files if present
|
||||
warn "sqlite3 not available in pod, falling back to file copy"
|
||||
kubectl exec "${POD_NAME}" -n "${NAMESPACE}" -- \
|
||||
sh -c "cp '${DB_PATH_IN_CONTAINER}/dashboard.db' '${remote_backup_path}'" \
|
||||
|| fatal "Failed to copy database file"
|
||||
|
||||
# Try to also get WAL and SHM files
|
||||
kubectl exec "${POD_NAME}" -n "${NAMESPACE}" -- \
|
||||
sh -c "cp '${DB_PATH_IN_CONTAINER}/dashboard.db-wal' '${remote_backup_path}-wal' 2>/dev/null || true"
|
||||
kubectl exec "${POD_NAME}" -n "${NAMESPACE}" -- \
|
||||
sh -c "cp '${DB_PATH_IN_CONTAINER}/dashboard.db-shm' '${remote_backup_path}-shm' 2>/dev/null || true"
|
||||
fi
|
||||
|
||||
# Copy backup from pod to local
|
||||
info "Copying backup to local filesystem..."
|
||||
kubectl cp "${NAMESPACE}/${POD_NAME}:${remote_backup_path}" "${OUTPUT_DIR}/${DB_FILENAME}" \
|
||||
|| fatal "Failed to copy backup from pod"
|
||||
|
||||
# Copy WAL if it exists
|
||||
kubectl cp "${NAMESPACE}/${POD_NAME}:${remote_backup_path}-wal" "${OUTPUT_DIR}/${DB_FILENAME}-wal" 2>/dev/null || true
|
||||
|
||||
# Cleanup remote temp file
|
||||
kubectl exec "${POD_NAME}" -n "${NAMESPACE}" -- \
|
||||
sh -c "rm -f '${remote_backup_path}' '${remote_backup_path}-wal' '${remote_backup_path}-shm'" 2>/dev/null || true
|
||||
|
||||
local file_size
|
||||
file_size=$(du -sh "${OUTPUT_DIR}/${DB_FILENAME}" 2>/dev/null | awk '{print $1}')
|
||||
ok "Backup saved: ${OUTPUT_DIR}/${DB_FILENAME} (${file_size})"
|
||||
}
|
||||
|
||||
# ── Validate backup integrity ──────────────────────────────────────────────
|
||||
validate_backup() {
|
||||
info "Validating backup integrity..."
|
||||
|
||||
local db_file="${OUTPUT_DIR}/${DB_FILENAME}"
|
||||
|
||||
if ! command -v sqlite3 &>/dev/null; then
|
||||
warn "sqlite3 not found locally – skipping integrity check"
|
||||
return
|
||||
fi
|
||||
|
||||
# Check integrity
|
||||
local integrity
|
||||
integrity=$(sqlite3 "${db_file}" "PRAGMA integrity_check;" 2>/dev/null || echo "error")
|
||||
|
||||
if [[ "$integrity" == "ok" ]]; then
|
||||
ok "SQLite integrity check: OK"
|
||||
else
|
||||
err "SQLite integrity check failed: ${integrity}"
|
||||
warn "Backup may be corrupted – consider re-running the backup"
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Show basic stats
|
||||
local table_count
|
||||
table_count=$(sqlite3 "${db_file}" "SELECT count(*) FROM sqlite_master WHERE type='table';" 2>/dev/null || echo "?")
|
||||
local page_count
|
||||
page_count=$(sqlite3 "${db_file}" "PRAGMA page_count;" 2>/dev/null || echo "?")
|
||||
local page_size
|
||||
page_size=$(sqlite3 "${db_file}" "PRAGMA page_size;" 2>/dev/null || echo "?")
|
||||
|
||||
info "Database stats: ${table_count} tables, ${page_count} pages × ${page_size} bytes"
|
||||
}
|
||||
|
||||
# ── Compress backup ────────────────────────────────────────────────────────
|
||||
compress_backup() {
|
||||
if [[ "$COMPRESS" == false ]]; then
|
||||
info "Compression skipped"
|
||||
return
|
||||
fi
|
||||
|
||||
local db_file="${OUTPUT_DIR}/${DB_FILENAME}"
|
||||
|
||||
if ! command -v gzip &>/dev/null; then
|
||||
warn "gzip not available – skipping compression"
|
||||
return
|
||||
fi
|
||||
|
||||
info "Compressing backup..."
|
||||
gzip -k "${db_file}"
|
||||
DB_FILENAME="${DB_FILENAME}.gz"
|
||||
|
||||
local compressed_size
|
||||
compressed_size=$(du -sh "${OUTPUT_DIR}/${DB_FILENAME}" 2>/dev/null | awk '{print $1}')
|
||||
ok "Compressed: ${OUTPUT_DIR}/${DB_FILENAME} (${compressed_size})"
|
||||
}
|
||||
|
||||
# ── Upload to cloud storage ────────────────────────────────────────────────
|
||||
upload_backup() {
|
||||
if [[ -z "$UPLOAD_DEST" ]]; then
|
||||
return
|
||||
fi
|
||||
|
||||
local backup_file="${OUTPUT_DIR}/${DB_FILENAME}"
|
||||
|
||||
info "Uploading backup to ${UPLOAD_DEST}..."
|
||||
|
||||
if [[ "$UPLOAD_DEST" == s3://* ]]; then
|
||||
if ! command -v aws &>/dev/null; then
|
||||
fatal "AWS CLI not found. Install it to upload to S3."
|
||||
fi
|
||||
aws s3 cp "${backup_file}" "${UPLOAD_DEST}${DB_FILENAME}" \
|
||||
--storage-class STANDARD_IA \
|
||||
|| fatal "S3 upload failed"
|
||||
ok "Uploaded to ${UPLOAD_DEST}${DB_FILENAME}"
|
||||
|
||||
elif [[ "$UPLOAD_DEST" == gs://* ]]; then
|
||||
if ! command -v gsutil &>/dev/null; then
|
||||
fatal "gsutil not found. Install Google Cloud SDK to upload to GCS."
|
||||
fi
|
||||
gsutil cp "${backup_file}" "${UPLOAD_DEST}${DB_FILENAME}" \
|
||||
|| fatal "GCS upload failed"
|
||||
ok "Uploaded to ${UPLOAD_DEST}${DB_FILENAME}"
|
||||
|
||||
elif [[ "$UPLOAD_DEST" == az://* ]] || [[ "$UPLOAD_DEST" == https://*.blob.core.windows.net/* ]]; then
|
||||
if ! command -v az &>/dev/null; then
|
||||
fatal "Azure CLI not found. Install it to upload to Azure Blob."
|
||||
fi
|
||||
local container_url="${UPLOAD_DEST}"
|
||||
az storage blob upload \
|
||||
--file "${backup_file}" \
|
||||
--name "${DB_FILENAME}" \
|
||||
--overwrite \
|
||||
|| fatal "Azure Blob upload failed"
|
||||
ok "Uploaded to Azure Blob Storage"
|
||||
|
||||
else
|
||||
warn "Unknown upload destination scheme: ${UPLOAD_DEST}"
|
||||
warn "Supported: s3://, gs://, az://"
|
||||
fi
|
||||
}
|
||||
|
||||
# ── Main ────────────────────────────────────────────────────────────────────
|
||||
main() {
|
||||
echo ""
|
||||
echo -e "${BOLD}${BLUE}╔══════════════════════════════════════════════════╗${NC}"
|
||||
echo -e "${BOLD}${BLUE}║ Claude Code Agent Monitor – DB Backup ║${NC}"
|
||||
echo -e "${BOLD}${BLUE}╚══════════════════════════════════════════════════╝${NC}"
|
||||
echo ""
|
||||
|
||||
parse_args "$@"
|
||||
|
||||
info "Configuration:"
|
||||
echo -e " ${BOLD}Environment:${NC} ${ENVIRONMENT}"
|
||||
echo -e " ${BOLD}Namespace:${NC} ${NAMESPACE}"
|
||||
echo -e " ${BOLD}Output:${NC} ${OUTPUT_DIR}"
|
||||
[[ -n "$UPLOAD_DEST" ]] && echo -e " ${BOLD}Upload:${NC} ${UPLOAD_DEST}"
|
||||
echo ""
|
||||
|
||||
find_pod
|
||||
create_backup
|
||||
validate_backup
|
||||
compress_backup
|
||||
upload_backup
|
||||
|
||||
echo ""
|
||||
ok "${BOLD}Backup complete!${NC}"
|
||||
echo -e " ${BOLD}File:${NC} ${OUTPUT_DIR}/${DB_FILENAME}"
|
||||
echo -e " ${BOLD}Timestamp:${NC} $(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||
echo ""
|
||||
}
|
||||
|
||||
main "$@"
|
||||
Executable
+396
@@ -0,0 +1,396 @@
|
||||
#!/usr/bin/env bash
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# db-restore.sh – SQLite database restore for Claude Code Agent Monitor
|
||||
#
|
||||
# Usage:
|
||||
# ./db-restore.sh --env production --input ./backups/agent-monitor_production_20240101_120000.db
|
||||
# ./db-restore.sh --help
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# @author Nguyễn Ngọc Trí Vĩ <vinnt@smartgift.vn>
|
||||
set -euo pipefail
|
||||
|
||||
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
readonly APP_NAME="agent-monitor"
|
||||
readonly APP_PORT=4820
|
||||
readonly DB_PATH_IN_CONTAINER="/app/data"
|
||||
|
||||
# ── Colors & logging ───────────────────────────────────────────────────────
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
CYAN='\033[0;36m'
|
||||
BOLD='\033[1m'
|
||||
NC='\033[0m'
|
||||
|
||||
log() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} $*"; }
|
||||
info() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${BLUE}ℹ${NC} $*"; }
|
||||
ok() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${GREEN}✔${NC} $*"; }
|
||||
warn() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${YELLOW}⚠${NC} $*" >&2; }
|
||||
err() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${RED}✖${NC} $*" >&2; }
|
||||
fatal() { err "$@"; exit 1; }
|
||||
|
||||
# ── Defaults ────────────────────────────────────────────────────────────────
|
||||
ENVIRONMENT=""
|
||||
INPUT_FILE=""
|
||||
NAMESPACE=""
|
||||
SKIP_HEALTH_CHECK=false
|
||||
FORCE=false
|
||||
BACKUP_BEFORE_RESTORE=true
|
||||
|
||||
# ── Usage ───────────────────────────────────────────────────────────────────
|
||||
usage() {
|
||||
cat <<EOF
|
||||
${BOLD}Usage:${NC}
|
||||
$(basename "$0") --env <environment> --input <backup-file> [options]
|
||||
|
||||
${BOLD}Required:${NC}
|
||||
--env, -e Environment: dev, staging, production
|
||||
--input, -i Path to backup file (.db or .db.gz)
|
||||
|
||||
${BOLD}Options:${NC}
|
||||
--namespace, -n Kubernetes namespace (default: agent-monitor-<env>)
|
||||
--no-backup Skip backing up current DB before restore
|
||||
--skip-health Skip post-restore health check
|
||||
--force Skip confirmation prompt
|
||||
--help, -h Show this help message
|
||||
|
||||
${BOLD}Examples:${NC}
|
||||
$(basename "$0") --env production --input ./backups/agent-monitor_production_20240101_120000.db
|
||||
$(basename "$0") --env staging --input ./backups/backup.db.gz --force
|
||||
|
||||
EOF
|
||||
exit 0
|
||||
}
|
||||
|
||||
# ── Argument parsing ────────────────────────────────────────────────────────
|
||||
parse_args() {
|
||||
[[ $# -eq 0 ]] && usage
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--env|-e) ENVIRONMENT="$2"; shift 2 ;;
|
||||
--input|-i) INPUT_FILE="$2"; shift 2 ;;
|
||||
--namespace|-n) NAMESPACE="$2"; shift 2 ;;
|
||||
--no-backup) BACKUP_BEFORE_RESTORE=false; shift ;;
|
||||
--skip-health) SKIP_HEALTH_CHECK=true; shift ;;
|
||||
--force) FORCE=true; shift ;;
|
||||
--help|-h) usage ;;
|
||||
*) fatal "Unknown option: $1" ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -z "$ENVIRONMENT" ]] && fatal "Missing required argument: --env"
|
||||
[[ -z "$INPUT_FILE" ]] && fatal "Missing required argument: --input"
|
||||
[[ -z "$NAMESPACE" ]] && NAMESPACE="agent-monitor-${ENVIRONMENT}"
|
||||
|
||||
# Validate input file exists
|
||||
[[ -f "$INPUT_FILE" ]] || fatal "Input file not found: ${INPUT_FILE}"
|
||||
}
|
||||
|
||||
# ── Validate backup file ───────────────────────────────────────────────────
|
||||
validate_input() {
|
||||
info "Validating input file: ${INPUT_FILE}"
|
||||
|
||||
local restore_file="${INPUT_FILE}"
|
||||
|
||||
# Decompress if needed
|
||||
if [[ "$INPUT_FILE" == *.gz ]]; then
|
||||
info "Decompressing gzipped backup..."
|
||||
restore_file="${INPUT_FILE%.gz}"
|
||||
if [[ -f "$restore_file" ]]; then
|
||||
warn "Decompressed file already exists: ${restore_file}"
|
||||
else
|
||||
gzip -dk "${INPUT_FILE}" || fatal "Failed to decompress ${INPUT_FILE}"
|
||||
fi
|
||||
fi
|
||||
|
||||
RESTORE_FILE="$restore_file"
|
||||
|
||||
# Validate with sqlite3 if available
|
||||
if command -v sqlite3 &>/dev/null; then
|
||||
local integrity
|
||||
integrity=$(sqlite3 "${RESTORE_FILE}" "PRAGMA integrity_check;" 2>/dev/null || echo "error")
|
||||
if [[ "$integrity" == "ok" ]]; then
|
||||
ok "SQLite integrity check passed"
|
||||
else
|
||||
fatal "Input file failed integrity check: ${integrity}"
|
||||
fi
|
||||
|
||||
local table_count
|
||||
table_count=$(sqlite3 "${RESTORE_FILE}" "SELECT count(*) FROM sqlite_master WHERE type='table';" 2>/dev/null || echo "?")
|
||||
info "Backup contains ${table_count} tables"
|
||||
else
|
||||
warn "sqlite3 not available – skipping integrity check"
|
||||
# Basic file header check
|
||||
local header
|
||||
header=$(head -c 16 "${RESTORE_FILE}" | strings 2>/dev/null || echo "")
|
||||
if echo "$header" | grep -q "SQLite format"; then
|
||||
ok "File appears to be a valid SQLite database"
|
||||
else
|
||||
fatal "File does not appear to be a SQLite database"
|
||||
fi
|
||||
fi
|
||||
|
||||
local file_size
|
||||
file_size=$(du -sh "${RESTORE_FILE}" 2>/dev/null | awk '{print $1}')
|
||||
info "Restore file size: ${file_size}"
|
||||
}
|
||||
|
||||
# ── Safety confirmation ─────────────────────────────────────────────────────
|
||||
confirm_restore() {
|
||||
if [[ "$FORCE" == true ]]; then
|
||||
return
|
||||
fi
|
||||
|
||||
echo ""
|
||||
warn "${BOLD}⚠ DATABASE RESTORE WARNING ⚠${NC}"
|
||||
echo ""
|
||||
echo -e " This will ${RED}${BOLD}REPLACE${NC} the current database in ${BOLD}${ENVIRONMENT}${NC}"
|
||||
echo -e " with the contents of: ${INPUT_FILE}"
|
||||
echo ""
|
||||
echo -e " The deployment will be ${BOLD}scaled down${NC} during restore."
|
||||
echo ""
|
||||
|
||||
if [[ "$ENVIRONMENT" == "production" ]]; then
|
||||
echo -e " ${RED}${BOLD}THIS IS A PRODUCTION ENVIRONMENT!${NC}"
|
||||
echo ""
|
||||
read -r -p "$(echo -e "${YELLOW}Type the environment name to confirm:${NC} ")" confirm
|
||||
[[ "$confirm" == "$ENVIRONMENT" ]] || fatal "Restore cancelled. You typed '${confirm}', expected '${ENVIRONMENT}'."
|
||||
else
|
||||
read -r -p "$(echo -e "${YELLOW}Type 'yes' to confirm:${NC} ")" confirm
|
||||
[[ "$confirm" == "yes" ]] || fatal "Restore cancelled."
|
||||
fi
|
||||
}
|
||||
|
||||
# ── Get deployment info ─────────────────────────────────────────────────────
|
||||
get_deployment_info() {
|
||||
info "Getting deployment info..."
|
||||
|
||||
DEPLOYMENT_NAME=$(kubectl get deployment -n "${NAMESPACE}" \
|
||||
-l "app.kubernetes.io/name=${APP_NAME}" \
|
||||
-o jsonpath='{.items[0].metadata.name}' 2>/dev/null || echo "")
|
||||
|
||||
if [[ -z "$DEPLOYMENT_NAME" ]]; then
|
||||
fatal "No deployment found for ${APP_NAME} in ${NAMESPACE}"
|
||||
fi
|
||||
|
||||
ORIGINAL_REPLICAS=$(kubectl get deployment "${DEPLOYMENT_NAME}" -n "${NAMESPACE}" \
|
||||
-o jsonpath='{.spec.replicas}' 2>/dev/null || echo "1")
|
||||
|
||||
info "Deployment: ${DEPLOYMENT_NAME} (${ORIGINAL_REPLICAS} replicas)"
|
||||
}
|
||||
|
||||
# ── Backup current DB before restore ───────────────────────────────────────
|
||||
backup_current() {
|
||||
if [[ "$BACKUP_BEFORE_RESTORE" == false ]]; then
|
||||
info "Skipping pre-restore backup (--no-backup)"
|
||||
return
|
||||
fi
|
||||
|
||||
info "Backing up current database before restore..."
|
||||
|
||||
if [[ -x "${SCRIPT_DIR}/db-backup.sh" ]]; then
|
||||
local backup_dir="${SCRIPT_DIR}/../../data/pre-restore-backups"
|
||||
"${SCRIPT_DIR}/db-backup.sh" \
|
||||
--env "${ENVIRONMENT}" \
|
||||
--output "${backup_dir}" \
|
||||
--namespace "${NAMESPACE}" \
|
||||
--no-compress \
|
||||
&& ok "Pre-restore backup created in ${backup_dir}" \
|
||||
|| warn "Pre-restore backup failed – proceeding anyway"
|
||||
else
|
||||
warn "db-backup.sh not found – skipping pre-restore backup"
|
||||
fi
|
||||
}
|
||||
|
||||
# ── Scale down deployment ───────────────────────────────────────────────────
|
||||
scale_down() {
|
||||
info "Scaling down deployment to 0 replicas..."
|
||||
|
||||
kubectl scale deployment "${DEPLOYMENT_NAME}" \
|
||||
--replicas=0 \
|
||||
-n "${NAMESPACE}" \
|
||||
|| fatal "Failed to scale down deployment"
|
||||
|
||||
# Wait for all pods to terminate
|
||||
info "Waiting for pods to terminate..."
|
||||
local wait_count=0
|
||||
while [[ $wait_count -lt 60 ]]; do
|
||||
local running
|
||||
running=$(kubectl get pods -n "${NAMESPACE}" \
|
||||
-l "app.kubernetes.io/name=${APP_NAME}" \
|
||||
--field-selector=status.phase=Running \
|
||||
--no-headers 2>/dev/null | wc -l | tr -d ' ')
|
||||
|
||||
if [[ "$running" == "0" ]]; then
|
||||
ok "All pods terminated"
|
||||
return
|
||||
fi
|
||||
|
||||
wait_count=$((wait_count + 1))
|
||||
sleep 2
|
||||
done
|
||||
|
||||
warn "Pods did not terminate within timeout"
|
||||
}
|
||||
|
||||
# ── Restore database ───────────────────────────────────────────────────────
|
||||
restore_database() {
|
||||
info "Restoring database..."
|
||||
|
||||
# We need a temporary pod to access the PVC
|
||||
# Create a helper pod that mounts the PVC
|
||||
local helper_pod="${APP_NAME}-db-restore-helper"
|
||||
|
||||
# Get PVC name
|
||||
local pvc_name
|
||||
pvc_name=$(kubectl get pvc -n "${NAMESPACE}" \
|
||||
-l "app.kubernetes.io/name=${APP_NAME}" \
|
||||
-o jsonpath='{.items[0].metadata.name}' 2>/dev/null || echo "agent-monitor-data")
|
||||
|
||||
info "Creating helper pod to access PVC: ${pvc_name}"
|
||||
|
||||
kubectl apply -n "${NAMESPACE}" -f - <<YAML
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: ${helper_pod}
|
||||
labels:
|
||||
app: db-restore-helper
|
||||
spec:
|
||||
restartPolicy: Never
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 1000
|
||||
runAsGroup: 1000
|
||||
fsGroup: 1000
|
||||
seccompProfile:
|
||||
type: RuntimeDefault
|
||||
containers:
|
||||
- name: helper
|
||||
image: alpine:3.19
|
||||
command: ["sleep", "3600"]
|
||||
securityContext:
|
||||
allowPrivilegeEscalation: false
|
||||
readOnlyRootFilesystem: true
|
||||
capabilities:
|
||||
drop: ["ALL"]
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /data
|
||||
volumes:
|
||||
- name: data
|
||||
persistentVolumeClaim:
|
||||
claimName: ${pvc_name}
|
||||
YAML
|
||||
|
||||
# Wait for helper pod to be ready
|
||||
info "Waiting for helper pod..."
|
||||
if ! kubectl wait --for=condition=ready "pod/${helper_pod}" -n "${NAMESPACE}" --timeout=120s; then
|
||||
kubectl delete pod "${helper_pod}" -n "${NAMESPACE}" --ignore-not-found=true
|
||||
fatal "Helper pod did not become ready"
|
||||
fi
|
||||
|
||||
# Backup existing DB in the PVC
|
||||
info "Moving existing database to .bak..."
|
||||
kubectl exec "${helper_pod}" -n "${NAMESPACE}" -- \
|
||||
sh -c "[ -f /data/dashboard.db ] && mv /data/dashboard.db /data/dashboard.db.bak || true"
|
||||
kubectl exec "${helper_pod}" -n "${NAMESPACE}" -- \
|
||||
sh -c "rm -f /data/dashboard.db-wal /data/dashboard.db-shm"
|
||||
|
||||
# Copy new database to pod, then to PVC path
|
||||
info "Uploading restore file..."
|
||||
kubectl cp "${RESTORE_FILE}" "${NAMESPACE}/${helper_pod}:/data/dashboard.db" \
|
||||
|| { kubectl delete pod "${helper_pod}" -n "${NAMESPACE}" --ignore-not-found=true; fatal "Failed to copy restore file"; }
|
||||
|
||||
# Verify copied file
|
||||
kubectl exec "${helper_pod}" -n "${NAMESPACE}" -- ls -la /data/dashboard.db
|
||||
|
||||
# Cleanup helper pod
|
||||
info "Removing helper pod..."
|
||||
kubectl delete pod "${helper_pod}" -n "${NAMESPACE}" --ignore-not-found=true --wait=false
|
||||
|
||||
ok "Database file restored"
|
||||
}
|
||||
|
||||
# ── Scale up deployment ─────────────────────────────────────────────────────
|
||||
scale_up() {
|
||||
info "Scaling deployment back to ${ORIGINAL_REPLICAS} replicas..."
|
||||
|
||||
kubectl scale deployment "${DEPLOYMENT_NAME}" \
|
||||
--replicas="${ORIGINAL_REPLICAS}" \
|
||||
-n "${NAMESPACE}" \
|
||||
|| fatal "Failed to scale up deployment"
|
||||
|
||||
# Wait for rollout
|
||||
info "Waiting for pods to start..."
|
||||
if ! kubectl rollout status "deployment/${DEPLOYMENT_NAME}" -n "${NAMESPACE}" --timeout=300s; then
|
||||
fatal "Deployment did not stabilize after restore!"
|
||||
fi
|
||||
|
||||
ok "Deployment scaled up successfully"
|
||||
}
|
||||
|
||||
# ── Post-restore health check ──────────────────────────────────────────────
|
||||
run_health_check() {
|
||||
if [[ "$SKIP_HEALTH_CHECK" == true ]]; then
|
||||
info "Skipping health check"
|
||||
return
|
||||
fi
|
||||
|
||||
info "Running post-restore health check..."
|
||||
|
||||
# Wait for pods to be ready
|
||||
if ! kubectl wait --for=condition=ready pod \
|
||||
-l "app.kubernetes.io/name=${APP_NAME}" \
|
||||
-n "${NAMESPACE}" --timeout=120s 2>/dev/null; then
|
||||
fatal "Pods did not become ready after restore!"
|
||||
fi
|
||||
|
||||
if [[ -x "${SCRIPT_DIR}/health-check.sh" ]]; then
|
||||
local local_port=14823
|
||||
kubectl port-forward "svc/${APP_NAME}" "${local_port}:${APP_PORT}" -n "${NAMESPACE}" &
|
||||
local pf_pid=$!
|
||||
sleep 3
|
||||
|
||||
if "${SCRIPT_DIR}/health-check.sh" --url "http://localhost:${local_port}" --retries 10 --interval 3; then
|
||||
ok "Post-restore health check passed"
|
||||
else
|
||||
err "Post-restore health check failed!"
|
||||
warn "The application may need manual investigation"
|
||||
fi
|
||||
|
||||
kill "$pf_pid" 2>/dev/null || true
|
||||
else
|
||||
ok "Pods are ready"
|
||||
fi
|
||||
}
|
||||
|
||||
# ── Main ────────────────────────────────────────────────────────────────────
|
||||
main() {
|
||||
echo ""
|
||||
echo -e "${BOLD}${YELLOW}╔══════════════════════════════════════════════════╗${NC}"
|
||||
echo -e "${BOLD}${YELLOW}║ Claude Code Agent Monitor – DB Restore ║${NC}"
|
||||
echo -e "${BOLD}${YELLOW}╚══════════════════════════════════════════════════╝${NC}"
|
||||
echo ""
|
||||
|
||||
parse_args "$@"
|
||||
validate_input
|
||||
confirm_restore
|
||||
get_deployment_info
|
||||
backup_current
|
||||
scale_down
|
||||
restore_database
|
||||
scale_up
|
||||
run_health_check
|
||||
|
||||
echo ""
|
||||
ok "${BOLD}Database restore complete!${NC}"
|
||||
echo -e " ${BOLD}Environment:${NC} ${ENVIRONMENT}"
|
||||
echo -e " ${BOLD}Source:${NC} ${INPUT_FILE}"
|
||||
echo -e " ${BOLD}Timestamp:${NC} $(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||
echo ""
|
||||
}
|
||||
|
||||
main "$@"
|
||||
Executable
+572
@@ -0,0 +1,572 @@
|
||||
#!/usr/bin/env bash
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# deploy.sh – Main deployment orchestrator for Claude Code Agent Monitor
|
||||
#
|
||||
# Usage:
|
||||
# ./deploy.sh --env dev|staging|production --method helm|kustomize|terraform
|
||||
# ./deploy.sh --env production --method helm --strategy blue-green|canary|rolling
|
||||
# ./deploy.sh --env staging --method helm --dry-run
|
||||
# ./deploy.sh --help
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# @author Nguyễn Ngọc Trí Vĩ <vinnt@smartgift.vn>
|
||||
set -euo pipefail
|
||||
|
||||
# ── Constants ───────────────────────────────────────────────────────────────
|
||||
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
readonly PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
|
||||
readonly DEPLOY_DIR="${PROJECT_ROOT}/deployments"
|
||||
readonly APP_NAME="agent-monitor"
|
||||
readonly APP_PORT=4820
|
||||
readonly DEFAULT_REGISTRY="ghcr.io"
|
||||
readonly DEFAULT_IMAGE_NAME="claude-code-agent-monitor"
|
||||
|
||||
# ── Colors & logging ───────────────────────────────────────────────────────
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
CYAN='\033[0;36m'
|
||||
BOLD='\033[1m'
|
||||
NC='\033[0m'
|
||||
|
||||
log() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} $*"; }
|
||||
info() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${BLUE}ℹ${NC} $*"; }
|
||||
ok() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${GREEN}✔${NC} $*"; }
|
||||
warn() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${YELLOW}⚠${NC} $*" >&2; }
|
||||
err() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${RED}✖${NC} $*" >&2; }
|
||||
fatal() { err "$@"; exit 1; }
|
||||
|
||||
banner() {
|
||||
echo ""
|
||||
echo -e "${BOLD}${BLUE}╔══════════════════════════════════════════════════╗${NC}"
|
||||
echo -e "${BOLD}${BLUE}║ Claude Code Agent Monitor – Deploy ║${NC}"
|
||||
echo -e "${BOLD}${BLUE}╚══════════════════════════════════════════════════╝${NC}"
|
||||
echo ""
|
||||
}
|
||||
|
||||
# ── Default parameter values ────────────────────────────────────────────────
|
||||
ENVIRONMENT=""
|
||||
METHOD=""
|
||||
STRATEGY="rolling"
|
||||
DRY_RUN=false
|
||||
IMAGE_TAG=""
|
||||
REGISTRY="${DOCKER_REGISTRY:-$DEFAULT_REGISTRY}"
|
||||
IMAGE_NAME="${DOCKER_IMAGE_NAME:-$DEFAULT_IMAGE_NAME}"
|
||||
NAMESPACE=""
|
||||
HELM_RELEASE="${APP_NAME}"
|
||||
HELM_CHART_DIR="${DEPLOY_DIR}/helm/agent-monitor"
|
||||
KUBE_CONTEXT=""
|
||||
SKIP_BUILD=false
|
||||
SKIP_HEALTH_CHECK=false
|
||||
HEALTH_CHECK_RETRIES=30
|
||||
HEALTH_CHECK_INTERVAL=5
|
||||
VALUES_FILE=""
|
||||
|
||||
# ── Usage ───────────────────────────────────────────────────────────────────
|
||||
usage() {
|
||||
cat <<EOF
|
||||
${BOLD}Usage:${NC}
|
||||
$(basename "$0") --env <environment> --method <method> [options]
|
||||
|
||||
${BOLD}Required:${NC}
|
||||
--env, -e Environment: dev, staging, production
|
||||
--method, -m Deployment method: helm, kustomize, terraform
|
||||
|
||||
${BOLD}Options:${NC}
|
||||
--strategy, -s Deployment strategy: rolling (default), blue-green, canary
|
||||
--tag, -t Docker image tag (default: git SHA)
|
||||
--registry Container registry (default: ${DEFAULT_REGISTRY})
|
||||
--image Image name (default: ${DEFAULT_IMAGE_NAME})
|
||||
--namespace, -n Kubernetes namespace (default: agent-monitor-<env>)
|
||||
--release Helm release name (default: ${APP_NAME})
|
||||
--context Kubernetes context to use
|
||||
--values Additional Helm values file
|
||||
--skip-build Skip container image build/push
|
||||
--skip-health Skip post-deploy health check
|
||||
--dry-run Preview changes without applying
|
||||
--help, -h Show this help message
|
||||
|
||||
${BOLD}Examples:${NC}
|
||||
$(basename "$0") --env dev --method helm
|
||||
$(basename "$0") --env production --method helm --strategy blue-green --tag v1.2.3
|
||||
$(basename "$0") --env staging --method kustomize --dry-run
|
||||
$(basename "$0") --env production --method terraform
|
||||
|
||||
EOF
|
||||
exit 0
|
||||
}
|
||||
|
||||
# ── Argument parsing ────────────────────────────────────────────────────────
|
||||
parse_args() {
|
||||
[[ $# -eq 0 ]] && usage
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--env|-e) ENVIRONMENT="$2"; shift 2 ;;
|
||||
--method|-m) METHOD="$2"; shift 2 ;;
|
||||
--strategy|-s) STRATEGY="$2"; shift 2 ;;
|
||||
--tag|-t) IMAGE_TAG="$2"; shift 2 ;;
|
||||
--registry) REGISTRY="$2"; shift 2 ;;
|
||||
--image) IMAGE_NAME="$2"; shift 2 ;;
|
||||
--namespace|-n) NAMESPACE="$2"; shift 2 ;;
|
||||
--release) HELM_RELEASE="$2"; shift 2 ;;
|
||||
--context) KUBE_CONTEXT="$2"; shift 2 ;;
|
||||
--values) VALUES_FILE="$2"; shift 2 ;;
|
||||
--skip-build) SKIP_BUILD=true; shift ;;
|
||||
--skip-health) SKIP_HEALTH_CHECK=true; shift ;;
|
||||
--dry-run) DRY_RUN=true; shift ;;
|
||||
--help|-h) usage ;;
|
||||
*) fatal "Unknown option: $1. Use --help for usage." ;;
|
||||
esac
|
||||
done
|
||||
}
|
||||
|
||||
# ── Validation ──────────────────────────────────────────────────────────────
|
||||
validate_args() {
|
||||
[[ -z "$ENVIRONMENT" ]] && fatal "Missing required argument: --env"
|
||||
[[ -z "$METHOD" ]] && fatal "Missing required argument: --method"
|
||||
|
||||
case "$ENVIRONMENT" in
|
||||
dev|staging|production) ;;
|
||||
*) fatal "Invalid environment: $ENVIRONMENT. Must be dev, staging, or production." ;;
|
||||
esac
|
||||
|
||||
case "$METHOD" in
|
||||
helm|kustomize|terraform) ;;
|
||||
*) fatal "Invalid method: $METHOD. Must be helm, kustomize, or terraform." ;;
|
||||
esac
|
||||
|
||||
case "$STRATEGY" in
|
||||
rolling|blue-green|canary) ;;
|
||||
*) fatal "Invalid strategy: $STRATEGY. Must be rolling, blue-green, or canary." ;;
|
||||
esac
|
||||
|
||||
# Default namespace
|
||||
[[ -z "$NAMESPACE" ]] && NAMESPACE="agent-monitor-${ENVIRONMENT}"
|
||||
|
||||
# Default image tag from git
|
||||
if [[ -z "$IMAGE_TAG" ]]; then
|
||||
IMAGE_TAG="$(git -C "$PROJECT_ROOT" rev-parse --short HEAD 2>/dev/null || echo 'latest')"
|
||||
fi
|
||||
|
||||
readonly FULL_IMAGE="${REGISTRY}/${IMAGE_NAME}:${IMAGE_TAG}"
|
||||
readonly MCP_IMAGE="${REGISTRY}/${IMAGE_NAME}-mcp:${IMAGE_TAG}"
|
||||
}
|
||||
|
||||
# ── Prerequisite checks ────────────────────────────────────────────────────
|
||||
check_prerequisites() {
|
||||
info "Checking prerequisites..."
|
||||
|
||||
local missing=()
|
||||
|
||||
# Always need docker for building
|
||||
if [[ "$SKIP_BUILD" == false ]]; then
|
||||
command -v docker &>/dev/null || missing+=("docker")
|
||||
fi
|
||||
|
||||
case "$METHOD" in
|
||||
helm)
|
||||
command -v kubectl &>/dev/null || missing+=("kubectl")
|
||||
command -v helm &>/dev/null || missing+=("helm")
|
||||
;;
|
||||
kustomize)
|
||||
command -v kubectl &>/dev/null || missing+=("kubectl")
|
||||
command -v kustomize &>/dev/null || missing+=("kustomize")
|
||||
;;
|
||||
terraform)
|
||||
command -v terraform &>/dev/null || missing+=("terraform")
|
||||
;;
|
||||
esac
|
||||
|
||||
if [[ ${#missing[@]} -gt 0 ]]; then
|
||||
fatal "Missing required tools: ${missing[*]}. Please install them and retry."
|
||||
fi
|
||||
|
||||
# Validate kube context if specified
|
||||
if [[ -n "$KUBE_CONTEXT" ]] && [[ "$METHOD" != "terraform" ]]; then
|
||||
if ! kubectl config get-contexts "$KUBE_CONTEXT" &>/dev/null; then
|
||||
fatal "Kubernetes context '$KUBE_CONTEXT' not found."
|
||||
fi
|
||||
kubectl config use-context "$KUBE_CONTEXT"
|
||||
fi
|
||||
|
||||
# Validate Helm chart exists
|
||||
if [[ "$METHOD" == "helm" ]] && [[ ! -f "${HELM_CHART_DIR}/Chart.yaml" ]]; then
|
||||
fatal "Helm chart not found at ${HELM_CHART_DIR}"
|
||||
fi
|
||||
|
||||
ok "All prerequisites satisfied"
|
||||
}
|
||||
|
||||
# ── Build & push container images ───────────────────────────────────────────
|
||||
build_and_push() {
|
||||
if [[ "$SKIP_BUILD" == true ]]; then
|
||||
info "Skipping image build (--skip-build)"
|
||||
return
|
||||
fi
|
||||
|
||||
info "Building container images..."
|
||||
|
||||
local docker_cmd="docker build"
|
||||
local push_cmd="docker push"
|
||||
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
info "[DRY-RUN] Would build: ${FULL_IMAGE}"
|
||||
info "[DRY-RUN] Would build: ${MCP_IMAGE}"
|
||||
return
|
||||
fi
|
||||
|
||||
# Build main application image
|
||||
log "Building main app image: ${FULL_IMAGE}"
|
||||
docker build \
|
||||
--file "${PROJECT_ROOT}/Dockerfile" \
|
||||
--tag "${FULL_IMAGE}" \
|
||||
--label "org.opencontainers.image.revision=$(git -C "$PROJECT_ROOT" rev-parse HEAD 2>/dev/null || echo 'unknown')" \
|
||||
--label "org.opencontainers.image.created=$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
|
||||
"${PROJECT_ROOT}"
|
||||
ok "Main app image built: ${FULL_IMAGE}"
|
||||
|
||||
# Build MCP sidecar image
|
||||
log "Building MCP sidecar image: ${MCP_IMAGE}"
|
||||
docker build \
|
||||
--file "${PROJECT_ROOT}/mcp/Dockerfile" \
|
||||
--tag "${MCP_IMAGE}" \
|
||||
"${PROJECT_ROOT}"
|
||||
ok "MCP sidecar image built: ${MCP_IMAGE}"
|
||||
|
||||
# Push images
|
||||
log "Pushing images to registry..."
|
||||
docker push "${FULL_IMAGE}"
|
||||
docker push "${MCP_IMAGE}"
|
||||
ok "Images pushed to ${REGISTRY}"
|
||||
}
|
||||
|
||||
# ── Helm deployment ─────────────────────────────────────────────────────────
|
||||
deploy_helm() {
|
||||
info "Deploying via Helm (strategy: ${STRATEGY})..."
|
||||
|
||||
local helm_args=(
|
||||
upgrade --install "${HELM_RELEASE}" "${HELM_CHART_DIR}"
|
||||
--namespace "${NAMESPACE}"
|
||||
--create-namespace
|
||||
--set "image.repository=${REGISTRY}/${IMAGE_NAME}"
|
||||
--set "image.tag=${IMAGE_TAG}"
|
||||
--set "environment=${ENVIRONMENT}"
|
||||
--set "mcp.image.repository=${REGISTRY}/${IMAGE_NAME}-mcp"
|
||||
--set "mcp.image.tag=${IMAGE_TAG}"
|
||||
--timeout 600s
|
||||
--wait
|
||||
--atomic
|
||||
)
|
||||
|
||||
# Environment-specific values
|
||||
local env_values="${HELM_CHART_DIR}/values-${ENVIRONMENT}.yaml"
|
||||
if [[ -f "$env_values" ]]; then
|
||||
helm_args+=(--values "$env_values")
|
||||
fi
|
||||
|
||||
# User-provided values file
|
||||
if [[ -n "$VALUES_FILE" ]] && [[ -f "$VALUES_FILE" ]]; then
|
||||
helm_args+=(--values "$VALUES_FILE")
|
||||
fi
|
||||
|
||||
# Strategy-specific settings
|
||||
case "$STRATEGY" in
|
||||
blue-green)
|
||||
helm_args+=(--set "strategy.type=blue-green")
|
||||
;;
|
||||
canary)
|
||||
helm_args+=(--set "strategy.type=canary")
|
||||
helm_args+=(--set "strategy.canary.weight=10")
|
||||
;;
|
||||
rolling)
|
||||
helm_args+=(--set "strategy.type=rolling")
|
||||
helm_args+=(--set "strategy.rolling.maxUnavailable=25%")
|
||||
helm_args+=(--set "strategy.rolling.maxSurge=25%")
|
||||
;;
|
||||
esac
|
||||
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
helm_args+=(--dry-run --debug)
|
||||
info "[DRY-RUN] Helm command:"
|
||||
echo " helm ${helm_args[*]}"
|
||||
helm "${helm_args[@]}" 2>&1 | head -100
|
||||
return
|
||||
fi
|
||||
|
||||
# Store current revision for rollback
|
||||
local current_revision
|
||||
current_revision=$(helm history "${HELM_RELEASE}" -n "${NAMESPACE}" --max 1 -o json 2>/dev/null \
|
||||
| grep -o '"revision":[0-9]*' | head -1 | cut -d: -f2 || echo "0")
|
||||
info "Current Helm revision: ${current_revision}"
|
||||
|
||||
# Execute deployment
|
||||
if ! helm "${helm_args[@]}"; then
|
||||
err "Helm deployment failed!"
|
||||
if [[ "$current_revision" != "0" ]]; then
|
||||
warn "Attempting auto-rollback to revision ${current_revision}..."
|
||||
helm rollback "${HELM_RELEASE}" "${current_revision}" -n "${NAMESPACE}" --wait --timeout 300s \
|
||||
&& ok "Auto-rollback to revision ${current_revision} succeeded" \
|
||||
|| fatal "Auto-rollback also failed! Manual intervention required."
|
||||
fi
|
||||
exit 1
|
||||
fi
|
||||
|
||||
ok "Helm deployment succeeded"
|
||||
}
|
||||
|
||||
# ── Kustomize deployment ───────────────────────────────────────────────────
|
||||
deploy_kustomize() {
|
||||
info "Deploying via Kustomize (overlay: ${ENVIRONMENT})..."
|
||||
|
||||
local overlay_dir="${DEPLOY_DIR}/kubernetes/overlays/${ENVIRONMENT}"
|
||||
if [[ ! -d "$overlay_dir" ]]; then
|
||||
fatal "Kustomize overlay not found at ${overlay_dir}"
|
||||
fi
|
||||
|
||||
# Set image in kustomization
|
||||
local kustomize_cmd="kubectl apply -k ${overlay_dir}"
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
info "[DRY-RUN] Kustomize output:"
|
||||
kubectl kustomize "${overlay_dir}" | head -100
|
||||
return
|
||||
fi
|
||||
|
||||
# Update image reference using kustomize edit
|
||||
pushd "${overlay_dir}" > /dev/null
|
||||
kustomize edit set image "${APP_NAME}=${FULL_IMAGE}" 2>/dev/null || true
|
||||
popd > /dev/null
|
||||
|
||||
# Apply with server-side apply for safety
|
||||
if ! kubectl apply -k "${overlay_dir}" --server-side --force-conflicts; then
|
||||
err "Kustomize deployment failed!"
|
||||
warn "Run: kubectl rollout undo deployment/${APP_NAME} -n ${NAMESPACE}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Wait for rollout
|
||||
info "Waiting for rollout to complete..."
|
||||
if ! kubectl rollout status "deployment/${APP_NAME}" -n "${NAMESPACE}" --timeout=600s; then
|
||||
err "Rollout did not complete in time!"
|
||||
warn "Attempting auto-rollback..."
|
||||
kubectl rollout undo "deployment/${APP_NAME}" -n "${NAMESPACE}" \
|
||||
&& ok "Auto-rollback succeeded" \
|
||||
|| fatal "Auto-rollback failed! Manual intervention required."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
ok "Kustomize deployment succeeded"
|
||||
}
|
||||
|
||||
# ── Terraform deployment ───────────────────────────────────────────────────
|
||||
deploy_terraform() {
|
||||
info "Deploying via Terraform (environment: ${ENVIRONMENT})..."
|
||||
|
||||
local tf_dir="${DEPLOY_DIR}/terraform"
|
||||
local env_vars_file="${tf_dir}/environments/${ENVIRONMENT}/terraform.tfvars"
|
||||
|
||||
if [[ ! -d "$tf_dir" ]]; then
|
||||
fatal "Terraform directory not found at ${tf_dir}"
|
||||
fi
|
||||
|
||||
pushd "${tf_dir}" > /dev/null
|
||||
|
||||
# Initialize
|
||||
info "Running terraform init..."
|
||||
terraform init -input=false
|
||||
|
||||
# Plan
|
||||
local plan_args=(-input=false -out=tfplan)
|
||||
if [[ -f "$env_vars_file" ]]; then
|
||||
plan_args+=(-var-file="$env_vars_file")
|
||||
fi
|
||||
plan_args+=(-var "app_container_image=${FULL_IMAGE}")
|
||||
plan_args+=(-var "environment=${ENVIRONMENT}")
|
||||
|
||||
info "Running terraform plan..."
|
||||
terraform plan "${plan_args[@]}"
|
||||
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
info "[DRY-RUN] Terraform plan complete. Skipping apply."
|
||||
rm -f tfplan
|
||||
popd > /dev/null
|
||||
return
|
||||
fi
|
||||
|
||||
# Apply
|
||||
info "Applying terraform plan..."
|
||||
if ! terraform apply -input=false tfplan; then
|
||||
err "Terraform apply failed!"
|
||||
fatal "Review state and run 'terraform plan' to diagnose."
|
||||
fi
|
||||
|
||||
rm -f tfplan
|
||||
popd > /dev/null
|
||||
|
||||
ok "Terraform deployment succeeded"
|
||||
}
|
||||
|
||||
# ── Post-deployment health check ───────────────────────────────────────────
|
||||
run_health_check() {
|
||||
if [[ "$SKIP_HEALTH_CHECK" == true ]] || [[ "$DRY_RUN" == true ]]; then
|
||||
info "Skipping health check"
|
||||
return
|
||||
fi
|
||||
|
||||
info "Running post-deployment health check..."
|
||||
|
||||
# Determine health check URL
|
||||
local health_url=""
|
||||
|
||||
if [[ "$METHOD" == "terraform" ]]; then
|
||||
info "For Terraform deployments, verify health via the load balancer URL in terraform output."
|
||||
return
|
||||
fi
|
||||
|
||||
# Try to get service URL from cluster
|
||||
local svc_type
|
||||
svc_type=$(kubectl get svc "${APP_NAME}" -n "${NAMESPACE}" -o jsonpath='{.spec.type}' 2>/dev/null || echo "")
|
||||
|
||||
case "$svc_type" in
|
||||
LoadBalancer)
|
||||
local lb_host
|
||||
lb_host=$(kubectl get svc "${APP_NAME}" -n "${NAMESPACE}" \
|
||||
-o jsonpath='{.status.loadBalancer.ingress[0].hostname}' 2>/dev/null || echo "")
|
||||
[[ -z "$lb_host" ]] && lb_host=$(kubectl get svc "${APP_NAME}" -n "${NAMESPACE}" \
|
||||
-o jsonpath='{.status.loadBalancer.ingress[0].ip}' 2>/dev/null || echo "")
|
||||
[[ -n "$lb_host" ]] && health_url="http://${lb_host}:${APP_PORT}/api/health"
|
||||
;;
|
||||
NodePort)
|
||||
local node_port
|
||||
node_port=$(kubectl get svc "${APP_NAME}" -n "${NAMESPACE}" \
|
||||
-o jsonpath='{.spec.ports[0].nodePort}' 2>/dev/null || echo "")
|
||||
[[ -n "$node_port" ]] && health_url="http://localhost:${node_port}/api/health"
|
||||
;;
|
||||
*)
|
||||
# Use port-forward for ClusterIP
|
||||
info "Service type is ClusterIP – using kubectl port-forward for health check"
|
||||
local local_port=14820
|
||||
kubectl port-forward "svc/${APP_NAME}" "${local_port}:${APP_PORT}" -n "${NAMESPACE}" &
|
||||
local pf_pid=$!
|
||||
sleep 3
|
||||
health_url="http://localhost:${local_port}/api/health"
|
||||
;;
|
||||
esac
|
||||
|
||||
if [[ -z "$health_url" ]]; then
|
||||
warn "Could not determine health check URL. Checking pod readiness instead."
|
||||
if kubectl wait --for=condition=ready pod -l "app.kubernetes.io/name=${APP_NAME}" \
|
||||
-n "${NAMESPACE}" --timeout=120s; then
|
||||
ok "Pods are ready"
|
||||
else
|
||||
err "Pods did not become ready"
|
||||
trigger_auto_rollback
|
||||
fi
|
||||
return
|
||||
fi
|
||||
|
||||
# Run health check script
|
||||
if [[ -x "${SCRIPT_DIR}/health-check.sh" ]]; then
|
||||
if ! "${SCRIPT_DIR}/health-check.sh" \
|
||||
--url "${health_url}" \
|
||||
--retries "${HEALTH_CHECK_RETRIES}" \
|
||||
--interval "${HEALTH_CHECK_INTERVAL}"; then
|
||||
err "Health check failed after deployment!"
|
||||
# Kill port-forward if running
|
||||
[[ -n "${pf_pid:-}" ]] && kill "$pf_pid" 2>/dev/null || true
|
||||
trigger_auto_rollback
|
||||
fi
|
||||
else
|
||||
# Inline health check
|
||||
local attempt=0
|
||||
while [[ $attempt -lt $HEALTH_CHECK_RETRIES ]]; do
|
||||
if curl -sf --max-time 5 "${health_url}" | grep -q '"status":"ok"'; then
|
||||
ok "Health check passed"
|
||||
[[ -n "${pf_pid:-}" ]] && kill "$pf_pid" 2>/dev/null || true
|
||||
return
|
||||
fi
|
||||
attempt=$((attempt + 1))
|
||||
info "Health check attempt ${attempt}/${HEALTH_CHECK_RETRIES}..."
|
||||
sleep "${HEALTH_CHECK_INTERVAL}"
|
||||
done
|
||||
err "Health check failed after ${HEALTH_CHECK_RETRIES} attempts!"
|
||||
[[ -n "${pf_pid:-}" ]] && kill "$pf_pid" 2>/dev/null || true
|
||||
trigger_auto_rollback
|
||||
fi
|
||||
|
||||
# Cleanup port-forward
|
||||
[[ -n "${pf_pid:-}" ]] && kill "$pf_pid" 2>/dev/null || true
|
||||
}
|
||||
|
||||
trigger_auto_rollback() {
|
||||
warn "Triggering auto-rollback..."
|
||||
case "$METHOD" in
|
||||
helm)
|
||||
helm rollback "${HELM_RELEASE}" -n "${NAMESPACE}" --wait --timeout 300s \
|
||||
&& ok "Auto-rollback succeeded" \
|
||||
|| fatal "Auto-rollback failed! Manual intervention required."
|
||||
;;
|
||||
kustomize)
|
||||
kubectl rollout undo "deployment/${APP_NAME}" -n "${NAMESPACE}" \
|
||||
&& ok "Auto-rollback succeeded" \
|
||||
|| fatal "Auto-rollback failed! Manual intervention required."
|
||||
;;
|
||||
terraform)
|
||||
warn "Terraform auto-rollback not supported. Review state manually."
|
||||
;;
|
||||
esac
|
||||
exit 1
|
||||
}
|
||||
|
||||
# ── Production safety gate ──────────────────────────────────────────────────
|
||||
confirm_production() {
|
||||
if [[ "$ENVIRONMENT" == "production" ]] && [[ "$DRY_RUN" == false ]]; then
|
||||
echo ""
|
||||
warn "You are about to deploy to ${BOLD}PRODUCTION${NC}"
|
||||
echo -e " ${BOLD}Method:${NC} ${METHOD}"
|
||||
echo -e " ${BOLD}Strategy:${NC} ${STRATEGY}"
|
||||
echo -e " ${BOLD}Image:${NC} ${FULL_IMAGE}"
|
||||
echo -e " ${BOLD}Namespace:${NC} ${NAMESPACE}"
|
||||
echo ""
|
||||
read -r -p "$(echo -e "${YELLOW}Type 'yes' to confirm:${NC} ")" confirm
|
||||
[[ "$confirm" == "yes" ]] || fatal "Deployment cancelled."
|
||||
fi
|
||||
}
|
||||
|
||||
# ── Main ────────────────────────────────────────────────────────────────────
|
||||
main() {
|
||||
banner
|
||||
parse_args "$@"
|
||||
validate_args
|
||||
|
||||
info "Deployment configuration:"
|
||||
echo -e " ${BOLD}Environment:${NC} ${ENVIRONMENT}"
|
||||
echo -e " ${BOLD}Method:${NC} ${METHOD}"
|
||||
echo -e " ${BOLD}Strategy:${NC} ${STRATEGY}"
|
||||
echo -e " ${BOLD}Image:${NC} ${FULL_IMAGE}"
|
||||
echo -e " ${BOLD}Namespace:${NC} ${NAMESPACE}"
|
||||
echo -e " ${BOLD}Dry run:${NC} ${DRY_RUN}"
|
||||
echo ""
|
||||
|
||||
check_prerequisites
|
||||
confirm_production
|
||||
build_and_push
|
||||
|
||||
case "$METHOD" in
|
||||
helm) deploy_helm ;;
|
||||
kustomize) deploy_kustomize ;;
|
||||
terraform) deploy_terraform ;;
|
||||
esac
|
||||
|
||||
run_health_check
|
||||
|
||||
echo ""
|
||||
ok "${BOLD}Deployment complete!${NC}"
|
||||
echo -e " ${BOLD}Environment:${NC} ${ENVIRONMENT}"
|
||||
echo -e " ${BOLD}Image:${NC} ${FULL_IMAGE}"
|
||||
echo -e " ${BOLD}Namespace:${NC} ${NAMESPACE}"
|
||||
echo -e " ${BOLD}Timestamp:${NC} $(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||
echo ""
|
||||
}
|
||||
|
||||
main "$@"
|
||||
Executable
+342
@@ -0,0 +1,342 @@
|
||||
#!/usr/bin/env bash
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# health-check.sh – Comprehensive health check for Claude Code Agent Monitor
|
||||
#
|
||||
# Usage:
|
||||
# ./health-check.sh --url http://localhost:4820
|
||||
# ./health-check.sh --url http://host:port --retries 30 --interval 5
|
||||
# ./health-check.sh --url http://host:port --json
|
||||
# ./health-check.sh --help
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# @author Nguyễn Ngọc Trí Vĩ <vinnt@smartgift.vn>
|
||||
set -euo pipefail
|
||||
|
||||
# ── Colors & logging ───────────────────────────────────────────────────────
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
CYAN='\033[0;36m'
|
||||
BOLD='\033[1m'
|
||||
NC='\033[0m'
|
||||
|
||||
log() { [[ "$JSON_OUTPUT" == true ]] && return; echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} $*"; }
|
||||
info() { [[ "$JSON_OUTPUT" == true ]] && return; echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${BLUE}ℹ${NC} $*"; }
|
||||
ok() { [[ "$JSON_OUTPUT" == true ]] && return; echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${GREEN}✔${NC} $*"; }
|
||||
warn() { [[ "$JSON_OUTPUT" == true ]] && return; echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${YELLOW}⚠${NC} $*" >&2; }
|
||||
err() { [[ "$JSON_OUTPUT" == true ]] && return; echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${RED}✖${NC} $*" >&2; }
|
||||
|
||||
# ── Defaults ────────────────────────────────────────────────────────────────
|
||||
BASE_URL=""
|
||||
RETRIES=30
|
||||
INTERVAL=5
|
||||
TIMEOUT=5
|
||||
RESPONSE_THRESHOLD=2000 # milliseconds
|
||||
JSON_OUTPUT=false
|
||||
CHECK_WEBSOCKET=true
|
||||
HEALTH_PATH="/api/health"
|
||||
|
||||
# ── Usage ───────────────────────────────────────────────────────────────────
|
||||
usage() {
|
||||
cat <<HELP
|
||||
${BOLD}Usage:${NC}
|
||||
$(basename "$0") --url <base-url> [options]
|
||||
|
||||
${BOLD}Required:${NC}
|
||||
--url, -u Base URL (e.g., http://localhost:4820)
|
||||
|
||||
${BOLD}Options:${NC}
|
||||
--retries, -r Max retry attempts (default: 30)
|
||||
--interval, -i Seconds between retries (default: 5)
|
||||
--timeout HTTP request timeout in seconds (default: 5)
|
||||
--threshold Max response time in ms (default: 2000)
|
||||
--path Health endpoint path (default: /api/health)
|
||||
--no-websocket Skip WebSocket connectivity check
|
||||
--json Output results as JSON
|
||||
--help, -h Show this help message
|
||||
|
||||
${BOLD}Exit codes:${NC}
|
||||
0 All checks passed
|
||||
1 One or more checks failed
|
||||
|
||||
${BOLD}Examples:${NC}
|
||||
$(basename "$0") --url http://localhost:4820
|
||||
$(basename "$0") --url https://monitor.example.com --retries 10 --json
|
||||
$(basename "$0") --url http://10.0.1.5:4820 --threshold 500 --no-websocket
|
||||
|
||||
HELP
|
||||
exit 0
|
||||
}
|
||||
|
||||
# ── Argument parsing ────────────────────────────────────────────────────────
|
||||
parse_args() {
|
||||
[[ $# -eq 0 ]] && usage
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--url|-u) BASE_URL="$2"; shift 2 ;;
|
||||
--retries|-r) RETRIES="$2"; shift 2 ;;
|
||||
--interval|-i) INTERVAL="$2"; shift 2 ;;
|
||||
--timeout) TIMEOUT="$2"; shift 2 ;;
|
||||
--threshold) RESPONSE_THRESHOLD="$2"; shift 2 ;;
|
||||
--path) HEALTH_PATH="$2"; shift 2 ;;
|
||||
--no-websocket) CHECK_WEBSOCKET=false; shift ;;
|
||||
--json) JSON_OUTPUT=true; shift ;;
|
||||
--help|-h) usage ;;
|
||||
*) echo "Unknown option: $1" >&2; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -z "$BASE_URL" ]] && { echo "Missing required argument: --url" >&2; exit 1; }
|
||||
|
||||
# Strip trailing slash
|
||||
BASE_URL="${BASE_URL%/}"
|
||||
}
|
||||
|
||||
# ── HTTP health check ──────────────────────────────────────────────────────
|
||||
check_http_health() {
|
||||
local url="${BASE_URL}${HEALTH_PATH}"
|
||||
local attempt=0
|
||||
local http_ok=false
|
||||
local status_code=""
|
||||
local response_body=""
|
||||
local response_time_ms=0
|
||||
|
||||
info "Checking HTTP health: ${url}"
|
||||
|
||||
while [[ $attempt -lt $RETRIES ]]; do
|
||||
attempt=$((attempt + 1))
|
||||
|
||||
# Measure response time and capture output
|
||||
local start_ns
|
||||
start_ns=$(date +%s%N 2>/dev/null || echo "0")
|
||||
|
||||
local http_response
|
||||
http_response=$(curl -sf \
|
||||
--max-time "${TIMEOUT}" \
|
||||
--write-out "\n%{http_code}\n%{time_total}" \
|
||||
"${url}" 2>/dev/null) || true
|
||||
|
||||
local end_ns
|
||||
end_ns=$(date +%s%N 2>/dev/null || echo "0")
|
||||
|
||||
if [[ -n "$http_response" ]]; then
|
||||
response_body=$(echo "$http_response" | head -n -2)
|
||||
status_code=$(echo "$http_response" | tail -2 | head -1)
|
||||
local time_total
|
||||
time_total=$(echo "$http_response" | tail -1)
|
||||
# Convert seconds to milliseconds
|
||||
response_time_ms=$(echo "$time_total" | awk '{printf "%.0f", $1 * 1000}' 2>/dev/null || echo "0")
|
||||
|
||||
if [[ "$status_code" == "200" ]] && echo "$response_body" | grep -q '"status":"ok"'; then
|
||||
http_ok=true
|
||||
break
|
||||
fi
|
||||
fi
|
||||
|
||||
if [[ $attempt -lt $RETRIES ]]; then
|
||||
info "Attempt ${attempt}/${RETRIES} – waiting ${INTERVAL}s..."
|
||||
sleep "${INTERVAL}"
|
||||
fi
|
||||
done
|
||||
|
||||
# Results
|
||||
HTTP_OK="$http_ok"
|
||||
HTTP_STATUS="$status_code"
|
||||
HTTP_BODY="$response_body"
|
||||
HTTP_RESPONSE_TIME_MS="$response_time_ms"
|
||||
HTTP_ATTEMPTS="$attempt"
|
||||
|
||||
if [[ "$http_ok" == true ]]; then
|
||||
ok "HTTP health check passed (${response_time_ms}ms, ${attempt} attempt(s))"
|
||||
else
|
||||
err "HTTP health check failed after ${attempt} attempts"
|
||||
fi
|
||||
}
|
||||
|
||||
# ── Response time check ────────────────────────────────────────────────────
|
||||
check_response_time() {
|
||||
if [[ "$HTTP_OK" != true ]]; then
|
||||
LATENCY_OK=false
|
||||
return
|
||||
fi
|
||||
|
||||
if [[ "$HTTP_RESPONSE_TIME_MS" -le "$RESPONSE_THRESHOLD" ]]; then
|
||||
LATENCY_OK=true
|
||||
ok "Response time ${HTTP_RESPONSE_TIME_MS}ms within threshold (${RESPONSE_THRESHOLD}ms)"
|
||||
else
|
||||
LATENCY_OK=false
|
||||
warn "Response time ${HTTP_RESPONSE_TIME_MS}ms exceeds threshold (${RESPONSE_THRESHOLD}ms)"
|
||||
fi
|
||||
}
|
||||
|
||||
# ── WebSocket connectivity check ───────────────────────────────────────────
|
||||
check_websocket() {
|
||||
WS_OK=false
|
||||
|
||||
if [[ "$CHECK_WEBSOCKET" == false ]]; then
|
||||
info "WebSocket check skipped"
|
||||
WS_OK=true # treat as pass when skipped
|
||||
return
|
||||
fi
|
||||
|
||||
# Construct WebSocket URL
|
||||
local ws_url="${BASE_URL}"
|
||||
ws_url="${ws_url/http:/ws:}"
|
||||
ws_url="${ws_url/https:/wss:}"
|
||||
ws_url="${ws_url}/ws"
|
||||
|
||||
info "Checking WebSocket: ${ws_url}"
|
||||
|
||||
# Check if we have a WebSocket testing tool
|
||||
if command -v websocat &>/dev/null; then
|
||||
if echo "" | websocat --one-message -t "${ws_url}" 2>/dev/null; then
|
||||
WS_OK=true
|
||||
ok "WebSocket connection succeeded (websocat)"
|
||||
return
|
||||
fi
|
||||
fi
|
||||
|
||||
# Fallback: use curl with upgrade headers to test the handshake
|
||||
local ws_status
|
||||
ws_status=$(curl -sf \
|
||||
--max-time "${TIMEOUT}" \
|
||||
-o /dev/null \
|
||||
-w "%{http_code}" \
|
||||
-H "Upgrade: websocket" \
|
||||
-H "Connection: Upgrade" \
|
||||
-H "Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==" \
|
||||
-H "Sec-WebSocket-Version: 13" \
|
||||
"${BASE_URL}/ws" 2>/dev/null) || ws_status="000"
|
||||
|
||||
# 101 = Switching Protocols (WebSocket upgrade success)
|
||||
# 400 = Bad Request (server recognized WS but rejected – still proves WS is available)
|
||||
case "$ws_status" in
|
||||
101)
|
||||
WS_OK=true
|
||||
ok "WebSocket handshake succeeded (HTTP 101)"
|
||||
;;
|
||||
400|426)
|
||||
WS_OK=true
|
||||
ok "WebSocket endpoint reachable (HTTP ${ws_status} – server recognized upgrade)"
|
||||
;;
|
||||
*)
|
||||
# Try Node.js one-liner as last resort
|
||||
if command -v node &>/dev/null; then
|
||||
local node_result
|
||||
node_result=$(node -e "
|
||||
const ws = new (require('ws'))('${ws_url}');
|
||||
const t = setTimeout(() => { process.stdout.write('timeout'); process.exit(1); }, 5000);
|
||||
ws.on('open', () => { clearTimeout(t); process.stdout.write('ok'); ws.close(); process.exit(0); });
|
||||
ws.on('error', (e) => { clearTimeout(t); process.stdout.write('error:' + e.message); process.exit(1); });
|
||||
" 2>/dev/null) || node_result="error"
|
||||
|
||||
if [[ "$node_result" == "ok" ]]; then
|
||||
WS_OK=true
|
||||
ok "WebSocket connection verified (node)"
|
||||
else
|
||||
WS_OK=false
|
||||
warn "WebSocket check failed: ${node_result}"
|
||||
fi
|
||||
else
|
||||
warn "WebSocket check inconclusive (no ws testing tool available, HTTP status: ${ws_status})"
|
||||
WS_OK=true # Don't fail the whole check for this
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
# ── Output results ──────────────────────────────────────────────────────────
|
||||
output_results() {
|
||||
local overall_healthy=true
|
||||
[[ "$HTTP_OK" != true ]] && overall_healthy=false
|
||||
[[ "$LATENCY_OK" != true ]] && overall_healthy=false
|
||||
[[ "$WS_OK" != true ]] && overall_healthy=false
|
||||
|
||||
if [[ "$JSON_OUTPUT" == true ]]; then
|
||||
cat <<JSON
|
||||
{
|
||||
"healthy": ${overall_healthy},
|
||||
"timestamp": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
|
||||
"url": "${BASE_URL}",
|
||||
"checks": {
|
||||
"http": {
|
||||
"passed": ${HTTP_OK},
|
||||
"status_code": "${HTTP_STATUS:-null}",
|
||||
"response_time_ms": ${HTTP_RESPONSE_TIME_MS:-0},
|
||||
"attempts": ${HTTP_ATTEMPTS:-0},
|
||||
"body": $(echo "${HTTP_BODY:-null}" | head -c 500 | python3 -c 'import sys,json; print(json.dumps(sys.stdin.read()))' 2>/dev/null || echo '"null"')
|
||||
},
|
||||
"latency": {
|
||||
"passed": ${LATENCY_OK},
|
||||
"response_time_ms": ${HTTP_RESPONSE_TIME_MS:-0},
|
||||
"threshold_ms": ${RESPONSE_THRESHOLD}
|
||||
},
|
||||
"websocket": {
|
||||
"passed": ${WS_OK},
|
||||
"checked": ${CHECK_WEBSOCKET}
|
||||
}
|
||||
}
|
||||
}
|
||||
JSON
|
||||
else
|
||||
echo ""
|
||||
echo -e "${BOLD}Health Check Summary${NC}"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
|
||||
local http_icon=$([[ "$HTTP_OK" == true ]] && echo "${GREEN}✔${NC}" || echo "${RED}✖${NC}")
|
||||
local latency_icon=$([[ "$LATENCY_OK" == true ]] && echo "${GREEN}✔${NC}" || echo "${YELLOW}⚠${NC}")
|
||||
local ws_icon=$([[ "$WS_OK" == true ]] && echo "${GREEN}✔${NC}" || echo "${RED}✖${NC}")
|
||||
|
||||
echo -e " ${http_icon} HTTP /api/health (${HTTP_STATUS:-???}, ${HTTP_RESPONSE_TIME_MS:-?}ms, ${HTTP_ATTEMPTS:-?} attempts)"
|
||||
echo -e " ${latency_icon} Response time (${HTTP_RESPONSE_TIME_MS:-?}ms / ${RESPONSE_THRESHOLD}ms threshold)"
|
||||
echo -e " ${ws_icon} WebSocket (checked: ${CHECK_WEBSOCKET})"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
|
||||
if [[ "$overall_healthy" == true ]]; then
|
||||
echo -e " ${GREEN}${BOLD}Overall: HEALTHY${NC}"
|
||||
else
|
||||
echo -e " ${RED}${BOLD}Overall: UNHEALTHY${NC}"
|
||||
fi
|
||||
echo ""
|
||||
fi
|
||||
|
||||
if [[ "$overall_healthy" == true ]]; then
|
||||
return 0
|
||||
else
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# ── Main ────────────────────────────────────────────────────────────────────
|
||||
main() {
|
||||
parse_args "$@"
|
||||
|
||||
# Initialize result variables
|
||||
HTTP_OK=false
|
||||
HTTP_STATUS=""
|
||||
HTTP_BODY=""
|
||||
HTTP_RESPONSE_TIME_MS=0
|
||||
HTTP_ATTEMPTS=0
|
||||
LATENCY_OK=false
|
||||
WS_OK=false
|
||||
|
||||
[[ "$JSON_OUTPUT" != true ]] && {
|
||||
echo ""
|
||||
echo -e "${BOLD}${GREEN}╔══════════════════════════════════════════════════╗${NC}"
|
||||
echo -e "${BOLD}${GREEN}║ Claude Code Agent Monitor – Health Check ║${NC}"
|
||||
echo -e "${BOLD}${GREEN}╚══════════════════════════════════════════════════╝${NC}"
|
||||
echo ""
|
||||
info "Target: ${BASE_URL}"
|
||||
info "Config: retries=${RETRIES}, interval=${INTERVAL}s, threshold=${RESPONSE_THRESHOLD}ms"
|
||||
echo ""
|
||||
}
|
||||
|
||||
check_http_health
|
||||
check_response_time
|
||||
check_websocket
|
||||
|
||||
output_results
|
||||
}
|
||||
|
||||
main "$@"
|
||||
Executable
+221
@@ -0,0 +1,221 @@
|
||||
#!/usr/bin/env bash
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# rollback.sh – Rollback deployments for Claude Code Agent Monitor
|
||||
#
|
||||
# Usage:
|
||||
# ./rollback.sh --env production --method helm --revision 5
|
||||
# ./rollback.sh --env staging --method kustomize
|
||||
# ./rollback.sh --help
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# @author Nguyễn Ngọc Trí Vĩ <vinnt@smartgift.vn>
|
||||
set -euo pipefail
|
||||
|
||||
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
readonly APP_NAME="agent-monitor"
|
||||
readonly APP_PORT=4820
|
||||
|
||||
# ── Colors & logging ───────────────────────────────────────────────────────
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
CYAN='\033[0;36m'
|
||||
BOLD='\033[1m'
|
||||
NC='\033[0m'
|
||||
|
||||
log() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} $*"; }
|
||||
info() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${BLUE}ℹ${NC} $*"; }
|
||||
ok() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${GREEN}✔${NC} $*"; }
|
||||
warn() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${YELLOW}⚠${NC} $*" >&2; }
|
||||
err() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${RED}✖${NC} $*" >&2; }
|
||||
fatal() { err "$@"; exit 1; }
|
||||
|
||||
# ── Defaults ────────────────────────────────────────────────────────────────
|
||||
ENVIRONMENT=""
|
||||
METHOD=""
|
||||
REVISION=""
|
||||
NAMESPACE=""
|
||||
HELM_RELEASE="${APP_NAME}"
|
||||
SKIP_HEALTH_CHECK=false
|
||||
|
||||
# ── Usage ───────────────────────────────────────────────────────────────────
|
||||
usage() {
|
||||
cat <<EOF
|
||||
${BOLD}Usage:${NC}
|
||||
$(basename "$0") --env <environment> --method <method> [options]
|
||||
|
||||
${BOLD}Required:${NC}
|
||||
--env, -e Environment: dev, staging, production
|
||||
--method, -m Method: helm, kustomize
|
||||
|
||||
${BOLD}Options:${NC}
|
||||
--revision, -r Helm revision or rollout history number to roll back to
|
||||
--namespace, -n Kubernetes namespace (default: agent-monitor-<env>)
|
||||
--release Helm release name (default: ${APP_NAME})
|
||||
--skip-health Skip post-rollback health check
|
||||
--help, -h Show this help message
|
||||
|
||||
${BOLD}Examples:${NC}
|
||||
$(basename "$0") --env production --method helm --revision 5
|
||||
$(basename "$0") --env staging --method kustomize
|
||||
$(basename "$0") --env production --method helm # rolls back to previous
|
||||
|
||||
EOF
|
||||
exit 0
|
||||
}
|
||||
|
||||
# ── Argument parsing ────────────────────────────────────────────────────────
|
||||
parse_args() {
|
||||
[[ $# -eq 0 ]] && usage
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--env|-e) ENVIRONMENT="$2"; shift 2 ;;
|
||||
--method|-m) METHOD="$2"; shift 2 ;;
|
||||
--revision|-r) REVISION="$2"; shift 2 ;;
|
||||
--namespace|-n) NAMESPACE="$2"; shift 2 ;;
|
||||
--release) HELM_RELEASE="$2"; shift 2 ;;
|
||||
--skip-health) SKIP_HEALTH_CHECK=true; shift ;;
|
||||
--help|-h) usage ;;
|
||||
*) fatal "Unknown option: $1" ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -z "$ENVIRONMENT" ]] && fatal "Missing required argument: --env"
|
||||
[[ -z "$METHOD" ]] && fatal "Missing required argument: --method"
|
||||
[[ -z "$NAMESPACE" ]] && NAMESPACE="agent-monitor-${ENVIRONMENT}"
|
||||
}
|
||||
|
||||
# ── Show release history ───────────────────────────────────────────────────
|
||||
show_history() {
|
||||
info "Release history:"
|
||||
case "$METHOD" in
|
||||
helm)
|
||||
helm history "${HELM_RELEASE}" -n "${NAMESPACE}" --max 10 2>/dev/null || warn "No history found"
|
||||
;;
|
||||
kustomize)
|
||||
kubectl rollout history "deployment/${APP_NAME}" -n "${NAMESPACE}" 2>/dev/null || warn "No history found"
|
||||
;;
|
||||
esac
|
||||
echo ""
|
||||
}
|
||||
|
||||
# ── Confirm rollback ───────────────────────────────────────────────────────
|
||||
confirm_rollback() {
|
||||
if [[ "$ENVIRONMENT" == "production" ]]; then
|
||||
local target_msg="previous revision"
|
||||
[[ -n "$REVISION" ]] && target_msg="revision ${REVISION}"
|
||||
|
||||
warn "Rolling back ${BOLD}PRODUCTION${NC} to ${target_msg}"
|
||||
read -r -p "$(echo -e "${YELLOW}Type 'yes' to confirm:${NC} ")" confirm
|
||||
[[ "$confirm" == "yes" ]] || fatal "Rollback cancelled."
|
||||
fi
|
||||
}
|
||||
|
||||
# ── Helm rollback ───────────────────────────────────────────────────────────
|
||||
rollback_helm() {
|
||||
info "Rolling back Helm release '${HELM_RELEASE}' in namespace '${NAMESPACE}'..."
|
||||
|
||||
local rollback_args=(rollback "${HELM_RELEASE}")
|
||||
[[ -n "$REVISION" ]] && rollback_args+=("${REVISION}")
|
||||
rollback_args+=(-n "${NAMESPACE}" --wait --timeout 300s)
|
||||
|
||||
if ! helm "${rollback_args[@]}"; then
|
||||
fatal "Helm rollback failed! Manual intervention required."
|
||||
fi
|
||||
|
||||
ok "Helm rollback completed"
|
||||
|
||||
# Show current status
|
||||
info "Current release status:"
|
||||
helm status "${HELM_RELEASE}" -n "${NAMESPACE}" 2>/dev/null || true
|
||||
}
|
||||
|
||||
# ── Kustomize rollback ─────────────────────────────────────────────────────
|
||||
rollback_kustomize() {
|
||||
info "Rolling back deployment '${APP_NAME}' in namespace '${NAMESPACE}'..."
|
||||
|
||||
local undo_args=(rollout undo "deployment/${APP_NAME}" -n "${NAMESPACE}")
|
||||
if [[ -n "$REVISION" ]]; then
|
||||
undo_args+=(--to-revision="${REVISION}")
|
||||
fi
|
||||
|
||||
if ! kubectl "${undo_args[@]}"; then
|
||||
fatal "Kubectl rollback failed! Manual intervention required."
|
||||
fi
|
||||
|
||||
# Wait for rollout
|
||||
info "Waiting for rollout to complete..."
|
||||
if ! kubectl rollout status "deployment/${APP_NAME}" -n "${NAMESPACE}" --timeout=300s; then
|
||||
fatal "Rollout did not complete in time!"
|
||||
fi
|
||||
|
||||
ok "Kustomize rollback completed"
|
||||
}
|
||||
|
||||
# ── Post-rollback health check ─────────────────────────────────────────────
|
||||
run_health_check() {
|
||||
if [[ "$SKIP_HEALTH_CHECK" == true ]]; then
|
||||
info "Skipping health check"
|
||||
return
|
||||
fi
|
||||
|
||||
info "Running post-rollback health check..."
|
||||
|
||||
# Wait for pods to be ready
|
||||
if ! kubectl wait --for=condition=ready pod \
|
||||
-l "app.kubernetes.io/name=${APP_NAME}" \
|
||||
-n "${NAMESPACE}" --timeout=120s 2>/dev/null; then
|
||||
fatal "Pods did not become ready after rollback!"
|
||||
fi
|
||||
|
||||
# Use health-check.sh if available
|
||||
if [[ -x "${SCRIPT_DIR}/health-check.sh" ]]; then
|
||||
# Port forward for check
|
||||
local local_port=14820
|
||||
kubectl port-forward "svc/${APP_NAME}" "${local_port}:${APP_PORT}" -n "${NAMESPACE}" &
|
||||
local pf_pid=$!
|
||||
sleep 3
|
||||
|
||||
if "${SCRIPT_DIR}/health-check.sh" --url "http://localhost:${local_port}" --retries 10 --interval 3; then
|
||||
ok "Health check passed after rollback"
|
||||
else
|
||||
err "Health check failed after rollback!"
|
||||
fi
|
||||
|
||||
kill "$pf_pid" 2>/dev/null || true
|
||||
else
|
||||
ok "Pods are ready"
|
||||
fi
|
||||
}
|
||||
|
||||
# ── Main ────────────────────────────────────────────────────────────────────
|
||||
main() {
|
||||
echo ""
|
||||
echo -e "${BOLD}${YELLOW}╔══════════════════════════════════════════════════╗${NC}"
|
||||
echo -e "${BOLD}${YELLOW}║ Claude Code Agent Monitor – Rollback ║${NC}"
|
||||
echo -e "${BOLD}${YELLOW}╚══════════════════════════════════════════════════╝${NC}"
|
||||
echo ""
|
||||
|
||||
parse_args "$@"
|
||||
show_history
|
||||
confirm_rollback
|
||||
|
||||
case "$METHOD" in
|
||||
helm) rollback_helm ;;
|
||||
kustomize) rollback_kustomize ;;
|
||||
*) fatal "Rollback not supported for method: ${METHOD}" ;;
|
||||
esac
|
||||
|
||||
run_health_check
|
||||
|
||||
echo ""
|
||||
ok "${BOLD}Rollback complete!${NC}"
|
||||
echo -e " ${BOLD}Environment:${NC} ${ENVIRONMENT}"
|
||||
echo -e " ${BOLD}Method:${NC} ${METHOD}"
|
||||
echo -e " ${BOLD}Revision:${NC} ${REVISION:-previous}"
|
||||
echo -e " ${BOLD}Timestamp:${NC} $(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||
echo ""
|
||||
}
|
||||
|
||||
main "$@"
|
||||
Executable
+335
@@ -0,0 +1,335 @@
|
||||
#!/usr/bin/env bash
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# teardown.sh – Clean removal of Claude Code Agent Monitor infrastructure
|
||||
#
|
||||
# Usage:
|
||||
# ./teardown.sh --env dev --method helm
|
||||
# ./teardown.sh --env production --method terraform
|
||||
# ./teardown.sh --help
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# @author Nguyễn Ngọc Trí Vĩ <vinnt@smartgift.vn>
|
||||
set -euo pipefail
|
||||
|
||||
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
readonly PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
|
||||
readonly DEPLOY_DIR="${PROJECT_ROOT}/deployments"
|
||||
readonly APP_NAME="agent-monitor"
|
||||
|
||||
# ── Colors & logging ───────────────────────────────────────────────────────
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
CYAN='\033[0;36m'
|
||||
BOLD='\033[1m'
|
||||
NC='\033[0m'
|
||||
|
||||
log() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} $*"; }
|
||||
info() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${BLUE}ℹ${NC} $*"; }
|
||||
ok() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${GREEN}✔${NC} $*"; }
|
||||
warn() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${YELLOW}⚠${NC} $*" >&2; }
|
||||
err() { echo -e "${CYAN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} ${RED}✖${NC} $*" >&2; }
|
||||
fatal() { err "$@"; exit 1; }
|
||||
|
||||
# ── Defaults ────────────────────────────────────────────────────────────────
|
||||
ENVIRONMENT=""
|
||||
METHOD=""
|
||||
NAMESPACE=""
|
||||
HELM_RELEASE="${APP_NAME}"
|
||||
SKIP_BACKUP=false
|
||||
FORCE=false
|
||||
DELETE_NAMESPACE=false
|
||||
DELETE_PVC=false
|
||||
|
||||
# ── Usage ───────────────────────────────────────────────────────────────────
|
||||
usage() {
|
||||
cat <<EOF
|
||||
${BOLD}Usage:${NC}
|
||||
$(basename "$0") --env <environment> --method <method> [options]
|
||||
|
||||
${BOLD}Required:${NC}
|
||||
--env, -e Environment: dev, staging, production
|
||||
--method, -m Method: helm, kustomize, terraform
|
||||
|
||||
${BOLD}Options:${NC}
|
||||
--namespace, -n Kubernetes namespace (default: agent-monitor-<env>)
|
||||
--release Helm release name (default: ${APP_NAME})
|
||||
--delete-namespace Also delete the Kubernetes namespace
|
||||
--delete-pvc Also delete PersistentVolumeClaims (data loss!)
|
||||
--skip-backup Skip data backup before teardown
|
||||
--force Skip all confirmation prompts
|
||||
--help, -h Show this help message
|
||||
|
||||
${BOLD}Examples:${NC}
|
||||
$(basename "$0") --env dev --method helm
|
||||
$(basename "$0") --env staging --method kustomize --delete-namespace
|
||||
$(basename "$0") --env production --method terraform
|
||||
|
||||
EOF
|
||||
exit 0
|
||||
}
|
||||
|
||||
# ── Argument parsing ────────────────────────────────────────────────────────
|
||||
parse_args() {
|
||||
[[ $# -eq 0 ]] && usage
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--env|-e) ENVIRONMENT="$2"; shift 2 ;;
|
||||
--method|-m) METHOD="$2"; shift 2 ;;
|
||||
--namespace|-n) NAMESPACE="$2"; shift 2 ;;
|
||||
--release) HELM_RELEASE="$2"; shift 2 ;;
|
||||
--delete-namespace) DELETE_NAMESPACE=true; shift ;;
|
||||
--delete-pvc) DELETE_PVC=true; shift ;;
|
||||
--skip-backup) SKIP_BACKUP=true; shift ;;
|
||||
--force) FORCE=true; shift ;;
|
||||
--help|-h) usage ;;
|
||||
*) fatal "Unknown option: $1" ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -z "$ENVIRONMENT" ]] && fatal "Missing required argument: --env"
|
||||
[[ -z "$METHOD" ]] && fatal "Missing required argument: --method"
|
||||
[[ -z "$NAMESPACE" ]] && NAMESPACE="agent-monitor-${ENVIRONMENT}"
|
||||
}
|
||||
|
||||
# ── Confirm teardown ───────────────────────────────────────────────────────
|
||||
confirm_teardown() {
|
||||
if [[ "$FORCE" == true ]]; then
|
||||
return
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo -e " ${RED}${BOLD}╔══════════════════════════════════════════╗${NC}"
|
||||
echo -e " ${RED}${BOLD}║ ⚠ TEARDOWN WARNING ⚠ ║${NC}"
|
||||
echo -e " ${RED}${BOLD}╚══════════════════════════════════════════╝${NC}"
|
||||
echo ""
|
||||
echo -e " This will ${RED}${BOLD}DESTROY${NC} the following resources:"
|
||||
echo -e " • Environment: ${BOLD}${ENVIRONMENT}${NC}"
|
||||
echo -e " • Method: ${BOLD}${METHOD}${NC}"
|
||||
echo -e " • Namespace: ${BOLD}${NAMESPACE}${NC}"
|
||||
[[ "$DELETE_NAMESPACE" == true ]] && echo -e " • ${RED}Namespace will be deleted${NC}"
|
||||
[[ "$DELETE_PVC" == true ]] && echo -e " • ${RED}PVCs will be deleted (DATA LOSS!)${NC}"
|
||||
echo ""
|
||||
|
||||
if [[ "$ENVIRONMENT" == "production" ]]; then
|
||||
echo -e " ${RED}${BOLD}THIS IS A PRODUCTION ENVIRONMENT!${NC}"
|
||||
echo ""
|
||||
read -r -p "$(echo -e "${RED}Type 'destroy ${ENVIRONMENT}' to confirm:${NC} ")" confirm
|
||||
[[ "$confirm" == "destroy ${ENVIRONMENT}" ]] || fatal "Teardown cancelled."
|
||||
|
||||
# Second confirmation for production
|
||||
echo ""
|
||||
read -r -p "$(echo -e "${RED}Are you absolutely sure? Type 'YES' in caps:${NC} ")" confirm2
|
||||
[[ "$confirm2" == "YES" ]] || fatal "Teardown cancelled."
|
||||
else
|
||||
read -r -p "$(echo -e "${YELLOW}Type 'yes' to confirm:${NC} ")" confirm
|
||||
[[ "$confirm" == "yes" ]] || fatal "Teardown cancelled."
|
||||
fi
|
||||
}
|
||||
|
||||
# ── Pre-teardown backup ────────────────────────────────────────────────────
|
||||
backup_data() {
|
||||
if [[ "$SKIP_BACKUP" == true ]]; then
|
||||
info "Skipping pre-teardown backup (--skip-backup)"
|
||||
return
|
||||
fi
|
||||
|
||||
info "Creating backup before teardown..."
|
||||
|
||||
if [[ -x "${SCRIPT_DIR}/db-backup.sh" ]]; then
|
||||
local backup_dir="${PROJECT_ROOT}/data/pre-teardown-backups"
|
||||
"${SCRIPT_DIR}/db-backup.sh" \
|
||||
--env "${ENVIRONMENT}" \
|
||||
--output "${backup_dir}" \
|
||||
--namespace "${NAMESPACE}" \
|
||||
&& ok "Pre-teardown backup created" \
|
||||
|| warn "Backup failed – continuing with teardown"
|
||||
else
|
||||
warn "db-backup.sh not found – skipping backup"
|
||||
fi
|
||||
}
|
||||
|
||||
# ── Show current resources ──────────────────────────────────────────────────
|
||||
show_resources() {
|
||||
info "Current resources in namespace '${NAMESPACE}':"
|
||||
echo ""
|
||||
|
||||
kubectl get all -n "${NAMESPACE}" 2>/dev/null || warn "Could not list resources"
|
||||
|
||||
if [[ "$METHOD" != "terraform" ]]; then
|
||||
echo ""
|
||||
info "PersistentVolumeClaims:"
|
||||
kubectl get pvc -n "${NAMESPACE}" 2>/dev/null || warn "No PVCs found"
|
||||
fi
|
||||
echo ""
|
||||
}
|
||||
|
||||
# ── Helm teardown ──────────────────────────────────────────────────────────
|
||||
teardown_helm() {
|
||||
info "Uninstalling Helm release '${HELM_RELEASE}'..."
|
||||
|
||||
if helm status "${HELM_RELEASE}" -n "${NAMESPACE}" &>/dev/null; then
|
||||
helm uninstall "${HELM_RELEASE}" -n "${NAMESPACE}" --wait --timeout 300s \
|
||||
|| fatal "Helm uninstall failed"
|
||||
ok "Helm release '${HELM_RELEASE}' uninstalled"
|
||||
else
|
||||
warn "Helm release '${HELM_RELEASE}' not found in namespace '${NAMESPACE}'"
|
||||
fi
|
||||
|
||||
# Also try uninstalling blue/green releases
|
||||
for color in blue green; do
|
||||
if helm status "${HELM_RELEASE}-${color}" -n "${NAMESPACE}" &>/dev/null; then
|
||||
info "Uninstalling ${color} slot release..."
|
||||
helm uninstall "${HELM_RELEASE}-${color}" -n "${NAMESPACE}" --wait --timeout 300s \
|
||||
&& ok "Release '${HELM_RELEASE}-${color}' uninstalled" \
|
||||
|| warn "Failed to uninstall ${color} release"
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
# ── Kustomize teardown ─────────────────────────────────────────────────────
|
||||
teardown_kustomize() {
|
||||
local overlay_dir="${DEPLOY_DIR}/kubernetes/overlays/${ENVIRONMENT}"
|
||||
|
||||
if [[ -d "$overlay_dir" ]]; then
|
||||
info "Deleting Kustomize resources..."
|
||||
kubectl delete -k "${overlay_dir}" --ignore-not-found=true --wait=true --timeout=300s \
|
||||
&& ok "Kustomize resources deleted" \
|
||||
|| warn "Some resources may not have been deleted"
|
||||
else
|
||||
warn "Kustomize overlay not found at ${overlay_dir}"
|
||||
info "Deleting resources by label..."
|
||||
kubectl delete all -l "app.kubernetes.io/name=${APP_NAME}" -n "${NAMESPACE}" --wait=true \
|
||||
|| warn "Could not delete resources by label"
|
||||
fi
|
||||
}
|
||||
|
||||
# ── Terraform teardown ─────────────────────────────────────────────────────
|
||||
teardown_terraform() {
|
||||
local tf_dir="${DEPLOY_DIR}/terraform"
|
||||
local env_vars_file="${tf_dir}/environments/${ENVIRONMENT}/terraform.tfvars"
|
||||
|
||||
info "Destroying Terraform-managed infrastructure..."
|
||||
|
||||
pushd "${tf_dir}" > /dev/null
|
||||
|
||||
terraform init -input=false
|
||||
|
||||
local destroy_args=(-input=false -auto-approve)
|
||||
if [[ -f "$env_vars_file" ]]; then
|
||||
destroy_args+=(-var-file="$env_vars_file")
|
||||
fi
|
||||
# Need to provide required variables that may not have defaults
|
||||
destroy_args+=(-var "environment=${ENVIRONMENT}")
|
||||
|
||||
if ! terraform destroy "${destroy_args[@]}"; then
|
||||
popd > /dev/null
|
||||
fatal "Terraform destroy failed! Review state manually."
|
||||
fi
|
||||
|
||||
popd > /dev/null
|
||||
ok "Terraform infrastructure destroyed"
|
||||
}
|
||||
|
||||
# ── Cleanup PVCs ────────────────────────────────────────────────────────────
|
||||
cleanup_pvcs() {
|
||||
if [[ "$DELETE_PVC" != true ]]; then
|
||||
local pvc_count
|
||||
pvc_count=$(kubectl get pvc -n "${NAMESPACE}" --no-headers 2>/dev/null | wc -l | tr -d ' ')
|
||||
if [[ "$pvc_count" -gt 0 ]]; then
|
||||
warn "PersistentVolumeClaims still exist. Use --delete-pvc to remove them."
|
||||
kubectl get pvc -n "${NAMESPACE}" 2>/dev/null
|
||||
fi
|
||||
return
|
||||
fi
|
||||
|
||||
info "Deleting PersistentVolumeClaims..."
|
||||
kubectl delete pvc --all -n "${NAMESPACE}" --wait=true \
|
||||
&& ok "PVCs deleted" \
|
||||
|| warn "Some PVCs could not be deleted"
|
||||
}
|
||||
|
||||
# ── Cleanup namespace ──────────────────────────────────────────────────────
|
||||
cleanup_namespace() {
|
||||
if [[ "$DELETE_NAMESPACE" != true ]]; then
|
||||
return
|
||||
fi
|
||||
|
||||
if [[ "$ENVIRONMENT" == "production" ]]; then
|
||||
warn "Refusing to delete production namespace automatically."
|
||||
warn "Delete manually: kubectl delete namespace ${NAMESPACE}"
|
||||
return
|
||||
fi
|
||||
|
||||
info "Deleting namespace '${NAMESPACE}'..."
|
||||
kubectl delete namespace "${NAMESPACE}" --ignore-not-found=true --wait=true --timeout=120s \
|
||||
&& ok "Namespace '${NAMESPACE}' deleted" \
|
||||
|| warn "Namespace deletion may be stuck. Check: kubectl get namespace ${NAMESPACE}"
|
||||
}
|
||||
|
||||
# ── Verify teardown ────────────────────────────────────────────────────────
|
||||
verify_teardown() {
|
||||
info "Verifying teardown..."
|
||||
|
||||
if [[ "$METHOD" == "terraform" ]]; then
|
||||
ok "Terraform state should reflect no resources"
|
||||
return
|
||||
fi
|
||||
|
||||
local remaining
|
||||
remaining=$(kubectl get all -n "${NAMESPACE}" --no-headers 2>/dev/null | wc -l | tr -d ' ')
|
||||
|
||||
if [[ "$remaining" -eq 0 ]]; then
|
||||
ok "No resources remaining in namespace '${NAMESPACE}'"
|
||||
else
|
||||
warn "${remaining} resources still exist in namespace '${NAMESPACE}':"
|
||||
kubectl get all -n "${NAMESPACE}" 2>/dev/null
|
||||
fi
|
||||
}
|
||||
|
||||
# ── Main ────────────────────────────────────────────────────────────────────
|
||||
main() {
|
||||
echo ""
|
||||
echo -e "${BOLD}${RED}╔══════════════════════════════════════════════════╗${NC}"
|
||||
echo -e "${BOLD}${RED}║ Claude Code Agent Monitor – Teardown ║${NC}"
|
||||
echo -e "${BOLD}${RED}╚══════════════════════════════════════════════════╝${NC}"
|
||||
echo ""
|
||||
|
||||
parse_args "$@"
|
||||
|
||||
info "Teardown configuration:"
|
||||
echo -e " ${BOLD}Environment:${NC} ${ENVIRONMENT}"
|
||||
echo -e " ${BOLD}Method:${NC} ${METHOD}"
|
||||
echo -e " ${BOLD}Namespace:${NC} ${NAMESPACE}"
|
||||
echo ""
|
||||
|
||||
if [[ "$METHOD" != "terraform" ]]; then
|
||||
show_resources
|
||||
fi
|
||||
|
||||
confirm_teardown
|
||||
backup_data
|
||||
|
||||
case "$METHOD" in
|
||||
helm) teardown_helm ;;
|
||||
kustomize) teardown_kustomize ;;
|
||||
terraform) teardown_terraform ;;
|
||||
*) fatal "Invalid method: ${METHOD}" ;;
|
||||
esac
|
||||
|
||||
if [[ "$METHOD" != "terraform" ]]; then
|
||||
cleanup_pvcs
|
||||
cleanup_namespace
|
||||
verify_teardown
|
||||
fi
|
||||
|
||||
echo ""
|
||||
ok "${BOLD}Teardown complete!${NC}"
|
||||
echo -e " ${BOLD}Environment:${NC} ${ENVIRONMENT}"
|
||||
echo -e " ${BOLD}Method:${NC} ${METHOD}"
|
||||
echo -e " ${BOLD}Timestamp:${NC} $(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||
echo ""
|
||||
}
|
||||
|
||||
main "$@"
|
||||
@@ -0,0 +1,88 @@
|
||||
# Terraform Infrastructure
|
||||
|
||||
Cloud-agnostic infrastructure modules for deploying the Claude Code Agent Monitor to AWS, GCP, Azure, or OCI.
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph Modules["Reusable Modules"]
|
||||
NET["networking"]
|
||||
COMP["compute"]
|
||||
DB["database"]
|
||||
LB["loadbalancer"]
|
||||
MON["monitoring"]
|
||||
SEC["secrets"]
|
||||
end
|
||||
|
||||
subgraph Providers["Provider Implementations"]
|
||||
AWS["aws/"]
|
||||
GCP["gcp/"]
|
||||
AZ["azure/"]
|
||||
OCI["oci/"]
|
||||
end
|
||||
|
||||
subgraph Envs["Environments"]
|
||||
DEV["dev/terraform.tfvars"]
|
||||
STG["staging/terraform.tfvars"]
|
||||
PRD["production/terraform.tfvars"]
|
||||
end
|
||||
|
||||
AWS --> NET & COMP & DB & LB & MON & SEC
|
||||
GCP --> NET & COMP & DB & LB & MON & SEC
|
||||
AZ --> NET & COMP & DB & LB & MON & SEC
|
||||
OCI --> NET & COMP & DB & LB & MON & SEC
|
||||
Envs -.->|var-file| Providers
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
# 1. Choose your provider
|
||||
cd providers/aws # or gcp, azure, oci
|
||||
|
||||
# 2. Initialize
|
||||
terraform init
|
||||
|
||||
# 3. Plan with environment
|
||||
terraform plan -var-file=../../environments/production/terraform.tfvars
|
||||
|
||||
# 4. Apply
|
||||
terraform apply -var-file=../../environments/production/terraform.tfvars
|
||||
|
||||
# 5. Get outputs
|
||||
terraform output
|
||||
```
|
||||
|
||||
## Module Reference
|
||||
|
||||
| Module | Purpose | Key Resources |
|
||||
|---|---|---|
|
||||
| `networking` | VPC/VNet, subnets, NAT, security groups | VPC, public/private subnets, NAT gateway, firewall rules |
|
||||
| `compute` | Container orchestration with blue-green slots | ECS tasks / Cloud Run / ACI / OKE deployments |
|
||||
| `database` | Persistent storage for SQLite | EFS / Filestore / Azure Files / FSS with encryption |
|
||||
| `loadbalancer` | Application LB with WebSocket + traffic splitting | ALB / GCLB / App Gateway / LBaaS, health checks |
|
||||
| `monitoring` | Metrics, logs, alerts, dashboards | CloudWatch / Cloud Monitoring / Azure Monitor / OCI Monitoring |
|
||||
| `secrets` | Secret management | Secrets Manager / Secret Manager / Key Vault / Vault |
|
||||
|
||||
## Remote State
|
||||
|
||||
Each provider is configured to use cloud-native remote state:
|
||||
|
||||
| Provider | Backend | Bucket |
|
||||
|---|---|---|
|
||||
| AWS | S3 + DynamoDB locking | `agent-monitor-tfstate-{account_id}` |
|
||||
| GCP | GCS | `agent-monitor-tfstate-{project_id}` |
|
||||
| Azure | Azure Blob Storage | `agentmonitortfstate` |
|
||||
| OCI | OCI Object Storage | `agent-monitor-tfstate` |
|
||||
|
||||
## Environment Sizing
|
||||
|
||||
| Resource | Dev | Staging | Production |
|
||||
|---|---|---|---|
|
||||
| Replicas | 1 | 2 | 3 (auto-scale to 10) |
|
||||
| CPU | 256 | 512 | 1024 |
|
||||
| Memory | 512 MB | 1 GB | 2 GB |
|
||||
| Storage | 5 GB | 10 GB | 50 GB (encrypted) |
|
||||
| Multi-AZ | No | Yes | Yes |
|
||||
| Monitoring | Basic | Standard | Full + alerts |
|
||||
@@ -0,0 +1,47 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Remote state backend – uncomment the block matching your cloud provider.
|
||||
# Only ONE backend may be active at a time.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
# ── AWS S3 ──────────────────────────────────────────────────────────────────
|
||||
# terraform {
|
||||
# backend "s3" {
|
||||
# bucket = "ccam-terraform-state"
|
||||
# key = "claude-agent-monitor/terraform.tfstate"
|
||||
# region = "us-east-1"
|
||||
# encrypt = true
|
||||
# dynamodb_table = "ccam-terraform-locks"
|
||||
# }
|
||||
# }
|
||||
|
||||
# ── GCP Cloud Storage ──────────────────────────────────────────────────────
|
||||
# terraform {
|
||||
# backend "gcs" {
|
||||
# bucket = "ccam-terraform-state"
|
||||
# prefix = "claude-agent-monitor"
|
||||
# }
|
||||
# }
|
||||
|
||||
# ── Azure Blob Storage ─────────────────────────────────────────────────────
|
||||
# terraform {
|
||||
# backend "azurerm" {
|
||||
# resource_group_name = "ccam-terraform-state-rg"
|
||||
# storage_account_name = "ccamtfstate"
|
||||
# container_name = "tfstate"
|
||||
# key = "claude-agent-monitor.tfstate"
|
||||
# }
|
||||
# }
|
||||
|
||||
# ── OCI Object Storage ─────────────────────────────────────────────────────
|
||||
# terraform {
|
||||
# backend "s3" {
|
||||
# bucket = "ccam-terraform-state"
|
||||
# key = "claude-agent-monitor/terraform.tfstate"
|
||||
# region = "us-ashburn-1"
|
||||
# endpoint = "https://<namespace>.compat.objectstorage.<region>.oraclecloud.com"
|
||||
# skip_region_validation = true
|
||||
# skip_credentials_validation = true
|
||||
# skip_metadata_api_check = true
|
||||
# force_path_style = true
|
||||
# }
|
||||
# }
|
||||
@@ -0,0 +1,67 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Development environment – terraform.tfvars
|
||||
#
|
||||
# Minimal resources for development/testing. Single replica, small compute,
|
||||
# monitoring disabled to reduce cost.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
# ── Provider ────────────────────────────────────────────────────────────────
|
||||
cloud_provider = "aws"
|
||||
region = "us-east-1"
|
||||
|
||||
# ── Project ─────────────────────────────────────────────────────────────────
|
||||
project_name = "claude-agent-monitor"
|
||||
environment = "dev"
|
||||
|
||||
tags = {
|
||||
team = "platform"
|
||||
cost_center = "engineering"
|
||||
}
|
||||
|
||||
# ── Networking ──────────────────────────────────────────────────────────────
|
||||
vpc_cidr = "10.0.0.0/16"
|
||||
public_subnet_cidrs = ["10.0.1.0/24", "10.0.2.0/24"]
|
||||
private_subnet_cidrs = ["10.0.11.0/24", "10.0.12.0/24"]
|
||||
|
||||
# ── Compute (small) ────────────────────────────────────────────────────────
|
||||
app_container_image = "ghcr.io/anthropics/claude-agent-monitor:latest"
|
||||
mcp_container_image = "" # MCP sidecar disabled in dev
|
||||
cpu = 256 # 0.25 vCPU
|
||||
memory = 512 # 512 MiB
|
||||
|
||||
min_replicas = 1
|
||||
max_replicas = 1
|
||||
desired_replicas = 1
|
||||
|
||||
environment_variables = {
|
||||
NODE_ENV = "development"
|
||||
DASHBOARD_PORT = "4820"
|
||||
LOG_LEVEL = "debug"
|
||||
}
|
||||
|
||||
# ── Deployment ──────────────────────────────────────────────────────────────
|
||||
deployment_strategy = "rolling"
|
||||
active_deployment_slot = "blue"
|
||||
blue_weight = 100
|
||||
green_weight = 0
|
||||
|
||||
# ── TLS (disabled in dev) ──────────────────────────────────────────────────
|
||||
domain_name = ""
|
||||
tls_certificate_arn = ""
|
||||
|
||||
# ── Storage ─────────────────────────────────────────────────────────────────
|
||||
storage_size_gb = 10
|
||||
enable_storage_backup = false
|
||||
|
||||
# ── Health check ────────────────────────────────────────────────────────────
|
||||
health_check_path = "/api/health"
|
||||
health_check_interval = 60 # Less frequent in dev
|
||||
|
||||
# ── Auto-scaling (disabled – single replica) ────────────────────────────────
|
||||
autoscaling_cpu_target = 80
|
||||
autoscaling_memory_target = 90
|
||||
|
||||
# ── Monitoring (minimal) ───────────────────────────────────────────────────
|
||||
enable_monitoring = false
|
||||
alert_email = ""
|
||||
log_retention_days = 7
|
||||
@@ -0,0 +1,80 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Production environment – terraform.tfvars
|
||||
#
|
||||
# Full production configuration. 3+ replicas with auto-scaling, large
|
||||
# compute, comprehensive monitoring, TLS, blue-green deployment ready.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
# ── Provider ────────────────────────────────────────────────────────────────
|
||||
cloud_provider = "aws"
|
||||
region = "us-east-1"
|
||||
|
||||
# ── Project ─────────────────────────────────────────────────────────────────
|
||||
project_name = "claude-agent-monitor"
|
||||
environment = "production"
|
||||
|
||||
tags = {
|
||||
team = "platform"
|
||||
cost_center = "engineering"
|
||||
criticality = "high"
|
||||
compliance = "soc2"
|
||||
}
|
||||
|
||||
# ── Networking (3 AZs for high availability) ───────────────────────────────
|
||||
vpc_cidr = "10.2.0.0/16"
|
||||
public_subnet_cidrs = ["10.2.1.0/24", "10.2.2.0/24", "10.2.3.0/24"]
|
||||
private_subnet_cidrs = ["10.2.11.0/24", "10.2.12.0/24", "10.2.13.0/24"]
|
||||
|
||||
# ── Compute (large) ────────────────────────────────────────────────────────
|
||||
app_container_image = "ghcr.io/anthropics/claude-agent-monitor:latest"
|
||||
mcp_container_image = "ghcr.io/anthropics/claude-agent-monitor-mcp:latest"
|
||||
cpu = 1024 # 1 vCPU
|
||||
memory = 2048 # 2 GiB
|
||||
|
||||
min_replicas = 3
|
||||
max_replicas = 10
|
||||
desired_replicas = 3
|
||||
|
||||
environment_variables = {
|
||||
NODE_ENV = "production"
|
||||
DASHBOARD_PORT = "4820"
|
||||
LOG_LEVEL = "warn"
|
||||
}
|
||||
|
||||
# ── Deployment (blue-green with canary support) ────────────────────────────
|
||||
deployment_strategy = "blue-green"
|
||||
active_deployment_slot = "blue"
|
||||
blue_weight = 100
|
||||
green_weight = 0
|
||||
|
||||
# During canary deployment, adjust weights:
|
||||
# blue_weight = 90
|
||||
# green_weight = 10
|
||||
# Then gradually shift to:
|
||||
# blue_weight = 0
|
||||
# green_weight = 100
|
||||
# Finally, flip active_deployment_slot = "green"
|
||||
|
||||
# ── TLS ─────────────────────────────────────────────────────────────────────
|
||||
domain_name = "" # Set to production FQDN (e.g. "monitor.example.com")
|
||||
tls_certificate_arn = "" # Set to existing ACM cert ARN or leave empty for auto
|
||||
|
||||
# ── Storage ─────────────────────────────────────────────────────────────────
|
||||
storage_size_gb = 50
|
||||
enable_storage_backup = true
|
||||
|
||||
# ── Health check (strict thresholds) ───────────────────────────────────────
|
||||
health_check_path = "/api/health"
|
||||
health_check_interval = 15
|
||||
health_check_timeout = 5
|
||||
health_check_healthy_threshold = 2
|
||||
health_check_unhealthy_threshold = 2
|
||||
|
||||
# ── Auto-scaling (aggressive) ──────────────────────────────────────────────
|
||||
autoscaling_cpu_target = 60
|
||||
autoscaling_memory_target = 70
|
||||
|
||||
# ── Monitoring (comprehensive) ─────────────────────────────────────────────
|
||||
enable_monitoring = true
|
||||
alert_email = "" # REQUIRED: Set to ops team email for production alerts
|
||||
log_retention_days = 90
|
||||
@@ -0,0 +1,70 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Staging environment – terraform.tfvars
|
||||
#
|
||||
# Production-like configuration with moderate resources. Two replicas,
|
||||
# medium compute, monitoring enabled with relaxed thresholds.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
# ── Provider ────────────────────────────────────────────────────────────────
|
||||
cloud_provider = "aws"
|
||||
region = "us-east-1"
|
||||
|
||||
# ── Project ─────────────────────────────────────────────────────────────────
|
||||
project_name = "claude-agent-monitor"
|
||||
environment = "staging"
|
||||
|
||||
tags = {
|
||||
team = "platform"
|
||||
cost_center = "engineering"
|
||||
}
|
||||
|
||||
# ── Networking ──────────────────────────────────────────────────────────────
|
||||
vpc_cidr = "10.1.0.0/16"
|
||||
public_subnet_cidrs = ["10.1.1.0/24", "10.1.2.0/24", "10.1.3.0/24"]
|
||||
private_subnet_cidrs = ["10.1.11.0/24", "10.1.12.0/24", "10.1.13.0/24"]
|
||||
|
||||
# ── Compute (medium) ───────────────────────────────────────────────────────
|
||||
app_container_image = "ghcr.io/anthropics/claude-agent-monitor:staging"
|
||||
mcp_container_image = "ghcr.io/anthropics/claude-agent-monitor-mcp:staging"
|
||||
cpu = 512 # 0.5 vCPU
|
||||
memory = 1024 # 1 GiB
|
||||
|
||||
min_replicas = 1
|
||||
max_replicas = 3
|
||||
desired_replicas = 2
|
||||
|
||||
environment_variables = {
|
||||
NODE_ENV = "production"
|
||||
DASHBOARD_PORT = "4820"
|
||||
LOG_LEVEL = "info"
|
||||
}
|
||||
|
||||
# ── Deployment (blue-green ready) ──────────────────────────────────────────
|
||||
deployment_strategy = "blue-green"
|
||||
active_deployment_slot = "blue"
|
||||
blue_weight = 100
|
||||
green_weight = 0
|
||||
|
||||
# ── TLS ─────────────────────────────────────────────────────────────────────
|
||||
domain_name = "" # Set to staging FQDN when available
|
||||
tls_certificate_arn = "" # Auto-created if domain_name is set
|
||||
|
||||
# ── Storage ─────────────────────────────────────────────────────────────────
|
||||
storage_size_gb = 20
|
||||
enable_storage_backup = true
|
||||
|
||||
# ── Health check ────────────────────────────────────────────────────────────
|
||||
health_check_path = "/api/health"
|
||||
health_check_interval = 30
|
||||
health_check_timeout = 5
|
||||
health_check_healthy_threshold = 2
|
||||
health_check_unhealthy_threshold = 3
|
||||
|
||||
# ── Auto-scaling ────────────────────────────────────────────────────────────
|
||||
autoscaling_cpu_target = 70
|
||||
autoscaling_memory_target = 80
|
||||
|
||||
# ── Monitoring ──────────────────────────────────────────────────────────────
|
||||
enable_monitoring = true
|
||||
alert_email = "" # Set to team email for staging alerts
|
||||
log_retention_days = 14
|
||||
@@ -0,0 +1,185 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Claude Code Agent Monitor – Root orchestration module
|
||||
#
|
||||
# Selects the cloud provider implementation via var.cloud_provider and wires
|
||||
# the generic modules together. Each provider directory contains a full,
|
||||
# opinionated implementation that composes the child modules.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
locals {
|
||||
common_tags = merge(
|
||||
{
|
||||
project = var.project_name
|
||||
environment = var.environment
|
||||
managed_by = "terraform"
|
||||
repository = "Claude-Code-Agent-Monitor"
|
||||
},
|
||||
var.tags,
|
||||
)
|
||||
|
||||
# Normalise resource name prefix (lowercase, hyphens)
|
||||
name_prefix = lower(replace("${var.project_name}-${var.environment}", "_", "-"))
|
||||
}
|
||||
|
||||
# ── Networking ──────────────────────────────────────────────────────────────
|
||||
|
||||
module "networking" {
|
||||
source = "./modules/networking"
|
||||
|
||||
project_name = var.project_name
|
||||
environment = var.environment
|
||||
cloud_provider = var.cloud_provider
|
||||
region = var.region
|
||||
vpc_cidr = var.vpc_cidr
|
||||
availability_zones = var.availability_zones
|
||||
public_subnet_cidrs = var.public_subnet_cidrs
|
||||
private_subnet_cidrs = var.private_subnet_cidrs
|
||||
app_port = var.app_port
|
||||
mcp_port = var.mcp_port
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# ── Persistent storage (SQLite DB) ─────────────────────────────────────────
|
||||
|
||||
module "database" {
|
||||
source = "./modules/database"
|
||||
|
||||
project_name = var.project_name
|
||||
environment = var.environment
|
||||
cloud_provider = var.cloud_provider
|
||||
region = var.region
|
||||
storage_size_gb = var.storage_size_gb
|
||||
enable_backup = var.enable_storage_backup
|
||||
private_subnet_ids = module.networking.private_subnet_ids
|
||||
vpc_id = module.networking.vpc_id
|
||||
allowed_security_group_ids = module.networking.storage_security_group_ids
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# ── Compute (Blue slot) ────────────────────────────────────────────────────
|
||||
|
||||
module "compute_blue" {
|
||||
source = "./modules/compute"
|
||||
|
||||
project_name = var.project_name
|
||||
environment = var.environment
|
||||
cloud_provider = var.cloud_provider
|
||||
region = var.region
|
||||
deployment_slot = "blue"
|
||||
container_image = var.app_container_image
|
||||
mcp_container_image = var.mcp_container_image
|
||||
app_port = var.app_port
|
||||
mcp_port = var.mcp_port
|
||||
cpu = var.cpu
|
||||
memory = var.memory
|
||||
desired_count = var.active_deployment_slot == "blue" ? var.desired_replicas : 0
|
||||
min_count = var.active_deployment_slot == "blue" ? var.min_replicas : 0
|
||||
max_count = var.active_deployment_slot == "blue" ? var.max_replicas : 0
|
||||
environment_variables = var.environment_variables
|
||||
health_check_path = var.health_check_path
|
||||
vpc_id = module.networking.vpc_id
|
||||
private_subnet_ids = module.networking.private_subnet_ids
|
||||
security_group_ids = module.networking.private_security_group_ids
|
||||
storage_filesystem_id = module.database.filesystem_id
|
||||
storage_mount_targets = module.database.mount_target_ids
|
||||
autoscaling_cpu_target = var.autoscaling_cpu_target
|
||||
autoscaling_memory_target = var.autoscaling_memory_target
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# ── Compute (Green slot) ───────────────────────────────────────────────────
|
||||
|
||||
module "compute_green" {
|
||||
source = "./modules/compute"
|
||||
|
||||
project_name = var.project_name
|
||||
environment = var.environment
|
||||
cloud_provider = var.cloud_provider
|
||||
region = var.region
|
||||
deployment_slot = "green"
|
||||
container_image = var.app_container_image
|
||||
mcp_container_image = var.mcp_container_image
|
||||
app_port = var.app_port
|
||||
mcp_port = var.mcp_port
|
||||
cpu = var.cpu
|
||||
memory = var.memory
|
||||
desired_count = var.active_deployment_slot == "green" ? var.desired_replicas : 0
|
||||
min_count = var.active_deployment_slot == "green" ? var.min_replicas : 0
|
||||
max_count = var.active_deployment_slot == "green" ? var.max_replicas : 0
|
||||
environment_variables = var.environment_variables
|
||||
health_check_path = var.health_check_path
|
||||
vpc_id = module.networking.vpc_id
|
||||
private_subnet_ids = module.networking.private_subnet_ids
|
||||
security_group_ids = module.networking.private_security_group_ids
|
||||
storage_filesystem_id = module.database.filesystem_id
|
||||
storage_mount_targets = module.database.mount_target_ids
|
||||
autoscaling_cpu_target = var.autoscaling_cpu_target
|
||||
autoscaling_memory_target = var.autoscaling_memory_target
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# ── Load balancer ───────────────────────────────────────────────────────────
|
||||
|
||||
module "loadbalancer" {
|
||||
source = "./modules/loadbalancer"
|
||||
|
||||
project_name = var.project_name
|
||||
environment = var.environment
|
||||
cloud_provider = var.cloud_provider
|
||||
region = var.region
|
||||
vpc_id = module.networking.vpc_id
|
||||
public_subnet_ids = module.networking.public_subnet_ids
|
||||
security_group_ids = module.networking.public_security_group_ids
|
||||
app_port = var.app_port
|
||||
mcp_port = var.mcp_port
|
||||
tls_certificate_arn = var.tls_certificate_arn
|
||||
domain_name = var.domain_name
|
||||
|
||||
blue_target_group_arn = module.compute_blue.target_group_arn
|
||||
green_target_group_arn = module.compute_green.target_group_arn
|
||||
blue_weight = var.blue_weight
|
||||
green_weight = var.green_weight
|
||||
|
||||
health_check_path = var.health_check_path
|
||||
health_check_interval = var.health_check_interval
|
||||
health_check_timeout = var.health_check_timeout
|
||||
health_check_healthy_threshold = var.health_check_healthy_threshold
|
||||
health_check_unhealthy_threshold = var.health_check_unhealthy_threshold
|
||||
|
||||
enable_deletion_protection = var.environment == "production"
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# ── Weight sum validation ───────────────────────────────────────────────────
|
||||
|
||||
check "blue_green_weight_sum" {
|
||||
assert {
|
||||
condition = var.blue_weight + var.green_weight == 100
|
||||
error_message = "blue_weight (${var.blue_weight}) + green_weight (${var.green_weight}) must sum to 100."
|
||||
}
|
||||
}
|
||||
|
||||
# ── Monitoring ──────────────────────────────────────────────────────────────
|
||||
|
||||
module "monitoring" {
|
||||
source = "./modules/monitoring"
|
||||
count = var.enable_monitoring ? 1 : 0
|
||||
|
||||
project_name = var.project_name
|
||||
environment = var.environment
|
||||
cloud_provider = var.cloud_provider
|
||||
region = var.region
|
||||
alert_email = var.alert_email
|
||||
log_retention_days = var.log_retention_days
|
||||
|
||||
loadbalancer_arn = module.loadbalancer.loadbalancer_arn
|
||||
target_group_arns = [
|
||||
module.compute_blue.target_group_arn,
|
||||
module.compute_green.target_group_arn,
|
||||
]
|
||||
compute_cluster_name = module.compute_blue.cluster_name
|
||||
filesystem_id = module.database.filesystem_id
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
@@ -0,0 +1,396 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Compute module – Container orchestration with blue/green slot support
|
||||
#
|
||||
# Provisions an ECS Fargate service with:
|
||||
# - Main application container (Express + React)
|
||||
# - MCP sidecar container
|
||||
# - EFS persistent volume for SQLite
|
||||
# - Auto-scaling policies
|
||||
# - Blue/green deployment slot via variable
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
locals {
|
||||
name_prefix = lower(replace("${var.project_name}-${var.environment}", "_", "-"))
|
||||
slot_name = "${local.name_prefix}-${var.deployment_slot}"
|
||||
|
||||
common_tags = merge(
|
||||
{
|
||||
module = "compute"
|
||||
deployment_slot = var.deployment_slot
|
||||
},
|
||||
var.tags,
|
||||
)
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# ECS Cluster (shared across slots)
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_ecs_cluster" "main" {
|
||||
name = "${local.name_prefix}-cluster"
|
||||
|
||||
setting {
|
||||
name = "containerInsights"
|
||||
value = "enabled"
|
||||
}
|
||||
|
||||
configuration {
|
||||
execute_command_configuration {
|
||||
logging = "DEFAULT"
|
||||
}
|
||||
}
|
||||
|
||||
tags = merge(local.common_tags, {
|
||||
Name = "${local.name_prefix}-cluster"
|
||||
})
|
||||
|
||||
lifecycle {
|
||||
prevent_destroy = false
|
||||
}
|
||||
}
|
||||
|
||||
resource "aws_ecs_cluster_capacity_providers" "main" {
|
||||
cluster_name = aws_ecs_cluster.main.name
|
||||
|
||||
capacity_providers = ["FARGATE", "FARGATE_SPOT"]
|
||||
|
||||
default_capacity_provider_strategy {
|
||||
base = 1
|
||||
weight = 1
|
||||
capacity_provider = "FARGATE"
|
||||
}
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# IAM roles
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
data "aws_region" "current" {}
|
||||
data "aws_caller_identity" "current" {}
|
||||
|
||||
resource "aws_iam_role" "task_execution" {
|
||||
name = "${local.slot_name}-task-exec"
|
||||
|
||||
assume_role_policy = jsonencode({
|
||||
Version = "2012-10-17"
|
||||
Statement = [{
|
||||
Action = "sts:AssumeRole"
|
||||
Effect = "Allow"
|
||||
Principal = {
|
||||
Service = "ecs-tasks.amazonaws.com"
|
||||
}
|
||||
}]
|
||||
})
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
resource "aws_iam_role_policy_attachment" "task_execution" {
|
||||
role = aws_iam_role.task_execution.name
|
||||
policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
|
||||
}
|
||||
|
||||
resource "aws_iam_role" "task" {
|
||||
name = "${local.slot_name}-task"
|
||||
|
||||
assume_role_policy = jsonencode({
|
||||
Version = "2012-10-17"
|
||||
Statement = [{
|
||||
Action = "sts:AssumeRole"
|
||||
Effect = "Allow"
|
||||
Principal = {
|
||||
Service = "ecs-tasks.amazonaws.com"
|
||||
}
|
||||
}]
|
||||
})
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# EFS access policy for the task role
|
||||
resource "aws_iam_role_policy" "task_efs" {
|
||||
name = "${local.slot_name}-efs-access"
|
||||
role = aws_iam_role.task.id
|
||||
|
||||
policy = jsonencode({
|
||||
Version = "2012-10-17"
|
||||
Statement = [{
|
||||
Effect = "Allow"
|
||||
Action = [
|
||||
"elasticfilesystem:ClientMount",
|
||||
"elasticfilesystem:ClientWrite",
|
||||
"elasticfilesystem:ClientRootAccess",
|
||||
]
|
||||
Resource = "arn:aws:elasticfilesystem:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:file-system/${var.storage_filesystem_id}"
|
||||
}]
|
||||
})
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# CloudWatch log group
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_cloudwatch_log_group" "app" {
|
||||
name = "/ecs/${local.slot_name}"
|
||||
retention_in_days = 30
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Task definition
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_ecs_task_definition" "app" {
|
||||
family = local.slot_name
|
||||
network_mode = "awsvpc"
|
||||
requires_compatibilities = ["FARGATE"]
|
||||
cpu = tostring(var.cpu)
|
||||
memory = tostring(var.memory)
|
||||
execution_role_arn = aws_iam_role.task_execution.arn
|
||||
task_role_arn = aws_iam_role.task.arn
|
||||
|
||||
container_definitions = jsonencode(concat(
|
||||
[
|
||||
{
|
||||
name = "app"
|
||||
image = var.container_image
|
||||
essential = true
|
||||
cpu = var.mcp_container_image != "" ? floor(var.cpu * 0.75) : var.cpu
|
||||
memory = var.mcp_container_image != "" ? floor(var.memory * 0.75) : var.memory
|
||||
|
||||
portMappings = [
|
||||
{
|
||||
containerPort = var.app_port
|
||||
protocol = "tcp"
|
||||
}
|
||||
]
|
||||
|
||||
environment = [
|
||||
for k, v in var.environment_variables : {
|
||||
name = k
|
||||
value = v
|
||||
}
|
||||
]
|
||||
|
||||
mountPoints = [
|
||||
{
|
||||
sourceVolume = "app-data"
|
||||
containerPath = "/app/data"
|
||||
readOnly = false
|
||||
}
|
||||
]
|
||||
|
||||
healthCheck = {
|
||||
command = ["CMD-SHELL", "wget --no-verbose --tries=1 --spider http://localhost:${var.app_port}${var.health_check_path} || exit 1"]
|
||||
interval = 30
|
||||
timeout = 5
|
||||
retries = 3
|
||||
startPeriod = 60
|
||||
}
|
||||
|
||||
logConfiguration = {
|
||||
logDriver = "awslogs"
|
||||
options = {
|
||||
"awslogs-group" = aws_cloudwatch_log_group.app.name
|
||||
"awslogs-region" = data.aws_region.current.name
|
||||
"awslogs-stream-prefix" = "app"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
var.mcp_container_image != "" ? [
|
||||
{
|
||||
name = "mcp-sidecar"
|
||||
image = var.mcp_container_image
|
||||
essential = false
|
||||
cpu = floor(var.cpu * 0.25)
|
||||
memory = floor(var.memory * 0.25)
|
||||
|
||||
portMappings = [
|
||||
{
|
||||
containerPort = var.mcp_port
|
||||
protocol = "tcp"
|
||||
}
|
||||
]
|
||||
|
||||
environment = [
|
||||
{
|
||||
name = "NODE_ENV"
|
||||
value = "production"
|
||||
},
|
||||
{
|
||||
name = "MCP_PORT"
|
||||
value = tostring(var.mcp_port)
|
||||
}
|
||||
]
|
||||
|
||||
logConfiguration = {
|
||||
logDriver = "awslogs"
|
||||
options = {
|
||||
"awslogs-group" = aws_cloudwatch_log_group.app.name
|
||||
"awslogs-region" = data.aws_region.current.name
|
||||
"awslogs-stream-prefix" = "mcp"
|
||||
}
|
||||
}
|
||||
}
|
||||
] : []
|
||||
))
|
||||
|
||||
volume {
|
||||
name = "app-data"
|
||||
|
||||
efs_volume_configuration {
|
||||
file_system_id = var.storage_filesystem_id
|
||||
root_directory = "/"
|
||||
transit_encryption = "ENABLED"
|
||||
|
||||
authorization_config {
|
||||
iam = "ENABLED"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
tags = local.common_tags
|
||||
|
||||
lifecycle {
|
||||
create_before_destroy = true
|
||||
}
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Target group (registered with LB by the loadbalancer module)
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_lb_target_group" "app" {
|
||||
name_prefix = substr(var.deployment_slot, 0, 5)
|
||||
port = var.app_port
|
||||
protocol = "HTTP"
|
||||
vpc_id = var.vpc_id
|
||||
target_type = "ip"
|
||||
|
||||
health_check {
|
||||
enabled = true
|
||||
path = var.health_check_path
|
||||
port = "traffic-port"
|
||||
protocol = "HTTP"
|
||||
healthy_threshold = 2
|
||||
unhealthy_threshold = 3
|
||||
timeout = 5
|
||||
interval = 30
|
||||
matcher = "200"
|
||||
}
|
||||
|
||||
stickiness {
|
||||
type = "lb_cookie"
|
||||
cookie_duration = 86400
|
||||
enabled = true
|
||||
}
|
||||
|
||||
deregistration_delay = 60
|
||||
|
||||
tags = merge(local.common_tags, {
|
||||
Name = "${local.slot_name}-tg"
|
||||
})
|
||||
|
||||
lifecycle {
|
||||
create_before_destroy = true
|
||||
}
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# ECS Service
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_ecs_service" "app" {
|
||||
name = local.slot_name
|
||||
cluster = aws_ecs_cluster.main.id
|
||||
task_definition = aws_ecs_task_definition.app.arn
|
||||
desired_count = var.desired_count
|
||||
launch_type = "FARGATE"
|
||||
platform_version = "LATEST"
|
||||
health_check_grace_period_seconds = 120
|
||||
enable_execute_command = var.environment != "production"
|
||||
|
||||
network_configuration {
|
||||
subnets = var.private_subnet_ids
|
||||
security_groups = var.security_group_ids
|
||||
assign_public_ip = false
|
||||
}
|
||||
|
||||
load_balancer {
|
||||
target_group_arn = aws_lb_target_group.app.arn
|
||||
container_name = "app"
|
||||
container_port = var.app_port
|
||||
}
|
||||
|
||||
deployment_configuration {
|
||||
maximum_percent = 200
|
||||
minimum_healthy_percent = 100
|
||||
}
|
||||
|
||||
deployment_circuit_breaker {
|
||||
enable = true
|
||||
rollback = true
|
||||
}
|
||||
|
||||
tags = local.common_tags
|
||||
|
||||
lifecycle {
|
||||
ignore_changes = [desired_count]
|
||||
}
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Auto-scaling
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_appautoscaling_target" "ecs" {
|
||||
count = var.max_count > 0 ? 1 : 0
|
||||
|
||||
max_capacity = var.max_count
|
||||
min_capacity = var.min_count
|
||||
resource_id = "service/${aws_ecs_cluster.main.name}/${aws_ecs_service.app.name}"
|
||||
scalable_dimension = "ecs:service:DesiredCount"
|
||||
service_namespace = "ecs"
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
resource "aws_appautoscaling_policy" "cpu" {
|
||||
count = var.max_count > 0 ? 1 : 0
|
||||
|
||||
name = "${local.slot_name}-cpu-scaling"
|
||||
policy_type = "TargetTrackingScaling"
|
||||
resource_id = aws_appautoscaling_target.ecs[0].resource_id
|
||||
scalable_dimension = aws_appautoscaling_target.ecs[0].scalable_dimension
|
||||
service_namespace = aws_appautoscaling_target.ecs[0].service_namespace
|
||||
|
||||
target_tracking_scaling_policy_configuration {
|
||||
predefined_metric_specification {
|
||||
predefined_metric_type = "ECSServiceAverageCPUUtilization"
|
||||
}
|
||||
target_value = var.autoscaling_cpu_target
|
||||
scale_in_cooldown = 300
|
||||
scale_out_cooldown = 60
|
||||
}
|
||||
}
|
||||
|
||||
resource "aws_appautoscaling_policy" "memory" {
|
||||
count = var.max_count > 0 ? 1 : 0
|
||||
|
||||
name = "${local.slot_name}-memory-scaling"
|
||||
policy_type = "TargetTrackingScaling"
|
||||
resource_id = aws_appautoscaling_target.ecs[0].resource_id
|
||||
scalable_dimension = aws_appautoscaling_target.ecs[0].scalable_dimension
|
||||
service_namespace = aws_appautoscaling_target.ecs[0].service_namespace
|
||||
|
||||
target_tracking_scaling_policy_configuration {
|
||||
predefined_metric_specification {
|
||||
predefined_metric_type = "ECSServiceAverageMemoryUtilization"
|
||||
}
|
||||
target_value = var.autoscaling_memory_target
|
||||
scale_in_cooldown = 300
|
||||
scale_out_cooldown = 60
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Compute module outputs
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
output "cluster_name" {
|
||||
description = "Name of the ECS cluster"
|
||||
value = aws_ecs_cluster.main.name
|
||||
}
|
||||
|
||||
output "cluster_arn" {
|
||||
description = "ARN of the ECS cluster"
|
||||
value = aws_ecs_cluster.main.arn
|
||||
}
|
||||
|
||||
output "service_name" {
|
||||
description = "Name of the ECS service for this slot"
|
||||
value = aws_ecs_service.app.name
|
||||
}
|
||||
|
||||
output "service_arn" {
|
||||
description = "ARN of the ECS service for this slot"
|
||||
value = aws_ecs_service.app.id
|
||||
}
|
||||
|
||||
output "task_definition_arn" {
|
||||
description = "ARN of the current task definition"
|
||||
value = aws_ecs_task_definition.app.arn
|
||||
}
|
||||
|
||||
output "target_group_arn" {
|
||||
description = "ARN of the target group for LB registration"
|
||||
value = aws_lb_target_group.app.arn
|
||||
}
|
||||
|
||||
output "target_group_name" {
|
||||
description = "Name of the target group"
|
||||
value = aws_lb_target_group.app.name
|
||||
}
|
||||
|
||||
output "log_group_name" {
|
||||
description = "CloudWatch log group name"
|
||||
value = aws_cloudwatch_log_group.app.name
|
||||
}
|
||||
|
||||
output "task_execution_role_arn" {
|
||||
description = "ARN of the task execution IAM role"
|
||||
value = aws_iam_role.task_execution.arn
|
||||
}
|
||||
|
||||
output "task_role_arn" {
|
||||
description = "ARN of the task IAM role"
|
||||
value = aws_iam_role.task.arn
|
||||
}
|
||||
@@ -0,0 +1,146 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Compute module variables
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
variable "project_name" {
|
||||
description = "Project identifier used in resource naming"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "environment" {
|
||||
description = "Deployment environment (dev, staging, production)"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "cloud_provider" {
|
||||
description = "Target cloud provider (aws, gcp, azure, oci)"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
description = "Cloud region for deployment"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "deployment_slot" {
|
||||
description = "Deployment slot identifier for blue-green: blue or green"
|
||||
type = string
|
||||
default = "blue"
|
||||
validation {
|
||||
condition = contains(["blue", "green"], var.deployment_slot)
|
||||
error_message = "deployment_slot must be blue or green."
|
||||
}
|
||||
}
|
||||
|
||||
variable "container_image" {
|
||||
description = "Container image URI for the main application"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "mcp_container_image" {
|
||||
description = "Container image URI for the MCP sidecar (empty to disable)"
|
||||
type = string
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "app_port" {
|
||||
description = "Port the application container listens on"
|
||||
type = number
|
||||
default = 4820
|
||||
}
|
||||
|
||||
variable "mcp_port" {
|
||||
description = "Port the MCP sidecar container listens on"
|
||||
type = number
|
||||
default = 8819
|
||||
}
|
||||
|
||||
variable "cpu" {
|
||||
description = "CPU units for the task (256, 512, 1024, 2048, 4096)"
|
||||
type = number
|
||||
default = 512
|
||||
validation {
|
||||
condition = contains([256, 512, 1024, 2048, 4096], var.cpu)
|
||||
error_message = "cpu must be one of: 256, 512, 1024, 2048, 4096."
|
||||
}
|
||||
}
|
||||
|
||||
variable "memory" {
|
||||
description = "Memory in MiB for the task"
|
||||
type = number
|
||||
default = 1024
|
||||
}
|
||||
|
||||
variable "desired_count" {
|
||||
description = "Desired number of running task instances"
|
||||
type = number
|
||||
default = 1
|
||||
}
|
||||
|
||||
variable "min_count" {
|
||||
description = "Minimum number of task instances for auto-scaling"
|
||||
type = number
|
||||
default = 1
|
||||
}
|
||||
|
||||
variable "max_count" {
|
||||
description = "Maximum number of task instances for auto-scaling"
|
||||
type = number
|
||||
default = 3
|
||||
}
|
||||
|
||||
variable "environment_variables" {
|
||||
description = "Map of environment variables for the application container"
|
||||
type = map(string)
|
||||
default = {}
|
||||
}
|
||||
|
||||
variable "health_check_path" {
|
||||
description = "HTTP path for container health checks"
|
||||
type = string
|
||||
default = "/api/health"
|
||||
}
|
||||
|
||||
variable "vpc_id" {
|
||||
description = "VPC ID for target group and networking"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "private_subnet_ids" {
|
||||
description = "Subnet IDs where tasks will be placed"
|
||||
type = list(string)
|
||||
}
|
||||
|
||||
variable "security_group_ids" {
|
||||
description = "Security group IDs attached to task ENIs"
|
||||
type = list(string)
|
||||
}
|
||||
|
||||
variable "storage_filesystem_id" {
|
||||
description = "EFS file system ID for persistent SQLite storage"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "storage_mount_targets" {
|
||||
description = "EFS mount target IDs (ensures mount targets exist before service)"
|
||||
type = list(string)
|
||||
default = []
|
||||
}
|
||||
|
||||
variable "autoscaling_cpu_target" {
|
||||
description = "Target CPU utilization percentage for auto-scaling"
|
||||
type = number
|
||||
default = 70
|
||||
}
|
||||
|
||||
variable "autoscaling_memory_target" {
|
||||
description = "Target memory utilization percentage for auto-scaling"
|
||||
type = number
|
||||
default = 80
|
||||
}
|
||||
|
||||
variable "tags" {
|
||||
description = "Resource tags"
|
||||
type = map(string)
|
||||
default = {}
|
||||
}
|
||||
@@ -0,0 +1,142 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Database module – Persistent storage for SQLite
|
||||
#
|
||||
# Creates a managed network file system (EFS on AWS) with:
|
||||
# - Encryption at rest and in transit
|
||||
# - Automated backup policy
|
||||
# - Mount targets in each private subnet
|
||||
# - Performance mode optimised for SQLite workloads
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
locals {
|
||||
name_prefix = lower(replace("${var.project_name}-${var.environment}", "_", "-"))
|
||||
|
||||
common_tags = merge(
|
||||
{
|
||||
module = "database"
|
||||
},
|
||||
var.tags,
|
||||
)
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# EFS file system
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_efs_file_system" "main" {
|
||||
creation_token = "${local.name_prefix}-data"
|
||||
encrypted = true
|
||||
|
||||
# General Purpose is optimal for SQLite (latency-sensitive small I/O)
|
||||
performance_mode = "generalPurpose"
|
||||
throughput_mode = "elastic"
|
||||
|
||||
lifecycle_policy {
|
||||
transition_to_ia = "AFTER_30_DAYS"
|
||||
}
|
||||
|
||||
lifecycle_policy {
|
||||
transition_to_primary_storage_class = "AFTER_1_ACCESS"
|
||||
}
|
||||
|
||||
tags = merge(local.common_tags, {
|
||||
Name = "${local.name_prefix}-efs"
|
||||
})
|
||||
|
||||
lifecycle {
|
||||
prevent_destroy = true
|
||||
}
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# EFS backup policy
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_efs_backup_policy" "main" {
|
||||
file_system_id = aws_efs_file_system.main.id
|
||||
|
||||
backup_policy {
|
||||
status = var.enable_backup ? "ENABLED" : "DISABLED"
|
||||
}
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# EFS mount targets (one per private subnet / AZ)
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_efs_mount_target" "main" {
|
||||
count = length(var.private_subnet_ids)
|
||||
|
||||
file_system_id = aws_efs_file_system.main.id
|
||||
subnet_id = var.private_subnet_ids[count.index]
|
||||
security_groups = var.allowed_security_group_ids
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# EFS access point – scoped to /app/data for the container workload
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_efs_access_point" "app_data" {
|
||||
file_system_id = aws_efs_file_system.main.id
|
||||
|
||||
posix_user {
|
||||
uid = 1000
|
||||
gid = 1000
|
||||
}
|
||||
|
||||
root_directory {
|
||||
path = "/app-data"
|
||||
|
||||
creation_info {
|
||||
owner_uid = 1000
|
||||
owner_gid = 1000
|
||||
permissions = "0755"
|
||||
}
|
||||
}
|
||||
|
||||
tags = merge(local.common_tags, {
|
||||
Name = "${local.name_prefix}-app-data-ap"
|
||||
})
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# EFS file system policy – enforce encryption in transit
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_efs_file_system_policy" "main" {
|
||||
file_system_id = aws_efs_file_system.main.id
|
||||
|
||||
policy = jsonencode({
|
||||
Version = "2012-10-17"
|
||||
Statement = [
|
||||
{
|
||||
Sid = "EnforceEncryptInTransit"
|
||||
Effect = "Deny"
|
||||
Principal = { AWS = "*" }
|
||||
Action = "*"
|
||||
Resource = aws_efs_file_system.main.arn
|
||||
Condition = {
|
||||
Bool = {
|
||||
"aws:SecureTransport" = "false"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
Sid = "AllowMountViaAccessPoint"
|
||||
Effect = "Allow"
|
||||
Principal = { AWS = "*" }
|
||||
Action = [
|
||||
"elasticfilesystem:ClientMount",
|
||||
"elasticfilesystem:ClientWrite",
|
||||
"elasticfilesystem:ClientRootAccess",
|
||||
]
|
||||
Resource = aws_efs_file_system.main.arn
|
||||
Condition = {
|
||||
Bool = {
|
||||
"elasticfilesystem:AccessedViaMountTarget" = "true"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Database module outputs
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
output "filesystem_id" {
|
||||
description = "ID of the EFS file system"
|
||||
value = aws_efs_file_system.main.id
|
||||
}
|
||||
|
||||
output "filesystem_arn" {
|
||||
description = "ARN of the EFS file system"
|
||||
value = aws_efs_file_system.main.arn
|
||||
}
|
||||
|
||||
output "filesystem_dns_name" {
|
||||
description = "DNS name of the EFS file system"
|
||||
value = aws_efs_file_system.main.dns_name
|
||||
}
|
||||
|
||||
output "mount_target_ids" {
|
||||
description = "IDs of the EFS mount targets"
|
||||
value = aws_efs_mount_target.main[*].id
|
||||
}
|
||||
|
||||
output "mount_target_ips" {
|
||||
description = "IP addresses of the EFS mount targets"
|
||||
value = aws_efs_mount_target.main[*].ip_address
|
||||
}
|
||||
|
||||
output "access_point_id" {
|
||||
description = "ID of the EFS access point for /app/data"
|
||||
value = aws_efs_access_point.app_data.id
|
||||
}
|
||||
|
||||
output "access_point_arn" {
|
||||
description = "ARN of the EFS access point"
|
||||
value = aws_efs_access_point.app_data.arn
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Database module variables
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
variable "project_name" {
|
||||
description = "Project identifier used in resource naming"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "environment" {
|
||||
description = "Deployment environment (dev, staging, production)"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "cloud_provider" {
|
||||
description = "Target cloud provider (aws, gcp, azure, oci)"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
description = "Cloud region for deployment"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "storage_size_gb" {
|
||||
description = "Storage allocation in GiB (used by providers with provisioned capacity)"
|
||||
type = number
|
||||
default = 20
|
||||
validation {
|
||||
condition = var.storage_size_gb >= 1
|
||||
error_message = "storage_size_gb must be at least 1 GiB."
|
||||
}
|
||||
}
|
||||
|
||||
variable "enable_backup" {
|
||||
description = "Enable automated backup of the file system"
|
||||
type = bool
|
||||
default = true
|
||||
}
|
||||
|
||||
variable "private_subnet_ids" {
|
||||
description = "Private subnet IDs for mount targets"
|
||||
type = list(string)
|
||||
}
|
||||
|
||||
variable "vpc_id" {
|
||||
description = "VPC ID for security group association"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "allowed_security_group_ids" {
|
||||
description = "Security group IDs allowed to mount the file system"
|
||||
type = list(string)
|
||||
}
|
||||
|
||||
variable "tags" {
|
||||
description = "Resource tags"
|
||||
type = map(string)
|
||||
default = {}
|
||||
}
|
||||
@@ -0,0 +1,247 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Load Balancer module – Application load balancer with WebSocket support
|
||||
#
|
||||
# Provisions:
|
||||
# - ALB in public subnets
|
||||
# - HTTPS listener with TLS termination
|
||||
# - HTTP → HTTPS redirect
|
||||
# - Weighted target groups for blue/green and canary deployments
|
||||
# - Sticky sessions for WebSocket connections
|
||||
# - Path-based routing for MCP sidecar (/mcp/*)
|
||||
# - Health checks at /api/health
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
# Production TLS enforcement — prevents deploying production without encryption
|
||||
check "production_tls_required" {
|
||||
assert {
|
||||
condition = var.environment != "production" || var.tls_certificate_arn != ""
|
||||
error_message = "Production deployments require TLS. Set tls_certificate_arn and domain_name."
|
||||
}
|
||||
}
|
||||
|
||||
check "production_domain_required" {
|
||||
assert {
|
||||
condition = var.environment != "production" || var.domain_name != ""
|
||||
error_message = "Production deployments require a domain name. Set domain_name."
|
||||
}
|
||||
}
|
||||
|
||||
locals {
|
||||
name_prefix = lower(replace("${var.project_name}-${var.environment}", "_", "-"))
|
||||
|
||||
# Determine whether TLS is configured
|
||||
has_tls = var.tls_certificate_arn != ""
|
||||
|
||||
common_tags = merge(
|
||||
{
|
||||
module = "loadbalancer"
|
||||
},
|
||||
var.tags,
|
||||
)
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Application Load Balancer
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_lb" "main" {
|
||||
name = "${local.name_prefix}-alb"
|
||||
internal = false
|
||||
load_balancer_type = "application"
|
||||
security_groups = var.security_group_ids
|
||||
subnets = var.public_subnet_ids
|
||||
|
||||
enable_deletion_protection = var.environment == "production" ? true : var.enable_deletion_protection
|
||||
enable_http2 = true
|
||||
idle_timeout = 300 # WebSocket connections may be long-lived
|
||||
|
||||
drop_invalid_header_fields = true
|
||||
|
||||
tags = merge(local.common_tags, {
|
||||
Name = "${local.name_prefix}-alb"
|
||||
})
|
||||
|
||||
lifecycle {
|
||||
prevent_destroy = false
|
||||
# For production, set enable_deletion_protection = true above (enforced automatically)
|
||||
}
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# HTTPS listener (primary – with weighted target groups for blue/green)
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_lb_listener" "https" {
|
||||
count = local.has_tls ? 1 : 0
|
||||
|
||||
load_balancer_arn = aws_lb.main.arn
|
||||
port = 443
|
||||
protocol = "HTTPS"
|
||||
ssl_policy = "ELBSecurityPolicy-TLS13-1-2-2021-06"
|
||||
certificate_arn = var.tls_certificate_arn
|
||||
|
||||
default_action {
|
||||
type = "forward"
|
||||
|
||||
forward {
|
||||
target_group {
|
||||
arn = var.blue_target_group_arn
|
||||
weight = var.blue_weight
|
||||
}
|
||||
|
||||
target_group {
|
||||
arn = var.green_target_group_arn
|
||||
weight = var.green_weight
|
||||
}
|
||||
|
||||
stickiness {
|
||||
enabled = true
|
||||
duration = 86400
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# HTTP listener – redirect to HTTPS when TLS is configured, else forward
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_lb_listener" "http_redirect" {
|
||||
count = local.has_tls ? 1 : 0
|
||||
|
||||
load_balancer_arn = aws_lb.main.arn
|
||||
port = 80
|
||||
protocol = "HTTP"
|
||||
|
||||
default_action {
|
||||
type = "redirect"
|
||||
|
||||
redirect {
|
||||
port = "443"
|
||||
protocol = "HTTPS"
|
||||
status_code = "HTTP_301"
|
||||
}
|
||||
}
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
resource "aws_lb_listener" "http_forward" {
|
||||
count = local.has_tls ? 0 : 1
|
||||
|
||||
load_balancer_arn = aws_lb.main.arn
|
||||
port = 80
|
||||
protocol = "HTTP"
|
||||
|
||||
default_action {
|
||||
type = "forward"
|
||||
|
||||
forward {
|
||||
target_group {
|
||||
arn = var.blue_target_group_arn
|
||||
weight = var.blue_weight
|
||||
}
|
||||
|
||||
target_group {
|
||||
arn = var.green_target_group_arn
|
||||
weight = var.green_weight
|
||||
}
|
||||
|
||||
stickiness {
|
||||
enabled = true
|
||||
duration = 86400
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# MCP sidecar target group
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_lb_target_group" "mcp" {
|
||||
name_prefix = "mcp-"
|
||||
port = var.mcp_port
|
||||
protocol = "HTTP"
|
||||
vpc_id = var.vpc_id
|
||||
target_type = "ip"
|
||||
|
||||
health_check {
|
||||
enabled = true
|
||||
path = "/"
|
||||
port = tostring(var.mcp_port)
|
||||
protocol = "HTTP"
|
||||
healthy_threshold = var.health_check_healthy_threshold
|
||||
unhealthy_threshold = var.health_check_unhealthy_threshold
|
||||
timeout = var.health_check_timeout
|
||||
interval = var.health_check_interval
|
||||
matcher = "200-404"
|
||||
}
|
||||
|
||||
tags = merge(local.common_tags, {
|
||||
Name = "${local.name_prefix}-mcp-tg"
|
||||
})
|
||||
|
||||
lifecycle {
|
||||
create_before_destroy = true
|
||||
}
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Path-based routing rule for MCP sidecar (/mcp/*)
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_lb_listener_rule" "mcp_https" {
|
||||
count = local.has_tls ? 1 : 0
|
||||
|
||||
listener_arn = aws_lb_listener.https[0].arn
|
||||
priority = 10
|
||||
|
||||
action {
|
||||
type = "forward"
|
||||
target_group_arn = aws_lb_target_group.mcp.arn
|
||||
}
|
||||
|
||||
condition {
|
||||
path_pattern {
|
||||
values = ["/mcp", "/mcp/*"]
|
||||
}
|
||||
}
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
resource "aws_lb_listener_rule" "mcp_http" {
|
||||
count = local.has_tls ? 0 : 1
|
||||
|
||||
listener_arn = aws_lb_listener.http_forward[0].arn
|
||||
priority = 10
|
||||
|
||||
action {
|
||||
type = "forward"
|
||||
target_group_arn = aws_lb_target_group.mcp.arn
|
||||
}
|
||||
|
||||
condition {
|
||||
path_pattern {
|
||||
values = ["/mcp", "/mcp/*"]
|
||||
}
|
||||
}
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Weight sum validation
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
check "lb_weight_sum" {
|
||||
assert {
|
||||
condition = var.blue_weight + var.green_weight == 100
|
||||
error_message = "blue_weight (${var.blue_weight}) + green_weight (${var.green_weight}) must sum to 100 for correct traffic routing."
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Load Balancer module outputs
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
output "loadbalancer_arn" {
|
||||
description = "ARN of the application load balancer"
|
||||
value = aws_lb.main.arn
|
||||
}
|
||||
|
||||
output "loadbalancer_id" {
|
||||
description = "ID of the application load balancer"
|
||||
value = aws_lb.main.id
|
||||
}
|
||||
|
||||
output "dns_name" {
|
||||
description = "DNS name of the application load balancer"
|
||||
value = aws_lb.main.dns_name
|
||||
}
|
||||
|
||||
output "zone_id" {
|
||||
description = "Route53 zone ID for the load balancer (alias records)"
|
||||
value = aws_lb.main.zone_id
|
||||
}
|
||||
|
||||
output "application_url" {
|
||||
description = "Full URL to access the application"
|
||||
value = local.has_tls ? "https://${var.domain_name != "" ? var.domain_name : aws_lb.main.dns_name}" : "http://${aws_lb.main.dns_name}"
|
||||
}
|
||||
|
||||
output "https_listener_arn" {
|
||||
description = "ARN of the HTTPS listener (empty if TLS not configured)"
|
||||
value = local.has_tls ? aws_lb_listener.https[0].arn : ""
|
||||
}
|
||||
|
||||
output "http_listener_arn" {
|
||||
description = "ARN of the HTTP listener"
|
||||
value = local.has_tls ? aws_lb_listener.http_redirect[0].arn : aws_lb_listener.http_forward[0].arn
|
||||
}
|
||||
|
||||
output "mcp_target_group_arn" {
|
||||
description = "ARN of the MCP sidecar target group"
|
||||
value = aws_lb_target_group.mcp.arn
|
||||
}
|
||||
@@ -0,0 +1,134 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Load Balancer module variables
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
variable "project_name" {
|
||||
description = "Project identifier used in resource naming"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "environment" {
|
||||
description = "Deployment environment (dev, staging, production)"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "cloud_provider" {
|
||||
description = "Target cloud provider (aws, gcp, azure, oci)"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
description = "Cloud region for deployment"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "vpc_id" {
|
||||
description = "VPC ID for target group association"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "public_subnet_ids" {
|
||||
description = "Public subnet IDs for load balancer placement"
|
||||
type = list(string)
|
||||
}
|
||||
|
||||
variable "security_group_ids" {
|
||||
description = "Security group IDs attached to the load balancer"
|
||||
type = list(string)
|
||||
}
|
||||
|
||||
variable "app_port" {
|
||||
description = "Application container port"
|
||||
type = number
|
||||
default = 4820
|
||||
}
|
||||
|
||||
variable "mcp_port" {
|
||||
description = "MCP sidecar container port"
|
||||
type = number
|
||||
default = 8819
|
||||
}
|
||||
|
||||
variable "tls_certificate_arn" {
|
||||
description = "ARN of the TLS certificate for HTTPS (required for production)"
|
||||
type = string
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "domain_name" {
|
||||
description = "Fully qualified domain name for the application (required for production)"
|
||||
type = string
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "blue_target_group_arn" {
|
||||
description = "ARN of the blue deployment target group"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "green_target_group_arn" {
|
||||
description = "ARN of the green deployment target group"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "blue_weight" {
|
||||
description = "Traffic weight for blue target group (0-100)"
|
||||
type = number
|
||||
default = 100
|
||||
validation {
|
||||
condition = var.blue_weight >= 0 && var.blue_weight <= 100
|
||||
error_message = "blue_weight must be between 0 and 100."
|
||||
}
|
||||
}
|
||||
|
||||
variable "green_weight" {
|
||||
description = "Traffic weight for green target group (0-100)"
|
||||
type = number
|
||||
default = 0
|
||||
validation {
|
||||
condition = var.green_weight >= 0 && var.green_weight <= 100
|
||||
error_message = "green_weight must be between 0 and 100."
|
||||
}
|
||||
}
|
||||
|
||||
variable "health_check_path" {
|
||||
description = "HTTP path for health checks"
|
||||
type = string
|
||||
default = "/api/health"
|
||||
}
|
||||
|
||||
variable "health_check_interval" {
|
||||
description = "Seconds between health checks"
|
||||
type = number
|
||||
default = 30
|
||||
}
|
||||
|
||||
variable "health_check_timeout" {
|
||||
description = "Seconds before a health check times out"
|
||||
type = number
|
||||
default = 5
|
||||
}
|
||||
|
||||
variable "health_check_healthy_threshold" {
|
||||
description = "Consecutive successes to mark healthy"
|
||||
type = number
|
||||
default = 2
|
||||
}
|
||||
|
||||
variable "health_check_unhealthy_threshold" {
|
||||
description = "Consecutive failures to mark unhealthy"
|
||||
type = number
|
||||
default = 3
|
||||
}
|
||||
|
||||
variable "tags" {
|
||||
description = "Resource tags"
|
||||
type = map(string)
|
||||
default = {}
|
||||
}
|
||||
|
||||
variable "enable_deletion_protection" {
|
||||
description = "Enable deletion protection on the load balancer (recommended for production)"
|
||||
type = bool
|
||||
default = false
|
||||
}
|
||||
@@ -0,0 +1,312 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Monitoring module – Observability, alerting, and dashboards
|
||||
#
|
||||
# Provisions:
|
||||
# - CloudWatch log groups for centralized log aggregation
|
||||
# - Metric alarms for error rate, latency, disk, unhealthy hosts
|
||||
# - SNS topic for alert notifications
|
||||
# - CloudWatch dashboard with key operational metrics
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
locals {
|
||||
name_prefix = lower(replace("${var.project_name}-${var.environment}", "_", "-"))
|
||||
|
||||
common_tags = merge(
|
||||
{
|
||||
module = "monitoring"
|
||||
},
|
||||
var.tags,
|
||||
)
|
||||
|
||||
# Parse ALB ARN suffix for CloudWatch metric dimensions
|
||||
alb_arn_suffix = try(
|
||||
regex("app/.*$", var.loadbalancer_arn),
|
||||
""
|
||||
)
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# SNS topic for alert notifications
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_sns_topic" "alerts" {
|
||||
name = "${local.name_prefix}-alerts"
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
resource "aws_sns_topic_subscription" "email" {
|
||||
count = var.alert_email != "" ? 1 : 0
|
||||
|
||||
topic_arn = aws_sns_topic.alerts.arn
|
||||
protocol = "email"
|
||||
endpoint = var.alert_email
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# CloudWatch log group (application-level)
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_cloudwatch_log_group" "application" {
|
||||
name = "/ccam/${local.name_prefix}"
|
||||
retention_in_days = var.log_retention_days
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Metric alarms
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
# High 5xx error rate from ALB
|
||||
resource "aws_cloudwatch_metric_alarm" "high_5xx_rate" {
|
||||
alarm_name = "${local.name_prefix}-high-5xx-error-rate"
|
||||
comparison_operator = "GreaterThanThreshold"
|
||||
evaluation_periods = 3
|
||||
metric_name = "HTTPCode_Target_5XX_Count"
|
||||
namespace = "AWS/ApplicationELB"
|
||||
period = 60
|
||||
statistic = "Sum"
|
||||
threshold = 10
|
||||
alarm_description = "High 5XX error rate detected on ${local.name_prefix} ALB"
|
||||
treat_missing_data = "notBreaching"
|
||||
|
||||
dimensions = {
|
||||
LoadBalancer = local.alb_arn_suffix
|
||||
}
|
||||
|
||||
alarm_actions = [aws_sns_topic.alerts.arn]
|
||||
ok_actions = [aws_sns_topic.alerts.arn]
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# High target response time (latency)
|
||||
resource "aws_cloudwatch_metric_alarm" "high_latency" {
|
||||
alarm_name = "${local.name_prefix}-high-latency"
|
||||
comparison_operator = "GreaterThanThreshold"
|
||||
evaluation_periods = 3
|
||||
metric_name = "TargetResponseTime"
|
||||
namespace = "AWS/ApplicationELB"
|
||||
period = 60
|
||||
statistic = "Average"
|
||||
threshold = 2.0 # seconds
|
||||
alarm_description = "High average latency (>2s) on ${local.name_prefix} ALB"
|
||||
treat_missing_data = "notBreaching"
|
||||
|
||||
dimensions = {
|
||||
LoadBalancer = local.alb_arn_suffix
|
||||
}
|
||||
|
||||
alarm_actions = [aws_sns_topic.alerts.arn]
|
||||
ok_actions = [aws_sns_topic.alerts.arn]
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# Unhealthy host count
|
||||
resource "aws_cloudwatch_metric_alarm" "unhealthy_hosts" {
|
||||
count = length(var.target_group_arns)
|
||||
|
||||
alarm_name = "${local.name_prefix}-unhealthy-hosts-${count.index}"
|
||||
comparison_operator = "GreaterThanThreshold"
|
||||
evaluation_periods = 2
|
||||
metric_name = "UnHealthyHostCount"
|
||||
namespace = "AWS/ApplicationELB"
|
||||
period = 60
|
||||
statistic = "Maximum"
|
||||
threshold = 0
|
||||
alarm_description = "Unhealthy targets detected in target group ${count.index}"
|
||||
treat_missing_data = "notBreaching"
|
||||
|
||||
dimensions = {
|
||||
LoadBalancer = local.alb_arn_suffix
|
||||
TargetGroup = try(regex("targetgroup/.*$", var.target_group_arns[count.index]), "")
|
||||
}
|
||||
|
||||
alarm_actions = [aws_sns_topic.alerts.arn]
|
||||
ok_actions = [aws_sns_topic.alerts.arn]
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# EFS burst credit balance (low disk throughput)
|
||||
resource "aws_cloudwatch_metric_alarm" "efs_burst_credits" {
|
||||
alarm_name = "${local.name_prefix}-efs-low-burst-credits"
|
||||
comparison_operator = "LessThanThreshold"
|
||||
evaluation_periods = 3
|
||||
metric_name = "BurstCreditBalance"
|
||||
namespace = "AWS/EFS"
|
||||
period = 300
|
||||
statistic = "Average"
|
||||
threshold = 1000000000 # 1 GiB in bytes
|
||||
alarm_description = "EFS burst credits running low for ${local.name_prefix}"
|
||||
treat_missing_data = "notBreaching"
|
||||
|
||||
dimensions = {
|
||||
FileSystemId = var.filesystem_id
|
||||
}
|
||||
|
||||
alarm_actions = [aws_sns_topic.alerts.arn]
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# ECS CPU utilisation (cluster-level)
|
||||
resource "aws_cloudwatch_metric_alarm" "ecs_high_cpu" {
|
||||
alarm_name = "${local.name_prefix}-ecs-high-cpu"
|
||||
comparison_operator = "GreaterThanThreshold"
|
||||
evaluation_periods = 3
|
||||
metric_name = "CPUUtilization"
|
||||
namespace = "AWS/ECS"
|
||||
period = 300
|
||||
statistic = "Average"
|
||||
threshold = 85
|
||||
alarm_description = "High ECS CPU utilisation (>85%) for cluster ${var.compute_cluster_name}"
|
||||
treat_missing_data = "notBreaching"
|
||||
|
||||
dimensions = {
|
||||
ClusterName = var.compute_cluster_name
|
||||
}
|
||||
|
||||
alarm_actions = [aws_sns_topic.alerts.arn]
|
||||
ok_actions = [aws_sns_topic.alerts.arn]
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# ECS Memory utilisation
|
||||
resource "aws_cloudwatch_metric_alarm" "ecs_high_memory" {
|
||||
alarm_name = "${local.name_prefix}-ecs-high-memory"
|
||||
comparison_operator = "GreaterThanThreshold"
|
||||
evaluation_periods = 3
|
||||
metric_name = "MemoryUtilization"
|
||||
namespace = "AWS/ECS"
|
||||
period = 300
|
||||
statistic = "Average"
|
||||
threshold = 85
|
||||
alarm_description = "High ECS memory utilisation (>85%) for cluster ${var.compute_cluster_name}"
|
||||
treat_missing_data = "notBreaching"
|
||||
|
||||
dimensions = {
|
||||
ClusterName = var.compute_cluster_name
|
||||
}
|
||||
|
||||
alarm_actions = [aws_sns_topic.alerts.arn]
|
||||
ok_actions = [aws_sns_topic.alerts.arn]
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# CloudWatch Dashboard
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_cloudwatch_dashboard" "main" {
|
||||
dashboard_name = local.name_prefix
|
||||
dashboard_body = jsonencode({
|
||||
widgets = [
|
||||
{
|
||||
type = "metric"
|
||||
x = 0
|
||||
y = 0
|
||||
width = 12
|
||||
height = 6
|
||||
properties = {
|
||||
title = "ALB Request Count"
|
||||
region = var.region
|
||||
metrics = [
|
||||
["AWS/ApplicationELB", "RequestCount", "LoadBalancer", local.alb_arn_suffix, { stat = "Sum", period = 60 }]
|
||||
]
|
||||
view = "timeSeries"
|
||||
stacked = false
|
||||
}
|
||||
},
|
||||
{
|
||||
type = "metric"
|
||||
x = 12
|
||||
y = 0
|
||||
width = 12
|
||||
height = 6
|
||||
properties = {
|
||||
title = "ALB Response Time"
|
||||
region = var.region
|
||||
metrics = [
|
||||
["AWS/ApplicationELB", "TargetResponseTime", "LoadBalancer", local.alb_arn_suffix, { stat = "Average", period = 60 }],
|
||||
["AWS/ApplicationELB", "TargetResponseTime", "LoadBalancer", local.alb_arn_suffix, { stat = "p99", period = 60 }],
|
||||
]
|
||||
view = "timeSeries"
|
||||
stacked = false
|
||||
}
|
||||
},
|
||||
{
|
||||
type = "metric"
|
||||
x = 0
|
||||
y = 6
|
||||
width = 12
|
||||
height = 6
|
||||
properties = {
|
||||
title = "HTTP Error Rates"
|
||||
region = var.region
|
||||
metrics = [
|
||||
["AWS/ApplicationELB", "HTTPCode_Target_4XX_Count", "LoadBalancer", local.alb_arn_suffix, { stat = "Sum", period = 60 }],
|
||||
["AWS/ApplicationELB", "HTTPCode_Target_5XX_Count", "LoadBalancer", local.alb_arn_suffix, { stat = "Sum", period = 60 }],
|
||||
]
|
||||
view = "timeSeries"
|
||||
stacked = false
|
||||
}
|
||||
},
|
||||
{
|
||||
type = "metric"
|
||||
x = 12
|
||||
y = 6
|
||||
width = 12
|
||||
height = 6
|
||||
properties = {
|
||||
title = "ECS CPU & Memory"
|
||||
region = var.region
|
||||
metrics = [
|
||||
["AWS/ECS", "CPUUtilization", "ClusterName", var.compute_cluster_name, { stat = "Average", period = 60 }],
|
||||
["AWS/ECS", "MemoryUtilization", "ClusterName", var.compute_cluster_name, { stat = "Average", period = 60 }],
|
||||
]
|
||||
view = "timeSeries"
|
||||
stacked = false
|
||||
}
|
||||
},
|
||||
{
|
||||
type = "metric"
|
||||
x = 0
|
||||
y = 12
|
||||
width = 12
|
||||
height = 6
|
||||
properties = {
|
||||
title = "EFS I/O"
|
||||
region = var.region
|
||||
metrics = [
|
||||
["AWS/EFS", "DataReadIOBytes", "FileSystemId", var.filesystem_id, { stat = "Sum", period = 60 }],
|
||||
["AWS/EFS", "DataWriteIOBytes", "FileSystemId", var.filesystem_id, { stat = "Sum", period = 60 }],
|
||||
]
|
||||
view = "timeSeries"
|
||||
stacked = false
|
||||
}
|
||||
},
|
||||
{
|
||||
type = "metric"
|
||||
x = 12
|
||||
y = 12
|
||||
width = 12
|
||||
height = 6
|
||||
properties = {
|
||||
title = "Healthy vs Unhealthy Hosts"
|
||||
region = var.region
|
||||
metrics = [
|
||||
["AWS/ApplicationELB", "HealthyHostCount", "LoadBalancer", local.alb_arn_suffix, { stat = "Average", period = 60 }],
|
||||
["AWS/ApplicationELB", "UnHealthyHostCount", "LoadBalancer", local.alb_arn_suffix, { stat = "Average", period = 60 }],
|
||||
]
|
||||
view = "timeSeries"
|
||||
stacked = false
|
||||
}
|
||||
},
|
||||
]
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Monitoring module outputs
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
output "sns_topic_arn" {
|
||||
description = "ARN of the SNS alert topic"
|
||||
value = aws_sns_topic.alerts.arn
|
||||
}
|
||||
|
||||
output "log_group_name" {
|
||||
description = "Name of the CloudWatch log group"
|
||||
value = aws_cloudwatch_log_group.application.name
|
||||
}
|
||||
|
||||
output "log_group_arn" {
|
||||
description = "ARN of the CloudWatch log group"
|
||||
value = aws_cloudwatch_log_group.application.arn
|
||||
}
|
||||
|
||||
output "dashboard_name" {
|
||||
description = "Name of the CloudWatch dashboard"
|
||||
value = aws_cloudwatch_dashboard.main.dashboard_name
|
||||
}
|
||||
|
||||
output "dashboard_url" {
|
||||
description = "URL to the CloudWatch dashboard in the AWS console"
|
||||
value = "https://${var.region}.console.aws.amazon.com/cloudwatch/home?region=${var.region}#dashboards:name=${aws_cloudwatch_dashboard.main.dashboard_name}"
|
||||
}
|
||||
|
||||
output "alarm_arns" {
|
||||
description = "ARNs of all configured CloudWatch alarms"
|
||||
value = concat(
|
||||
[aws_cloudwatch_metric_alarm.high_5xx_rate.arn],
|
||||
[aws_cloudwatch_metric_alarm.high_latency.arn],
|
||||
[aws_cloudwatch_metric_alarm.efs_burst_credits.arn],
|
||||
[aws_cloudwatch_metric_alarm.ecs_high_cpu.arn],
|
||||
[aws_cloudwatch_metric_alarm.ecs_high_memory.arn],
|
||||
aws_cloudwatch_metric_alarm.unhealthy_hosts[*].arn,
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,66 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Monitoring module variables
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
variable "project_name" {
|
||||
description = "Project identifier used in resource naming"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "environment" {
|
||||
description = "Deployment environment (dev, staging, production)"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "cloud_provider" {
|
||||
description = "Target cloud provider (aws, gcp, azure, oci)"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
description = "Cloud region for deployment"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "alert_email" {
|
||||
description = "Email address for alert notifications (empty to skip)"
|
||||
type = string
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "log_retention_days" {
|
||||
description = "Number of days to retain application logs"
|
||||
type = number
|
||||
default = 30
|
||||
validation {
|
||||
condition = contains([1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, 3653], var.log_retention_days)
|
||||
error_message = "log_retention_days must be a valid CloudWatch retention period."
|
||||
}
|
||||
}
|
||||
|
||||
variable "loadbalancer_arn" {
|
||||
description = "ARN of the application load balancer to monitor"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "target_group_arns" {
|
||||
description = "ARNs of target groups to monitor for unhealthy hosts"
|
||||
type = list(string)
|
||||
default = []
|
||||
}
|
||||
|
||||
variable "compute_cluster_name" {
|
||||
description = "Name of the ECS cluster for compute metrics"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "filesystem_id" {
|
||||
description = "EFS file system ID for storage metrics"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "tags" {
|
||||
description = "Resource tags"
|
||||
type = map(string)
|
||||
default = {}
|
||||
}
|
||||
@@ -0,0 +1,279 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Networking module – Cloud-agnostic VPC / VNet / VCN abstraction
|
||||
#
|
||||
# Creates the foundational network topology: virtual network, public and
|
||||
# private subnets across availability zones, NAT gateway, internet gateway,
|
||||
# route tables, and security groups / firewall rules.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
locals {
|
||||
name_prefix = lower(replace("${var.project_name}-${var.environment}", "_", "-"))
|
||||
|
||||
# Default AZs when none provided – derive from region
|
||||
default_azs = [
|
||||
"${var.region}a",
|
||||
"${var.region}b",
|
||||
"${var.region}c",
|
||||
]
|
||||
|
||||
availability_zones = length(var.availability_zones) > 0 ? var.availability_zones : local.default_azs
|
||||
|
||||
# Number of AZs determines subnet count
|
||||
az_count = min(length(local.availability_zones), length(var.public_subnet_cidrs), length(var.private_subnet_cidrs))
|
||||
|
||||
common_tags = merge(
|
||||
{
|
||||
module = "networking"
|
||||
},
|
||||
var.tags,
|
||||
)
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# VPC
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_vpc" "main" {
|
||||
cidr_block = var.vpc_cidr
|
||||
enable_dns_support = true
|
||||
enable_dns_hostnames = true
|
||||
|
||||
tags = merge(local.common_tags, {
|
||||
Name = "${local.name_prefix}-vpc"
|
||||
})
|
||||
|
||||
lifecycle {
|
||||
prevent_destroy = false
|
||||
}
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Internet gateway
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_internet_gateway" "main" {
|
||||
vpc_id = aws_vpc.main.id
|
||||
|
||||
tags = merge(local.common_tags, {
|
||||
Name = "${local.name_prefix}-igw"
|
||||
})
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Public subnets
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_subnet" "public" {
|
||||
count = local.az_count
|
||||
|
||||
vpc_id = aws_vpc.main.id
|
||||
cidr_block = var.public_subnet_cidrs[count.index]
|
||||
availability_zone = local.availability_zones[count.index]
|
||||
map_public_ip_on_launch = true
|
||||
|
||||
tags = merge(local.common_tags, {
|
||||
Name = "${local.name_prefix}-public-${local.availability_zones[count.index]}"
|
||||
tier = "public"
|
||||
})
|
||||
}
|
||||
|
||||
resource "aws_route_table" "public" {
|
||||
vpc_id = aws_vpc.main.id
|
||||
|
||||
route {
|
||||
cidr_block = "0.0.0.0/0"
|
||||
gateway_id = aws_internet_gateway.main.id
|
||||
}
|
||||
|
||||
tags = merge(local.common_tags, {
|
||||
Name = "${local.name_prefix}-public-rt"
|
||||
})
|
||||
}
|
||||
|
||||
resource "aws_route_table_association" "public" {
|
||||
count = local.az_count
|
||||
|
||||
subnet_id = aws_subnet.public[count.index].id
|
||||
route_table_id = aws_route_table.public.id
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# NAT gateway (single, in first public subnet – cost-conscious default)
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_eip" "nat" {
|
||||
domain = "vpc"
|
||||
|
||||
tags = merge(local.common_tags, {
|
||||
Name = "${local.name_prefix}-nat-eip"
|
||||
})
|
||||
}
|
||||
|
||||
resource "aws_nat_gateway" "main" {
|
||||
allocation_id = aws_eip.nat.id
|
||||
subnet_id = aws_subnet.public[0].id
|
||||
|
||||
tags = merge(local.common_tags, {
|
||||
Name = "${local.name_prefix}-nat"
|
||||
})
|
||||
|
||||
depends_on = [aws_internet_gateway.main]
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Private subnets
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_subnet" "private" {
|
||||
count = local.az_count
|
||||
|
||||
vpc_id = aws_vpc.main.id
|
||||
cidr_block = var.private_subnet_cidrs[count.index]
|
||||
availability_zone = local.availability_zones[count.index]
|
||||
|
||||
tags = merge(local.common_tags, {
|
||||
Name = "${local.name_prefix}-private-${local.availability_zones[count.index]}"
|
||||
tier = "private"
|
||||
})
|
||||
}
|
||||
|
||||
resource "aws_route_table" "private" {
|
||||
vpc_id = aws_vpc.main.id
|
||||
|
||||
route {
|
||||
cidr_block = "0.0.0.0/0"
|
||||
nat_gateway_id = aws_nat_gateway.main.id
|
||||
}
|
||||
|
||||
tags = merge(local.common_tags, {
|
||||
Name = "${local.name_prefix}-private-rt"
|
||||
})
|
||||
}
|
||||
|
||||
resource "aws_route_table_association" "private" {
|
||||
count = local.az_count
|
||||
|
||||
subnet_id = aws_subnet.private[count.index].id
|
||||
route_table_id = aws_route_table.private.id
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Security groups
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
# Public (load-balancer-facing)
|
||||
resource "aws_security_group" "public" {
|
||||
name_prefix = "${local.name_prefix}-public-"
|
||||
description = "Allow HTTPS/HTTP inbound and all outbound"
|
||||
vpc_id = aws_vpc.main.id
|
||||
|
||||
ingress {
|
||||
description = "HTTPS"
|
||||
from_port = 443
|
||||
to_port = 443
|
||||
protocol = "tcp"
|
||||
cidr_blocks = ["0.0.0.0/0"]
|
||||
}
|
||||
|
||||
ingress {
|
||||
description = "HTTP (redirect)"
|
||||
from_port = 80
|
||||
to_port = 80
|
||||
protocol = "tcp"
|
||||
cidr_blocks = ["0.0.0.0/0"]
|
||||
}
|
||||
|
||||
egress {
|
||||
description = "All outbound"
|
||||
from_port = 0
|
||||
to_port = 0
|
||||
protocol = "-1"
|
||||
cidr_blocks = ["0.0.0.0/0"]
|
||||
}
|
||||
|
||||
tags = merge(local.common_tags, {
|
||||
Name = "${local.name_prefix}-public-sg"
|
||||
})
|
||||
|
||||
lifecycle {
|
||||
create_before_destroy = true
|
||||
}
|
||||
}
|
||||
|
||||
# Private (container-facing)
|
||||
resource "aws_security_group" "private" {
|
||||
name_prefix = "${local.name_prefix}-private-"
|
||||
description = "Allow traffic from public SG to app and MCP ports"
|
||||
vpc_id = aws_vpc.main.id
|
||||
|
||||
ingress {
|
||||
description = "Application port from LB"
|
||||
from_port = var.app_port
|
||||
to_port = var.app_port
|
||||
protocol = "tcp"
|
||||
security_groups = [aws_security_group.public.id]
|
||||
}
|
||||
|
||||
ingress {
|
||||
description = "MCP sidecar port from LB"
|
||||
from_port = var.mcp_port
|
||||
to_port = var.mcp_port
|
||||
protocol = "tcp"
|
||||
security_groups = [aws_security_group.public.id]
|
||||
}
|
||||
|
||||
ingress {
|
||||
description = "NFS (EFS) within VPC"
|
||||
from_port = 2049
|
||||
to_port = 2049
|
||||
protocol = "tcp"
|
||||
self = true
|
||||
}
|
||||
|
||||
egress {
|
||||
description = "All outbound"
|
||||
from_port = 0
|
||||
to_port = 0
|
||||
protocol = "-1"
|
||||
cidr_blocks = ["0.0.0.0/0"]
|
||||
}
|
||||
|
||||
tags = merge(local.common_tags, {
|
||||
Name = "${local.name_prefix}-private-sg"
|
||||
})
|
||||
|
||||
lifecycle {
|
||||
create_before_destroy = true
|
||||
}
|
||||
}
|
||||
|
||||
# EFS security group
|
||||
resource "aws_security_group" "storage" {
|
||||
name_prefix = "${local.name_prefix}-storage-"
|
||||
description = "Allow NFS access from private security group"
|
||||
vpc_id = aws_vpc.main.id
|
||||
|
||||
ingress {
|
||||
description = "NFS from private subnets"
|
||||
from_port = 2049
|
||||
to_port = 2049
|
||||
protocol = "tcp"
|
||||
security_groups = [aws_security_group.private.id]
|
||||
}
|
||||
|
||||
egress {
|
||||
description = "All outbound"
|
||||
from_port = 0
|
||||
to_port = 0
|
||||
protocol = "-1"
|
||||
cidr_blocks = ["0.0.0.0/0"]
|
||||
}
|
||||
|
||||
tags = merge(local.common_tags, {
|
||||
Name = "${local.name_prefix}-storage-sg"
|
||||
})
|
||||
|
||||
lifecycle {
|
||||
create_before_destroy = true
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Networking module outputs
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
output "vpc_id" {
|
||||
description = "ID of the provisioned VPC"
|
||||
value = aws_vpc.main.id
|
||||
}
|
||||
|
||||
output "vpc_cidr" {
|
||||
description = "CIDR block of the VPC"
|
||||
value = aws_vpc.main.cidr_block
|
||||
}
|
||||
|
||||
output "public_subnet_ids" {
|
||||
description = "IDs of the public subnets"
|
||||
value = aws_subnet.public[*].id
|
||||
}
|
||||
|
||||
output "private_subnet_ids" {
|
||||
description = "IDs of the private subnets"
|
||||
value = aws_subnet.private[*].id
|
||||
}
|
||||
|
||||
output "public_security_group_ids" {
|
||||
description = "Security group IDs for public-facing resources (LB)"
|
||||
value = [aws_security_group.public.id]
|
||||
}
|
||||
|
||||
output "private_security_group_ids" {
|
||||
description = "Security group IDs for private resources (containers)"
|
||||
value = [aws_security_group.private.id]
|
||||
}
|
||||
|
||||
output "storage_security_group_ids" {
|
||||
description = "Security group IDs for persistent storage"
|
||||
value = [aws_security_group.storage.id]
|
||||
}
|
||||
|
||||
output "nat_gateway_ip" {
|
||||
description = "Public IP of the NAT gateway"
|
||||
value = aws_eip.nat.public_ip
|
||||
}
|
||||
|
||||
output "internet_gateway_id" {
|
||||
description = "ID of the internet gateway"
|
||||
value = aws_internet_gateway.main.id
|
||||
}
|
||||
|
||||
output "availability_zones" {
|
||||
description = "Availability zones used for deployment"
|
||||
value = local.availability_zones
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Networking module variables
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
variable "project_name" {
|
||||
description = "Project identifier used in resource naming"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "environment" {
|
||||
description = "Deployment environment (dev, staging, production)"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "cloud_provider" {
|
||||
description = "Target cloud provider (aws, gcp, azure, oci)"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
description = "Cloud region for deployment"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "vpc_cidr" {
|
||||
description = "CIDR block for the virtual network"
|
||||
type = string
|
||||
default = "10.0.0.0/16"
|
||||
validation {
|
||||
condition = can(cidrhost(var.vpc_cidr, 0))
|
||||
error_message = "vpc_cidr must be a valid CIDR block."
|
||||
}
|
||||
}
|
||||
|
||||
variable "availability_zones" {
|
||||
description = "List of availability zones for multi-AZ deployment"
|
||||
type = list(string)
|
||||
default = []
|
||||
}
|
||||
|
||||
variable "public_subnet_cidrs" {
|
||||
description = "CIDR blocks for public subnets (one per AZ)"
|
||||
type = list(string)
|
||||
default = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
|
||||
}
|
||||
|
||||
variable "private_subnet_cidrs" {
|
||||
description = "CIDR blocks for private subnets (one per AZ)"
|
||||
type = list(string)
|
||||
default = ["10.0.11.0/24", "10.0.12.0/24", "10.0.13.0/24"]
|
||||
}
|
||||
|
||||
variable "app_port" {
|
||||
description = "Application container port"
|
||||
type = number
|
||||
default = 4820
|
||||
}
|
||||
|
||||
variable "mcp_port" {
|
||||
description = "MCP sidecar container port"
|
||||
type = number
|
||||
default = 8819
|
||||
}
|
||||
|
||||
variable "tags" {
|
||||
description = "Resource tags"
|
||||
type = map(string)
|
||||
default = {}
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Root module outputs – Claude Code Agent Monitor
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
output "application_url" {
|
||||
description = "Public URL of the Claude Code Agent Monitor dashboard"
|
||||
value = module.loadbalancer.application_url
|
||||
}
|
||||
|
||||
output "loadbalancer_dns" {
|
||||
description = "DNS name of the application load balancer"
|
||||
value = module.loadbalancer.dns_name
|
||||
}
|
||||
|
||||
output "vpc_id" {
|
||||
description = "ID of the provisioned VPC / VNet / VCN"
|
||||
value = module.networking.vpc_id
|
||||
}
|
||||
|
||||
output "private_subnet_ids" {
|
||||
description = "IDs of the private subnets hosting compute workloads"
|
||||
value = module.networking.private_subnet_ids
|
||||
}
|
||||
|
||||
output "public_subnet_ids" {
|
||||
description = "IDs of the public subnets hosting the load balancer"
|
||||
value = module.networking.public_subnet_ids
|
||||
}
|
||||
|
||||
output "filesystem_id" {
|
||||
description = "ID of the persistent file system for SQLite storage"
|
||||
value = module.database.filesystem_id
|
||||
}
|
||||
|
||||
output "blue_service_name" {
|
||||
description = "Name of the blue deployment compute service"
|
||||
value = module.compute_blue.service_name
|
||||
}
|
||||
|
||||
output "green_service_name" {
|
||||
description = "Name of the green deployment compute service"
|
||||
value = module.compute_green.service_name
|
||||
}
|
||||
|
||||
output "active_slot" {
|
||||
description = "Currently active deployment slot"
|
||||
value = var.active_deployment_slot
|
||||
}
|
||||
|
||||
output "monitoring_dashboard_url" {
|
||||
description = "URL of the monitoring dashboard (if enabled)"
|
||||
value = var.enable_monitoring ? module.monitoring[0].dashboard_url : "monitoring disabled"
|
||||
}
|
||||
@@ -0,0 +1,264 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# AWS Provider – Full implementation for Claude Code Agent Monitor
|
||||
#
|
||||
# Composes the generic modules into a production-ready AWS stack:
|
||||
# VPC → ECS Fargate → EFS → ALB → CloudWatch → ACM
|
||||
#
|
||||
# Features:
|
||||
# - Multi-AZ deployment
|
||||
# - Blue/green deployment slots
|
||||
# - EFS for persistent SQLite storage
|
||||
# - ALB with WebSocket support and sticky sessions
|
||||
# - Auto-scaling with CPU/memory targets
|
||||
# - CloudWatch monitoring, alarms, and dashboards
|
||||
# - IAM least-privilege roles
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
provider "aws" {
|
||||
region = var.region
|
||||
|
||||
default_tags {
|
||||
tags = local.common_tags
|
||||
}
|
||||
}
|
||||
|
||||
# ── Data sources ────────────────────────────────────────────────────────────
|
||||
|
||||
data "aws_caller_identity" "current" {}
|
||||
data "aws_region" "current" {}
|
||||
|
||||
data "aws_availability_zones" "available" {
|
||||
state = "available"
|
||||
}
|
||||
|
||||
# ── Locals ──────────────────────────────────────────────────────────────────
|
||||
|
||||
locals {
|
||||
name_prefix = lower(replace("${var.project_name}-${var.environment}", "_", "-"))
|
||||
|
||||
common_tags = merge(
|
||||
{
|
||||
project = var.project_name
|
||||
environment = var.environment
|
||||
managed_by = "terraform"
|
||||
cloud_provider = "aws"
|
||||
repository = "Claude-Code-Agent-Monitor"
|
||||
},
|
||||
var.tags,
|
||||
)
|
||||
|
||||
# Use first 3 available AZs when none specified
|
||||
availability_zones = length(var.availability_zones) > 0 ? var.availability_zones : slice(data.aws_availability_zones.available.names, 0, min(3, length(data.aws_availability_zones.available.names)))
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Networking
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
module "networking" {
|
||||
source = "../../modules/networking"
|
||||
|
||||
project_name = var.project_name
|
||||
environment = var.environment
|
||||
cloud_provider = "aws"
|
||||
region = var.region
|
||||
vpc_cidr = var.vpc_cidr
|
||||
availability_zones = local.availability_zones
|
||||
public_subnet_cidrs = var.public_subnet_cidrs
|
||||
private_subnet_cidrs = var.private_subnet_cidrs
|
||||
app_port = var.app_port
|
||||
mcp_port = var.mcp_port
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Database (EFS for SQLite persistence)
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
module "database" {
|
||||
source = "../../modules/database"
|
||||
|
||||
project_name = var.project_name
|
||||
environment = var.environment
|
||||
cloud_provider = "aws"
|
||||
region = var.region
|
||||
storage_size_gb = var.storage_size_gb
|
||||
enable_backup = var.enable_storage_backup
|
||||
private_subnet_ids = module.networking.private_subnet_ids
|
||||
vpc_id = module.networking.vpc_id
|
||||
allowed_security_group_ids = module.networking.storage_security_group_ids
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Compute – Blue slot
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
module "compute_blue" {
|
||||
source = "../../modules/compute"
|
||||
|
||||
project_name = var.project_name
|
||||
environment = var.environment
|
||||
cloud_provider = "aws"
|
||||
region = var.region
|
||||
deployment_slot = "blue"
|
||||
container_image = var.app_container_image
|
||||
mcp_container_image = var.mcp_container_image
|
||||
app_port = var.app_port
|
||||
mcp_port = var.mcp_port
|
||||
cpu = var.cpu
|
||||
memory = var.memory
|
||||
desired_count = var.active_deployment_slot == "blue" ? var.desired_replicas : 0
|
||||
min_count = var.active_deployment_slot == "blue" ? var.min_replicas : 0
|
||||
max_count = var.active_deployment_slot == "blue" ? var.max_replicas : 0
|
||||
environment_variables = var.environment_variables
|
||||
health_check_path = var.health_check_path
|
||||
vpc_id = module.networking.vpc_id
|
||||
private_subnet_ids = module.networking.private_subnet_ids
|
||||
security_group_ids = module.networking.private_security_group_ids
|
||||
storage_filesystem_id = module.database.filesystem_id
|
||||
storage_mount_targets = module.database.mount_target_ids
|
||||
autoscaling_cpu_target = var.autoscaling_cpu_target
|
||||
autoscaling_memory_target = var.autoscaling_memory_target
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Compute – Green slot
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
module "compute_green" {
|
||||
source = "../../modules/compute"
|
||||
|
||||
project_name = var.project_name
|
||||
environment = var.environment
|
||||
cloud_provider = "aws"
|
||||
region = var.region
|
||||
deployment_slot = "green"
|
||||
container_image = var.app_container_image
|
||||
mcp_container_image = var.mcp_container_image
|
||||
app_port = var.app_port
|
||||
mcp_port = var.mcp_port
|
||||
cpu = var.cpu
|
||||
memory = var.memory
|
||||
desired_count = var.active_deployment_slot == "green" ? var.desired_replicas : 0
|
||||
min_count = var.active_deployment_slot == "green" ? var.min_replicas : 0
|
||||
max_count = var.active_deployment_slot == "green" ? var.max_replicas : 0
|
||||
environment_variables = var.environment_variables
|
||||
health_check_path = var.health_check_path
|
||||
vpc_id = module.networking.vpc_id
|
||||
private_subnet_ids = module.networking.private_subnet_ids
|
||||
security_group_ids = module.networking.private_security_group_ids
|
||||
storage_filesystem_id = module.database.filesystem_id
|
||||
storage_mount_targets = module.database.mount_target_ids
|
||||
autoscaling_cpu_target = var.autoscaling_cpu_target
|
||||
autoscaling_memory_target = var.autoscaling_memory_target
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# ACM Certificate (optional – when domain_name is specified)
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "aws_acm_certificate" "main" {
|
||||
count = var.domain_name != "" && var.tls_certificate_arn == "" ? 1 : 0
|
||||
|
||||
domain_name = var.domain_name
|
||||
validation_method = "DNS"
|
||||
|
||||
tags = merge(local.common_tags, {
|
||||
Name = "${local.name_prefix}-cert"
|
||||
})
|
||||
|
||||
lifecycle {
|
||||
create_before_destroy = true
|
||||
}
|
||||
}
|
||||
|
||||
locals {
|
||||
tls_cert_arn = var.tls_certificate_arn != "" ? var.tls_certificate_arn : (
|
||||
length(aws_acm_certificate.main) > 0 ? aws_acm_certificate.main[0].arn : ""
|
||||
)
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Load Balancer
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
module "loadbalancer" {
|
||||
source = "../../modules/loadbalancer"
|
||||
|
||||
project_name = var.project_name
|
||||
environment = var.environment
|
||||
cloud_provider = "aws"
|
||||
region = var.region
|
||||
vpc_id = module.networking.vpc_id
|
||||
public_subnet_ids = module.networking.public_subnet_ids
|
||||
security_group_ids = module.networking.public_security_group_ids
|
||||
app_port = var.app_port
|
||||
mcp_port = var.mcp_port
|
||||
tls_certificate_arn = local.tls_cert_arn
|
||||
domain_name = var.domain_name
|
||||
|
||||
blue_target_group_arn = module.compute_blue.target_group_arn
|
||||
green_target_group_arn = module.compute_green.target_group_arn
|
||||
blue_weight = var.blue_weight
|
||||
green_weight = var.green_weight
|
||||
|
||||
health_check_path = var.health_check_path
|
||||
health_check_interval = var.health_check_interval
|
||||
health_check_timeout = var.health_check_timeout
|
||||
health_check_healthy_threshold = var.health_check_healthy_threshold
|
||||
health_check_unhealthy_threshold = var.health_check_unhealthy_threshold
|
||||
|
||||
enable_deletion_protection = var.environment == "production"
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Monitoring
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
module "monitoring" {
|
||||
source = "../../modules/monitoring"
|
||||
count = var.enable_monitoring ? 1 : 0
|
||||
|
||||
project_name = var.project_name
|
||||
environment = var.environment
|
||||
cloud_provider = "aws"
|
||||
region = var.region
|
||||
alert_email = var.alert_email
|
||||
log_retention_days = var.log_retention_days
|
||||
|
||||
loadbalancer_arn = module.loadbalancer.loadbalancer_arn
|
||||
target_group_arns = [module.compute_blue.target_group_arn, module.compute_green.target_group_arn]
|
||||
compute_cluster_name = module.compute_blue.cluster_name
|
||||
filesystem_id = module.database.filesystem_id
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Route53 DNS record (optional)
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
data "aws_route53_zone" "main" {
|
||||
count = var.domain_name != "" && var.route53_zone_id != "" ? 1 : 0
|
||||
|
||||
zone_id = var.route53_zone_id
|
||||
}
|
||||
|
||||
resource "aws_route53_record" "app" {
|
||||
count = var.domain_name != "" && var.route53_zone_id != "" ? 1 : 0
|
||||
|
||||
zone_id = data.aws_route53_zone.main[0].zone_id
|
||||
name = var.domain_name
|
||||
type = "A"
|
||||
|
||||
alias {
|
||||
name = module.loadbalancer.dns_name
|
||||
zone_id = module.loadbalancer.zone_id
|
||||
evaluate_target_health = true
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# AWS provider outputs
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
output "application_url" {
|
||||
description = "Public URL of the Claude Code Agent Monitor dashboard"
|
||||
value = module.loadbalancer.application_url
|
||||
}
|
||||
|
||||
output "alb_dns_name" {
|
||||
description = "DNS name of the Application Load Balancer"
|
||||
value = module.loadbalancer.dns_name
|
||||
}
|
||||
|
||||
output "vpc_id" {
|
||||
description = "ID of the VPC"
|
||||
value = module.networking.vpc_id
|
||||
}
|
||||
|
||||
output "ecs_cluster_name" {
|
||||
description = "Name of the ECS cluster"
|
||||
value = module.compute_blue.cluster_name
|
||||
}
|
||||
|
||||
output "blue_service_name" {
|
||||
description = "Name of the blue ECS service"
|
||||
value = module.compute_blue.service_name
|
||||
}
|
||||
|
||||
output "green_service_name" {
|
||||
description = "Name of the green ECS service"
|
||||
value = module.compute_green.service_name
|
||||
}
|
||||
|
||||
output "efs_filesystem_id" {
|
||||
description = "ID of the EFS file system"
|
||||
value = module.database.filesystem_id
|
||||
}
|
||||
|
||||
output "acm_certificate_arn" {
|
||||
description = "ARN of the ACM certificate (if auto-created)"
|
||||
value = length(aws_acm_certificate.main) > 0 ? aws_acm_certificate.main[0].arn : var.tls_certificate_arn
|
||||
}
|
||||
|
||||
output "monitoring_dashboard_url" {
|
||||
description = "CloudWatch dashboard URL"
|
||||
value = var.enable_monitoring ? module.monitoring[0].dashboard_url : "monitoring disabled"
|
||||
}
|
||||
|
||||
output "account_id" {
|
||||
description = "AWS account ID"
|
||||
value = data.aws_caller_identity.current.account_id
|
||||
}
|
||||
|
||||
output "region" {
|
||||
description = "AWS region"
|
||||
value = data.aws_region.current.name
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# AWS provider – Terraform and provider constraints
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
terraform {
|
||||
required_version = ">= 1.5.0"
|
||||
|
||||
required_providers {
|
||||
aws = {
|
||||
source = "hashicorp/aws"
|
||||
version = "~> 5.0"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,254 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# AWS provider variables
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
# ── Core ────────────────────────────────────────────────────────────────────
|
||||
|
||||
variable "project_name" {
|
||||
description = "Project identifier used in resource naming and tagging"
|
||||
type = string
|
||||
default = "claude-agent-monitor"
|
||||
}
|
||||
|
||||
variable "environment" {
|
||||
description = "Deployment environment: dev, staging, or production"
|
||||
type = string
|
||||
validation {
|
||||
condition = contains(["dev", "staging", "production"], var.environment)
|
||||
error_message = "environment must be one of: dev, staging, production."
|
||||
}
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
description = "AWS region for resource deployment"
|
||||
type = string
|
||||
default = "us-east-1"
|
||||
}
|
||||
|
||||
variable "tags" {
|
||||
description = "Additional tags to apply to all resources"
|
||||
type = map(string)
|
||||
default = {}
|
||||
}
|
||||
|
||||
# ── Networking ──────────────────────────────────────────────────────────────
|
||||
|
||||
variable "vpc_cidr" {
|
||||
description = "CIDR block for the VPC"
|
||||
type = string
|
||||
default = "10.0.0.0/16"
|
||||
}
|
||||
|
||||
variable "availability_zones" {
|
||||
description = "List of AZs (auto-detected if empty)"
|
||||
type = list(string)
|
||||
default = []
|
||||
}
|
||||
|
||||
variable "public_subnet_cidrs" {
|
||||
description = "CIDR blocks for public subnets"
|
||||
type = list(string)
|
||||
default = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
|
||||
}
|
||||
|
||||
variable "private_subnet_cidrs" {
|
||||
description = "CIDR blocks for private subnets"
|
||||
type = list(string)
|
||||
default = ["10.0.11.0/24", "10.0.12.0/24", "10.0.13.0/24"]
|
||||
}
|
||||
|
||||
# ── Compute ─────────────────────────────────────────────────────────────────
|
||||
|
||||
variable "app_container_image" {
|
||||
description = "Docker image URI for the main application"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "mcp_container_image" {
|
||||
description = "Docker image URI for the MCP sidecar (empty to disable)"
|
||||
type = string
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "app_port" {
|
||||
description = "Application container port"
|
||||
type = number
|
||||
default = 4820
|
||||
}
|
||||
|
||||
variable "mcp_port" {
|
||||
description = "MCP sidecar container port"
|
||||
type = number
|
||||
default = 8819
|
||||
}
|
||||
|
||||
variable "cpu" {
|
||||
description = "CPU units for Fargate tasks (256, 512, 1024, 2048, 4096)"
|
||||
type = number
|
||||
default = 512
|
||||
}
|
||||
|
||||
variable "memory" {
|
||||
description = "Memory in MiB for Fargate tasks"
|
||||
type = number
|
||||
default = 1024
|
||||
}
|
||||
|
||||
variable "min_replicas" {
|
||||
description = "Minimum number of ECS tasks"
|
||||
type = number
|
||||
default = 1
|
||||
}
|
||||
|
||||
variable "max_replicas" {
|
||||
description = "Maximum number of ECS tasks for auto-scaling"
|
||||
type = number
|
||||
default = 3
|
||||
}
|
||||
|
||||
variable "desired_replicas" {
|
||||
description = "Desired number of ECS tasks at steady state"
|
||||
type = number
|
||||
default = 1
|
||||
}
|
||||
|
||||
variable "environment_variables" {
|
||||
description = "Environment variables for the application container"
|
||||
type = map(string)
|
||||
default = {
|
||||
NODE_ENV = "production"
|
||||
DASHBOARD_PORT = "4820"
|
||||
}
|
||||
}
|
||||
|
||||
# ── Deployment ──────────────────────────────────────────────────────────────
|
||||
|
||||
variable "active_deployment_slot" {
|
||||
description = "Active deployment slot: blue or green"
|
||||
type = string
|
||||
default = "blue"
|
||||
validation {
|
||||
condition = contains(["blue", "green"], var.active_deployment_slot)
|
||||
error_message = "active_deployment_slot must be blue or green."
|
||||
}
|
||||
}
|
||||
|
||||
variable "blue_weight" {
|
||||
description = "Traffic weight for blue target group (0-100)"
|
||||
type = number
|
||||
default = 100
|
||||
validation {
|
||||
condition = var.blue_weight >= 0 && var.blue_weight <= 100
|
||||
error_message = "blue_weight must be between 0 and 100."
|
||||
}
|
||||
}
|
||||
|
||||
variable "green_weight" {
|
||||
description = "Traffic weight for green target group (0-100)"
|
||||
type = number
|
||||
default = 0
|
||||
validation {
|
||||
condition = var.green_weight >= 0 && var.green_weight <= 100
|
||||
error_message = "green_weight must be between 0 and 100."
|
||||
}
|
||||
}
|
||||
|
||||
# ── TLS / Domain ────────────────────────────────────────────────────────────
|
||||
|
||||
variable "domain_name" {
|
||||
description = "FQDN for the application (empty to skip DNS/TLS)"
|
||||
type = string
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "tls_certificate_arn" {
|
||||
description = "ARN of an existing ACM certificate (auto-created if domain_name set)"
|
||||
type = string
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "route53_zone_id" {
|
||||
description = "Route53 hosted zone ID for DNS records (empty to skip)"
|
||||
type = string
|
||||
default = ""
|
||||
}
|
||||
|
||||
# ── Storage ─────────────────────────────────────────────────────────────────
|
||||
|
||||
variable "storage_size_gb" {
|
||||
description = "EFS storage does not require pre-provisioning; kept for interface compatibility"
|
||||
type = number
|
||||
default = 20
|
||||
}
|
||||
|
||||
variable "enable_storage_backup" {
|
||||
description = "Enable AWS Backup for EFS"
|
||||
type = bool
|
||||
default = true
|
||||
}
|
||||
|
||||
# ── Health check ────────────────────────────────────────────────────────────
|
||||
|
||||
variable "health_check_path" {
|
||||
description = "HTTP path for health checks"
|
||||
type = string
|
||||
default = "/api/health"
|
||||
}
|
||||
|
||||
variable "health_check_interval" {
|
||||
description = "Seconds between health checks"
|
||||
type = number
|
||||
default = 30
|
||||
}
|
||||
|
||||
variable "health_check_timeout" {
|
||||
description = "Seconds before a health check request times out"
|
||||
type = number
|
||||
default = 5
|
||||
}
|
||||
|
||||
variable "health_check_healthy_threshold" {
|
||||
description = "Consecutive successes to mark target healthy"
|
||||
type = number
|
||||
default = 2
|
||||
}
|
||||
|
||||
variable "health_check_unhealthy_threshold" {
|
||||
description = "Consecutive failures to mark target unhealthy"
|
||||
type = number
|
||||
default = 3
|
||||
}
|
||||
|
||||
# ── Auto-scaling ────────────────────────────────────────────────────────────
|
||||
|
||||
variable "autoscaling_cpu_target" {
|
||||
description = "Target CPU utilization percentage for auto-scaling"
|
||||
type = number
|
||||
default = 70
|
||||
}
|
||||
|
||||
variable "autoscaling_memory_target" {
|
||||
description = "Target memory utilization percentage for auto-scaling"
|
||||
type = number
|
||||
default = 80
|
||||
}
|
||||
|
||||
# ── Monitoring ──────────────────────────────────────────────────────────────
|
||||
|
||||
variable "enable_monitoring" {
|
||||
description = "Enable CloudWatch monitoring, alarms, and dashboards"
|
||||
type = bool
|
||||
default = true
|
||||
}
|
||||
|
||||
variable "alert_email" {
|
||||
description = "Email address for SNS alert notifications"
|
||||
type = string
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "log_retention_days" {
|
||||
description = "CloudWatch log retention in days"
|
||||
type = number
|
||||
default = 30
|
||||
}
|
||||
@@ -0,0 +1,615 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Azure Provider – Full implementation for Claude Code Agent Monitor
|
||||
#
|
||||
# Architecture:
|
||||
# VNet → ACI (Container Instances) or AKS → Azure Files → Application
|
||||
# Gateway → Azure Monitor → Key Vault
|
||||
#
|
||||
# Azure Container Instances is chosen for simplicity; for production at
|
||||
# scale, AKS is recommended. Application Gateway provides L7 LB with
|
||||
# WebSocket support and SSL termination.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
provider "azurerm" {
|
||||
features {
|
||||
resource_group {
|
||||
prevent_deletion_if_contains_resources = true
|
||||
}
|
||||
key_vault {
|
||||
purge_soft_delete_on_destroy = false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# ── Data sources ────────────────────────────────────────────────────────────
|
||||
|
||||
data "azurerm_client_config" "current" {}
|
||||
|
||||
# ── Locals ──────────────────────────────────────────────────────────────────
|
||||
|
||||
locals {
|
||||
name_prefix = lower(replace("${var.project_name}-${var.environment}", "_", "-"))
|
||||
|
||||
# Azure resource names (alphanumeric for storage accounts)
|
||||
storage_account_name = lower(replace(substr("ccam${var.environment}${substr(md5(var.project_name), 0, 8)}", 0, 24), "-", ""))
|
||||
|
||||
common_tags = merge(
|
||||
{
|
||||
project = var.project_name
|
||||
environment = var.environment
|
||||
managed_by = "terraform"
|
||||
cloud_provider = "azure"
|
||||
repository = "Claude-Code-Agent-Monitor"
|
||||
},
|
||||
var.tags,
|
||||
)
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Resource Group
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "azurerm_resource_group" "main" {
|
||||
name = "${local.name_prefix}-rg"
|
||||
location = var.region
|
||||
|
||||
tags = local.common_tags
|
||||
|
||||
lifecycle {
|
||||
prevent_destroy = false
|
||||
}
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Virtual Network
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "azurerm_virtual_network" "main" {
|
||||
name = "${local.name_prefix}-vnet"
|
||||
resource_group_name = azurerm_resource_group.main.name
|
||||
location = azurerm_resource_group.main.location
|
||||
address_space = [var.vpc_cidr]
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
resource "azurerm_subnet" "public" {
|
||||
name = "${local.name_prefix}-public"
|
||||
resource_group_name = azurerm_resource_group.main.name
|
||||
virtual_network_name = azurerm_virtual_network.main.name
|
||||
address_prefixes = [var.public_subnet_cidrs[0]]
|
||||
}
|
||||
|
||||
resource "azurerm_subnet" "private" {
|
||||
name = "${local.name_prefix}-private"
|
||||
resource_group_name = azurerm_resource_group.main.name
|
||||
virtual_network_name = azurerm_virtual_network.main.name
|
||||
address_prefixes = [var.private_subnet_cidrs[0]]
|
||||
|
||||
delegation {
|
||||
name = "aci-delegation"
|
||||
service_delegation {
|
||||
name = "Microsoft.ContainerInstance/containerGroups"
|
||||
actions = ["Microsoft.Network/virtualNetworks/subnets/action"]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
resource "azurerm_subnet" "appgw" {
|
||||
name = "${local.name_prefix}-appgw"
|
||||
resource_group_name = azurerm_resource_group.main.name
|
||||
virtual_network_name = azurerm_virtual_network.main.name
|
||||
address_prefixes = ["10.0.100.0/24"]
|
||||
}
|
||||
|
||||
# NSG for private subnet
|
||||
resource "azurerm_network_security_group" "private" {
|
||||
name = "${local.name_prefix}-private-nsg"
|
||||
location = azurerm_resource_group.main.location
|
||||
resource_group_name = azurerm_resource_group.main.name
|
||||
|
||||
security_rule {
|
||||
name = "allow-app-port"
|
||||
priority = 100
|
||||
direction = "Inbound"
|
||||
access = "Allow"
|
||||
protocol = "Tcp"
|
||||
source_port_range = "*"
|
||||
destination_port_range = tostring(var.app_port)
|
||||
source_address_prefix = var.vpc_cidr
|
||||
destination_address_prefix = "*"
|
||||
}
|
||||
|
||||
security_rule {
|
||||
name = "allow-mcp-port"
|
||||
priority = 110
|
||||
direction = "Inbound"
|
||||
access = "Allow"
|
||||
protocol = "Tcp"
|
||||
source_port_range = "*"
|
||||
destination_port_range = tostring(var.mcp_port)
|
||||
source_address_prefix = var.vpc_cidr
|
||||
destination_address_prefix = "*"
|
||||
}
|
||||
|
||||
security_rule {
|
||||
name = "allow-smb"
|
||||
priority = 120
|
||||
direction = "Inbound"
|
||||
access = "Allow"
|
||||
protocol = "Tcp"
|
||||
source_port_range = "*"
|
||||
destination_port_range = "445"
|
||||
source_address_prefix = var.vpc_cidr
|
||||
destination_address_prefix = "*"
|
||||
}
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
resource "azurerm_subnet_network_security_group_association" "private" {
|
||||
subnet_id = azurerm_subnet.private.id
|
||||
network_security_group_id = azurerm_network_security_group.private.id
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Azure Files (persistent storage for SQLite)
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "azurerm_storage_account" "main" {
|
||||
name = local.storage_account_name
|
||||
resource_group_name = azurerm_resource_group.main.name
|
||||
location = azurerm_resource_group.main.location
|
||||
account_tier = var.environment == "production" ? "Premium" : "Standard"
|
||||
account_replication_type = var.environment == "production" ? "ZRS" : "LRS"
|
||||
account_kind = var.environment == "production" ? "FileStorage" : "StorageV2"
|
||||
|
||||
min_tls_version = "TLS1_2"
|
||||
|
||||
network_rules {
|
||||
default_action = "Deny"
|
||||
virtual_network_subnet_ids = [azurerm_subnet.private.id]
|
||||
}
|
||||
|
||||
tags = local.common_tags
|
||||
|
||||
lifecycle {
|
||||
prevent_destroy = true
|
||||
}
|
||||
}
|
||||
|
||||
resource "azurerm_storage_share" "appdata" {
|
||||
name = "appdata"
|
||||
storage_account_name = azurerm_storage_account.main.name
|
||||
quota = var.storage_size_gb
|
||||
access_tier = var.environment == "production" ? "Premium" : "Hot"
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Key Vault (for secrets management)
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "azurerm_key_vault" "main" {
|
||||
name = substr("${local.name_prefix}-kv", 0, 24)
|
||||
location = azurerm_resource_group.main.location
|
||||
resource_group_name = azurerm_resource_group.main.name
|
||||
tenant_id = data.azurerm_client_config.current.tenant_id
|
||||
sku_name = "standard"
|
||||
purge_protection_enabled = var.environment == "production"
|
||||
|
||||
access_policy {
|
||||
tenant_id = data.azurerm_client_config.current.tenant_id
|
||||
object_id = data.azurerm_client_config.current.object_id
|
||||
|
||||
secret_permissions = [
|
||||
"Get", "List", "Set", "Delete", "Purge",
|
||||
]
|
||||
}
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Container Instances (Blue / Green)
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "azurerm_container_group" "blue" {
|
||||
name = "${local.name_prefix}-blue"
|
||||
location = azurerm_resource_group.main.location
|
||||
resource_group_name = azurerm_resource_group.main.name
|
||||
os_type = "Linux"
|
||||
ip_address_type = "Private"
|
||||
subnet_ids = [azurerm_subnet.private.id]
|
||||
restart_policy = "Always"
|
||||
|
||||
container {
|
||||
name = "app"
|
||||
image = var.app_container_image
|
||||
cpu = var.cpu / 1000.0
|
||||
memory = var.memory / 1024.0
|
||||
|
||||
ports {
|
||||
port = var.app_port
|
||||
protocol = "TCP"
|
||||
}
|
||||
|
||||
dynamic "environment_variables" {
|
||||
for_each = var.environment_variables
|
||||
content {
|
||||
name = environment_variables.key
|
||||
value = environment_variables.value
|
||||
}
|
||||
}
|
||||
|
||||
volume {
|
||||
name = "app-data"
|
||||
mount_path = "/app/data"
|
||||
read_only = false
|
||||
storage_account_name = azurerm_storage_account.main.name
|
||||
storage_account_key = azurerm_storage_account.main.primary_access_key
|
||||
share_name = azurerm_storage_share.appdata.name
|
||||
}
|
||||
|
||||
liveness_probe {
|
||||
http_get {
|
||||
path = var.health_check_path
|
||||
port = var.app_port
|
||||
scheme = "Http"
|
||||
}
|
||||
initial_delay_seconds = 30
|
||||
period_seconds = 30
|
||||
failure_threshold = 3
|
||||
}
|
||||
|
||||
readiness_probe {
|
||||
http_get {
|
||||
path = var.health_check_path
|
||||
port = var.app_port
|
||||
scheme = "Http"
|
||||
}
|
||||
initial_delay_seconds = 10
|
||||
period_seconds = 10
|
||||
failure_threshold = 3
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "container" {
|
||||
for_each = var.mcp_container_image != "" ? [1] : []
|
||||
content {
|
||||
name = "mcp-sidecar"
|
||||
image = var.mcp_container_image
|
||||
cpu = 0.25
|
||||
memory = 0.25
|
||||
|
||||
ports {
|
||||
port = var.mcp_port
|
||||
protocol = "TCP"
|
||||
}
|
||||
|
||||
environment_variables = {
|
||||
NODE_ENV = "production"
|
||||
MCP_PORT = tostring(var.mcp_port)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
tags = merge(local.common_tags, {
|
||||
deployment_slot = "blue"
|
||||
})
|
||||
|
||||
lifecycle {
|
||||
ignore_changes = [
|
||||
tags["last_deployed"],
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
resource "azurerm_container_group" "green" {
|
||||
count = var.active_deployment_slot == "green" || var.green_weight > 0 ? 1 : 0
|
||||
|
||||
name = "${local.name_prefix}-green"
|
||||
location = azurerm_resource_group.main.location
|
||||
resource_group_name = azurerm_resource_group.main.name
|
||||
os_type = "Linux"
|
||||
ip_address_type = "Private"
|
||||
subnet_ids = [azurerm_subnet.private.id]
|
||||
restart_policy = "Always"
|
||||
|
||||
container {
|
||||
name = "app"
|
||||
image = var.app_container_image
|
||||
cpu = var.cpu / 1000.0
|
||||
memory = var.memory / 1024.0
|
||||
|
||||
ports {
|
||||
port = var.app_port
|
||||
protocol = "TCP"
|
||||
}
|
||||
|
||||
dynamic "environment_variables" {
|
||||
for_each = var.environment_variables
|
||||
content {
|
||||
name = environment_variables.key
|
||||
value = environment_variables.value
|
||||
}
|
||||
}
|
||||
|
||||
volume {
|
||||
name = "app-data"
|
||||
mount_path = "/app/data"
|
||||
read_only = false
|
||||
storage_account_name = azurerm_storage_account.main.name
|
||||
storage_account_key = azurerm_storage_account.main.primary_access_key
|
||||
share_name = azurerm_storage_share.appdata.name
|
||||
}
|
||||
|
||||
liveness_probe {
|
||||
http_get {
|
||||
path = var.health_check_path
|
||||
port = var.app_port
|
||||
scheme = "Http"
|
||||
}
|
||||
initial_delay_seconds = 30
|
||||
period_seconds = 30
|
||||
failure_threshold = 3
|
||||
}
|
||||
}
|
||||
|
||||
tags = merge(local.common_tags, {
|
||||
deployment_slot = "green"
|
||||
})
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Application Gateway (L7 load balancer with WebSocket + SSL)
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "azurerm_public_ip" "appgw" {
|
||||
name = "${local.name_prefix}-appgw-pip"
|
||||
location = azurerm_resource_group.main.location
|
||||
resource_group_name = azurerm_resource_group.main.name
|
||||
allocation_method = "Static"
|
||||
sku = "Standard"
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
resource "azurerm_application_gateway" "main" {
|
||||
name = "${local.name_prefix}-appgw"
|
||||
resource_group_name = azurerm_resource_group.main.name
|
||||
location = azurerm_resource_group.main.location
|
||||
enable_http2 = true
|
||||
|
||||
sku {
|
||||
name = var.environment == "production" ? "WAF_v2" : "Standard_v2"
|
||||
tier = var.environment == "production" ? "WAF_v2" : "Standard_v2"
|
||||
capacity = var.environment == "production" ? 2 : 1
|
||||
}
|
||||
|
||||
gateway_ip_configuration {
|
||||
name = "gateway-ip"
|
||||
subnet_id = azurerm_subnet.appgw.id
|
||||
}
|
||||
|
||||
frontend_ip_configuration {
|
||||
name = "frontend-ip"
|
||||
public_ip_address_id = azurerm_public_ip.appgw.id
|
||||
}
|
||||
|
||||
frontend_port {
|
||||
name = "http"
|
||||
port = 80
|
||||
}
|
||||
|
||||
frontend_port {
|
||||
name = "https"
|
||||
port = 443
|
||||
}
|
||||
|
||||
# Blue backend pool
|
||||
backend_address_pool {
|
||||
name = "blue-pool"
|
||||
ip_addresses = [azurerm_container_group.blue.ip_address]
|
||||
}
|
||||
|
||||
# Green backend pool
|
||||
dynamic "backend_address_pool" {
|
||||
for_each = length(azurerm_container_group.green) > 0 ? [1] : []
|
||||
content {
|
||||
name = "green-pool"
|
||||
ip_addresses = [azurerm_container_group.green[0].ip_address]
|
||||
}
|
||||
}
|
||||
|
||||
backend_http_settings {
|
||||
name = "app-settings"
|
||||
cookie_based_affinity = "Enabled"
|
||||
port = var.app_port
|
||||
protocol = "Http"
|
||||
request_timeout = 300 # WebSocket support
|
||||
pick_host_name_from_backend_address = false
|
||||
|
||||
connection_draining {
|
||||
enabled = true
|
||||
drain_timeout_sec = 60
|
||||
}
|
||||
|
||||
probe_name = "app-health"
|
||||
}
|
||||
|
||||
probe {
|
||||
name = "app-health"
|
||||
protocol = "Http"
|
||||
path = var.health_check_path
|
||||
host = "127.0.0.1"
|
||||
interval = var.health_check_interval
|
||||
timeout = var.health_check_timeout
|
||||
unhealthy_threshold = var.health_check_unhealthy_threshold
|
||||
|
||||
match {
|
||||
status_code = ["200"]
|
||||
}
|
||||
}
|
||||
|
||||
# HTTP listener
|
||||
http_listener {
|
||||
name = "http-listener"
|
||||
frontend_ip_configuration_name = "frontend-ip"
|
||||
frontend_port_name = "http"
|
||||
protocol = "Http"
|
||||
}
|
||||
|
||||
# Routing rule – HTTP to blue pool
|
||||
request_routing_rule {
|
||||
name = "http-routing"
|
||||
priority = 100
|
||||
rule_type = "Basic"
|
||||
http_listener_name = "http-listener"
|
||||
backend_address_pool_name = var.active_deployment_slot == "blue" ? "blue-pool" : "green-pool"
|
||||
backend_http_settings_name = "app-settings"
|
||||
}
|
||||
|
||||
tags = local.common_tags
|
||||
|
||||
lifecycle {
|
||||
prevent_destroy = false
|
||||
ignore_changes = [
|
||||
tags["last_deployed"],
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Azure Monitor (alerts and diagnostics)
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "azurerm_monitor_action_group" "main" {
|
||||
count = var.enable_monitoring ? 1 : 0
|
||||
|
||||
name = "${local.name_prefix}-alerts"
|
||||
resource_group_name = azurerm_resource_group.main.name
|
||||
short_name = substr(local.name_prefix, 0, 12)
|
||||
|
||||
dynamic "email_receiver" {
|
||||
for_each = var.alert_email != "" ? [1] : []
|
||||
content {
|
||||
name = "email-alert"
|
||||
email_address = var.alert_email
|
||||
}
|
||||
}
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
resource "azurerm_monitor_metric_alert" "appgw_unhealthy" {
|
||||
count = var.enable_monitoring ? 1 : 0
|
||||
|
||||
name = "${local.name_prefix}-unhealthy-backend"
|
||||
resource_group_name = azurerm_resource_group.main.name
|
||||
scopes = [azurerm_application_gateway.main.id]
|
||||
description = "Alert when backend health drops below threshold"
|
||||
severity = 1
|
||||
|
||||
criteria {
|
||||
metric_namespace = "Microsoft.Network/applicationGateways"
|
||||
metric_name = "UnhealthyHostCount"
|
||||
aggregation = "Average"
|
||||
operator = "GreaterThan"
|
||||
threshold = 0
|
||||
}
|
||||
|
||||
action {
|
||||
action_group_id = azurerm_monitor_action_group.main[0].id
|
||||
}
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
resource "azurerm_monitor_metric_alert" "appgw_5xx" {
|
||||
count = var.enable_monitoring ? 1 : 0
|
||||
|
||||
name = "${local.name_prefix}-high-5xx"
|
||||
resource_group_name = azurerm_resource_group.main.name
|
||||
scopes = [azurerm_application_gateway.main.id]
|
||||
description = "High 5xx error rate on Application Gateway"
|
||||
severity = 2
|
||||
|
||||
criteria {
|
||||
metric_namespace = "Microsoft.Network/applicationGateways"
|
||||
metric_name = "ResponseStatus"
|
||||
aggregation = "Count"
|
||||
operator = "GreaterThan"
|
||||
threshold = 10
|
||||
|
||||
dimension {
|
||||
name = "HttpStatusGroup"
|
||||
operator = "Include"
|
||||
values = ["5xx"]
|
||||
}
|
||||
}
|
||||
|
||||
action {
|
||||
action_group_id = azurerm_monitor_action_group.main[0].id
|
||||
}
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
resource "azurerm_monitor_metric_alert" "appgw_latency" {
|
||||
count = var.enable_monitoring ? 1 : 0
|
||||
|
||||
name = "${local.name_prefix}-high-latency"
|
||||
resource_group_name = azurerm_resource_group.main.name
|
||||
scopes = [azurerm_application_gateway.main.id]
|
||||
description = "High backend response latency"
|
||||
severity = 2
|
||||
|
||||
criteria {
|
||||
metric_namespace = "Microsoft.Network/applicationGateways"
|
||||
metric_name = "BackendLastByteResponseTime"
|
||||
aggregation = "Average"
|
||||
operator = "GreaterThan"
|
||||
threshold = 2000 # ms
|
||||
}
|
||||
|
||||
action {
|
||||
action_group_id = azurerm_monitor_action_group.main[0].id
|
||||
}
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# Log Analytics Workspace
|
||||
resource "azurerm_log_analytics_workspace" "main" {
|
||||
count = var.enable_monitoring ? 1 : 0
|
||||
|
||||
name = "${local.name_prefix}-logs"
|
||||
location = azurerm_resource_group.main.location
|
||||
resource_group_name = azurerm_resource_group.main.name
|
||||
sku = "PerGB2018"
|
||||
retention_in_days = var.log_retention_days
|
||||
|
||||
tags = local.common_tags
|
||||
}
|
||||
|
||||
# Diagnostic settings for App Gateway
|
||||
resource "azurerm_monitor_diagnostic_setting" "appgw" {
|
||||
count = var.enable_monitoring ? 1 : 0
|
||||
|
||||
name = "${local.name_prefix}-appgw-diag"
|
||||
target_resource_id = azurerm_application_gateway.main.id
|
||||
log_analytics_workspace_id = azurerm_log_analytics_workspace.main[0].id
|
||||
|
||||
enabled_log {
|
||||
category = "ApplicationGatewayAccessLog"
|
||||
}
|
||||
|
||||
enabled_log {
|
||||
category = "ApplicationGatewayPerformanceLog"
|
||||
}
|
||||
|
||||
metric {
|
||||
category = "AllMetrics"
|
||||
enabled = true
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Azure provider outputs
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
output "application_url" {
|
||||
description = "Public URL of the Claude Code Agent Monitor dashboard"
|
||||
value = "http://${azurerm_public_ip.appgw.ip_address}"
|
||||
}
|
||||
|
||||
output "public_ip" {
|
||||
description = "Public IP address of the Application Gateway"
|
||||
value = azurerm_public_ip.appgw.ip_address
|
||||
}
|
||||
|
||||
output "resource_group_name" {
|
||||
description = "Name of the Azure resource group"
|
||||
value = azurerm_resource_group.main.name
|
||||
}
|
||||
|
||||
output "vnet_id" {
|
||||
description = "ID of the Virtual Network"
|
||||
value = azurerm_virtual_network.main.id
|
||||
}
|
||||
|
||||
output "blue_container_group_id" {
|
||||
description = "ID of the blue container group"
|
||||
value = azurerm_container_group.blue.id
|
||||
}
|
||||
|
||||
output "green_container_group_id" {
|
||||
description = "ID of the green container group (if deployed)"
|
||||
value = length(azurerm_container_group.green) > 0 ? azurerm_container_group.green[0].id : ""
|
||||
}
|
||||
|
||||
output "storage_account_name" {
|
||||
description = "Name of the Azure Storage Account"
|
||||
value = azurerm_storage_account.main.name
|
||||
}
|
||||
|
||||
output "key_vault_uri" {
|
||||
description = "URI of the Azure Key Vault"
|
||||
value = azurerm_key_vault.main.vault_uri
|
||||
}
|
||||
|
||||
output "app_gateway_id" {
|
||||
description = "ID of the Application Gateway"
|
||||
value = azurerm_application_gateway.main.id
|
||||
}
|
||||
|
||||
output "log_analytics_workspace_id" {
|
||||
description = "ID of the Log Analytics workspace (if monitoring enabled)"
|
||||
value = var.enable_monitoring ? azurerm_log_analytics_workspace.main[0].id : ""
|
||||
}
|
||||
|
||||
output "region" {
|
||||
description = "Azure region"
|
||||
value = var.region
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Azure provider – Terraform and provider constraints
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
terraform {
|
||||
required_version = ">= 1.5.0"
|
||||
|
||||
required_providers {
|
||||
azurerm = {
|
||||
source = "hashicorp/azurerm"
|
||||
version = "~> 3.80"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,190 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Azure provider variables
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
variable "project_name" {
|
||||
description = "Project identifier used in resource naming"
|
||||
type = string
|
||||
default = "claude-agent-monitor"
|
||||
}
|
||||
|
||||
variable "environment" {
|
||||
description = "Deployment environment: dev, staging, or production"
|
||||
type = string
|
||||
validation {
|
||||
condition = contains(["dev", "staging", "production"], var.environment)
|
||||
error_message = "environment must be one of: dev, staging, production."
|
||||
}
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
description = "Azure region for resource deployment"
|
||||
type = string
|
||||
default = "eastus"
|
||||
}
|
||||
|
||||
variable "tags" {
|
||||
description = "Additional tags to apply to all resources"
|
||||
type = map(string)
|
||||
default = {}
|
||||
}
|
||||
|
||||
# ── Networking ──────────────────────────────────────────────────────────────
|
||||
|
||||
variable "vpc_cidr" {
|
||||
description = "Address space for the Virtual Network"
|
||||
type = string
|
||||
default = "10.0.0.0/16"
|
||||
}
|
||||
|
||||
variable "public_subnet_cidrs" {
|
||||
description = "Address prefixes for the public subnet"
|
||||
type = list(string)
|
||||
default = ["10.0.1.0/24"]
|
||||
}
|
||||
|
||||
variable "private_subnet_cidrs" {
|
||||
description = "Address prefixes for the private subnet"
|
||||
type = list(string)
|
||||
default = ["10.0.11.0/24"]
|
||||
}
|
||||
|
||||
# ── Compute ─────────────────────────────────────────────────────────────────
|
||||
|
||||
variable "app_container_image" {
|
||||
description = "Container image URI for the main application"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "mcp_container_image" {
|
||||
description = "Container image URI for the MCP sidecar (empty to disable)"
|
||||
type = string
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "app_port" {
|
||||
description = "Application container port"
|
||||
type = number
|
||||
default = 4820
|
||||
}
|
||||
|
||||
variable "mcp_port" {
|
||||
description = "MCP sidecar container port"
|
||||
type = number
|
||||
default = 8819
|
||||
}
|
||||
|
||||
variable "cpu" {
|
||||
description = "CPU millicores for container instances"
|
||||
type = number
|
||||
default = 512
|
||||
}
|
||||
|
||||
variable "memory" {
|
||||
description = "Memory in MiB for container instances"
|
||||
type = number
|
||||
default = 1024
|
||||
}
|
||||
|
||||
variable "environment_variables" {
|
||||
description = "Environment variables for the application container"
|
||||
type = map(string)
|
||||
default = {
|
||||
NODE_ENV = "production"
|
||||
DASHBOARD_PORT = "4820"
|
||||
}
|
||||
}
|
||||
|
||||
# ── Deployment ──────────────────────────────────────────────────────────────
|
||||
|
||||
variable "active_deployment_slot" {
|
||||
description = "Active deployment slot: blue or green"
|
||||
type = string
|
||||
default = "blue"
|
||||
validation {
|
||||
condition = contains(["blue", "green"], var.active_deployment_slot)
|
||||
error_message = "active_deployment_slot must be blue or green."
|
||||
}
|
||||
}
|
||||
|
||||
variable "blue_weight" {
|
||||
description = "Traffic weight for blue backend (0-100)"
|
||||
type = number
|
||||
default = 100
|
||||
validation {
|
||||
condition = var.blue_weight >= 0 && var.blue_weight <= 100
|
||||
error_message = "blue_weight must be between 0 and 100."
|
||||
}
|
||||
}
|
||||
|
||||
variable "green_weight" {
|
||||
description = "Traffic weight for green backend (0-100)"
|
||||
type = number
|
||||
default = 0
|
||||
validation {
|
||||
condition = var.green_weight >= 0 && var.green_weight <= 100
|
||||
error_message = "green_weight must be between 0 and 100."
|
||||
}
|
||||
}
|
||||
|
||||
# ── TLS / Domain ────────────────────────────────────────────────────────────
|
||||
|
||||
variable "domain_name" {
|
||||
description = "FQDN for the application (empty to skip)"
|
||||
type = string
|
||||
default = ""
|
||||
}
|
||||
|
||||
# ── Storage ─────────────────────────────────────────────────────────────────
|
||||
|
||||
variable "storage_size_gb" {
|
||||
description = "Azure Files share quota in GiB"
|
||||
type = number
|
||||
default = 20
|
||||
}
|
||||
|
||||
# ── Health check ────────────────────────────────────────────────────────────
|
||||
|
||||
variable "health_check_path" {
|
||||
description = "HTTP path for health checks"
|
||||
type = string
|
||||
default = "/api/health"
|
||||
}
|
||||
|
||||
variable "health_check_interval" {
|
||||
description = "Seconds between health checks"
|
||||
type = number
|
||||
default = 30
|
||||
}
|
||||
|
||||
variable "health_check_timeout" {
|
||||
description = "Seconds before a health check times out"
|
||||
type = number
|
||||
default = 5
|
||||
}
|
||||
|
||||
variable "health_check_unhealthy_threshold" {
|
||||
description = "Consecutive failures to mark unhealthy"
|
||||
type = number
|
||||
default = 3
|
||||
}
|
||||
|
||||
# ── Monitoring ──────────────────────────────────────────────────────────────
|
||||
|
||||
variable "enable_monitoring" {
|
||||
description = "Enable Azure Monitor alerts and diagnostics"
|
||||
type = bool
|
||||
default = true
|
||||
}
|
||||
|
||||
variable "alert_email" {
|
||||
description = "Email address for Azure Monitor alerts"
|
||||
type = string
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "log_retention_days" {
|
||||
description = "Log Analytics workspace retention in days"
|
||||
type = number
|
||||
default = 30
|
||||
}
|
||||
@@ -0,0 +1,678 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# GCP Provider – Full implementation for Claude Code Agent Monitor
|
||||
#
|
||||
# Architecture:
|
||||
# VPC → Cloud Run (blue/green) → Filestore → Cloud Load Balancer
|
||||
# → Cloud Monitoring → Managed SSL Certificate
|
||||
#
|
||||
# Cloud Run is chosen over GKE for cost efficiency and operational simplicity
|
||||
# for this containerised workload. Filestore provides NFS for SQLite.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
provider "google" {
|
||||
project = var.gcp_project_id
|
||||
region = var.region
|
||||
}
|
||||
|
||||
provider "google-beta" {
|
||||
project = var.gcp_project_id
|
||||
region = var.region
|
||||
}
|
||||
|
||||
# ── Locals ──────────────────────────────────────────────────────────────────
|
||||
|
||||
locals {
|
||||
name_prefix = lower(replace("${var.project_name}-${var.environment}", "_", "-"))
|
||||
|
||||
common_labels = merge(
|
||||
{
|
||||
project = replace(var.project_name, "-", "_")
|
||||
environment = var.environment
|
||||
managed_by = "terraform"
|
||||
cloud_provider = "gcp"
|
||||
},
|
||||
{ for k, v in var.tags : replace(k, "-", "_") => replace(v, "-", "_") },
|
||||
)
|
||||
}
|
||||
|
||||
# ── Enable required APIs ───────────────────────────────────────────────────
|
||||
|
||||
resource "google_project_service" "apis" {
|
||||
for_each = toset([
|
||||
"run.googleapis.com",
|
||||
"compute.googleapis.com",
|
||||
"file.googleapis.com",
|
||||
"vpcaccess.googleapis.com",
|
||||
"monitoring.googleapis.com",
|
||||
"logging.googleapis.com",
|
||||
"certificatemanager.googleapis.com",
|
||||
])
|
||||
|
||||
service = each.value
|
||||
disable_on_destroy = false
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# VPC Network
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "google_compute_network" "main" {
|
||||
name = "${local.name_prefix}-vpc"
|
||||
auto_create_subnetworks = false
|
||||
|
||||
depends_on = [google_project_service.apis]
|
||||
}
|
||||
|
||||
resource "google_compute_subnetwork" "private" {
|
||||
name = "${local.name_prefix}-private"
|
||||
ip_cidr_range = var.private_subnet_cidrs[0]
|
||||
region = var.region
|
||||
network = google_compute_network.main.id
|
||||
|
||||
private_ip_google_access = true
|
||||
|
||||
log_config {
|
||||
aggregation_interval = "INTERVAL_5_SEC"
|
||||
flow_sampling = 0.5
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_compute_subnetwork" "proxy" {
|
||||
name = "${local.name_prefix}-proxy"
|
||||
ip_cidr_range = "10.0.100.0/24"
|
||||
region = var.region
|
||||
network = google_compute_network.main.id
|
||||
purpose = "REGIONAL_MANAGED_PROXY"
|
||||
role = "ACTIVE"
|
||||
}
|
||||
|
||||
# Cloud NAT for outbound internet
|
||||
resource "google_compute_router" "main" {
|
||||
name = "${local.name_prefix}-router"
|
||||
region = var.region
|
||||
network = google_compute_network.main.id
|
||||
}
|
||||
|
||||
resource "google_compute_router_nat" "main" {
|
||||
name = "${local.name_prefix}-nat"
|
||||
router = google_compute_router.main.name
|
||||
region = var.region
|
||||
nat_ip_allocate_option = "AUTO_ONLY"
|
||||
source_subnetwork_ip_ranges_to_nat = "ALL_SUBNETWORKS_ALL_IP_RANGES"
|
||||
|
||||
log_config {
|
||||
enable = true
|
||||
filter = "ERRORS_ONLY"
|
||||
}
|
||||
}
|
||||
|
||||
# VPC Connector for Cloud Run → Filestore
|
||||
resource "google_vpc_access_connector" "main" {
|
||||
name = "${local.name_prefix}-conn"
|
||||
region = var.region
|
||||
network = google_compute_network.main.id
|
||||
ip_cidr_range = "10.0.200.0/28"
|
||||
min_instances = 2
|
||||
max_instances = var.environment == "production" ? 10 : 3
|
||||
|
||||
depends_on = [google_project_service.apis]
|
||||
}
|
||||
|
||||
# Firewall rules
|
||||
resource "google_compute_firewall" "allow_health_checks" {
|
||||
name = "${local.name_prefix}-allow-health-checks"
|
||||
network = google_compute_network.main.id
|
||||
|
||||
allow {
|
||||
protocol = "tcp"
|
||||
ports = [tostring(var.app_port), tostring(var.mcp_port)]
|
||||
}
|
||||
|
||||
source_ranges = ["130.211.0.0/22", "35.191.0.0/16"] # GCP health check ranges
|
||||
target_tags = ["${local.name_prefix}-app"]
|
||||
}
|
||||
|
||||
resource "google_compute_firewall" "allow_internal" {
|
||||
name = "${local.name_prefix}-allow-internal"
|
||||
network = google_compute_network.main.id
|
||||
|
||||
allow {
|
||||
protocol = "tcp"
|
||||
ports = [tostring(var.app_port), tostring(var.mcp_port), "2049"]
|
||||
}
|
||||
|
||||
source_ranges = [var.vpc_cidr]
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Filestore (NFS for SQLite persistence)
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "google_filestore_instance" "main" {
|
||||
name = "${local.name_prefix}-data"
|
||||
location = "${var.region}-b"
|
||||
tier = var.environment == "production" ? "BASIC_SSD" : "BASIC_HDD"
|
||||
|
||||
file_shares {
|
||||
name = "appdata"
|
||||
capacity_gb = var.storage_size_gb
|
||||
}
|
||||
|
||||
networks {
|
||||
network = google_compute_network.main.name
|
||||
modes = ["MODE_IPV4"]
|
||||
}
|
||||
|
||||
labels = local.common_labels
|
||||
|
||||
depends_on = [google_project_service.apis]
|
||||
|
||||
lifecycle {
|
||||
prevent_destroy = true
|
||||
}
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Cloud Run services (Blue / Green)
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "google_cloud_run_v2_service" "blue" {
|
||||
name = "${local.name_prefix}-blue"
|
||||
location = var.region
|
||||
ingress = "INGRESS_TRAFFIC_INTERNAL_LOAD_BALANCER"
|
||||
|
||||
template {
|
||||
scaling {
|
||||
min_instance_count = var.active_deployment_slot == "blue" ? var.min_replicas : 0
|
||||
max_instance_count = var.active_deployment_slot == "blue" ? var.max_replicas : 1
|
||||
}
|
||||
|
||||
vpc_access {
|
||||
connector = google_vpc_access_connector.main.id
|
||||
egress = "ALL_TRAFFIC"
|
||||
}
|
||||
|
||||
containers {
|
||||
image = var.app_container_image
|
||||
name = "app"
|
||||
|
||||
ports {
|
||||
container_port = var.app_port
|
||||
}
|
||||
|
||||
resources {
|
||||
limits = {
|
||||
cpu = "${var.cpu}m"
|
||||
memory = "${var.memory}Mi"
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "env" {
|
||||
for_each = var.environment_variables
|
||||
content {
|
||||
name = env.key
|
||||
value = env.value
|
||||
}
|
||||
}
|
||||
|
||||
env {
|
||||
name = "FILESTORE_IP"
|
||||
value = google_filestore_instance.main.networks[0].ip_addresses[0]
|
||||
}
|
||||
|
||||
startup_probe {
|
||||
http_get {
|
||||
path = var.health_check_path
|
||||
port = var.app_port
|
||||
}
|
||||
initial_delay_seconds = 10
|
||||
period_seconds = 10
|
||||
failure_threshold = 5
|
||||
}
|
||||
|
||||
liveness_probe {
|
||||
http_get {
|
||||
path = var.health_check_path
|
||||
port = var.app_port
|
||||
}
|
||||
period_seconds = 30
|
||||
failure_threshold = 3
|
||||
}
|
||||
|
||||
volume_mounts {
|
||||
name = "app-data"
|
||||
mount_path = "/app/data"
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "containers" {
|
||||
for_each = var.mcp_container_image != "" ? [1] : []
|
||||
content {
|
||||
image = var.mcp_container_image
|
||||
name = "mcp-sidecar"
|
||||
|
||||
ports {
|
||||
container_port = var.mcp_port
|
||||
}
|
||||
|
||||
resources {
|
||||
limits = {
|
||||
cpu = "250m"
|
||||
memory = "256Mi"
|
||||
}
|
||||
}
|
||||
|
||||
env {
|
||||
name = "NODE_ENV"
|
||||
value = "production"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
volumes {
|
||||
name = "app-data"
|
||||
nfs {
|
||||
server = google_filestore_instance.main.networks[0].ip_addresses[0]
|
||||
path = "/appdata"
|
||||
read_only = false
|
||||
}
|
||||
}
|
||||
|
||||
session_affinity = true
|
||||
timeout = "300s"
|
||||
}
|
||||
|
||||
labels = local.common_labels
|
||||
|
||||
depends_on = [google_project_service.apis]
|
||||
|
||||
lifecycle {
|
||||
ignore_changes = [
|
||||
client,
|
||||
client_version,
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_cloud_run_v2_service" "green" {
|
||||
name = "${local.name_prefix}-green"
|
||||
location = var.region
|
||||
ingress = "INGRESS_TRAFFIC_INTERNAL_LOAD_BALANCER"
|
||||
|
||||
template {
|
||||
scaling {
|
||||
min_instance_count = var.active_deployment_slot == "green" ? var.min_replicas : 0
|
||||
max_instance_count = var.active_deployment_slot == "green" ? var.max_replicas : 1
|
||||
}
|
||||
|
||||
vpc_access {
|
||||
connector = google_vpc_access_connector.main.id
|
||||
egress = "ALL_TRAFFIC"
|
||||
}
|
||||
|
||||
containers {
|
||||
image = var.app_container_image
|
||||
name = "app"
|
||||
|
||||
ports {
|
||||
container_port = var.app_port
|
||||
}
|
||||
|
||||
resources {
|
||||
limits = {
|
||||
cpu = "${var.cpu}m"
|
||||
memory = "${var.memory}Mi"
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "env" {
|
||||
for_each = var.environment_variables
|
||||
content {
|
||||
name = env.key
|
||||
value = env.value
|
||||
}
|
||||
}
|
||||
|
||||
env {
|
||||
name = "FILESTORE_IP"
|
||||
value = google_filestore_instance.main.networks[0].ip_addresses[0]
|
||||
}
|
||||
|
||||
startup_probe {
|
||||
http_get {
|
||||
path = var.health_check_path
|
||||
port = var.app_port
|
||||
}
|
||||
initial_delay_seconds = 10
|
||||
period_seconds = 10
|
||||
failure_threshold = 5
|
||||
}
|
||||
|
||||
liveness_probe {
|
||||
http_get {
|
||||
path = var.health_check_path
|
||||
port = var.app_port
|
||||
}
|
||||
period_seconds = 30
|
||||
failure_threshold = 3
|
||||
}
|
||||
|
||||
volume_mounts {
|
||||
name = "app-data"
|
||||
mount_path = "/app/data"
|
||||
}
|
||||
}
|
||||
|
||||
volumes {
|
||||
name = "app-data"
|
||||
nfs {
|
||||
server = google_filestore_instance.main.networks[0].ip_addresses[0]
|
||||
path = "/appdata"
|
||||
read_only = false
|
||||
}
|
||||
}
|
||||
|
||||
session_affinity = true
|
||||
timeout = "300s"
|
||||
}
|
||||
|
||||
labels = local.common_labels
|
||||
|
||||
depends_on = [google_project_service.apis]
|
||||
|
||||
lifecycle {
|
||||
ignore_changes = [
|
||||
client,
|
||||
client_version,
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
# IAM – restrict access to load balancer service account only
|
||||
# Cloud Run ingress is set to INGRESS_TRAFFIC_INTERNAL_LOAD_BALANCER,
|
||||
# so public IAM bindings are not needed. The LB routes traffic internally.
|
||||
# To grant specific service account access, replace with:
|
||||
# member = "serviceAccount:<your-lb-service-account>@<project>.iam.gserviceaccount.com"
|
||||
#
|
||||
# resource "google_cloud_run_v2_service_iam_member" "blue_invoker" {
|
||||
# name = google_cloud_run_v2_service.blue.name
|
||||
# location = var.region
|
||||
# role = "roles/run.invoker"
|
||||
# member = "serviceAccount:${var.project_id}-compute@developer.gserviceaccount.com"
|
||||
# }
|
||||
#
|
||||
# resource "google_cloud_run_v2_service_iam_member" "green_invoker" {
|
||||
# name = google_cloud_run_v2_service.green.name
|
||||
# location = var.region
|
||||
# role = "roles/run.invoker"
|
||||
# member = "serviceAccount:${var.project_id}-compute@developer.gserviceaccount.com"
|
||||
# }
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# External Application Load Balancer
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
# Serverless NEGs for Cloud Run
|
||||
resource "google_compute_region_network_endpoint_group" "blue" {
|
||||
name = "${local.name_prefix}-blue-neg"
|
||||
region = var.region
|
||||
network_endpoint_type = "SERVERLESS"
|
||||
|
||||
cloud_run {
|
||||
service = google_cloud_run_v2_service.blue.name
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_compute_region_network_endpoint_group" "green" {
|
||||
name = "${local.name_prefix}-green-neg"
|
||||
region = var.region
|
||||
network_endpoint_type = "SERVERLESS"
|
||||
|
||||
cloud_run {
|
||||
service = google_cloud_run_v2_service.green.name
|
||||
}
|
||||
}
|
||||
|
||||
# Backend service with weighted backends for blue/green
|
||||
resource "google_compute_backend_service" "main" {
|
||||
name = "${local.name_prefix}-backend"
|
||||
protocol = "HTTP"
|
||||
load_balancing_scheme = "EXTERNAL_MANAGED"
|
||||
timeout_sec = 300 # WebSocket support
|
||||
|
||||
session_affinity = "GENERATED_COOKIE"
|
||||
|
||||
backend {
|
||||
group = google_compute_region_network_endpoint_group.blue.id
|
||||
capacity_scaler = var.blue_weight / 100
|
||||
}
|
||||
|
||||
backend {
|
||||
group = google_compute_region_network_endpoint_group.green.id
|
||||
capacity_scaler = var.green_weight / 100
|
||||
}
|
||||
|
||||
health_checks = [google_compute_health_check.main.id]
|
||||
|
||||
log_config {
|
||||
enable = true
|
||||
sample_rate = 1.0
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_compute_health_check" "main" {
|
||||
name = "${local.name_prefix}-hc"
|
||||
|
||||
http_health_check {
|
||||
port = var.app_port
|
||||
request_path = var.health_check_path
|
||||
}
|
||||
|
||||
check_interval_sec = var.health_check_interval
|
||||
timeout_sec = var.health_check_timeout
|
||||
healthy_threshold = var.health_check_healthy_threshold
|
||||
unhealthy_threshold = var.health_check_unhealthy_threshold
|
||||
}
|
||||
|
||||
# URL map
|
||||
resource "google_compute_url_map" "main" {
|
||||
name = "${local.name_prefix}-urlmap"
|
||||
default_service = google_compute_backend_service.main.id
|
||||
}
|
||||
|
||||
# Managed SSL certificate (optional)
|
||||
resource "google_compute_managed_ssl_certificate" "main" {
|
||||
count = var.domain_name != "" ? 1 : 0
|
||||
|
||||
name = "${local.name_prefix}-cert"
|
||||
|
||||
managed {
|
||||
domains = [var.domain_name]
|
||||
}
|
||||
}
|
||||
|
||||
# HTTPS proxy
|
||||
resource "google_compute_target_https_proxy" "main" {
|
||||
count = var.domain_name != "" ? 1 : 0
|
||||
|
||||
name = "${local.name_prefix}-https-proxy"
|
||||
url_map = google_compute_url_map.main.id
|
||||
ssl_certificates = [google_compute_managed_ssl_certificate.main[0].id]
|
||||
}
|
||||
|
||||
# HTTP proxy (for redirect or direct access)
|
||||
resource "google_compute_target_http_proxy" "main" {
|
||||
name = "${local.name_prefix}-http-proxy"
|
||||
url_map = google_compute_url_map.main.id
|
||||
}
|
||||
|
||||
# Global forwarding rules
|
||||
resource "google_compute_global_forwarding_rule" "https" {
|
||||
count = var.domain_name != "" ? 1 : 0
|
||||
|
||||
name = "${local.name_prefix}-https"
|
||||
target = google_compute_target_https_proxy.main[0].id
|
||||
port_range = "443"
|
||||
ip_protocol = "TCP"
|
||||
load_balancing_scheme = "EXTERNAL_MANAGED"
|
||||
}
|
||||
|
||||
resource "google_compute_global_forwarding_rule" "http" {
|
||||
name = "${local.name_prefix}-http"
|
||||
target = google_compute_target_http_proxy.main.id
|
||||
port_range = "80"
|
||||
ip_protocol = "TCP"
|
||||
load_balancing_scheme = "EXTERNAL_MANAGED"
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Cloud Monitoring – Alert policies
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "google_monitoring_notification_channel" "email" {
|
||||
count = var.alert_email != "" ? 1 : 0
|
||||
|
||||
display_name = "${local.name_prefix}-email"
|
||||
type = "email"
|
||||
|
||||
labels = {
|
||||
email_address = var.alert_email
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_monitoring_alert_policy" "high_latency" {
|
||||
count = var.enable_monitoring ? 1 : 0
|
||||
|
||||
display_name = "${local.name_prefix}-high-latency"
|
||||
combiner = "OR"
|
||||
|
||||
conditions {
|
||||
display_name = "Cloud Run request latency > 2s"
|
||||
|
||||
condition_threshold {
|
||||
filter = "resource.type=\"cloud_run_revision\" AND metric.type=\"run.googleapis.com/request_latencies\""
|
||||
comparison = "COMPARISON_GT"
|
||||
duration = "300s"
|
||||
|
||||
threshold_value = 2000 # ms
|
||||
|
||||
aggregations {
|
||||
alignment_period = "60s"
|
||||
per_series_aligner = "ALIGN_PERCENTILE_99"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
notification_channels = var.alert_email != "" ? [google_monitoring_notification_channel.email[0].id] : []
|
||||
|
||||
alert_strategy {
|
||||
auto_close = "604800s"
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_monitoring_alert_policy" "high_error_rate" {
|
||||
count = var.enable_monitoring ? 1 : 0
|
||||
|
||||
display_name = "${local.name_prefix}-high-error-rate"
|
||||
combiner = "OR"
|
||||
|
||||
conditions {
|
||||
display_name = "Cloud Run 5xx error rate"
|
||||
|
||||
condition_threshold {
|
||||
filter = "resource.type=\"cloud_run_revision\" AND metric.type=\"run.googleapis.com/request_count\" AND metric.labels.response_code_class=\"5xx\""
|
||||
comparison = "COMPARISON_GT"
|
||||
duration = "300s"
|
||||
|
||||
threshold_value = 10
|
||||
|
||||
aggregations {
|
||||
alignment_period = "60s"
|
||||
per_series_aligner = "ALIGN_RATE"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
notification_channels = var.alert_email != "" ? [google_monitoring_notification_channel.email[0].id] : []
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Cloud Monitoring Dashboard
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
resource "google_monitoring_dashboard" "main" {
|
||||
count = var.enable_monitoring ? 1 : 0
|
||||
|
||||
dashboard_json = jsonencode({
|
||||
displayName = "${local.name_prefix} Dashboard"
|
||||
gridLayout = {
|
||||
columns = 2
|
||||
widgets = [
|
||||
{
|
||||
title = "Request Count"
|
||||
xyChart = {
|
||||
dataSets = [{
|
||||
timeSeriesQuery = {
|
||||
timeSeriesFilter = {
|
||||
filter = "resource.type=\"cloud_run_revision\" AND metric.type=\"run.googleapis.com/request_count\""
|
||||
aggregation = {
|
||||
alignmentPeriod = "60s"
|
||||
perSeriesAligner = "ALIGN_RATE"
|
||||
}
|
||||
}
|
||||
}
|
||||
}]
|
||||
}
|
||||
},
|
||||
{
|
||||
title = "Request Latency (p99)"
|
||||
xyChart = {
|
||||
dataSets = [{
|
||||
timeSeriesQuery = {
|
||||
timeSeriesFilter = {
|
||||
filter = "resource.type=\"cloud_run_revision\" AND metric.type=\"run.googleapis.com/request_latencies\""
|
||||
aggregation = {
|
||||
alignmentPeriod = "60s"
|
||||
perSeriesAligner = "ALIGN_PERCENTILE_99"
|
||||
}
|
||||
}
|
||||
}
|
||||
}]
|
||||
}
|
||||
},
|
||||
{
|
||||
title = "Instance Count"
|
||||
xyChart = {
|
||||
dataSets = [{
|
||||
timeSeriesQuery = {
|
||||
timeSeriesFilter = {
|
||||
filter = "resource.type=\"cloud_run_revision\" AND metric.type=\"run.googleapis.com/container/instance_count\""
|
||||
aggregation = {
|
||||
alignmentPeriod = "60s"
|
||||
perSeriesAligner = "ALIGN_MEAN"
|
||||
}
|
||||
}
|
||||
}
|
||||
}]
|
||||
}
|
||||
},
|
||||
{
|
||||
title = "CPU Utilization"
|
||||
xyChart = {
|
||||
dataSets = [{
|
||||
timeSeriesQuery = {
|
||||
timeSeriesFilter = {
|
||||
filter = "resource.type=\"cloud_run_revision\" AND metric.type=\"run.googleapis.com/container/cpu/utilizations\""
|
||||
aggregation = {
|
||||
alignmentPeriod = "60s"
|
||||
perSeriesAligner = "ALIGN_PERCENTILE_99"
|
||||
}
|
||||
}
|
||||
}
|
||||
}]
|
||||
}
|
||||
},
|
||||
]
|
||||
}
|
||||
})
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user