> For AI agents: the complete documentation index is available at /llms.txt, the full documentation bundle is available at /llms-full.txt. # Use K8s Only :::info This guide is intended for users who are familiar with **Kubernetes**. It does not cover how to set up a Kubernetes cluster or provide tutorials on using `kubectl`. Additionally, this guide may include some advanced Kubernetes terminology, so a certain level of familiarity is required for reading. This article focuses on deploying GZCTF in a Kubernetes cluster. For configuration instructions specific to GZCTF itself, please refer to the [Quick Start](/guide/start/quick-start.md) guide. ::: ## Deployment Notes 1. GZCTF supports multi-instance deployment, but based on testing, currently the most stable deployment method is a single-instance deployment with the database on the same node. Therefore, this article will focus on single-instance deployment as an example. 2. For multi-instance deployment, all instances need to use S3/Object Storage to ensure file consistency and write concurrency (See [#365](https://github.com/GZTimeWalker/GZCTF/issues/365)). Additionally, Redis needs to be deployed to ensure cache consistency among multiple instances. Refer to [appsettings](/config/appsettings.md) documentation to configure both. Multiple providers are supported such as Azure/AWS/Minio. 3. For multi-instance deployment, the load balancer needs to be configured with sticky sessions to enable real-time data retrieval using websockets. 4. **If you prefer a simpler deployment, go for a single-instance deployment!** 5. Since you have chosen to deploy with Kubernetes, it implies that you need a larger number of Pods. Please pay attention to the following configuration: - Specify `--kube-controller-manager-arg=node-cidr-mask-size=22` during installation. The default is `/24`, supporting up to 255 Pods per node. We recommend `/22` (about 1024 Pods per node), or `/20` (about 4096 Pods per node) if needed. Using a smaller mask (e.g. `/16`) only reserves an oversized address block per node and reduces the cluster's ability to scale out workers. **This cannot be changed after installation.** ```bash INSTALL_K3S_EXEC="--kube-controller-manager-arg=node-cidr-mask-size=22" ``` - Adjust the value of `maxPods` accordingly, otherwise you may reach the Pod limit and be unable to schedule more Pods. ```yaml apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration maxPods: 800 ``` ## Deploying GZCTF 1. Create namespaces and configuration files. See [appsettings](/config/appsettings.md) ```yaml apiVersion: v1 kind: Namespace metadata: name: gzctf-server --- apiVersion: v1 kind: ConfigMap metadata: name: gzctf-config namespace: gzctf-server data: appsettings.json: | { "ConnectionStrings": { "Database": "Host=gzctf-db:5432;Database=ctf;Username=postgres;Password=...", "RedisCache": "gzctf-garnet:6379,abortConnect=false", "Storage": "s3://accessKey=...;secretKey=...;bucket=...;endpoint=...;forcePathStyle=true" }, ... } --- apiVersion: v1 kind: ServiceAccount metadata: name: gzctf-sa namespace: gzctf-server --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: gzctf-crb roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin # use to access the Kubernetes API subjects: - kind: ServiceAccount name: gzctf-sa namespace: gzctf-server --- apiVersion: v1 kind: Secret metadata: name: gzctf-tls namespace: gzctf-server type: kubernetes.io/tls data: tls.crt: ... # base64 encoded TLS certificate tls.key: ... # base64 encoded TLS private key ``` 2. Create local PV (if you need to share storage among multiple instances, please change the configuration yourself) ```yaml apiVersion: v1 kind: PersistentVolume metadata: name: gzctf-files-pv namespace: gzctf-server spec: capacity: storage: 2Gi # If you configured S3 storage then this PV will be used for storing logs only, so 2Gi is too much accessModes: - ReadWriteOnce hostPath: path: /mnt/path/to/gzctf/files # local path --- apiVersion: v1 kind: PersistentVolume metadata: name: gzctf-db-pv namespace: gzctf-server spec: capacity: storage: 1Gi accessModes: - ReadWriteOnce hostPath: path: /mnt/path/to/gzctf/db # local path --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: gzctf-files namespace: gzctf-server spec: accessModes: - ReadWriteOnce resources: requests: storage: 2Gi # If you configured S3 storage then this PV will be used for storing logs only, so 2Gi is too much volumeName: gzctf-files-pv --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: gzctf-db namespace: gzctf-server spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi volumeName: gzctf-db-pv ``` 3. Create the Deployment manifest of GZCTF ```yaml apiVersion: apps/v1 kind: Deployment metadata: name: gzctf namespace: gzctf-server labels: app: gzctf spec: replicas: 1 strategy: type: RollingUpdate selector: matchLabels: app: gzctf template: metadata: labels: app: gzctf spec: serviceAccountName: gzctf-sa nodeSelector: kubernetes.io/hostname: xxx # Specify the deployment node, forcing it to be on the same node as the database containers: - name: gzctf image: gztime/gzctf:latest imagePullPolicy: Always env: - name: GZCTF_ADMIN_PASSWORD value: xxx # Admin password # choose your backend language `en_US` / `zh_CN` / `ja_JP` - name: LC_ALL value: en_US.UTF-8 ports: - containerPort: 8080 name: http - containerPort: 3000 name: metrics volumeMounts: - name: gzctf-files mountPath: /app/files - name: gzctf-config mountPath: /app/appsettings.json subPath: appsettings.json resources: requests: cpu: 1000m memory: 384Mi volumes: - name: gzctf-files persistentVolumeClaim: claimName: gzctf-files - name: gzctf-config configMap: name: gzctf-config --- apiVersion: apps/v1 kind: Deployment metadata: name: gzctf-garnet namespace: gzctf-server labels: app: gzctf-garnet spec: replicas: 1 selector: matchLabels: app: gzctf-garnet template: metadata: labels: app: gzctf-garnet spec: containers: - name: gzctf-garnet image: ghcr.io/microsoft/garnet-alpine:latest imagePullPolicy: Always ports: - containerPort: 6379 name: garnet args: ["--bind", "0.0.0.0"] resources: requests: cpu: 10m memory: 64Mi --- apiVersion: apps/v1 kind: Deployment metadata: name: gzctf-db namespace: gzctf-server labels: app: gzctf-db spec: replicas: 1 selector: matchLabels: app: gzctf-db template: metadata: labels: app: gzctf-db spec: nodeSelector: kubernetes.io/hostname: xxx # Specify the deployment node, need to be on the same node as GZCTF containers: - name: gzctf-db image: postgres:alpine imagePullPolicy: Always ports: - containerPort: 5432 name: postgres env: - name: POSTGRES_PASSWORD value: xxx # Database password, needs to be consistent with the database password in appsettings.json volumeMounts: - name: gzctf-db mountPath: /var/lib/postgresql resources: requests: cpu: 500m memory: 512Mi volumes: - name: gzctf-db persistentVolumeClaim: claimName: gzctf-db ``` 4. Create Service and Ingress ```yaml apiVersion: v1 kind: Service metadata: name: gzctf namespace: gzctf-server annotations: # Enable Traefik Sticky Session traefik.ingress.kubernetes.io/service.sticky.cookie: "true" traefik.ingress.kubernetes.io/service.sticky.cookie.name: "LB_Session" traefik.ingress.kubernetes.io/service.sticky.cookie.httponly: "true" spec: selector: app: gzctf ports: - protocol: TCP port: 8080 targetPort: 8080 --- apiVersion: v1 kind: Service metadata: name: gzctf-db namespace: gzctf-server spec: selector: app: gzctf-db ports: - protocol: TCP port: 5432 targetPort: 5432 --- apiVersion: v1 kind: Service metadata: name: gzctf-garnet namespace: gzctf-server spec: selector: app: gzctf-garnet ports: - protocol: TCP port: 6379 targetPort: 6379 --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: gzctf namespace: gzctf-server annotations: # Some TLS settings for Traefik, you can modify them to satisfy your needs kubernetes.io/ingress.class: "traefik" traefik.ingress.kubernetes.io/router.tls: "true" ingress.kubernetes.io/force-ssl-redirect: "true" spec: tls: - hosts: - ctf.example.com # Domain name secretName: gzctf-tls # Certificate name, you need to create the corresponding Secret yourself rules: - host: ctf.example.com # Domain name http: paths: - path: / pathType: Prefix backend: service: name: gzctf port: number: 8080 ``` 5. Additional Configuration for Traefik In order to make GZCTF able to obtain the real IP address of users through XFF, Traefik needs to be able to add the XFF header correctly. Please note that the following content may not always be up-to-date and applicable to all versions of Traefik. This is an example of helm values, please search for the latest configuration method yourself. ```yaml service: spec: externalTrafficPolicy: Local # To make XFF work properly, set externalTrafficPolicy to Local deployment: kind: DaemonSet ports: web: redirectTo: websecure # Redirect HTTP to HTTPS additionalArguments: - "--entryPoints.web.proxyProtocol.insecure" - "--entryPoints.web.forwardedHeaders.insecure" - "--entryPoints.websecure.proxyProtocol.insecure" - "--entryPoints.websecure.forwardedHeaders.insecure" ``` ## Deployment Tips 1. If you want GZCTF to automatically create an admin account during initialization, make sure to pass the `GZCTF_ADMIN_PASSWORD` environment variable. Otherwise, you will need to manually create the admin account. 2. Please debug and verify if GZCTF can correctly retrieve the real IP address of users on the system log page. If not, check if the Traefik configuration is correct. 3. If you want to monitor resource usage, please deploy Prometheus and Grafana yourself, enable Prometheus support in Traefik, and use node exporter to monitor the resource usage of challenge containers. 4. If you need to automatically update the deployment of GZCTF based on the configuration file, please refer to [Reloader](https://github.com/stakater/Reloader). 5. Inside the cluster, you can use `https://kubernetes.default.svc.cluster.local:443` as the `server` field in the cluster configuration file.