In modern software development, CI/CD (Continuous Integration/Continuous Deployment) is an indispensable process. In this article, I'll show you how to create a fully automated deployment pipeline by setting up a Kubernetes cluster on Proxmox virtual server infrastructure and integrating it with GitLab CI/CD.
Architecture Overview
Main components of our system:
- Proxmox VE: Hypervisor and VM management
- Kubernetes: Container orchestration
- GitLab: CI/CD and source control
- Docker Registry: Container image storage
- Cloudflare: DNS and CDN
Proxmox Setup and Configuration
Proxmox is a powerful KVM-based virtual server platform. First, we'll create our VMs.
Creating VM Template
# Download Ubuntu cloud image
wget https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64.img
# Create VM
qm create 9000 --memory 2048 --cores 2 --name ubuntu-cloud --net0 virtio,bridge=vmbr0
qm importdisk 9000 jammy-server-cloudimg-amd64.img local-lvm
qm set 9000 --scsihw virtio-scsi-pci --scsi0 local-lvm:vm-9000-disk-0
qm set 9000 --boot c --bootdisk scsi0
qm set 9000 --ide2 local-lvm:cloudinit
qm set 9000 --serial0 socket --vga serial0
qm set 9000 --agent enabled=1
# Convert to template
qm template 9000
Kubernetes Cluster Setup
Let's create our Kubernetes nodes using the template.
Master Node Setup
# Install kubeadm, kubelet, kubectl
sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.28/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.28/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
# Initialize cluster
sudo kubeadm init --pod-network-cidr=10.244.0.0/16
# Configure kubectl
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
CNI Plugin (Calico) Installation
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.26.1/manifests/tigera-operator.yaml
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.26.1/manifests/custom-resources.yaml
GitLab Runner Installation
We'll run GitLab Runner on Kubernetes.
apiVersion: v1
kind: Namespace
metadata:
name: gitlab-runner
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: gitlab-runner
namespace: gitlab-runner
spec:
replicas: 1
selector:
matchLabels:
app: gitlab-runner
template:
metadata:
labels:
app: gitlab-runner
spec:
containers:
- name: gitlab-runner
image: gitlab/gitlab-runner:latest
env:
- name: RUNNER_EXECUTOR
value: "kubernetes"
volumeMounts:
- name: config
mountPath: /etc/gitlab-runner
volumes:
- name: config
configMap:
name: gitlab-runner-config
CI/CD Pipeline Configuration
Our .gitlab-ci.yml file:
stages:
- build
- test
- deploy
variables:
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: ""
build:
stage: build
image: docker:latest
services:
- docker:dind
script:
- docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA .
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
deploy:
stage: deploy
image: bitnami/kubectl:latest
script:
- kubectl set image deployment/myapp myapp=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA -n production
- kubectl rollout status deployment/myapp -n production
Docker Registry Integration
Let's run our private Docker registry on Kubernetes:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: docker-registry-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: docker-registry
spec:
replicas: 1
selector:
matchLabels:
app: docker-registry
template:
metadata:
labels:
app: docker-registry
spec:
containers:
- name: registry
image: registry:2
ports:
- containerPort: 5000
volumeMounts:
- name: storage
mountPath: /var/lib/registry
volumes:
- name: storage
persistentVolumeClaim:
claimName: docker-registry-pvc
Cloudflare DNS and CDN
Domain management and CDN setup via Cloudflare:
- Add domain to Cloudflare
- Update DNS records
- Activate SSL/TLS certificate
- Set cache strategy with Page Rules
# Add Cloudflare API token to Kubernetes
kubectl create secret generic cloudflare-api-token \
--from-literal=api-token=YOUR_TOKEN \
-n kube-system
Monitoring and Logging
Monitoring with Prometheus and Grafana:
# Install Prometheus Operator
kubectl create -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/main/bundle.yaml
# Install Grafana
helm repo add grafana https://grafana.github.io/helm-charts
helm install grafana grafana/grafana -n monitoring --create-namespace
Auto Scaling
Horizontal Pod Autoscaler (HPA):
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: myapp-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: myapp
minReplicas: 2
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
Backup Strategy
Kubernetes backup with Velero:
# Install Velero
velero install \
--provider aws \
--plugins velero/velero-plugin-for-aws:v1.8.0 \
--bucket velero-backups \
--backup-location-config region=eu-west-1 \
--snapshot-location-config region=eu-west-1
# Schedule automatic backup
velero schedule create daily-backup --schedule="0 2 * * *"
Security Measures
- Network Policies: Restrict inter-pod communication
- RBAC: Apply role-based access control
- Pod Security Standards: Enforce security contexts
- Secret Encryption: Encrypt secrets
- Image Scanning: Scan container images
Conclusion
With this architecture:
- ✅ Fully automated CI/CD pipeline
- ✅ Scalable Kubernetes cluster
- ✅ Monitoring and alerting
- ✅ Automatic backup
- ✅ CDN and DNS management
The Proxmox + Kubernetes combination provides a powerful and flexible infrastructure. With GitLab CI/CD integration, the entire process from development to production is automated.
"Everything that can be automated should be automated. Time is our most valuable resource."