en:kubernetes:deployment [Colocall FAQ]

Instructions for Deploying a Service in a Virtual Cluster

Install K8up, which we will use for volume backups:

Create namespace
```
kubectl create ns k8up
```

Add helm repository

helm repo add k8up-io https://k8up-io.github.io/k8up

Install k8up release and k8up-crd

helm install k8up k8up-io/k8up -n k8up
kubectl apply -f https://github.com/k8up-io/k8up/releases/download/k8up-4.8.4/k8up-crd.yaml --server-side

Next, install cert-manager and ingress controller:

Install cert-manager

helm repo add jetstack https://charts.jetstack.io --force-update
helm install cert-manager oci://quay.io/jetstack/charts/cert-manager --version v1.18.2 --namespace cert-manager --create-namespace --set crds.enabled=true

Install ingress controller traefik

kubectl create namespace traefik-namespace
helm repo add traefik https://helm.traefik.io/traefik
helm repo update
helm install --namespace=traefik-namespace traefik traefik/traefik

Edit traefik service and add annotations
```
annotations:
  "lbipam.cilium.io/sharing-key": "password"
```
Password is taken from service parameters

Configure LetsEncrypt certificate generation. Email must be real:

Create test issuer

letsencrypt-staging.yaml

---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    email: hello@example.com
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-prod-key
    solvers:
      - http01:
          ingress:
            class: traefik
            serviceType: ClusterIP

Create production issuer

letsencrypt-prod.yaml

---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    email: hello@example.com
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-prod-key
    solvers:
      - http01:
          ingress:
            class: traefik
            serviceType: ClusterIP

Install MySQL:

Install Percona MySQL operator. Documentation: https://docs.percona.com/percona-operator-for-mysql/pxc/index.html

helm repo add percona https://percona.github.io/percona-helm-charts/
helm repo update
kubectl create namespace mysql-operator
helm install mysql-operator percona/pxc-operator --namespace mysql-operator

Create database:

Get helm chart config

helm show values percona/pxc-db > values.yaml

Set resource limits/requests, storageclass, s3 endpoint and backup

values.yaml

# example configuration
...
pxc:
  size: 3
  resources:
    requests:
      memory: 1G
      cpu: 600m
    limits:
      memory: 1G
      # cpu: 600m
  persistence:
    enabled: true
    storageClass: "sc-name"
    accessMode: ReadWriteOnce
    size: 8Gi
...
haproxy:
  enabled: true
  size: 3
  resources:
    requests:
      memory: 200Mi
      cpu: 200m
    limits:
      memory: 400Mi
      # cpu: 400m
...
logcollector:
  enabled: true
  resources:
    requests:
      memory: 100M
      cpu: 200m
    limits:
      memory: 100M
  # A custom Kubernetes Security Context for a Container to be used instead of the default one
  containerSecurityContext:
    privileged: false
...
backup:
  enabled: true
  storages:
    fs-pvc:
      type: filesystem
      volume:
        persistentVolumeClaim:
          storageClassName: sc-name
          accessModes: ["ReadWriteOnce"]
          resources:
            requests:
              storage: 10Gi
    minio:
      type: s3
      verifyTLS: false # if self sighned cert
      resources:
        requests:
          memory: 1Gi
          cpu: 600m
      s3:
        bucket: S3-BACKUP-BUCKET-NAME-HERE
        # Use credentialsSecret OR credentialsAccessKey/credentialsSecretKey
    #     credentialsSecret: my-cluster-name-backup-s3
        credentialsAccessKey: <s3-login-key>
        credentialsSecretKey: <s3-secret-key>
    #     region: us-west-2
        endpointUrl: https://<s3-url>

  schedule:
    - name: "daily-backup"
      schedule: "0 0 * * *"
      retention:
        type: "count"
        count: 3
        deleteFromStorage: true
      storageName: fs-pvc
    - name: "s3-backup"
      schedule: "0 1 * * *"
      retention:
        type: "count"
        count: 3
        deleteFromStorage: true
      storageName: minio

Percona operator allows backups to persistent volume or s3. Both can be used simultaneously.

Install database
```
helm install wordpress-db percona/pxc-db --namespace mysql-operator -f values.yaml
```
Root password is in secret `wordpress-db-pxc-db-secrets` in namespace mysql-operator

Install WordPress:

Create deployment manifest

wordpress.yaml

---
apiVersion: v1
kind: Secret
metadata:
  name: mysql-secret
type: Opaque
data:
  # root DB password from secret wordpress-db-pxc-db-secrets
  MYSQL_ROOT_PASSWORD: SDc4XjFRMW1zUWgraD1XWiVZXw==  
 
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: wordpress-pvc
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: "sc-name"
 
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: wordpress
spec:
  replicas: 1
  selector:
    matchLabels:
      app: wordpress
  template:
    metadata:
      labels:
        app: wordpress
    spec:
      containers:
        - name: wordpress
          image: wordpress:5.8.3-php7.4-apache
          resources:
            limits:
              memory: 1Gi
            requests:
              cpu: 500m
              memory: 1Gi
          ports:
            - containerPort: 80
              name: wordpress
          volumeMounts:
            - name: wordpress-data
              mountPath: /var/www/html
          env:
            - name: WORDPRESS_DB_HOST
              # FQDN is service-name.service-namespace.svc.cluster.local
              value: wordpress-mysql-haproxy.mysql-operator.svc.cluster.local 
            - name: WORDPRESS_DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: mysql-secret
                  key: MYSQL_ROOT_PASSWORD
            - name: WORDPRESS_DB_USER
              value: root
            - name: WORDPRESS_DB_NAME
              value: mysql
      volumes:
        - name: wordpress-data
          persistentVolumeClaim:
            claimName: wordpress-pvc
---
kind: Service
apiVersion: v1
metadata:
  name: wordpress-service
spec:
  selector:
    app: wordpress
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 80
 
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: wordpress-ingress
  namespace: wordpress
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: websecure
    traefik.ingress.kubernetes.io/router.tls: "true"
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  rules:
    - host: example.com.ua
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: wordpress-service
                port:
                  number: 80
  tls:
    - secretName: web-app-cert
      hosts:
        - example.com.ua

Create namespace and deploy

kubectl create ns wordpress
kubectl apply -f wordpress.yaml -n wordpress

Backup volumes with k8up:

Prepare S3 access parameters

echo -n '<s3-secret>' > S3_ACCOUNT_KEY
echo -n '<s3-key>' > S3_ACCOUNT_NAME
echo -n '<password>' > RESTIC_PASSWORD
kubectl create secret generic -n wordpress secret-backup-host --from-file=./RESTIC_PASSWORD --from-file=./S3_ACCOUNT_NAME --from-file=./S3_ACCOUNT_KEY

If S3 uses self-signed cert, add CA

kubectl create secret generic -n wordpress ca-tls --from-file=./CA.crt

Create backup manifest

wordpress-backup.yaml

---
apiVersion: k8up.io/v1
kind: Backup
metadata:
  name: wordpress-backup

spec:
  failedJobsHistoryLimit: 2
  successfulJobsHistoryLimit: 2
  backend:
    repoPasswordSecretRef:
      name: secret-test-backup
      key: RESTIC_PASSWORD
    s3:
      endpoint: <s3-url>
      bucket: test-1
      accessKeyIDSecretRef:
        name: secret-test-backup
        key: S3_ACCOUNT_NAME
      secretAccessKeySecretRef:
        name: secret-test-backup
        key: S3_ACCOUNT_KEY
  # if self sighned cert
    tlsOptions:
      caCert: /mnt/ca/CA.crt
    volumeMounts:
      - name: ca-tls
        mountPath: /mnt/ca/
  volumes:
    - name: ca-tls
      secret:
        secretName: ca-tls
        defaultMode: 420

Deploy manifest

kubectl apply -f wordpress-backup.yaml -n wordpress

Restore data from backup:

View snapshots

kubectl get snapshots -A
kubectl get snapshot <snapshot_name> -n wordpress -o yaml

Create restore manifest

wordpress-restore.yaml

---
apiVersion: k8up.io/v1
kind: Restore
metadata:
  name: restore-test-backup
spec:
  snapshot: b8528e47712b2c24a35a3e7c6edb553804206b0ea31e97a654e7545a8ec71c67 # id from snapshot manifest
  restoreMethod:
    folder:
      claimName: "pvc-name"
  backend:
    repoPasswordSecretRef:
      name: secret-test-backup
      key: RESTIC_PASSWORD
    s3:
      endpoint: <s3-url>
      bucket: test-1
      accessKeyIDSecretRef:
        name: secret-test-backup
        key: S3_ACCOUNT_NAME
      secretAccessKeySecretRef:
        name: secret-test-backup
        key: S3_ACCOUNT_KEY
  # if self sighned cert
    tlsOptions:
      caCert: /mnt/ca/CA.crt
    volumeMounts:
      - name: ca-tls
        mountPath: /mnt/ca/
  volumes:
    - name: ca-tls
      secret:
        secretName: ca-tls
        defaultMode: 420

Deploy restore

kubectl apply -f restore-test.yaml -n wordpress

Database backup/restore with Percona operator

Docs: https://docs.percona.com/percona-operator-for-mysql/pxc/backups.html https://docs.percona.com/percona-operator-for-mysql/pxc/backups-restore.html

Restore from PVC:

mysql-restore-pvc.yaml

---
apiVersion: pxc.percona.com/v1
kind: PerconaXtraDBClusterRestore
metadata:
  name: mysql-restore-pvc
spec:
  pxcCluster: wordpress-db
  backupName: daily-backup
  storageName: fs-pvc

 kubectl apply -f mysql-restore-pvc.yaml -n percona-operator

Restore from S3:

mysql-restore-s3.yaml

---
apiVersion: pxc.percona.com/v1
kind: PerconaXtraDBClusterRestore
metadata:
  name: mysql-restore-s3
spec:
  pxcCluster: wordpress-db
  backupName: sat-night-backup
  storageName: minio

 kubectl apply -f mysql-restore-s3.yaml -n percona-operator

Private registry

Create secret

kubectl create secret docker-registry my-registry-secret \
  --docker-server=https://registry.kube.colocall.net \
  --docker-username=<user> \
  --docker-password=<password> \
  --docker-email=<email> -n <namespace>

Use secret in pod config

apiVersion: v1
kind: Pod
metadata:
  name: private-image-pod
spec:
  containers:
  - name: app
    image: registry.kube.colocall.net/user/image-name[:TAG]
  imagePullSecrets:
  - name: my-registry-secret

Docker push/pull

docker tag image-name registry.kube.colocall.net/user/image-name:version
docker push registry.kube.colocall.net/user/image-name:version
docker pull registry.kube.colocall.net/user/image-name:version

Links to the documentation of the tools used in this guide:

traefik-ingress

cert-manager

k8up

percona-mysql-operator

Instructions for Deploying a Service in a Virtual Cluster

Table of Contents