Hetzer Infra

This repository is meant to setup my kubernetes-cluster on Hetzner Cloud. The setup is split into 2 dedicated parts:

1. Create necessary machines in Hetzner Cloud via terraform.

2. Install/Upgrade kubernetes-cluster and other software via ansible.

TL;DR

cd infra
terraform init
terraform apply
sleep 300 # Wait 5 minutes since the machines start _slow_ sometimes
cd ../k3s
ansible-galaxy install -r requirements.yml
ansible-playbook site.yml

Preparation

1. Ensure terraform is installed

2. Ensure ansible is installed

Setup

The project is split into different directories, each responsible for another task. For the initial setup it is mandatory to follow the instructions of each directory in the following order. Subsequent changes may only require a subset of these instructions. Though, it should be safe to run each step multiple times, they should all be indempotent.

infra

Run this setup in the infra/ directory.

Make sure config.auto.tfvars with all the needed configuration-secrets is present otherwise the module cannot be applied! The file is savely stored in the password-manager.

terraform init (1)
terraform apply (2)

1	Init the terraform modules if necessary
2	Setup infrastructure and create/update inventory.ini

The setup will take longer than just the terraform apply, since it will install some. As a rule of thumb wait ca. 5 minutes after the apply to do other work.

k3s

Run this setup in the k3s/ directory.

The k3s-setup requires a inventory.ini which is automatically created by the infra. So, make sure to apply the infra at least once, before running these playbooks.

ansible-galaxy install -r requirements.yml (1)
ansible-playbook site.yml (2)

1	Install required ansible collections to create a k3s-cluster (can be omitted in subsequent runs)
2	Install k3s and download kube-config to .kube/config

The second step will override any existing kube config, this might destroy any existing settings!

To apply the playbook you may need to install additional packages: helm helm-diff python3-kubernetes (Debian/Ubuntu)

Enlarge / Reduce size of cluster

Increase

1. Simply adjust the number of agents/servers in your infra/config.auto.tfvars.

2. Then run the ansible-playbook of k3s again

Decrease

If you want shrink the cluster DO NOT reduce the agent-amount directly! Instead proceed as the following:

1. Open k9s and go to :nodes

2. Select the highest agent and press r to drain it

3. Afterward that succeeded delete it with Ctrl-d

4. Finally reduce the amount of agents in terraform and apply the change

Responsibilities

infra/

• Creation of network for the kubernetes-cluster

  • A public subnet exposed to the internet for the kubernetes-servers

  • A private subnet for the kubernetes-agents

• Routing between the networks

• Firewall rules to block everything from the servers except of:

  • ping (protocol: icmp)

  • kubernetes api (Usually port 6443)

  • ssh (I prefer to use a non-standard port (usually port 1022)

  • public services, e.g. http and https (port 80 and 443) but also git-ssh (port 22)

• Creating the machines for kubernetes-servers in the public subnet

• Creating the machines for kubernetes-agents in the private subnet

• Creating DNS-records in Hetzer Cloud

k3s/

• Setup SSH-connections

• Setting up routing on all servers

• Installing k3s

• Keep the software up-to-date

• Add foundational services to the cluster