nemoinho/base-infra
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
fef383fed4e1ffac0f8f1e787d8e7af9f36bc28a
base-infra/README.adoc
Felix Nehrke fef383fed4 Move setup of foundational service from k8s to k3s 
It simply doesn't make sense to split the installation of the
kubernetes-cluster from the provisioning of foundational services.
Therefore I drop the idea to organise these services in another
terraform-setup and instead ensure their presence with ansible, as it's
already responsible for setting up the cluster and keep it up-to.date.
2025-11-28 00:23:36 +01:00

2.6 KiB
Raw Blame History

Hetzer Infra

This repository is meant to setup my kubernetes-cluster on Hetzner Cloud. The setup is split into 2 dedicated parts:

  1. Create necessary machines in Hetzner Cloud via terraform.

  2. Install/Upgrade kubernetes-cluster and other software via ansible.

TL;DR

cd infra
terraform init
terraform apply
sleep 300 # Wait 5 minutes since the machines start _slow_ sometimes
cd ../k3s
ansible-galaxy install -r requirements.yml
ansible-playbook site.yml

Preparation

  1. Ensure terraform is installed

  2. Ensure ansible is installed

Setup

The project is split into different directories, each responsible for another task. For the initial setup it is mandatory to follow the instructions of each directory in the following order. Subsequent changes may only require a subset of these instructions. Though, it should be safe to run each step multiple times, they should all be indempotent.

infra

Run this setup in the infra/ directory.

Make sure config.auto.tfvars with all the needed configuration-secrets is present otherwise the module cannot be applied! The file is savely stored in the password-manager. 
terraform init (1)
terraform apply (2)
1 Init the terraform modules if necessary
2 Setup infrastructure and create/update inventory.ini
The setup will take longer than just the terraform apply, since it will install some. As a rule of thumb wait ca. 5 minutes after the apply to do other work.

k3s

Run this setup in the k3s/ directory.

The k3s-setup requires a inventory.ini which is automatically created by the infra. So, make sure to apply the infra at least once, before running these playbooks. 
ansible-galaxy install -r requirements.yml (1)
ansible-playbook site.yml (2)
1 Install required ansible collections to create a k3s-cluster (can be omitted in subsequent runs)
2 Install k3s and download kube-config to .kube/config
The second step will override any existing kube config, this might destroy any existing settings!

To apply the playbook you may need to install additional packages:

Enlarge / Reduce size of cluster

Increase

  1. Simply adjust the number of agents/servers in your infra/config.auto.tfvars.

  2. Then run the ansible-playbook of k3s again

Decrease

If you want shrink the cluster DO NOT reduce the agent-amount directly! Instead proceed as the following:

  1. Open k9s and go to :nodes

  2. Select the highest agent and press r to drain it

  3. Afterward that succeeded delete it with Ctrl-d

  4. Finally reduce the amount of agents in terraform and apply the change

Responsibilities

infra/

  • Creation of network for the kubernetes-cluster

    • A public subnet exposed to the internet for the kubernetes-servers

    • A private subnet for the kubernetes-agents

  • Routing between the networks

  • Firewall rules to block everything from the servers except of:

    • ping (protocol: icmp)

    • kubernetes api (Usually port 6443)

    • ssh (I prefer to use a non-standard port (usually port 1022)

    • public services, e.g. http and https (port 80 and 443) but also git-ssh (port 22)

  • Creating the machines for kubernetes-servers in the public subnet

  • Creating the machines for kubernetes-agents in the private subnet

  • Creating DNS-records in Hetzer Cloud

k3s/

  • Setup SSH-connections

  • Setting up routing on all servers

  • Installing k3s

  • Keep the software up-to-date

  • Add foundational services to the cluster