Felix Nehrke
4beb9e2844
With this change we no longer use user-data scripts on the provided machines. That makes it way easier for me to handle all the configuration, since I only have to run ansible. Furthermore this the burdon to think what may went wrong, since ansible is easier to debug than some arbitrary scripts which run at provisioning-time on the machines. With this change I should also think about restructuring the code a bit as well. Since it's actually easier to provide the initial software-stack for the cluster via ansible than via terraform, at least as far as I can tell right now.
101 lines
2.7 KiB
Plaintext
101 lines
2.7 KiB
Plaintext
= Hetzer Infra
|
|
:icons: font
|
|
|
|
This repository is meant to setup my kubernetes-cluster on Hetzner Cloud.
|
|
The setup is split into 2 dedicated parts:
|
|
|
|
. Create necessary machines in Hetzner Cloud via terraform.
|
|
. Install/Upgrade kubernetes-cluster and other software via ansible.
|
|
|
|
== TL;DR
|
|
|
|
[source,bash]
|
|
----
|
|
cd infra
|
|
terraform init
|
|
terraform apply
|
|
sleep 300 # Wait 5 minutes since the machines start _slow_ sometimes
|
|
cd ../k3s
|
|
ansible-galaxy install -r requirements.yml
|
|
ansible-playbook site.yml
|
|
cd ../k8s
|
|
terraform init
|
|
terraform apply
|
|
----
|
|
|
|
== Preparation
|
|
|
|
. Ensure `terraform` is installed
|
|
. Ensure `ansible` is installed
|
|
|
|
== Setup
|
|
|
|
The project is split into different directories, each responsible for another task.
|
|
For the initial setup it is mandatory to follow the instructions of each directory in the following order.
|
|
Subsequent changes may only require a subset of these instructions.
|
|
Though, it should be safe to run each step multiple times, they should all be indempotent.
|
|
|
|
=== infra
|
|
|
|
Run this setup in the `infra/` directory.
|
|
|
|
include::./infra/README.adoc[tag=setup]
|
|
|
|
=== k3s
|
|
|
|
Run this setup in the `k3s/` directory.
|
|
|
|
[NOTE]
|
|
The k3s-setup requires a `inventory.ini` which is automatically created by the infra.
|
|
So, make sure to apply the infra at least once, before running these playbooks.
|
|
|
|
include::./k3s/README.adoc[tag=setup]
|
|
|
|
=== k8s
|
|
|
|
Run this setup in the `k8s/` directory.
|
|
|
|
include::./k8s/README.adoc[tag=setup]
|
|
|
|
== Enlarge / Reduce size of cluster
|
|
|
|
Increase::
|
|
--
|
|
. Simply adjust the number of agents/servers in your `infra/config.auto.tfvars`.
|
|
. Run steps 3 & 4 of the setup again
|
|
--
|
|
Decrease::
|
|
--
|
|
If you want shrink the cluster **DO NOT** reduce the agent-amount directly!
|
|
Instead proceed as the following:
|
|
|
|
. Open k9s and go to `:nodes`
|
|
. Select the highest agent and press `r` to drain it
|
|
. Afterward that succeeded delete it with `Ctrl-d`
|
|
. Finally reduce the amount of agents in terraform and apply the change
|
|
--
|
|
|
|
== Responsibilities
|
|
|
|
`infra/`::
|
|
* Creation of network for the kubernetes-cluster
|
|
** A public subnet exposed to the internet for the kubernetes-servers
|
|
** A private subnet for the kubernetes-agents
|
|
* Routing between the networks
|
|
* Firewall rules to block everything from the servers except of:
|
|
** ping (protocol: `icmp`)
|
|
** kubernetes api (Usually port `6443`)
|
|
** ssh (I prefer to use a non-standard port since I want to provide a git-server on port `22`)
|
|
** public services, e.g. http and https (port `80` and `443`)
|
|
* Creating the kubernetes-servers in the public subnet
|
|
* Creating the kubernetes-agents in the private subnet
|
|
* Setting up routing on all servers
|
|
* Setup SSH-connections
|
|
* Creating DNS-records in Hetzer Cloud
|
|
|
|
`k3s/`::
|
|
* Installing k3s
|
|
* Keep the software up-to-date
|
|
|
|
|