Move infra-setup and k3s-setup into dedicated directories

This changes makes it easier to differentiate and understand the
different parts of the kubernetes setup. On one hand we have the bare
infrastructure (servers, network, etc), on the other hand we have the
software (k3s in this case).

In the future we'll have a few more parts, like the minimal
configuration of the kubernetes cluster, e.g. with a cert-manager. This
is easier to manage with helm or terraform than with ansible. Therefore
it makes even more sense to split the responsibilities into dedicated
directories.
This commit is contained in:
2025-09-15 12:45:50 +02:00
parent dfcdc9797a
commit d96523a071
24 changed files with 89 additions and 59 deletions

View File

@@ -11,36 +11,36 @@ The setup is split into 2 dedicated parts:
. Ensure `terraform` is installed . Ensure `terraform` is installed
. Ensure `ansible` is installed . Ensure `ansible` is installed
. Create `config.auto.tfvars` with all the needed configuration-secrets (Stored in password-manager)
== Setup == Setup
In most cases it should be save to follow this guide: The project is split into different directories, each responsible for another task.
For the initial setup it is mandatory to follow the instructions of each directory in the following order.
Subsequent changes may only require a subset of these instructions.
Though, it should be safe to run each step multiple times, they should all be indempotent.
[source,bash] === infra
----
ansible-galaxy install -r requirements.yml # <1>
terraform apply # <2>
ansible-playbook k3s.orchestration.site -i inventory.ini # <3>
ansible-playbook download-kube-config.yml -i inventory.ini # <4>
ansible-playbook k3s.orchestration.upgrade -i inventory.ini # <5>
----
<1> Install required ansible collections to create a k3s-cluster (can be omitted in subsequent runs) Run this setup in the `infra/` directory.
<2> Setup infrastructure and create/update inventory.ini (This might take some time, even after it's "ready")
<3> Install k3s
<4> Download the kube-config to .kube/config
<5> Update k3s when necessary
[IMPORTANT] include::infra/README.adoc[tag=setup]
Step 4 will override any existing kube config this might destroy any existing settings!
=== k3s
Run this setup in the `k3s/` directory.
[NOTE]
The k3s-setup requires a `inventory.ini` which is automatically created by the infra.
So, make sure to apply the infra at least once, before running these playbooks.
include::k3s/README.adoc[tag=setup]
== Enlarge / Reduce size of cluster == Enlarge / Reduce size of cluster
Increase:: Increase::
-- --
. Simply adjust the number of agents/servers in your `config.auto.tfvars`. . Simply adjust the number of agents/servers in your `infra/config.auto.tfvars`.
. Run steps 2 & 3 of the setup again . Run steps 3 & 4 of the setup again
-- --
Decrease:: Decrease::
-- --
@@ -55,8 +55,7 @@ Instead proceed as the following:
== Responsibilities == Responsibilities
The terraform scripts are responsible for: `infra/`::
* Creation of network for the kubernetes-cluster * Creation of network for the kubernetes-cluster
** A public subnet exposed to the internet for the kubernetes-servers ** A public subnet exposed to the internet for the kubernetes-servers
** A private subnet for the kubernetes-agents ** A private subnet for the kubernetes-agents
@@ -72,8 +71,7 @@ The terraform scripts are responsible for:
* Setup SSH-connections * Setup SSH-connections
* Creating DNS-records in Hetzer Cloud * Creating DNS-records in Hetzer Cloud
The ansible scripts are responsible for: `k3s/`::
* Installing k3s * Installing k3s
* Keep the software up-to-date * Keep the software up-to-date

21
infra/README.adoc Normal file
View File

@@ -0,0 +1,21 @@
= infra
:icons: font
This project is responsible for providing the required infra to run a kubernetes-cluster.
== Setup
// tag::setup[]
[WARNING]
Make sure `config.auto.tfvars` with all the needed configuration-secrets is present otherwise the module cannot be applied!
The file is savely stored in the password-manager.
[source,bash]
----
terraform init # <1>
terraform apply # <2>
----
<1> Init the terraform modules if necessary
<2> Setup infrastructure and create/update inventory.ini (This might take some time, even after it's "ready")
// end::setup[]

13
infra/inventory.ini.tftpl Normal file
View File

@@ -0,0 +1,13 @@
[server]
%{for ip in server_ips~}
${ip}
%{endfor~}
[agent]
%{for ip in agent_ips~}
${ip}
%{endfor~}
[k3s_cluster:children]
server
agent

View File

@@ -56,11 +56,10 @@ module "k8s" {
} }
resource "local_file" "ansible_inventory" { resource "local_file" "ansible_inventory" {
filename = "${path.module}/inventory.ini" filename = "${path.module}/../k3s/inventory.ini"
content = templatefile("./inventory.ini.tftpl", { content = templatefile("./inventory.ini.tftpl", {
server_ips = module.k8s.server_ips_v4, server_ips = module.k8s.server_ips_v4,
agent_ips = module.k8s.agent_ips_v4, agent_ips = module.k8s.agent_ips_v4,
k3s_version = var.k3s_version,
}) })
} }

View File

@@ -72,8 +72,3 @@ variable "k8s_agent_type" {
type = string type = string
default = "cax11" default = "cax11"
} }
variable "k3s_version" {
type = string
description = "The k3s version to use."
}

View File

@@ -1,29 +0,0 @@
[server]
%{for ip in server_ips~}
${ip}
%{endfor~}
[server:vars]
ansible_user=root
ansible_ssh_common_args='-o StrictHostKeyChecking=accept-new'
%{if length(server_ips) > 0~}
api_endpoint=${server_ips[0]}
%{endif~}
k3s_version=${k3s_version}
[agent]
%{for ip in agent_ips~}
${ip}
%{endfor~}
[agent:vars]
ansible_user=root
%{if length(server_ips) > 0~}
ansible_ssh_common_args='-o StrictHostKeyChecking=accept-new -o ProxyCommand="ssh -p 22 -W %h:%p -q root@${server_ips[0]}"'
api_endpoint=${server_ips[0]}
%{endif~}
k3s_version=${k3s_version}
[k3s_cluster:children]
server
agent

26
k3s/README.adoc Normal file
View File

@@ -0,0 +1,26 @@
= k3s
:icons: font
This project is responsible for setting up a k3s installation.
== Setup
The setup requires a `inventory.ini` this should be provided by a previous step.
// tag::setup[]
[source,bash]
----
ansible-galaxy install -r requirements.yml # <1>
ansible-playbook k3s.orchestration.site -i inventory.ini # <2>
ansible-playbook download-kube-config.yml -i inventory.ini # <3>
ansible-playbook k3s.orchestration.upgrade -i inventory.ini # <4>
----
<1> Install required ansible collections to create a k3s-cluster (can be omitted in subsequent runs)
<2> Install k3s
<3> Download the kube-config to .kube/config
<4> Update k3s when necessary
[IMPORTANT]
Step 3 will override any existing kube config this might destroy any existing settings!
// end::setup[]

View File

@@ -0,0 +1,2 @@
ansible_ssh_common_args: '-o StrictHostKeyChecking=accept-new -o ProxyCommand="ssh -p 22 -W %h:%p -q root@188.245.199.197"'
k3s_version: v1.31.6+k3s1

View File

@@ -0,0 +1,3 @@
ansible_user: root
api_endpoint: "{{ hostvars[groups['server'][0]]['ansible_host'] | default(groups['server'][0]) }}"

View File

@@ -0,0 +1,2 @@
ansible_ssh_common_args: '-o StrictHostKeyChecking=accept-new'
k3s_version: v1.31.6+k3s1