feat(provision): automate the Fedora lab
This commit is contained in:
parent
0dd6f003f3
commit
fd7608227d
15 changed files with 644 additions and 0 deletions
97
scripts/provision/README.md
Normal file
97
scripts/provision/README.md
Normal file
|
|
@ -0,0 +1,97 @@
|
|||
# Fedora host provisioning
|
||||
|
||||
This directory configures already-installed Fedora 44 hosts. Infrastructure can
|
||||
be created manually or by the hand-maintained Terraform configuration; both
|
||||
paths produce the same input: two reachable hosts in `inventory.yml`.
|
||||
|
||||
Node 1 runs the k3s server. Node 2 runs a k3s agent. The playbook is idempotent
|
||||
and may be rerun to converge package, firewall, k3s, and service state.
|
||||
|
||||
## Automated path
|
||||
|
||||
Copy `inventory.example.yml` to `inventory.yml`, replace the addresses and node
|
||||
IP values, set `k3s_node_cidr` to the network containing only cluster nodes,
|
||||
set `k3s_operator_cidrs` to the authorized administration networks, and set the
|
||||
SSH user and connection options required by those hosts.
|
||||
The managed machines may be physical servers, VMs from any provider, or
|
||||
manually installed systems; the playbook does not depend on libvirt or the
|
||||
local lab subnet. Then provide the existing cluster token only in the process
|
||||
environment:
|
||||
|
||||
```bash
|
||||
K3S_TOKEN="$(openssl rand -hex 32)" ./bootstrap.sh inventory.yml
|
||||
```
|
||||
|
||||
The bootstrap checks for `ansible-playbook` and installs the declared Ansible
|
||||
collections. It does not install system packages on the operator machine.
|
||||
|
||||
Do not save the token in the inventory or repository. Preserve it in the
|
||||
operator's secret manager so a replacement agent can join the same cluster.
|
||||
|
||||
## Manual path
|
||||
|
||||
Use `manual-checklist.md` when configuration must be performed interactively.
|
||||
It describes the same end state as the playbook, so a manually prepared host
|
||||
can later be managed by Ansible without rebuilding it.
|
||||
|
||||
## Terraform path
|
||||
|
||||
Terraform is responsible only for creating machines, networks, and addresses.
|
||||
After `terraform apply`, put its resulting addresses into `inventory.yml` and
|
||||
run this playbook. Keeping configuration out of provisioner hooks makes the
|
||||
same Ansible workflow usable for physical hardware, VMs, and manually created
|
||||
hosts.
|
||||
|
||||
## One-command local QEMU lab
|
||||
|
||||
`lab.sh` is only a disposable integration harness for this workstation. Its
|
||||
`virbr0` interface, fixed test addresses, UFW forwarding rules, cloud image,
|
||||
and NetworkManager profiles are deliberately kept out of the reusable Ansible
|
||||
roles. Do not run it on the two production machines; put their real addresses
|
||||
in an inventory and run `bootstrap.sh` instead.
|
||||
|
||||
On an x86_64 Fedora or Arch-family workstation with hardware virtualization enabled:
|
||||
|
||||
```bash
|
||||
./lab.sh
|
||||
```
|
||||
|
||||
To reuse an existing Fedora 44 Cloud Base Generic QCOW2 image instead of
|
||||
downloading another copy:
|
||||
|
||||
```bash
|
||||
FEDORA_IMAGE=/path/to/Fedora-Cloud-Base-Generic-44.x86_64.qcow2 ./lab.sh up
|
||||
```
|
||||
|
||||
The lab intentionally supports x86_64 only. Both target Fedora hosts and the
|
||||
workstation used for the final project run x86_64, so maintaining a separate
|
||||
aarch64 image, firmware, and verification path would add an untested platform
|
||||
without helping the deployment demonstration.
|
||||
|
||||
The Fedora Server Guest Generic image is not suitable for this workflow because
|
||||
it starts the interactive initial-setup program instead of accepting cloud-init
|
||||
configuration.
|
||||
|
||||
The command installs missing host packages, enables libvirt, uses the selected
|
||||
local image or downloads the Fedora 44 cloud image, creates two reusable VMs,
|
||||
waits for ping and SSH, runs the Ansible configuration, and verifies
|
||||
applications, services, directories, ports, and Kubernetes node readiness. Its
|
||||
VM metadata and SSH key live under
|
||||
`${XDG_STATE_HOME:-$HOME/.local/state}/nereus-lab`, outside the repository.
|
||||
|
||||
Subsequent operations are `./lab.sh check`, `./lab.sh stop`, and the explicitly
|
||||
destructive `./lab.sh destroy`.
|
||||
|
||||
Deleting a VM also deletes any k3s local-path volumes stored on that machine.
|
||||
The lab recreates the host and its Kubernetes identity, but stateful demo data
|
||||
on the deleted disk must be recreated separately. This is acceptable for the
|
||||
disposable harness and is not a backup strategy.
|
||||
|
||||
### Tested lab capacity
|
||||
|
||||
The complete two-node stack passed provisioning, reboot, node-disconnect
|
||||
recovery, workload readiness, and public API checks on 2026-08-24 with 2 vCPUs,
|
||||
3 GiB RAM, and a 30 GiB virtual disk per node. These are the lowest settings
|
||||
tested for this project, not a production sizing recommendation. The thin
|
||||
QCOW2 files used approximately 2.0 GiB for node 1 and 3.3 GiB for node 2 during
|
||||
that verification.
|
||||
8
scripts/provision/ansible.cfg
Normal file
8
scripts/provision/ansible.cfg
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
[defaults]
|
||||
inventory = inventory.yml
|
||||
interpreter_python = auto_silent
|
||||
retry_files_enabled = False
|
||||
stdout_callback = default
|
||||
|
||||
[ssh_connection]
|
||||
pipelining = True
|
||||
25
scripts/provision/bootstrap.sh
Executable file
25
scripts/provision/bootstrap.sh
Executable file
|
|
@ -0,0 +1,25 @@
|
|||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
script_dir=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)
|
||||
inventory=${1:-"$script_dir/inventory.yml"}
|
||||
|
||||
if [[ ! -f "$inventory" ]]; then
|
||||
printf 'Inventory not found: %s\nCopy %s/inventory.example.yml and edit it first.\n' "$inventory" "$script_dir" >&2
|
||||
exit 1
|
||||
fi
|
||||
inventory=$(realpath -- "$inventory")
|
||||
|
||||
if [[ -z "${K3S_TOKEN:-}" ]]; then
|
||||
printf 'K3S_TOKEN must be supplied through the environment.\n' >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if ! command -v ansible-playbook >/dev/null 2>&1; then
|
||||
printf 'ansible-playbook is required. Install ansible-core on this operator machine.\n' >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cd "$script_dir"
|
||||
ansible-galaxy collection install --requirements-file requirements.yml
|
||||
ansible-playbook --inventory "$inventory" site.yml
|
||||
20
scripts/provision/inventory.example.yml
Normal file
20
scripts/provision/inventory.example.yml
Normal file
|
|
@ -0,0 +1,20 @@
|
|||
all:
|
||||
vars:
|
||||
ansible_user: fedora
|
||||
k3s_version: v1.33.4+k3s1
|
||||
k3s_cluster_cidr: 10.42.0.0/16
|
||||
k3s_service_cidr: 10.43.0.0/16
|
||||
k3s_node_cidr: 192.0.2.0/24
|
||||
k3s_operator_cidrs:
|
||||
- 198.51.100.0/24
|
||||
children:
|
||||
k3s_server:
|
||||
hosts:
|
||||
node1:
|
||||
ansible_host: 192.0.2.10
|
||||
k3s_node_ip: 192.0.2.10
|
||||
k3s_agent:
|
||||
hosts:
|
||||
node2:
|
||||
ansible_host: 192.0.2.11
|
||||
k3s_node_ip: 192.0.2.11
|
||||
232
scripts/provision/lab.sh
Executable file
232
scripts/provision/lab.sh
Executable file
|
|
@ -0,0 +1,232 @@
|
|||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
action=${1:-up}
|
||||
script_dir=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)
|
||||
state_root=${XDG_STATE_HOME:-"$HOME/.local/state"}/nereus-lab
|
||||
image_root=/var/lib/libvirt/images/nereus-lab
|
||||
inventory="$state_root/inventory.yml"
|
||||
key="$state_root/id_ed25519"
|
||||
known_hosts="$state_root/known_hosts"
|
||||
token_file="$state_root/k3s-token"
|
||||
fedora_release=44
|
||||
nodes=(nereus-node1 nereus-node2)
|
||||
created_nodes=()
|
||||
|
||||
log() { printf '[nereus-lab] %s\n' "$*"; }
|
||||
fail() { printf '[nereus-lab] ERROR: %s\n' "$*" >&2; exit 1; }
|
||||
|
||||
install_dependencies() {
|
||||
local missing=()
|
||||
for tool in qemu-img virsh virt-install virt-customize cloud-localds ansible-playbook curl ssh-keygen openssl ping; do
|
||||
command -v "$tool" >/dev/null 2>&1 || missing+=("$tool")
|
||||
done
|
||||
((${#missing[@]} == 0)) && return
|
||||
[[ -r /etc/os-release ]] || fail "missing tools: ${missing[*]}"
|
||||
# shellcheck disable=SC1091
|
||||
. /etc/os-release
|
||||
log "installing virtualization, cloud-init, and Ansible tooling"
|
||||
case ${ID:-} in
|
||||
fedora)
|
||||
sudo dnf install -y @virtualization cloud-utils guestfs-tools ansible-core openssl iputils
|
||||
;;
|
||||
arch|cachyos)
|
||||
sudo pacman -S --needed --noconfirm qemu-full libvirt virt-install cloud-image-utils guestfs-tools ansible-core openssl iputils dnsmasq
|
||||
;;
|
||||
*)
|
||||
fail "automatic dependency installation does not support ${ID:-this host}; missing: ${missing[*]}"
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
ensure_host() {
|
||||
local uplink
|
||||
install_dependencies
|
||||
[[ $(uname -m) == x86_64 ]] || fail "the bundled Fedora image workflow currently supports x86_64 hosts"
|
||||
[[ -e /dev/kvm ]] || fail "/dev/kvm is unavailable; enable CPU virtualization in firmware"
|
||||
sudo systemctl enable --now libvirtd
|
||||
if ! sudo virsh net-info default >/dev/null 2>&1; then
|
||||
fail "libvirt's default network is missing"
|
||||
fi
|
||||
sudo virsh net-start default >/dev/null 2>&1 || true
|
||||
sudo virsh net-autostart default >/dev/null
|
||||
if command -v ufw >/dev/null 2>&1 && sudo ufw status | grep -q '^Status: active'; then
|
||||
uplink=$(ip route show default | awk '/default/ {print $5; exit}')
|
||||
[[ -n "$uplink" ]] || fail "could not determine the host's default network interface"
|
||||
sudo ufw allow in on virbr0 to any port 53 proto udp >/dev/null
|
||||
sudo ufw allow in on virbr0 to any port 53 proto tcp >/dev/null
|
||||
sudo ufw allow in on virbr0 to any port 67 proto udp >/dev/null
|
||||
sudo ufw route allow in on virbr0 out on virbr0 from 192.168.122.0/24 to 192.168.122.0/24 >/dev/null
|
||||
sudo ufw route allow in on virbr0 out on "$uplink" from 192.168.122.0/24 >/dev/null
|
||||
fi
|
||||
mkdir -p "$state_root"
|
||||
chmod 0700 "$state_root"
|
||||
if [[ ! -f "$key" ]]; then
|
||||
ssh-keygen -q -t ed25519 -N '' -f "$key"
|
||||
fi
|
||||
touch "$known_hosts"
|
||||
chmod 0600 "$key" "$known_hosts"
|
||||
if [[ ! -f "$token_file" ]]; then
|
||||
umask 077
|
||||
openssl rand -hex 32 >"$token_file"
|
||||
fi
|
||||
chmod 0600 "$token_file"
|
||||
sudo install -d -m 0755 "$image_root"
|
||||
}
|
||||
|
||||
image_url() {
|
||||
local arch=x86_64 listing filename
|
||||
listing="https://download.fedoraproject.org/pub/fedora/linux/releases/${fedora_release}/Cloud/${arch}/images/"
|
||||
filename=$(curl -fsSL "$listing" | sed -nE "s/.*href=\"(Fedora-Cloud-Base-Generic-${fedora_release}-[0-9.]+\.${arch}\.qcow2)\".*/\1/p" | sort -V | tail -n1)
|
||||
[[ -n "$filename" ]] || fail "could not discover the Fedora ${fedora_release} cloud image"
|
||||
printf '%s%s\n' "$listing" "$filename"
|
||||
}
|
||||
|
||||
ensure_base_image() {
|
||||
local cached="$state_root/fedora-${fedora_release}-cloud-base.qcow2" source=${FEDORA_IMAGE:-}
|
||||
if [[ ! -s "$cached" ]]; then
|
||||
if [[ -n "$source" ]]; then
|
||||
[[ -f "$source" ]] || fail "FEDORA_IMAGE does not exist: $source"
|
||||
[[ $(basename "$source") == Fedora-Cloud-Base-Generic-* ]] || fail "FEDORA_IMAGE must be a Fedora Cloud Base Generic QCOW2 image"
|
||||
qemu-img check -q "$source" || fail "FEDORA_IMAGE is not a valid QCOW2 image"
|
||||
log "using local Fedora ${fedora_release} image: $source"
|
||||
install -m 0644 "$source" "$cached"
|
||||
else
|
||||
log "downloading Fedora ${fedora_release} cloud image"
|
||||
curl -fL --retry 3 --continue-at - -o "$cached.partial" "$(image_url)"
|
||||
if ! qemu-img check -q "$cached.partial"; then
|
||||
rm -f "$cached.partial"
|
||||
fail "downloaded Fedora image is not a valid QCOW2 image"
|
||||
fi
|
||||
mv "$cached.partial" "$cached"
|
||||
fi
|
||||
fi
|
||||
sudo install -m 0644 "$cached" "$image_root/fedora-${fedora_release}-cloud-base.qcow2"
|
||||
}
|
||||
|
||||
node_mac() {
|
||||
case $1 in
|
||||
nereus-node1) printf '52:54:00:6e:01:01\n' ;;
|
||||
nereus-node2) printf '52:54:00:6e:01:02\n' ;;
|
||||
*) fail "no MAC address assigned for $1" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
create_seed() {
|
||||
local node=$1 seed="$state_root/${node}-seed.iso" user_data="$state_root/${node}-user-data"
|
||||
{
|
||||
printf '#cloud-config\nhostname: %s\nmanage_etc_hosts: true\nusers:\n' "$node"
|
||||
printf ' - name: fedora\n groups: [wheel]\n sudo: ALL=(ALL) NOPASSWD:ALL\n shell: /bin/bash\n ssh_authorized_keys:\n'
|
||||
printf ' - %s\n' "$(<"$key.pub")"
|
||||
printf 'ssh_pwauth: false\npackages: [qemu-guest-agent]\nruncmd:\n - [systemctl, enable, --now, qemu-guest-agent]\n'
|
||||
} >"$user_data"
|
||||
chmod 0600 "$user_data"
|
||||
cloud-localds --dsmode local --hostname "$node" "$seed" "$user_data"
|
||||
sudo install -m 0644 "$seed" "$image_root/${node}-seed.iso"
|
||||
}
|
||||
|
||||
install_network_service() {
|
||||
local node=$1 disk=$2
|
||||
sudo virt-customize -q -a "$disk" \
|
||||
--copy-in "$script_dir/nereus-network.service:/etc/systemd/system" \
|
||||
--copy-in "$script_dir/${node}.nmconnection:/etc/NetworkManager/system-connections" \
|
||||
--chmod "0600:/etc/NetworkManager/system-connections/${node}.nmconnection" \
|
||||
--run-command "chown root:root /etc/NetworkManager/system-connections/${node}.nmconnection" \
|
||||
--run-command 'rm -f /etc/NetworkManager/system-connections/cloud-init-ens2.nmconnection' \
|
||||
--run-command 'systemctl enable nereus-network.service'
|
||||
}
|
||||
|
||||
create_vm() {
|
||||
local node=$1 disk="$image_root/${node}.qcow2" mac
|
||||
mac=$(node_mac "$node")
|
||||
if sudo virsh dominfo "$node" >/dev/null 2>&1; then
|
||||
if [[ $(sudo virsh domstate "$node") == "shut off" ]]; then
|
||||
install_network_service "$node" "$disk"
|
||||
fi
|
||||
sudo virsh start "$node" >/dev/null 2>&1 || true
|
||||
return
|
||||
fi
|
||||
log "creating $node"
|
||||
created_nodes+=("$node")
|
||||
ssh-keygen -q -f "$known_hosts" -R "$(node_ip "$node")" >/dev/null 2>&1 || true
|
||||
create_seed "$node"
|
||||
sudo qemu-img create -q -f qcow2 -F qcow2 -b "$image_root/fedora-${fedora_release}-cloud-base.qcow2" "$disk" 30G
|
||||
install_network_service "$node" "$disk"
|
||||
sudo virt-install --name "$node" --memory 3072 --vcpus 2 --import \
|
||||
--disk "path=$disk,format=qcow2,bus=virtio" \
|
||||
--disk "path=$image_root/${node}-seed.iso,format=raw,bus=virtio,readonly=on" \
|
||||
--network "network=default,model=virtio,mac=$mac" --graphics none --noautoconsole \
|
||||
--boot uefi --osinfo detect=on,require=off
|
||||
}
|
||||
|
||||
node_ip() {
|
||||
case $1 in
|
||||
nereus-node1) printf '192.168.122.10\n' ;;
|
||||
nereus-node2) printf '192.168.122.11\n' ;;
|
||||
*) fail "no address assigned for $1" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
wait_ssh() {
|
||||
local ip=$1
|
||||
for _ in $(seq 1 90); do
|
||||
if ssh -i "$key" -o BatchMode=yes -o ConnectTimeout=2 -o StrictHostKeyChecking=accept-new -o UserKnownHostsFile="$known_hosts" "fedora@$ip" true 2>/dev/null; then return; fi
|
||||
sleep 2
|
||||
done
|
||||
fail "SSH did not become ready at $ip"
|
||||
}
|
||||
|
||||
wait_ping() {
|
||||
local node=$1 ip=$2
|
||||
for _ in $(seq 1 60); do
|
||||
ping -c 1 -W 1 "$ip" >/dev/null 2>&1 && return
|
||||
sleep 2
|
||||
done
|
||||
fail "$node does not answer ping at $ip"
|
||||
}
|
||||
|
||||
write_inventory() {
|
||||
local ip1=$1 ip2=$2
|
||||
umask 077
|
||||
printf 'all:\n vars:\n ansible_user: fedora\n ansible_ssh_private_key_file: %s\n ansible_ssh_common_args: "-o StrictHostKeyChecking=accept-new -o UserKnownHostsFile=%s"\n k3s_version: v1.33.4+k3s1\n k3s_cluster_cidr: 10.42.0.0/16\n k3s_service_cidr: 10.43.0.0/16\n k3s_node_cidr: 192.168.122.0/24\n k3s_operator_cidrs:\n - 192.168.122.1/32\n children:\n k3s_server:\n hosts:\n node1:\n ansible_host: %s\n k3s_node_ip: %s\n k3s_agent:\n hosts:\n node2:\n ansible_host: %s\n k3s_node_ip: %s\n' "$key" "$known_hosts" "$ip1" "$ip1" "$ip2" "$ip2" >"$inventory"
|
||||
}
|
||||
|
||||
check_lab() {
|
||||
[[ -f "$inventory" ]] || fail "run '$0 up' first"
|
||||
log "checking reachability, services, ports, packages, and directories"
|
||||
ansible all -i "$inventory" -m ansible.builtin.ping
|
||||
ansible all -i "$inventory" -b -m ansible.builtin.shell -a 'for i in $(seq 1 60); do test -d /var/lib/rancher/k3s && command -v k3s >/dev/null && systemctl is-active --quiet firewalld && ss -lnt | grep -q ":10250 " && exit 0; sleep 2; done; exit 1'
|
||||
ansible k3s_server -i "$inventory" -b -m ansible.builtin.shell -a 'for i in $(seq 1 60); do systemctl is-active --quiet k3s && ss -lnt | grep -q ":6443 " && exit 0; sleep 2; done; exit 1'
|
||||
ansible k3s_agent -i "$inventory" -b -m ansible.builtin.shell -a 'for i in $(seq 1 60); do systemctl is-active --quiet k3s-agent && exit 0; sleep 2; done; exit 1'
|
||||
ansible k3s_server -i "$inventory" -b -m ansible.builtin.command -a 'k3s kubectl get nodes -o wide'
|
||||
ansible k3s_server -i "$inventory" -b -m ansible.builtin.command -a 'k3s kubectl wait --for=condition=Ready nodes --all --timeout=60s'
|
||||
log "all checks passed"
|
||||
}
|
||||
|
||||
case "$action" in
|
||||
up)
|
||||
ensure_host
|
||||
ensure_base_image
|
||||
for node in "${nodes[@]}"; do create_vm "$node"; done
|
||||
ip1=$(node_ip "${nodes[0]}"); ip2=$(node_ip "${nodes[1]}")
|
||||
log "node1=$ip1 node2=$ip2"
|
||||
wait_ping node1 "$ip1"
|
||||
wait_ping node2 "$ip2"
|
||||
wait_ssh "$ip1"; wait_ssh "$ip2"
|
||||
for node in "${created_nodes[@]}"; do
|
||||
if [[ "$node" != "${nodes[0]}" ]]; then
|
||||
ssh -i "$key" -o BatchMode=yes -o StrictHostKeyChecking=accept-new -o UserKnownHostsFile="$known_hosts" \
|
||||
"fedora@$ip1" "sudo -n k3s kubectl delete node '$node' --ignore-not-found=true"
|
||||
fi
|
||||
done
|
||||
write_inventory "$ip1" "$ip2"
|
||||
K3S_TOKEN=$(<"$token_file") "$script_dir/bootstrap.sh" "$inventory"
|
||||
check_lab
|
||||
;;
|
||||
check) check_lab ;;
|
||||
stop) for node in "${nodes[@]}"; do sudo virsh shutdown "$node" >/dev/null 2>&1 || true; done ;;
|
||||
destroy)
|
||||
for node in "${nodes[@]}"; do sudo virsh destroy "$node" >/dev/null 2>&1 || true; sudo virsh undefine "$node" --remove-all-storage; done
|
||||
;;
|
||||
*) fail "usage: $0 [up|check|stop|destroy]" ;;
|
||||
esac
|
||||
24
scripts/provision/manual-checklist.md
Normal file
24
scripts/provision/manual-checklist.md
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
# Manual Fedora 44 checklist
|
||||
|
||||
Use the values in `inventory.yml` as the source of truth. The manual result must
|
||||
match the Ansible result: one server named `node1`, one agent named `node2`, the
|
||||
same pinned k3s version, and both nodes reporting Ready.
|
||||
|
||||
1. Install Fedora Server 44 on both hosts, apply system updates, assign stable
|
||||
addresses, and confirm SSH access with sudo privileges.
|
||||
2. Install `curl`, `firewalld`, and `policycoreutils-python-utils` on both hosts.
|
||||
3. Enable firewalld. Trust the configured pod and service CIDRs, then allow
|
||||
`8472/udp` and `10250/tcp` between the two nodes. Allow `6443/tcp` to node 1
|
||||
from node 2 and authorized operator networks.
|
||||
4. Generate a cluster token in a secret manager. Never place it in shell
|
||||
history, an inventory file, or this repository.
|
||||
5. Download the official installer from `https://get.k3s.io`. On node 1, install
|
||||
the pinned version in server mode with its stable node IP.
|
||||
6. On node 2, run the same pinned installer in agent mode with node 1's
|
||||
`https://ADDRESS:6443` URL and the runtime cluster token.
|
||||
7. Enable and start `k3s.service` on node 1 and `k3s-agent.service` on node 2.
|
||||
8. On node 1, run `k3s kubectl get nodes -o wide` and verify both nodes are
|
||||
Ready before installing platform components.
|
||||
|
||||
Prefer the playbook for the actual command details. This checklist deliberately
|
||||
does not encourage copying a cluster token into an interactive command line.
|
||||
14
scripts/provision/nereus-network.service
Normal file
14
scripts/provision/nereus-network.service
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
[Unit]
|
||||
Description=Activate the Nereus NetworkManager profile
|
||||
After=NetworkManager.service cloud-init-local.service
|
||||
Requires=NetworkManager.service
|
||||
Before=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/usr/bin/nmcli connection reload
|
||||
ExecStart=/usr/bin/nmcli connection up nereus-static
|
||||
RemainAfterExit=yes
|
||||
|
||||
[Install]
|
||||
WantedBy=network-online.target
|
||||
22
scripts/provision/nereus-node1.nmconnection
Normal file
22
scripts/provision/nereus-node1.nmconnection
Normal file
|
|
@ -0,0 +1,22 @@
|
|||
[connection]
|
||||
id=nereus-static
|
||||
uuid=9b589a0e-fca8-4a73-82eb-e42601dbfa01
|
||||
type=ethernet
|
||||
interface-name=ens2
|
||||
autoconnect=true
|
||||
autoconnect-priority=999
|
||||
|
||||
[ethernet]
|
||||
mac-address=52:54:00:6E:01:01
|
||||
|
||||
[ipv4]
|
||||
address1=192.168.122.10/24
|
||||
dns=192.168.122.1;
|
||||
gateway=192.168.122.1
|
||||
method=manual
|
||||
|
||||
[ipv6]
|
||||
addr-gen-mode=default
|
||||
method=disabled
|
||||
|
||||
[proxy]
|
||||
22
scripts/provision/nereus-node2.nmconnection
Normal file
22
scripts/provision/nereus-node2.nmconnection
Normal file
|
|
@ -0,0 +1,22 @@
|
|||
[connection]
|
||||
id=nereus-static
|
||||
uuid=9b589a0e-fca8-4a73-82eb-e42601dbfa02
|
||||
type=ethernet
|
||||
interface-name=ens2
|
||||
autoconnect=true
|
||||
autoconnect-priority=999
|
||||
|
||||
[ethernet]
|
||||
mac-address=52:54:00:6E:01:02
|
||||
|
||||
[ipv4]
|
||||
address1=192.168.122.11/24
|
||||
dns=192.168.122.1;
|
||||
gateway=192.168.122.1
|
||||
method=manual
|
||||
|
||||
[ipv6]
|
||||
addr-gen-mode=default
|
||||
method=disabled
|
||||
|
||||
[proxy]
|
||||
3
scripts/provision/requirements.yml
Normal file
3
scripts/provision/requirements.yml
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
---
|
||||
collections:
|
||||
- name: ansible.posix
|
||||
68
scripts/provision/roles/common/tasks/main.yml
Normal file
68
scripts/provision/roles/common/tasks/main.yml
Normal file
|
|
@ -0,0 +1,68 @@
|
|||
---
|
||||
- name: Require a node address
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- k3s_node_ip is defined
|
||||
- k3s_node_ip | string | length > 0
|
||||
- k3s_node_cidr is defined
|
||||
- k3s_node_cidr | string | length > 0
|
||||
- k3s_operator_cidrs is defined
|
||||
- k3s_operator_cidrs | length > 0
|
||||
|
||||
- name: Install host dependencies
|
||||
ansible.builtin.dnf:
|
||||
name:
|
||||
- curl
|
||||
- firewalld
|
||||
- policycoreutils-python-utils
|
||||
state: present
|
||||
|
||||
- name: Enable firewalld
|
||||
ansible.builtin.systemd_service:
|
||||
name: firewalld
|
||||
enabled: true
|
||||
state: started
|
||||
|
||||
- name: Trust the k3s pod network
|
||||
ansible.posix.firewalld:
|
||||
source: "{{ k3s_cluster_cidr }}"
|
||||
zone: trusted
|
||||
permanent: true
|
||||
immediate: true
|
||||
state: enabled
|
||||
|
||||
- name: Trust the k3s service network
|
||||
ansible.posix.firewalld:
|
||||
source: "{{ k3s_service_cidr }}"
|
||||
zone: trusted
|
||||
permanent: true
|
||||
immediate: true
|
||||
state: enabled
|
||||
|
||||
- name: Remove globally open node-to-node k3s ports
|
||||
ansible.posix.firewalld:
|
||||
port: "{{ item }}"
|
||||
permanent: true
|
||||
immediate: true
|
||||
state: disabled
|
||||
loop:
|
||||
- 8472/udp
|
||||
- 10250/tcp
|
||||
|
||||
- name: Allow node-to-node k3s ports from the node network
|
||||
ansible.posix.firewalld:
|
||||
rich_rule: 'rule family="ipv4" source address="{{ k3s_node_cidr }}" port port="{{ item.port }}" protocol="{{ item.protocol }}" accept'
|
||||
permanent: true
|
||||
immediate: true
|
||||
state: enabled
|
||||
loop:
|
||||
- {port: "8472", protocol: udp}
|
||||
- {port: "10250", protocol: tcp}
|
||||
|
||||
- name: Download the official k3s installer
|
||||
ansible.builtin.get_url:
|
||||
url: https://get.k3s.io
|
||||
dest: /var/tmp/k3s-install.sh
|
||||
mode: "0755"
|
||||
owner: root
|
||||
group: root
|
||||
22
scripts/provision/roles/k3s_agent/tasks/main.yml
Normal file
22
scripts/provision/roles/k3s_agent/tasks/main.yml
Normal file
|
|
@ -0,0 +1,22 @@
|
|||
---
|
||||
- name: Require the runtime cluster token
|
||||
ansible.builtin.assert:
|
||||
that: lookup('ansible.builtin.env', 'K3S_TOKEN') | length >= 32
|
||||
fail_msg: "K3S_TOKEN must contain at least 32 characters."
|
||||
no_log: true
|
||||
|
||||
- name: Install or reconcile the k3s agent
|
||||
ansible.builtin.command: /var/tmp/k3s-install.sh
|
||||
environment:
|
||||
INSTALL_K3S_VERSION: "{{ k3s_version }}"
|
||||
INSTALL_K3S_EXEC: "agent --node-ip {{ k3s_node_ip }}"
|
||||
K3S_URL: "https://{{ hostvars[groups['k3s_server'][0]].k3s_node_ip }}:6443"
|
||||
K3S_TOKEN: "{{ lookup('ansible.builtin.env', 'K3S_TOKEN') }}"
|
||||
changed_when: false
|
||||
no_log: true
|
||||
|
||||
- name: Enable the k3s agent
|
||||
ansible.builtin.systemd_service:
|
||||
name: k3s-agent
|
||||
enabled: true
|
||||
state: started
|
||||
36
scripts/provision/roles/k3s_server/tasks/main.yml
Normal file
36
scripts/provision/roles/k3s_server/tasks/main.yml
Normal file
|
|
@ -0,0 +1,36 @@
|
|||
---
|
||||
- name: Require the runtime cluster token
|
||||
ansible.builtin.assert:
|
||||
that: lookup('ansible.builtin.env', 'K3S_TOKEN') | length >= 32
|
||||
fail_msg: "K3S_TOKEN must contain at least 32 characters."
|
||||
no_log: true
|
||||
|
||||
- name: Remove the globally open k3s API port
|
||||
ansible.posix.firewalld:
|
||||
port: 6443/tcp
|
||||
permanent: true
|
||||
immediate: true
|
||||
state: disabled
|
||||
|
||||
- name: Allow the k3s API from node and operator networks
|
||||
ansible.posix.firewalld:
|
||||
rich_rule: 'rule family="ipv4" source address="{{ item }}" port port="6443" protocol="tcp" accept'
|
||||
permanent: true
|
||||
immediate: true
|
||||
state: enabled
|
||||
loop: "{{ [k3s_node_cidr] + k3s_operator_cidrs }}"
|
||||
|
||||
- name: Install or reconcile the k3s server
|
||||
ansible.builtin.command: /var/tmp/k3s-install.sh
|
||||
environment:
|
||||
INSTALL_K3S_VERSION: "{{ k3s_version }}"
|
||||
INSTALL_K3S_EXEC: "server --node-ip {{ k3s_node_ip }} --write-kubeconfig-mode 0640"
|
||||
K3S_TOKEN: "{{ lookup('ansible.builtin.env', 'K3S_TOKEN') }}"
|
||||
changed_when: false
|
||||
no_log: true
|
||||
|
||||
- name: Enable the k3s server
|
||||
ansible.builtin.systemd_service:
|
||||
name: k3s
|
||||
enabled: true
|
||||
state: started
|
||||
13
scripts/provision/roles/validation/tasks/main.yml
Normal file
13
scripts/provision/roles/validation/tasks/main.yml
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
---
|
||||
- name: Wait for both nodes to report Ready
|
||||
ansible.builtin.command: k3s kubectl wait --for=condition=Ready nodes --all --timeout=180s
|
||||
changed_when: false
|
||||
|
||||
- name: Read the node list
|
||||
ansible.builtin.command: k3s kubectl get nodes -o wide
|
||||
register: node_list
|
||||
changed_when: false
|
||||
|
||||
- name: Show the resulting cluster
|
||||
ansible.builtin.debug:
|
||||
var: node_list.stdout_lines
|
||||
38
scripts/provision/site.yml
Normal file
38
scripts/provision/site.yml
Normal file
|
|
@ -0,0 +1,38 @@
|
|||
---
|
||||
- name: Prepare Fedora k3s nodes
|
||||
hosts: all
|
||||
become: true
|
||||
gather_facts: true
|
||||
pre_tasks:
|
||||
- name: Require Fedora 44
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- ansible_distribution == "Fedora"
|
||||
- ansible_distribution_major_version == "44"
|
||||
fail_msg: "This playbook supports Fedora 44 only."
|
||||
- name: Require one server and at least one agent
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- groups['k3s_server'] | length == 1
|
||||
- groups['k3s_agent'] | length >= 1
|
||||
run_once: true
|
||||
roles:
|
||||
- common
|
||||
|
||||
- name: Configure the k3s server
|
||||
hosts: k3s_server
|
||||
become: true
|
||||
roles:
|
||||
- k3s_server
|
||||
|
||||
- name: Configure k3s agents
|
||||
hosts: k3s_agent
|
||||
become: true
|
||||
roles:
|
||||
- k3s_agent
|
||||
|
||||
- name: Validate the cluster
|
||||
hosts: k3s_server
|
||||
become: true
|
||||
roles:
|
||||
- validation
|
||||
Loading…
Add table
Reference in a new issue