k3s bootstrap on Alpine Linux

D. Heinrich

3 min readFeb 15, 2020

I tried to setup k3s on alpinelinux, this is what I came up with..

Explanation section

Experienced Users can skip ahead to Prepare alpine Linux

What is Alpine Linux?

ABOUT
Alpine Linux is an independent, non-commercial, general purpose Linux distribution designed for power users who appreciate security, simplicity and resource efficiency.
SMALL
Alpine Linux is built around musl libc and busybox. This makes it smaller and more resource efficient than traditional GNU/Linux distributions. A container requires no more than 8 MB and a minimal installation to disk requires around 130 MB of storage. Not only do you get a fully-fledged Linux environment but a large selection of packages from the repository.
Binary packages are thinned out and split, giving you even more control over what you install, which in turn keeps your environment as small and efficient as possible.
SIMPLE
Alpine Linux is a very simple distribution that will try to stay out of your way. It uses its own package manager called apk, the OpenRC init system, script driven set-ups and that’s it! This provides you with a simple, crystal-clear Linux environment without all the noise. You can then add on top of that just the packages you need for your project, so whether it’s building a home PVR, or an iSCSI storage controller, a wafer-thin mail server container, or a rock-solid embedded switch, nothing else will get in the way.
SECURE
Alpine Linux was designed with security in mind. All userland binaries are compiled as Position Independent Executables (PIE) with stack smashing protection. These proactive security features prevent exploitation of entire classes of zero-day and other vulnerabilities.

What is k3s?

Perfect for Edge
K3S is a highly available, certified Kubernetes distribution designed for production workloads in unattended, resource-constrained, remote locations or inside IoT appliances.
Simplified & Secure
K3S is packaged as a single <40MB binary that reduces the dependencies and steps needed to install, run and auto-update a production Kubernetes cluster.
Optimized for ARM
Both ARM64 and ARMv7 are supported with binaries and multiarch images available for both. K3S works great from something as small as a Raspberry Pi to an AWS a1.4xlarge 32GiB server.

Prepare alpine Linux:

Alpine Linux uses openrc (which is a Unix-like systems, a dependency-based init — the first process started during booting of the computer system) instead of systemd.

First we need to add the cgroup mount point:

echo "cgroup /sys/fs/cgroup cgroup defaults 0 0" >> /etc/fstab

Now we edit the cgconfig.conf

cgconfig.conf is the configuration file used by libcgroup to define control groups, their parameters and also mount points. The file consists of mount and group sections. These sections can be in arbitrary order. Any line starting with ‘#’ is considered as comment line and is ignored.

cat > /etc/cgconfig.conf <<EOF
mount {
  cpuacct = /cgroup/cpuacct;
  memory = /cgroup/memory;
  devices = /cgroup/devices;
  freezer = /cgroup/freezer;
  net_cls = /cgroup/net_cls;
  blkio = /cgroup/blkio;
  cpuset = /cgroup/cpuset;
  cpu = /cgroup/cpu;
}
EOF

After preparing all cgroups we’ve to edit /etc/update-extlinux.conf
Append the following cgroup params to the line:

default_kernel_opts="...  cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory"

After those settings where made we can update the bootconfig and reboot

update-extlinux
reboot

Initializing the k3s-server

Install cni-plugins and iptables:

apk add --no-cache cni-plugins --repository=http://dl-cdn.alpinelinux.org/alpine/edge/testing
PATH=/usr/share/cni-plugins/bin:$PATHapk add iptables

Initialize the cluster:

K3S_TOKEN=SECRET k3s server --cluster-init

Join a new server:

K3S_TOKEN=SECRET k3s server --server https://<ip or hostname of server1>:6443

check if its running (needs to be executed on one of the servers):

kubectl get nodes

Troubleshooting k3s:

cni-plugin not yet installed:

ERRO[0816] failed to find host-local: exec: "host-local": executable file not found in $PATH

iptables not yet installed:

iptables not found

Cheers!

Sources: