arrow-down coffee engineering consultancy development remote-management support linkedin twitter youtube email phone gitlab github

How to run Kubernetes on Raspberry Pi cluster

Submitted by gavriel on July 11, 2018

For our booth at DevOpsDays Amsterdam in early July we decided we wanted to present a challenge to the booth visitors. Since Kubernetes is all the hype, we settled on kubernetes on a cluster of raspberry pi's. This blog is the outline of what we built and why.

What is Docker?

Docker allows administrators or developers to package these applications into something called containers. container image is a lightweight, stand-alone, executable package of a piece of software that includes everything needed to run it: code, runtime, system tools, system libraries, settings. Available for both Linux and Windows-based apps, the containerized software will always run the same, regardless of the environment. Containers isolate software from its surroundings, for example, differences between development and staging environments and help reduce conflicts between teams running different software on the same infrastructure.

There are many more advantages of using Docker; the details of which can be found in the official Docker Documentation.

docker architecture

Ok I know what is Docker now... Kubernetes?

TL; DR: Kubernetes is a way to manage Docker containers on multiple nodes

Kubernetes is a portable, extensible open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation.

for more info kubernetes Documentation.

kubernetes flower

Why would you do this ?

For me there are 3 main reasons:

  • Because we can.

  • I wanted to experiment with container technology at scale on real hardware. To explore what that’d be like, we built a Raspberry Pi cluster just like we would build a cluster of machines in a production datacenter. This allowed us to understand and simulate how Kubernetes would work when we move it to our data centers.

  • there's something more visceral about learning it this way

Grocery list

We used the following components for the cluster.

(Disclaimer: These are amazon affiliate links. We don't want to make money off them, it's just for convenience)

What to do with the network

let's talk a bit about network

You can connect to your own Wifi and access them that way, but you need to think about how they get their IP address. its all about how portable you need the cluster

I recommend buying a small router

I will release in the future post with Introduction to networking & Networking Basics

Lets put everything together

Building the Raspberry Pi cluster is pretty straightforward. Most of the work is putting the stackable casing together...after that just connect all the network & power cables.

Now that we have assembled everything, let's continue to the software and configuration

SD-cards flashing and OS selection

what OS to use? ... I chose to go with HypriotOS its have all the dependencies for docker and k8s out of the box The following section has to be repeated for every node you want to join your cluster.

Download the latest SD card image from the download page.

I use etcher by Resin.io for flashing the SD-cards.

etcher workflow

Just select the .zip file you just downloaded, and insert your SD-card, select the drive, and click “Flash!”. Easy, right?

Once the flashing process is finished, add the following file to /boot if you need to pre-config wi-fi network

$ vim /Volumes/boot/wpa_supplicant.conf

with the following content:

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
network={
   ssid="wifi_name"
   psk="password"
   key_mgmt=WPA-PSK
}

HypriotOS by default enables ssh.

Now, take out the SD-card and insert it into the Raspberry Pi.

After a minute or so, you should be able to access the Raspberry Pi from the same network.

$ ssh pirate@hypriotos.local

The default password is: hypriot

we need to install some software on the Raspberry Pi.

docker, kubernetes, and some config change

Therefore, create a new file on the Raspberry Pi:

$ nano init.sh
#!/bin/sh
​
# disable swap
sudo dphys-swapfile swapoff && \
sudo dphys-swapfile uninstall && \
sudo update-rc.d dphys-swapfile remove
​
#install kube
​
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - && \
echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list && \
sudo apt-get update -q && \
sudo apt-get install -qy kubeadm

Run the script,

$ sh init.sh

Reboot the machine and SSH back into the Raspberry Pi once the reboot has finished.

$ sudo reboot

Setting the Kubernetes master

run on the master_pi

$ sudo kubeadm init --apiserver-advertise-address <local ip>

you should get something like this after a couple of min

kubeadm join 192.168.1.234:6443 --token bfubpj.en6t5abj9doxrvw7 --discovery-token-ca-cert-hash sha256:d84c1f7ee30947f00c4579f272fbd9643d67bbcd9a80b59d107c90884baa34d7

this is the token we need to add the other 3 nodes to our raspberry cluster, (you can always generate a new one with kubeadm token create --print-join-command)

and for start using the master, you need to enter

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Install Weave network driver

(more info regarding Cluster Networking)

kubectl apply -f \
"https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

Check everything worked:

$ kubectl get pods --namespace=kube-system
​
NAME                           READY STATUS RESTARTS AGE
​
etcd-of-2                      1/1 Running 0 12m
kube-apiserver-of-2            1/1 Running 2 12m
kube-controller-manager-of-2   1/1 Running 1 11m
kube-dns-66ffd5c588-d8292      3/3 Running 0 11m
kube-proxy-xcj5h               1/1 Running 0 11m
kube-scheduler-of-2            1/1 Running 0 11m
weave-net-zz9rz                2/2 Running 0 5m

 

Setting up the worker nodes

we basically only need to run the kubeadm join on each of your worker nodes,

kubeadm join 192.168.1.234:6443 --token bfubpj.en6t5abj9doxrvw7 --discovery-token-ca-cert-hash sha256:d84c1f7ee30947f00c4579f272fbd9643d67bbcd9a80b59d107c90884baa34d7

I just used pssh with host list to run it

#list
#slave
​
192.168.1.212
192.168.1.213
192.168.1.214
pssh -h list -A "kubeadm join 192.168.1.234:6443 --token bfubpj.en6t5abj9doxrvw7 --discovery-token-ca-cert-hash sha256:d84c1f7ee30947f00c4579f272fbd9643d67bbcd9a80b59d107c90884baa34d7"

​
[1] 15:54:55 [SUCCESS] 192.168.1.212
[2] 15:54:56 [SUCCESS] 192.168.1.213
[3] 15:54:55 [SUCCESS] 192.168.1.214

now we have 3 slaves connected to the muster you can run "kubectl get nodes" on the master

$ kubectl get nodes
​
NAME      STATUS AGE       VERSION
k8s-1     Ready 10m       v1.7.4
k8s-2     Ready 10m       v1.7.4
k8s-3     Ready 10m       v1.7.4

 

We're done

That was it. You should now have a fully functioning Kubernetes cluster.