Deploying Traefik on Kubernetes helps simplify ingress routing and application networking. But when it comes to Kubernetes itself, the word “simple” is seldom used. Creating and configuring production clusters can be tricky and time-consuming.
Thanks to the open source community, however, a number of projects have emerged that make it easy to get up and running with Kubernetes. Today, you can start experimenting with containers on a simple cluster in minutes and in your choice of environment, which can even be the laptop in front of you. Add Traefik and you have a bona fide development workbench, created with minimal hassle.
In this post, I’ll compare some of the available options to help you decide which might be the best way for you to start working with Kubernetes and Traefik quickly. I’ll highlight some of the prominent characteristics of each and give my own impression of who might be that tool’s best audience.
A Wealth of Options
Before I start, I’d like to give an honorable mention to kubeadm, a classic means of creating Kubernetes clusters. It’s a fine tool and it gets the job done, but it’s too cumbersome for simple uses. It’s best viewed as a way to educate yourself on the inner workings of Kubernetes. Instead, here I’ll concentrate on tools that give you more of a turnkey experience.
You can think of the projects I’ll discuss as broadly belonging to one of two categories. First are the true desktop tools, which chiefly target developers. Next come lightweight Kubernetes distributions, which can be used for local experimentation but may also be suitable for production deployments.
When combined with Traefik as a Kubernetes ingress controller – which you can install using the official Helm chart, among other means – any of these options is a great way to develop and test your routing rules and configurations before moving on to staging.
Desktop Sandboxes
These tools are designed primarily for individual use. They emphasize ease of installation and setup, including on mainstream desktop operating systems. They are not suitable for production deployments, but you’ll still need some computing oomph: At least one dedicated CPU core and 1-2GB of RAM are good rules of thumb. All require 64-bit processors.
Platforms: macOS, Windows, Linux (Intel and ARM)
Pros:
- Create and destroy clusters with a single command
- Supports various hypervisors and container runtimes as backends
- Extensible via addons
Cons:
- Does not support every Kubernetes feature or configuration
- Performance limited when running clusters inside hypervisors
The original Kubernetes sandbox (circa 2016) still ranks among the best. A Kubernetes SIG project, its chief virtue is versatility. It bundles drivers for various hypervisors and container runtimes as backends, and its add-on mechanism lets you enable external resources and advanced features such as Nvidia GPU support. Recent releases have added the ability to create multi-node clusters. Binaries are readily available as installers and from most package managers, including Homebrew and Chocolatey.
Platforms: macOS, Windows, Linux (Intel, ARM, PPC)
Pros:
- Enables programmatic creation of clusters using public Go packages
- Allows loading of local container images directly into the cluster
Cons:
- Requires Docker, supports container backend only
- Performance limited when running clusters inside hypervisors
Kind is another project developed by Kubernetes maintainers. It was the first project to run clusters with Docker (kind stands for “Kubernetes in Docker”). Its first use was for testing Kubernetes itself, but it’s also useful for testing apps locally. One interesting feature is that it provides a set of Go packages that you can use to create and configure clusters programmatically. It uses kubeadm to do the heavy lifting of creating nodes.
Platforms: macOS (Intel and Apple silicon), Windows
Pros:
- Easy-to-use GUI tool with installers for macOS and Windows
- Bundles Docker Engine, Kubernetes, CLI tools
- Deploy workloads on Kubernetes, Swarm, and plain containers simultaneously
Cons:
- GUI tools not open source, no Linux version available
- Single-node cluster only
- Kubernetes server is not configurable
Docker has a lot of competition in the container ecosystem today, but it still offers the easiest GUI-driven way for developers to test apps on Kubernetes. With Docker Desktop, you can spin up a cluster with a single checkbox. There are no frills: It’s a single-node cluster running on Docker and you can’t configure the server. But if all you need is to occasionally create, suspend, restart, and destroy a cluster, its simplicity can’t be beat – particularly if you also need to test against Docker Swarm Mode or plain-vanilla containers.
Lightweight Kubernetes Distros
These projects track upstream Kubernetes, but they streamline and repackage the code with an emphasis on features like low memory footprint and minimal configuration. As such, they can be useful for individual experimentation, but they are also suitable for some production scenarios. They're generally built for Linux but can be made to work on macOS and Windows with the help of a hypervisor or container runtime.
Pros:
- Ships as a single binary for most modern Linux distros
- Easily create local clusters with Docker and companion k3d tool
- CNCF Sandbox project
Cons:
- All cluster functions run within a single process
- Bundles deprecated Traefik version as the default Ingress controller
If “k8s” stands for the ten letters in “Kubernetes,” then “k3s” is meant to be half that size. Rancher Labs created k3s as a way to quickly deploy Kubernetes while keeping clusters as lightweight as possible. As such, it’s billed as useful for production edge computing and IoT projects, and it’s optimized for ARM chips. When used for local testing, it’s best when paired with k3d, which spins up clusters on Docker with a single command. As of this writing, it comes bundled with the older Traefik 1.7 as its default ingress controller, but you can replace that with Traefik 2.x.
Pros:
- Packaged as a single, self-extracting binary
- Included-but-swappable container runtime, networking, and storage
- No dependency on host OS distro except kernel
Cons:
- Still in early phases of development
- Available only for Linux/amd64
- Control nodes contain no kubelet or container runtime
Despite the similarity in their names, k0s versus k3s isn’t an apples-to-apples comparison. Mirantis, which created k0s, describes it as “zero-friction Kubernetes.” It’s a single binary that’s designed to run anywhere, from clouds to IoT to bare metal. One unusual feature is that the control plane is completely isolated from the worker nodes, for security reasons. K0s is still in its early days, but version 1.0 is expected in early 2021. Read more about running Traefik on k0s in our deep dive.
Pros:
- Opinionated defaults for minimal or zero configuration
- Optional automatic, autonomous, and self-healing high availability
- Bundles Prometheus, Istio, Jaeger, other components
Cons:
- Installation requires snap support
- Larger memory footprint than other options
Canonical says its goal for MicroK8s is to eliminate everyday administration for Kubernetes clusters, which includes production clusters in addition to local testing environments. MicroK8s supports the full complement of Kubernetes APIs and it even offers built-in high availability when three or more nodes are deployed. The cluster runs inside an immutable container and security updates are installed automatically by default. MicroK8s also comes bundled with add-ons that enable such features as monitoring, tracing, and service mesh.
Wrap Up
Any of these options could make an excellent addition to your toolbox. Although I've tried to identify pros and cons for each tool, in every case it was easier to find benefits than drawbacks. For some of the tools, deficiencies that were present in early versions have already been addressed in subsequent releases.
Which tool is right for you will naturally depend on your needs. The first question to ask is whether your goal is to test applications in a model Kubernetes environment, or whether it's to develop and debug the Kubernetes infrastructure itself. Traefik fits well into either scenario.
Here at Traefik Labs, we've made good use of K3s and K3d for our own testing and sandboxing needs. As Kubernetes grows increasingly important to our customers, however, we're always looking for new ways to explore use cases for this important tool.
What tools are you using to develop and test with Kubernetes? Share your thoughts in our community forums.
If you'd like to learn more about how Kubernetes fits into a modern, cloud-native infrastructure, check out this white paper from Traefik Labs that explores the crucial role that Traefik and application networking play in this new model.
This is a companion discussion topic for the original entry at https://traefik.io/blog/six-ways-to-kickstart-development-with-kubernetes-and-traefik/