Topi Kettunen

Why Not Kubernetes?

First, I would like to say I think Kubernetes is an excellent platform for its intended purposes. It provides excellent fault tolerance all over the cluster, a fast and easy way to run updates on your deployments, great tools for managing services, volumes, metrics, and more, each has its own lifecycle to manage. Also, implementing your tooling by extending the Kubernetes’ API is a trivial task, so you can easily leverage the great tooling to make your own for whatever you might need.

Today it’s also effortless to spin up a Kubernetes cluster with various installers and managed options. While complex, it’s still a step closer to the idea of “just run my code and make it work”. Also, with containers in the picture, we are pretty close to the magical situation where we can run the same application similarly on the laptop and in one cloud.

For me, issues start rising when we use Kubernetes for something other than its intended purposes. While I don’t have any statistics on this, I have a pretty strong gut feeling that most of the people running Kubernetes are using it as a glorified scheduler for placing containers on nodes as fast as possible. While it’s an excellent and pretty easy tool to use for orchestrating containers. Its fundamental purpose is to orchestrate anything crucial to your infrastructure, like network, storage, and other dependencies.

Kubernetes allows the complete user freedom to run your infrastructure as you see fit. Despite sounding like a cliche, this kind of freedom can bear huge responsibilities. I would dare to say that most developers and system administrators don’t want to make these decisions. What if, at some point in the development, you wish to change your networking interface or maybe some dynamic storage provider? Can you even do such a thing in that stage of development if the decision was made before you even had anything running in Kubernetes?

Kelsey Hightower put it nicely a while back when he described Kubernetes isn’t meant for being a developer platform but a framework for creating platforms. So while it can work as a developer platform, and overall it’s pretty easy to get started, kubectl run and kubectl expose, and you’re good to go. That being said, all the API designs in Kubernetes are created for clusters and how to manage these. So while containers are part of this, much more must be leveraged. So should application developers, startups or small businesses use something like this? Probably not unless they are developing a platform product.

When we get into cluster management, we need to start thinking about managing the lifecycle of everything running inside the cluster. Unfortunately, this is also when things start to get hard. For example, what to do if something inside the cluster dies? What if I need to provision something dynamically? Kubernetes is pretty good at simplifying many of these topics, but all this complexity cannot be simplified away due to the complexity of things happening behind the scenes.

Kubernetes has a high entry threshold, and it’s a very complex project, but still, way too often, I see it marketed as a simple solution for many problems. While you can effortlessly use Kubernetes and get lots of stuff done, eventually, you will hit a wall. Deploying fault-tolerant distributed applications that are scalable against a pool of machines with dependencies in networking, storage, and more that’s a complicated problem.

Kubernetes is built for production workloads and running infrastructure beyond your demo application. For this reason, the complexity of Kubernetes is justified and should be approached with that mindset.