Build the platform you need

Where we are

As an industry we have decided that everyone needs a developer platform, infrastructure platform, some platform to run your mission critical applications in a stable and secure manner. This is a very broad statement but I tend to agree.

Afterwards it seems we all decided that what our developers need to be productive is just Kubernetes. This is I do not agree with, even though Kubernetes has been paying my bills over the last 7 years or so.

So what problems do most platforms have that I encountered in my career so far?

  • Managed Kubernetes is seen as the solution to making developers productive.
  • Certain managed services being offered other than Kubernetes, like databases, CI/CD systems etc. are sometimes offered but can only be acquired through a different mechanism than the KaaS features.
  • There is often no easy way to manage exposure of the application for the developers. DNS, loadbalancing are seen as something seperate from the KaaS features.
  • Security is usually an afterthought and only enforced by applying frameworks that have almost no applicable requirements for cloud-native environments.
  • When running in a private cloud environment, there is almost always a complete disconnect to how the datacenter and hardware is managed.
  • And this leads to datacenters and hardware being managed still with Excel sheets for configuration and week-long lead times

This might not sound surprising (I’m sure many of you encounter similar environments) but when you hear the business’ reasoning for investing non-trivial amounts of budget into building these platforms it is very surprising. Usually the goals from the business side are something like:

  • Bring new applications or new features to production faster
  • Scale to meet business demands
  • Reduce operational overhead and cost

Somewhere between these overarching goals and the reality of the platforms underneath, a disconnect happened. There must be a feedback problem.
A lot of reasons are found in the organizational and communication structures and dynamics within them but that is probably worth a whole other series of blog posts to dive into. I want to describe here what I think an organization should focus on first to not fall into the same traps again.

The “just follow the hype” fallacy and it’s bigger brother “we’ve always done it this way”

We all know and laugh about the “hype-driven development” jokes being thrown around in meetings and we probably all despise the “this is how we’ve always done this” comments from some colleagues running systems following some ancient Excel-based process. We intuitively know both extremes are not solving our problems, but then we seem to find a compromise that is just using both extremes at the same time rather than finding a new, more productive way.
We can see this when we build Kubernetes clusters with applications that automatically scale on top of a vSphere infrastructure for which you need to create a ticket to get a new VM. Or you enjoy running a modern infrastructure stack in the public cloud, but with so many strict constraints, that you’re not able to expose any of your endpoints to the internet or customers and definitely not to the company’s intranet.

We need a third way, a new focus. Ultimately, new value is only added to the business if a new capability is running in production or a new feature is being accessible to the customers. The platform’s role is to empower the developers to deliver these capabilities and features in a fast, reliable and secure way. Focus on this first. Repeat it often. Repeat it loud.

Make developers faster and better in delivering features to production.

How to do it better: build the platform you need

Now comes the hard part, the part only you can do.

Build the platform you need.

Raw access to Kubernetes is probably not the best way to enable your developers. Find out what kind of systems you run and find the patterns. 90% of your applications are consisting of a frontend, backend and database? Cool, that’s what you want to automate first. Build a platform that enables developers to quickly expose new development versions of their app in a way that they can have very quick feedback loops (I’m talking seconds here, not minutes or hours). Offer an easy and scalable way to run needed databases. And so on…
For most of these requirements the CNCF ecosystem, the public cloud providers and many other open-source and vendor solutions are available, so we don’t need to build everything from scratch. But still there is no vendor that has a product that is the exact platform you need. Customization is always needed. So focus on assembling all needed products into something accelerating your specific business.

But this sounds like something we do already, right? It’s just developing a new product, right? Yes! And your developers are your customers! Listen to your customers, find out what they really need, but like every other successful product, keep in mind what drives the business, which happens to be the same business as your customer’s.

Recognize your value streams

Infrastructure teams have, in my experience at least, a tendency to focus on regulation as they’re often responsible for the security and availability of the systems while being mostly recognized as a cost center in the organization. To be successful with a platform, this self-perception needs to change.

First an infrastructure or platform organization has to recognize that they’re an integral part of every value stream depending on software. Then they have to work aggressively to be recognized as such in the organization.

In parallel the internal focus needs to shift to a customer-centric view. We need to ask how we can enable the developers while still being secure and reliable.

With this in mind it should now be possible to shift investments into product development of the platform in addition to the ongoing operational cost.

The gist of it

We need to recognize as an industry that platforms are there to enable our developers to deliver faster.

We need to run our platform development as a product development.

We need to make our place in the value stream of our business seen and make it easier to secure the financial structures needed to be able to do the work.

In the next blog post I will dive deeper into some loose platform patterns that I hope make it easier to talk about these systems and make it easier for management to categorize platforms.