When authenticating users in Kubernetes, especially in a multi-tenancy environment, you want a simple entrypoint for your users to get their credentials. Maintaining static password or token files might be possible in Kubernetes but is probably not what you want to maintain in the long run. Instead Kubernetes offers the ability to integrate with an OpenID Connect service (OIDC). You can configure your kube-apiserver to trust a specific root CA used by the OIDC provider. OIDC is a standard built atop of the OAuth2 standard, so client apps will have to follow the well known OAuth2 flow when integrating with a OIDC provider.

When you don’t want or simply can’t use a public OIDC provider like Google, you can run your own. CoreOS made this quite simple with dex. Dex even has a stable LDAP integration for authenticating users, which is quite useful.

In this article I want to concentrate on implementing a small app that works as a OAuth2 client and as such integrates with Dex to get a signed JWT that then can be used in a kubeconfig for authentication at the kube-apiserver.

First things first, you’ll need a working dex setup. The “Getting started” tutorial from dex will guide you through all that. An example config for dex can be found here. It is up to you which authentication backend you use for dex and it won’t be important for the scope of this article.

The important part of the dex configuration is the following:

staticClients:
- id: kauthz
  redirectURIs:
  - 'http://127.0.0.1:8000/callback/dex'
  name: 'KAuthz'
  secret: ThisSuperSecretSecretIsSecret

This says dex that there is a client (which we statically configure here) with the client id ‘kauthz’ and the client secret ‘ThisSuperSecretSecretIsSecret’. Requests redirected from this client will be redirected back after authentication to the configured callback URI ‘http://127.0.0.1:8000/callback/dex’.

With dex running somewhere we can now start hacking away! We will now setup a very basic service that starts a webserver with two endpoints. The first one will immediatly redirect to the dex authentication url and the second one will take the callback from dex and output the valid JWT to the user.

Here is the main.go for now:

package main

import (
	"github.com/coreos/go-oidc"
	log "github.com/sirupsen/logrus"
	"golang.org/x/oauth2"
	"github.com/spf13/viper"
	"net/http"
	"golang.org/x/net/context"
)

func init() {
	viper.SetEnvPrefix("kauthz")
	viper.BindEnv("clientId")
	viper.BindEnv("clientSecret")
	viper.BindEnv("providerURL")
	viper.BindEnv("listenAddress")
}

func main() {
	clientID := viper.GetString("clientId")
	providerURL := viper.GetString("providerURL")
	listenAddress := viper.GetString("listenAddress")

	var clientSecret string
	if viper.IsSet("clientSecret") {
		clientSecret = viper.GetString("clientSecret")
	} else {
		log.Panic("Could not determine clientSecret! Please make sure to set env variable KAUTHZ_CLIENTSECRET.")
	}

	ctx := context.Background()

	provider, err := oidc.NewProvider(ctx, providerURL)
	if err != nil {
		log.Panicf("Could not create new OIDC provider: %s", err)
	}
	oidcConfig := &oidc.Config{
		ClientID: clientID,
	}
	verifier := provider.Verifier(oidcConfig)

	oauthConfig := &oauth2.Config{
		ClientID:     clientID,
		ClientSecret: clientSecret,
		Endpoint:     provider.Endpoint(),
		RedirectURL:  "http://127.0.0.1:8000/callback/dex",
		Scopes:       []string{oidc.ScopeOpenID, "profile", "email", "groups"},
	}

	http.HandleFunc("/", requestToken(oauthConfig))
	http.HandleFunc("/callback/dex", handleCallback(oauthConfig, verifier))

	err = http.ListenAndServe(listenAddress, nil)
	if err != nil {
		log.Fatal("%s", err)
	}
}

We register the functions handling requests to our two endpoints with statements like http.HandleFunc("/callback/dex", handleCallback()), where the first argument is the URI path for the requests we want to handle and the second argument is the handler function that is to be called by the router. The http.ListenAndServer(listenAddress, nil) will start the webserver listening on whatever IP and port we specified. Here we can use a notation like :8000 to specify listening on all IPs on port 8000. One should probably also mention that we configure our app via environment variables and we use the viper library to manage access to our config.

Much more interesting for our purposes is this part:

provider, err := oidc.NewProvider(ctx, providerURL)
if err != nil {
	logger.Panicf("Could not create new OIDC provider: %s", err)
}
oidcConfig = &oidc.Config{
	ClientID: clientID,
}
verifier = provider.Verifier(oidcConfig)

oauthConfig = &oauth2.Config{
	ClientID:     clientID,
	ClientSecret: clientSecret,
	Endpoint:     provider.Endpoint(),
	RedirectURL:  "http://127.0.0.1:8000/callback/dex",
	Scopes:       []string{oidc.ScopeOpenID, "profile", "email", "groups"},
}

Generally this configures and tests a connection to our OIDC provider. This is done by first discovering the capabilities of the OIDC provider via its ‘/.well-known/openid-configuration’ endpoint and populating a Provider struct with oidc.NewProvider(ctx, providerURL). Then we create a new Verifier with provider.Verifier(oidcConfig) that enables us to verify a JWT later on. And last but not least we populate an oauth2.Config struct to use in our handlers later on. Here we use our provider struct to get the auth and token endpoints for our oauth2 provider with provider.Endpoint() and define which OIDC scopes we want to request from the OIDC provider. This is just a string slice with the scopes we want. We could probably omit “profile” here, most important are “email” and “groups” as these are the ones used by Kubernetes by default. The scope oidc.ScopeOpenID is not optional here, since this tells the provider to include the OpenID Connect specific data in the token data later on.

Now we can have a look at our handler functions which I put in a file named handlers.go:

package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	log "github.com/sirupsen/logrus"
	"github.com/coreos/go-oidc"
	"golang.org/x/oauth2"
	"net/http"
	"net/url"
	"sync/atomic"
	"text/template"
)

func requestToken(config *oauth2.Config) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		state := "thisshouldactuallyberandomandforeveryrequestuniquestatehash"
		http.Redirect(w, r, config.AuthCodeURL(state), http.StatusFound)
	})
}

func callback(oauth2Config *oauth2.Config, verifier *oidc.IDTokenVerifier) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		state := r.URL.Query().Get("state")

		if state != "thisshouldactuallyberandomandforeveryrequestuniquestatehash" {
			http.Error(w, "state did not match", http.StatusBadRequest)
			log.Errorf("Request state '%s' did not match any state '%s'!\n", r.URL.Query().Get("state"), state)
			return
		}

		oauth2Token, err := oauth2Config.Exchange(ctx, r.URL.Query().Get("code"))
		if err != nil {
			http.Error(w, "Failed to exchange token: "+err.Error(), http.StatusInternalServerError)
			log.Errorf("Failed to exchange token! %s\n", err)
			return
		}
		rawIDToken, ok := oauth2Token.Extra("id_token").(string)
		if !ok {
			http.Error(w, "No id_token field in oauth2 token.", http.StatusInternalServerError)
			log.Errorf("No id_token field in oauth2 token!\n")
			return
		}
		_, err := verifier.Verify(ctx, rawIDToken)
		if err != nil {
			http.Error(w, "Failed to verify ID Token: "+err.Error(), http.StatusInternalServerError)
			log.Errorf("Failed to verify ID token! %s\n", err)
			return
		}

		w.Header().Add("Content-Type", "text/plain")
		w.Write([]byte(rawIDToken))
	})
}

The first handler I called requestToken as the user requests a new token this way. It’s very simply just a redirect with a HTTP status 302 “Found” to our OIDC provider. The URL we’re redirecting to is generated with config.AuthCodeURL(state). This uses the Config struct method AuthCodeURL() from the oauth2 library. We pass a string to this function that is used as the state parameter. This should be a unique string hash for every new request to an OAuht2 server. The server will send the state back when calling the callback endpoint of our client, where we can verify it’s integrity. This is used to mitigate some forms of CSRF attacks (cross-site request forgery). We will use a static value as state parameter for the sake of simplicity. In an upcoming article I will speak more about how we can use a local cache to store a state for later verification. More on the state parameter and the Oauth2 standard can be found here.

The second handler is the callback handler. This endpoint is the one the user is getting redirected back to by the OIDC provider. The parameters we pass to the function are access to the oauth2.Config struct and to the oidc.IDTokenVerifier. We use these to get the real token now and verify it before displaying it to the user. First of all let’s check if nobody CSRFed our request by checking if the state parameter is what we would expect. With state := r.URL.Query().Get("state") we extract the state parameter from the request struct and then we just compare it with our static value. When everything seems to be ok, we now have to get the real token. The redirected request does not contain the token we want, because OAuth2 works by the provider redirecting the user back to the client and not communicating directly with the client on a second channel. Instead it contains an authcode that the client has to exchange for the real token with the provider. We extract the code from the request and call the provider to get the token here oauth2Token, err := oauth2Config.Exchange(ctx, r.URL.Query().Get("code")). Now we need to get our token. OAuth2 does not include support for the JWT style ID-tokens. That’s why we passed the OIDC-specific scope when the user got redirected to the auth endpoint of the provider in our requestToken handler. By that the provider knew to include the extra id_token field when we exchange the authcode for the actual data. This field gets extracted and converted to string directly here: rawIDToken, ok := oauth2Token.Extra("id_token").(string). Since this is now explicitly conforming with the OpenID Connect standard, we can go ahead and verify the integrity of the token and if it’s not redacted already with the provider by calling idToken, err := verifier.Verify(ctx, rawIDToken). Now that we know that everything is ok and verified, we can respond to the user with the actual token. For that we set the content-type to “text/plain”, so that browsers and other programs play nicely and write the token to the response here: w.Write([]byte(rawIDToken)).

This is it! When we now start our service on localhost:8000, we can access our client with http://localhost:8000/ and will be redirected to dex. After logging in, dex will redirect us to our client again, specifically to the callback endpoint and our client will ouput a singed JWT for us to use in our kubeconfig file.

In the next article I’ll show how to use Golang text/template to output a functioning kubeconfig directly and we’ll extend the code with some more small features.