Published on

From Zero to Zero Trust: Securing Microservices with SPIFFE (The Fun Way!)

Authors

A few years ago, when someone needed to get two microservices to talk to each other securely, they would probably generate a long-lived API key, paste it into a config file, and commit it — maybe even to a public repo (oops). Then they’d cross their fingers and hope nobody found it.

It feels a bit like leaving your house key under the doormat. It’s simple, sure, but is it secure? Not really.

Welcome to the wonderful, chaotic world of microservice security. In my market, getting developers to encrypt internal traffic is a weekly adventure. We have teams building awesome things with Java Spring Boot and using Nacos for service discovery, but the communication between them is often plain, unencrypted HTTP.

After watching a fantastic gRPC Conf talk, Fortifying gRPC Microservices, I was inspired. The speaker explained these complex topics in such a clear, story-like way. I thought, "I have to try this myself!" So, I fired up my laptop, spun up Minikube, and built a working Go demo. The process was incredibly enlightening, and I want to share that journey with you.

So grab a coffee, and let's go from zero to Zero Trust. All the code is available on my GitHub: https://github.com/liweijian/spire-demo

"It's Behind a Firewall"

First question: why do we even need to secure communication inside our own network? Isn't that what the firewall is for?

Relying solely on a firewall is like having a world-class bouncer at the front door of a nightclub, but once you're inside, it's a total free-for-all. No one checks your ID again, so anyone could potentially impersonate a staff member and walk into the cash room.

In the digital world, if an attacker gets past your firewall (your "bouncer"), they can listen to all your internal plaintext traffic. As the Snowden leaks famously revealed with a slide saying "SSL added and removed here", even giants like Google had their internal traffic tapped.

In today's world of dynamic, cloud-native environments, services are ephemeral. Their IP addresses change constantly. Using network location as a security measure is like trying to identify a friend based on the parking spot they're in. Tomorrow, someone else will be in that spot.

This is the core idea of Zero Trust: Never trust, always verify.

The Painful Journey: API Keys and the mTLS Headache

Okay, so we need to secure our internal traffic. What are our options?

  1. API Keys & Access Tokens: This is often the first step. We issue a secret key or a short-lived JWT token to a client. The client flashes this token, and the server says, "Okay, you're in." This is better than nothing, but if the communication channel isn't encrypted, an attacker can just snatch the token off the wire and use it. We've solved one problem but ignored the bigger one: encryption.

  2. Traditional mTLS: The next logical step is mutual TLS. It’s like a secret handshake where both the client and server show their IDs (certificates) to prove who they are before they start talking. This is great! But doing it manually in a microservices world is a nightmare.

    • Certificate Management is a Pain: Who creates the certificates? Who signs them? How do you get them onto every single pod? How do you rotate them before they expire? It quickly becomes a full-time job.

    • Identity is Flimsy: What do you put in the certificate as the "name"? An IP address? A DNS name? As we discussed, these things are meaningless for ephemeral workloads. We once had to disable the default gRPC name validation and write our own custom, brittle code to check the identity. It worked, but it felt like holding things together with duct tape.

There had to be a better way. And there is.

The Hero Arrives: SPIFFE and SPIRE

This is where our story gets its heroes.

SPIFFE (Secure Production Identity Framework For Everyone) is not a tool, but a standard. It’s a set of rules for how to give a software service a strong, verifiable identity, no matter where it's running.

The core concept is the SPIFFE ID, a unique name for your service that looks like this: spiffe://your-company.com/your-service.

  • The your-company.com part is the "trust domain." Think of it as the country that issues the passport.

  • The /your-service part is the unique identity within that country.

This ID is then embedded into a short-lived cryptographic document called an SVID (Spiffe Verifiable Identity Document), which is usually just a standard X.509 certificate or JWT.

SPIRE (SPIFFE Runtime Environment) is the implementation of the SPIFFE standard. It’s the machinery that actually does the work. Think of SPIRE as the passport office for your services. It consists of two main parts:

  • SPIRE Server: The central authority. It manages which service gets which SPIFFE ID and signs the certificates.

  • SPIRE Agent: A little helper that runs on every node (or machine/pod). Its job is to figure out which services are running on its node and securely deliver the correct "passport" (SVID) to them.

The magic is in how the Agent verifies a workload's identity, a process called Workload Attestation. In Kubernetes, the Agent can ask the kubelet, "Hey, tell me about the process that's calling me. What's its service account? What's its namespace?" Based on these trusted proofs, the Agent gets the correct SVID from the SPIRE Server.

The result? Every service gets a strong, short-lived, automatically rotated identity without you having to lift a finger.

Let's Get Our Hands Dirty: A Live Demo in Minikube!

Talk is cheap. Let me show you how this actually works. I've put together a simple Go-based gRPC client/server demo that runs on Minikube.

Credit goes to Mehrdad Afshari

setup-spire.sh (The Magic Wand)

This script installs SPIRE using Helm and registers the identities for our client and server. We are telling SPIRE: "Any pod running with the Kubernetes service account my-server-sa should be given the identity spiffe://supasaf.com/server."

#!/bin/bash
set -e

KUBECTL="minikube kubectl --"

echo "🚀 Setting up SPIRE demo environment..."

echo "👤 Creating Service Accounts..."
$KUBECTL apply -f - << ENDSA
apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-server-sa
  namespace: default
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-client-sa
  namespace: default
ENDSA

echo "📦 Installing SPIRE with Helm..."
helm repo add spiffe https://spiffe.github.io/helm-charts/ 2>/dev/null || true
helm repo update

echo "📦 Installing SPIRE with custom values..."
if helm list -n spire-system | grep -q spire; then
    echo "⚠️  SPIRE already installed, upgrading with custom values..."
    helm upgrade spire spiffe/spire \
        --namespace spire-system \
        -f spire-values.yaml \
        --wait \
        --timeout 5m
else
    helm install spire spiffe/spire \
        --namespace spire-system \
        --create-namespace \
        -f spire-values.yaml \
        --wait \
        --timeout 5m
fi

echo "⏳ Waiting for SPIRE components to be ready..."
$KUBECTL wait --for=condition=ready pod -l app.kubernetes.io/name=server -n spire-system --timeout=300s
$KUBECTL wait --for=condition=ready pod -l app.kubernetes.io/name=agent -n spire-system --timeout=300s

SERVER_POD=$($KUBECTL get pod -n spire-system -l app.kubernetes.io/name=server -o jsonpath='{.items[0].metadata.name}')
echo "✅ SPIRE Server pod: $SERVER_POD"

echo "🔐 Registering node identity..."
$KUBECTL exec -n spire-system $SERVER_POD -c spire-server -- \
    /opt/spire/bin/spire-server entry create \
    -node \
    -spiffeID spiffe://supasaf.com/minikube-node \
    -selector k8s_psat:cluster:minikube \
    -selector k8s_psat:agent_ns:spire-system \
    -selector k8s_psat:agent_sa:spire-agent 2>/dev/null || echo "⚠️  Node entry already exists"

echo "🔐 Registering server workload identity..."
$KUBECTL exec -n spire-system $SERVER_POD -c spire-server -- \
    /opt/spire/bin/spire-server entry create \
    -parentID spiffe://supasaf.com/minikube-node \
    -spiffeID spiffe://supasaf.com/server \
    -selector k8s:ns:default \
    -selector k8s:sa:my-server-sa 2>/dev/null || echo "⚠️  Server entry already exists"

echo "🔐 Registering client workload identity..."
$KUBECTL exec -n spire-system $SERVER_POD -c spire-server -- \
    /opt/spire/bin/spire-server entry create \
    -parentID spiffe://supasaf.com/minikube-node \
    -spiffeID spiffe://supasaf.com/client \
    -selector k8s:ns:default \
    -selector k8s:sa:my-client-sa 2>/dev/null || echo "⚠️  Client entry already exists"

echo ""
echo "📋 Registered entries:"
$KUBECTL exec -n spire-system $SERVER_POD -c spire-server -- \
    /opt/spire/bin/spire-server entry show

echo ""
echo "✅ SPIRE setup complete!"
echo ""
echo "📝 Registered SPIFFE IDs:"
echo "   Node:   spiffe://supasaf.com/minikube-node"
echo "   Server: spiffe://supasaf.com/server (SA: my-server-sa)"
echo "   Client: spiffe://supasaf.com/client (SA: my-client-sa)"
echo ""
echo "Next steps:"
echo "1. Build: ./build.sh"
echo "2. Deploy: kubectl apply -f deployment.yaml"
echo "3. Check logs: kubectl logs -l app=my-server -f"

We will get the following outputs

✅ SPIRE Server pod: spire-server-0
🔐 Registering node identity...
Entry ID         : 11223f64-8be3-4d84-9d7e-1827b599c5ae
SPIFFE ID        : spiffe://supasaf.com/minikube-node
Parent ID        : spiffe://supasaf.com/spire/server
Revision         : 0
X509-SVID TTL    : default
JWT-SVID TTL     : default
Selector         : k8s_psat:agent_ns:spire-system
Selector         : k8s_psat:agent_sa:spire-agent
Selector         : k8s_psat:cluster:minikube

🔐 Registering server workload identity...
Entry ID         : 977ab667-1001-476f-bb4d-838bb4fc556b
SPIFFE ID        : spiffe://supasaf.com/server
Parent ID        : spiffe://supasaf.com/minikube-node
Revision         : 0
X509-SVID TTL    : default
JWT-SVID TTL     : default
Selector         : k8s:ns:default
Selector         : k8s:sa:my-server-sa

🔐 Registering client workload identity...
Entry ID         : 06a80986-983e-401e-9545-e71dfc2aecde
SPIFFE ID        : spiffe://supasaf.com/client
Parent ID        : spiffe://supasaf.com/minikube-node
Revision         : 0
X509-SVID TTL    : default
JWT-SVID TTL     : default
Selector         : k8s:ns:default
Selector         : k8s:sa:my-client-sa


📋 Registered entries:
Found 3 entries
Entry ID         : 06a80986-983e-401e-9545-e71dfc2aecde
SPIFFE ID        : spiffe://supasaf.com/client
Parent ID        : spiffe://supasaf.com/minikube-node
Revision         : 0
X509-SVID TTL    : default
JWT-SVID TTL     : default
Selector         : k8s:ns:default
Selector         : k8s:sa:my-client-sa

Entry ID         : 11223f64-8be3-4d84-9d7e-1827b599c5ae
SPIFFE ID        : spiffe://supasaf.com/minikube-node
Parent ID        : spiffe://supasaf.com/spire/server
Revision         : 0
X509-SVID TTL    : default
JWT-SVID TTL     : default
Selector         : k8s_psat:agent_ns:spire-system
Selector         : k8s_psat:agent_sa:spire-agent
Selector         : k8s_psat:cluster:minikube

Entry ID         : 977ab667-1001-476f-bb4d-838bb4fc556b
SPIFFE ID        : spiffe://supasaf.com/server
Parent ID        : spiffe://supasaf.com/minikube-node
Revision         : 0
X509-SVID TTL    : default
JWT-SVID TTL     : default
Selector         : k8s:ns:default
Selector         : k8s:sa:my-server-sa


✅ SPIRE setup complete!

📝 Registered SPIFFE IDs:
   Node:   spiffe://supasaf.com/minikube-node
   Server: spiffe://supasaf.com/server (SA: my-server-sa)
   Client: spiffe://supasaf.com/client (SA: my-client-sa)

deployment.yaml (The Blueprint)

This file tells Kubernetes how to run our apps. The most important part is the volumes section. We use the csi.spiffe.io driver to mount the SPIRE Agent's socket directly into our pod. This is how our app talks to the agent.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-server-sa
  namespace: default
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-client-sa
  namespace: default
---
apiVersion: v1
kind: Service
metadata:
  name: my-server-svc
  namespace: default
spec:
  selector:
    app: my-server
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-server-deployment
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-server
  template:
    metadata:
      labels:
        app: my-server
    spec:
      serviceAccountName: my-server-sa
      containers:
      - name: my-server
        image: my-server:latest
        imagePullPolicy: Never
        ports:
        - containerPort: 8080
        env:
        - name: SPIFFE_ENDPOINT_SOCKET
          value: unix:///spiffe-workload-api/spire-agent.sock
        volumeMounts:
        - name: spiffe-workload-api
          mountPath: /spiffe-workload-api
          readOnly: true
      volumes:
      - name: spiffe-workload-api
        csi:
          driver: "csi.spiffe.io"
          readOnly: true
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-client-deployment
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-client
  template:
    metadata:
      labels:
        app: my-client
    spec:
      serviceAccountName: my-client-sa
      restartPolicy: Always
      containers:
      - name: my-client
        image: my-client:latest
        imagePullPolicy: Never
        env:
        - name: SPIFFE_ENDPOINT_SOCKET
          value: unix:///spiffe-workload-api/spire-agent.sock
        volumeMounts:
        - name: spiffe-workload-api
          mountPath: /spiffe-workload-api
          readOnly: true
      volumes:
      - name: spiffe-workload-api
        csi:
          driver: "csi.spiffe.io"
          readOnly: true

server/main.go (The Wise Guardian)

Look how simple the Go code is! We don't handle any private keys or certificate files. We just ask the workloadapi for a Source, and it handles all the magic of talking to the SPIRE Agent behind the scenes. When a request comes in, we can inspect the peer's certificate and pull out its verified SPIFFE ID.

package main

import (
        "context"
        "log"
        "net"

        "github.com/spiffe/go-spiffe/v2/spiffetls/tlsconfig"
        "github.com/spiffe/go-spiffe/v2/workloadapi"
        "google.golang.org/grpc"
        "google.golang.org/grpc/credentials"
        "google.golang.org/grpc/peer"

        "spire-demo/proto"
)

type Server struct {
        proto.UnimplementedEchoServer
}

func (s *Server) SayHello(ctx context.Context, in *proto.Request) (*proto.Response, error) {
        p, ok := peer.FromContext(ctx)
        if !ok {
                log.Printf("Error: could not get peer from context")
                return &proto.Response{Message: "ERROR: no peer info"}, nil
        }

        tlsInfo, ok := p.AuthInfo.(credentials.TLSInfo)
        if !ok {
                log.Printf("Error: peer auth info is not TLS")
                return &proto.Response{Message: "ERROR: no TLS info"}, nil
        }

        if len(tlsInfo.State.PeerCertificates) == 0 {
                log.Printf("Error: no peer certificates")
                return &proto.Response{Message: "ERROR: no certificates"}, nil
        }

        cert := tlsInfo.State.PeerCertificates[0]
        if len(cert.URIs) == 0 {
                log.Printf("Error: no URIs in certificate")
                return &proto.Response{Message: "ERROR: no SPIFFE ID"}, nil
        }

        spiffeID := cert.URIs[0].String()
        log.Printf("✅ Received a request from client with SPIFFE ID: %s", spiffeID)

        return &proto.Response{Message: "Hello " + in.Name + " from " + spiffeID}, nil
}

func main() {
        ctx := context.Background()

        log.Println("🚀 Starting server, waiting for SPIRE agent...")

        source, err := workloadapi.NewX509Source(ctx)
        if err != nil {
                log.Fatalf("❌ Unable to create X509Source: %v", err)
        }
        defer source.Close()

        log.Println("✅ Successfully connected to SPIRE agent")

        tlsConfig := tlsconfig.MTLSServerConfig(source, source, tlsconfig.AuthorizeAny())
        creds := credentials.NewTLS(tlsConfig)

        server := grpc.NewServer(grpc.Creds(creds))
        proto.RegisterEchoServer(server, &Server{})

        lis, err := net.Listen("tcp", ":8080")
        if err != nil {
                log.Fatalf("❌ Failed to listen: %v", err)
        }

        log.Println("🎧 Server listening on :8080 with mTLS enabled")
        if err := server.Serve(lis); err != nil {
                log.Fatalf("❌ Failed to serve: %v", err)
        }
}

The "Aha!" Moment: Checking the Logs

After running ./setup-spire.sh, ./build.sh, and kubectl apply -f deployment.yaml, let's look at the logs.

Server Logs:

ss@ss:~/spire-demo$ kubectl logs -l app=my-server -f
2025/10/06 08:51:24 🚀 Starting server, waiting for SPIRE agent...
2025/10/06 08:51:25 ✅ Successfully connected to SPIRE agent
2025/10/06 08:51:25 🎧 Server listening on :8080 with mTLS enabled
2025/10/06 08:51:25 ✅ Received a request from client with SPIFFE ID: spiffe://supasaf.com/client
2025/10/06 08:51:31 ✅ Received a request from client with SPIFFE ID: spiffe://supasaf.com/client
2025/10/06 08:51:50 ✅ Received a request from client with SPIFFE ID: spiffe://supasaf.com/client
2025/10/06 08:52:22 ✅ Received a request from client with SPIFFE ID: spiffe://supasaf.com/client

Client Logs:

ss@ss:~/spire-demo$ kubectl logs -l app=my-client -f
2025/10/06 08:51:30 🚀 Starting client, waiting for SPIRE agent...
2025/10/06 08:51:31 ✅ Successfully connected to SPIRE agent
2025/10/06 08:51:31 🔄 Attempting to connect to server (attempt 1/10)...
2025/10/06 08:51:31 ✅ Connected to server
2025/10/06 08:51:31 📤 Sending request to server...
2025/10/06 08:51:31 ✅ Response from server: Hello SPIFFE from spiffe://supasaf.com/client
2025/10/06 08:51:31 🎉 Zero Trust communication successful!

Look at that beautiful server log! Received a request from client with SPIFFE ID: spiffe://supasaf.com/client.

The server doesn't know the client's IP address, nor does it care. It knows its true, cryptographically verified identity. This is the foundation of Zero Trust security. We've successfully established a secure, authenticated, and encrypted channel between two microservices, and it was almost entirely automatic.

Wrapping Up

Building this demo was a real eye-opener. The concepts from the gRPC Conf talk came to life when I could actually see the SPIFFE IDs in my logs, watch certificates rotate automatically, and know that every connection was authenticated and encrypted.

In my market, where English documentation can be a barrier and plaintext HTTP is still common, I hope this practical example helps show that zero trust isn't as scary as it sounds. SPIRE handles the complexity - certificate management, rotation, attestation - so we can focus on building secure applications.

Now if you'll excuse me, I have another meeting about microservice security next week. But this time, I'm bringing a working demo. 🚀