Securing Kubernetes Services with OAuth2-Proxy: A Practical Guide to Single Sign-On

Adding authentication to internal services can be challenging. While many applications come with built-in auth systems, managing separate user databases quickly becomes unwieldy. Today, I'll share how I've implemented a unified authentication system using OAuth2-Proxy as a sidecar container alongside my services, with Zitadel as the central identity provider.

The Authentication Challenge

When running multiple services like Grafana, DNS servers, or custom applications, each service traditionally needs its authentication system. This leads to several problems: users need various accounts, administrators have to manage access in multiple places, and security policies become hard to enforce consistently.

Enter the Dynamic Duo: OAuth2-Proxy and Zitadel

The solution I've implemented uses two key components. Zitadel serves as our Identity Provider (IdP), centrally managing all user accounts and access policies. OAuth2-Proxy acts as a sidecar container that sits in front of each service, handling the authentication flow and ensuring users are adequately authenticated before accessing the service.

Advanced Patterns with OAuth2-Proxy and Zitadel

Securing access becomes critical when managing infrastructure as code through tools like Atlantis. Today, I'll share how we've implemented a sophisticated authentication layer using OAuth2-Proxy and Zitadel, incorporating event-driven autoscaling for optimal resource usage.

Why This Matters

Atlantis is a powerful tool for Terraform automation, but it needs careful security consideration since it has significant permissions to modify infrastructure. By implementing SSO through Zitadel, we ensure that only authorized team members can access these powerful capabilities while maintaining a seamless user experience.

The Implementation Architecture

Let's examine how we've secured Atlantis using OAuth2-Proxy as a sidecar container while maintaining its webhook functionality for GitHub integration. This setup demonstrates several advanced patterns:

StatefulSet with Sidecar Pattern

The core of our implementation uses a StatefulSet rather than a Deployment:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: atlantis
spec:
  serviceName: atlantis
  template:
    spec:
      containers:
      - name: atlantis
        image: ghcr.io/runatlantis/atlantis:latest
        # Atlantis configuration here...
      - name: oauth2-proxy
        image: quay.io/oauth2-proxy/oauth2-proxy:v7.7.1
        args:
        - --http-address=0.0.0.0:4180
        - --upstream=http://localhost:80

What makes this implementation particularly sophisticated is how it handles different types of traffic:

Dual Ingress Configuration

We've implemented a dual-ingress approach to handle different access patterns:

# Public GitHub webhook endpoint
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: atlantis
  annotations:
    kubernetes.io/ingress.class: cloudflared 
spec:
  rules:
  - host: <HOST-URL>
    http:
      paths:
      - path: /events
        pathType: Exact
        backend:
          service:
            name: atlantis-keda-interceptor
            port:
              number: 8080

# Protected internal access
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: "zerossl-issuer"
    konghq.com/protocols: "https"
  name: atlantis-tailscale
spec:
  ingressClassName: kong
  rules:
  - host: <HOST-URL>

Selective Authentication with Skip Routes

One of the most interesting aspects is how we handle authentication bypass for webhooks:

data:
  OAUTH2_PROXY_SKIP_AUTH_ROUTES: "/healthz,/events"

This configuration allows GitHub webhooks to bypass authentication while ensuring all user access is authenticated. The webhook is made accessible through Cloudflared using the 'Exact' path setting. This setup ensures that any traffic not directed to this specific path will be rejected.

Event-Driven Scaling with KEDA

We've also implemented sophisticated scaling behaviour using KEDA's HTTP add-on:

kind: HTTPScaledObject
apiVersion: http.keda.sh/v1alpha1
metadata:
    name: atlantis
spec:
    hosts:
        - <HOST-URL>
        - <HOST-URL>
    replicas:
        min: 0
        max: 1
    scalingMetric:
        requestRate:
            targetValue: 5
            window: 1m
    scaledownPeriod: 3600  # 1hr cooldown

This configuration allows the Atlantis instance to scale to zero when not in use while ensuring responses when needed.

Security in Depth

The implementation uses several layers of security:

1. GitHub App Authentication: Stored securely using SealedSecrets

apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  name: atlantis-app

2. OAuth2-Proxy Configuration: Carefully configured session management

OAUTH2_PROXY_COOKIE_CSRF: "true"
OAUTH2_PROXY_COOKIE_CSRF_PER_REQUEST: "true"
OAUTH2_PROXY_COOKIE_REFRESH: "1h"
OAUTH2_PROXY_COOKIE_EXPIRE: "4h"

3. Repository Constraints: Controlled through Atlantis configuration

data:
  repo.yaml: |
    repos:
    - id: /.*/
      apply_requirements: [mergeable, undiverged]

GitOps Integration

The entire setup is managed through ArgoCD, leveraging GitOps principles:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: atlantis
spec:
  project: devops
  syncPolicy:
    automated:
      selfHeal: true
      prune: true

Key Learning Points

This implementation teaches us several valuable lessons about securing Kubernetes services:

1. Selective Authentication: Not all endpoints need authentication. The skip routes configuration shows how to maintain security while allowing necessary automation.
2. Resource Efficiency: Using KEDA for HTTP-based scaling demonstrates how to optimize resource usage without compromising availability.
3. Security Layering: The combination of GitHub App authentication, OAuth2-Proxy, and Zitadel creates comprehensive security without sacrificing usability.
4. Operational Considerations: Using a StatefulSet instead of a Deployment ensures persistent storage for Atlantis data while maintaining scalability.

Advanced Configurations and Considerations

Health Monitoring

The implementation includes comprehensive health checking:

livenessProbe:
  periodSeconds: 30
  httpGet:
    path: /healthz
    port: 80

Resource Management

Careful consideration has been given to resource allocation:

resources:
  requests:
    memory: 512Mi
    cpu: 250m
  limits:
    memory: 512Mi

Persistent Storage

The implementation uses volume claims to ensure data persistence:

volumeClaimTemplates:
- metadata:
    name: atlantis-data
  spec:
    accessModes: ["ReadWriteOnce"]
    resources:
      requests:
        storage: 3Gi

Conclusion

This implementation of OAuth2-Proxy with Atlantis demonstrates how to secure critical infrastructure tools while maintaining their functionality. The combination of KEDA for scaling, dual ingress configurations for different access patterns, and careful security considerations creates a robust and efficient system.

Whether you're implementing similar security patterns for other services or specifically securing Atlantis, these patterns provide a solid foundation for building secure, scalable Kubernetes services.