How To Replace Static Passwords With Dynamic, Ephemeral Credentials Using Vault And Sidecar Injection To Automate Database Security In Kubernetes Workloads

|

Modern Kubernetes workloads often rely on databases for stateful operations, but many deployments still use static database passwords stored in environment variables, configuration files, or Kubernetes Secrets. Static credentials introduce major security risks: they can be leaked, reused indefinitely, and are difficult to rotate consistently across distributed workloads.

A more secure approach is to replace static credentials with dynamic, ephemeral database credentials generated on demand. By using HashiCorp Vault with sidecar injection, Kubernetes workloads can automatically receive short-lived credentials without developers manually managing secrets.

This article explains how to replace static passwords with dynamic credentials using Vault and sidecar injection, including architecture design, configuration steps, and coding examples.

Why Static Passwords Are Dangerous

Static credentials create several security problems: 

1. Long-lived exposure
If a password leaks, attackers can use it indefinitely until it is manually rotated.

2. Manual rotation
Rotating database credentials across microservices is slow and error-prone.

3. Secret sprawl
Passwords often end up in:

    • Git repositories
    • CI/CD pipelines
    • Environment variables
    • Configuration files

4. Lack of auditing
It is difficult to track which service used a credential.

Dynamic credentials solve these problems by generating temporary access tokens tied to a workload identity.

What Are Dynamic Ephemeral Credentials

Dynamic credentials are automatically generated usernames and passwords with: 

    • Short Time-To-Live (TTL)
    • Automatic expiration
    • Automatic revocation
    • Least-privilege permissions

Instead of storing: 

DB_USER=appuser
DB_PASSWORD=SuperSecret123

Vault generates credentials like:

username: v-token-readonly-abc123
password: KJhsd7s9s8sd98s
ttl: 1 hour

After expiration, the credentials are automatically revoked.

Architecture Overview

The solution involves four main components: 

    1. Kubernetes Pod
    2. Vault Server
    3. Vault Agent Sidecar
    4. Database

Flow: 

    1. Pod starts
    2. Vault sidecar authenticates using Kubernetes ServiceAccount
    3. Vault generates dynamic credentials
    4. Sidecar writes credentials to a shared volume
    5. Application reads credentials
    6. Credentials rotate automatically

Example architecture: 

Application Container
        |
Shared Volume (/vault/secrets)
        |
Vault Agent Sidecar
        |
Vault Server
        |
Database

Installing Vault on Kubernetes

Vault can be installed using Helm: 

helm repo add hashicorp https://helm.releases.hashicorp.com
helm install vault hashicorp/vault \
  --set "server.dev.enabled=true"

For production: 

helm install vault hashicorp/vault \
  --set "injector.enabled=true"

This enables the Vault Agent Injector which automatically injects sidecars.

Enabling Kubernetes Authentication

Vault must trust Kubernetes.

Enable auth method: 

vault auth enable kubernetes

Configure Kubernetes auth: 

vault write auth/kubernetes/config \
    token_reviewer_jwt="$SA_JWT" \
    kubernetes_host="$KUBE_HOST" \
    kubernetes_ca_cert="$KUBE_CA_CERT"

This allows Vault to authenticate Pods.

Configuring the Database Engine

Enable the database secrets engine: 

vault secrets enable database

Example configuration for PostgreSQL: 

vault write database/config/postgres \
    plugin_name=postgresql-database-plugin \
    allowed_roles="app-role" \
    connection_url="postgresql://{{username}}:{{password}}@postgres:5432/appdb?sslmode=disable" \
    username="vaultadmin" \
    password="vaultadminpassword"

Vault uses the admin account to generate users dynamically.

Creating Dynamic Credential Roles

Define a role that generates credentials: 

vault write database/roles/app-role \
    db_name=postgres \
    creation_statements="
    CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}';
    GRANT SELECT,INSERT,UPDATE ON ALL TABLES IN SCHEMA public TO \"{{name}}\";
    " \
    default_ttl="1h" \
    max_ttl="24h"

This role: 

    • Creates database users
    • Sets expiration
    • Limits privileges

Creating Vault Policies

Create policy: 

path "database/creds/app-role" {
  capabilities = ["read"]
}

Apply policy:

vault policy write app-policy app-policy.hcl

Creating Kubernetes Roles

Bind Kubernetes ServiceAccount to Vault role: 

vault write auth/kubernetes/role/app-role \
    bound_service_account_names=app-sa \
    bound_service_account_namespaces=default \
    policies=app-policy \
    ttl=1h

Now Pods using app-sa can request credentials.

Configuring Sidecar Injection

Vault Agent Injector uses Pod annotations.

Example Pod YAML: 

apiVersion: v1
kind: Pod
metadata:
  name: app-pod
  annotations:
    vault.hashicorp.com/agent-inject: "true"
    vault.hashicorp.com/role: "app-role"
    vault.hashicorp.com/agent-inject-secret-db: "database/creds/app-role"
spec:
  serviceAccountName: app-sa

  containers:
  - name: app
    image: myapp:latest

Vault automatically injects a sidecar.

Rendering Credentials to Files

Add template annotation: 

vault.hashicorp.com/agent-inject-template-db: |
  {{- with secret "database/creds/app-role" -}}
  username={{ .Data.username }}
  password={{ .Data.password }}
  {{- end }}

Vault writes file: 

/vault/secrets/db

Example contents: 

username=v-token-app123
password=SDs8sdf78sdf

Application Code Example (Python)

Example application reading credentials dynamically: 

import psycopg2
import time

def read_credentials():
    with open('/vault/secrets/db') as f:
        lines = f.readlines()

    creds = {}
    for line in lines:
        key, value = line.strip().split('=')
        creds[key] = value

    return creds

while True:
    creds = read_credentials()

    conn = psycopg2.connect(
        host="postgres",
        database="appdb",
        user=creds['username'],
        password=creds['password']
    )

    cur = conn.cursor()
    cur.execute("SELECT NOW()")
    print(cur.fetchone())

    conn.close()

    time.sleep(60)

The application automatically picks up rotated credentials.

Automatic Credential Rotation

Vault Agent renews leases automatically.

When credentials expire: 

    1. Vault generates new credentials
    2. Sidecar updates file
    3. Application reloads credentials

No manual intervention required.

Example TTL settings: 

default_ttl = "1h"
max_ttl = "24h"

Short TTL improves security.

Advanced Template Configuration

More advanced template: 

vault.hashicorp.com/agent-inject-template-db: |
  {{- with secret "database/creds/app-role" -}}
  {
    "username": "{{ .Data.username }}",
    "password": "{{ .Data.password }}",
    "ttl": "{{ .LeaseDuration }}"
  }
  {{- end }}

Generated file: 

{
  "username": "v-app-123",
  "password": "sdkfjsd8fs",
  "ttl": "3600"
}

JSON format simplifies parsing.

Handling Credential Reloading

Applications should reload credentials safely.

Example (Node.js): 

const fs = require('fs');
const { Client } = require('pg');

function getCredentials() {
    const content = fs.readFileSync('/vault/secrets/db', 'utf8');
    const lines = content.split('\n');

    let creds = {};

    lines.forEach(line => {
        if(line){
            let parts = line.split('=');
            creds[parts[0]] = parts[1];
        }
    });

    return creds;
}

async function queryDB() {

    const creds = getCredentials();

    const client = new Client({
        host: 'postgres',
        database: 'appdb',
        user: creds.username,
        password: creds.password
    });

    await client.connect();

    const res = await client.query('SELECT NOW()');

    console.log(res.rows);

    await client.end();
}

setInterval(queryDB, 60000);

This approach ensures continuous operation.

Security Benefits

Dynamic credentials improve security significantly.

1. Zero hardcoded secrets

No credentials stored in:

    • Git
    • Images
    • ConfigMaps

2. Automatic rotation

Passwords rotate without downtime.

3. Fine-grained access

Each Pod gets unique credentials.

4. Automatic revocation

When Pod terminates: 

    • Lease revoked
    • User deleted

5. Audit logging

Vault logs: 

    • Credential generation
    • Usage
    • Revocation

Production Best Practices

Use short TTL values

Recommended: 

15m – 1h

Short lifetimes reduce risk.

Use least privilege roles

Avoid: 

GRANT ALL PRIVILEGES

Prefer: 

GRANT SELECT,INSERT

Enable TLS everywhere

Secure: 

    • Vault communication
    • Database connections
    • Kubernetes API

Separate roles per service

Example: 

orders-service-role
billing-service-role
analytics-service-role

This limits blast radius.

Use Readiness Probes

Ensure application waits for credentials: 

readinessProbe:
  exec:
    command:
    - cat
    - /vault/secrets/db

Common Pitfalls

1. Credential caching 

Applications sometimes cache credentials forever.

Always reload periodically.

2. Connection pools

Pools may hold expired credentials.

Solutions: 

    • Pool refresh
    • TTL-aware pools

3. Vault rate limits

Large clusters may overload Vault.

Solutions: 

    • Horizontal scaling
    • Performance standby nodes
    • Caching agent

4. Missing renewals

If the sidecar dies: 

    • Credentials expire
    • App fails

Use: 

restartPolicy: Always

Migration Strategy from Static Secrets

Step-by-step approach:

Step 1

Keep static secrets but introduce Vault.

Step 2

Test dynamic credentials in staging.

Step 3

Switch production gradually.

Step 4

Remove Kubernetes Secrets.

Comparison: Static vs Dynamic Credentials

Feature Static Dynamic
Rotation Manual Automatic
TTL Unlimited Configurable
Revocation Manual Automatic
Security Medium High
Auditability Low High

Dynamic credentials clearly provide superior security.

Conclusion

Replacing static database passwords with dynamic, ephemeral credentials using Vault and sidecar injection represents one of the most impactful security upgrades for Kubernetes environments. Static credentials are inherently fragile because they create long-lived trust relationships that attackers can exploit. Once exposed, static passwords often remain valid for weeks or months, dramatically increasing the risk of unauthorized database access.

Dynamic credentials fundamentally change this model by introducing short-lived, automatically generated identities tied directly to Kubernetes workloads. Each Pod receives its own unique database user with precisely scoped permissions and a limited lifetime. Even if credentials are compromised, they quickly expire and become useless, significantly reducing the attack window.

The integration of Vault Agent sidecars makes this system practical and scalable. Instead of requiring developers to implement complex authentication flows, sidecar injection provides a transparent mechanism for secret delivery. Applications simply read credentials from files, while Vault handles authentication, rotation, and revocation behind the scenes. This separation of concerns allows development teams to focus on business logic while security policies are centrally enforced.

Operationally, this architecture improves reliability as well as security. Automated credential rotation eliminates the risks associated with manual password updates and reduces the chance of outages caused by expired secrets. Fine-grained Vault policies ensure that each microservice receives only the access it requires, implementing a strong least-privilege model across the cluster.

From a compliance perspective, Vault introduces comprehensive audit logging and traceability. Security teams gain visibility into when credentials are issued, which workloads use them, and when they expire. This level of accountability is extremely difficult to achieve with traditional Kubernetes Secrets or environment-variable-based credentials.

Sidecar-based secret injection also aligns naturally with Kubernetes design principles. The ephemeral nature of Pods matches the short-lived nature of dynamic credentials, creating a consistent and secure lifecycle for both compute and identity resources.

In modern cloud-native environments, database credentials should no longer be treated as static configuration. Instead, they should be dynamic infrastructure components generated on demand and destroyed automatically. Vault and sidecar injection provide a mature and production-proven approach for achieving this goal.

Organizations adopting this pattern gain: 

    • Stronger zero-trust security
    • Reduced secret exposure
    • Automated credential lifecycle management
    • Improved auditability
    • Simplified operations
    • Better compliance posture

As Kubernetes deployments continue to scale, the use of dynamic ephemeral credentials will become the standard approach for securing databases and other sensitive services. Implementing Vault with sidecar injection today positions organizations for a future where no static secrets exist inside containerized workloads, dramatically reducing one of the most common and dangerous attack vectors in cloud-native infrastructure.