Secret Management Guide¶

This document describes the secret management system for the DSTA trading platform.

Overview¶

The DSTA secret management system provides secure storage and retrieval of sensitive credentials like API keys, database passwords, and encryption keys. It supports multiple backends with automatic fallback:

HashiCorp Vault (Recommended for production)
AWS Secrets Manager (For AWS deployments)
Environment Variables (Fallback/Development)

Architecture¶

Application Code
      │
      ▼
SecretManager
      │
      ├─► Vault Backend (Primary)
      │
      ├─► AWS Secrets Manager (Secondary)
      │
      └─► Environment Variables (Fallback)

Quick Start¶

Basic Usage¶

from infrastructure.secrets import get_secret, get_exchange_credentials

# Get a single secret
api_key = get_secret('exchange/binance/api_key')

# Get exchange credentials
creds = get_exchange_credentials('binance')
client = BinanceClient(creds['api_key'], creds['api_secret'])

# With fallback default
db_password = get_secret('database/password', default='dev_password')

# Force specific backend
vault_secret = get_secret('path/to/secret', backend='vault')

Advanced Usage¶

from infrastructure.secrets import SecretManager

# Create manager with specific backend
manager = SecretManager(backend='vault')

# Get single secret
api_key = manager.get_secret('exchange/binance/api_key')

# Get all secrets from a path
binance_secrets = manager.get_secrets('exchange/binance')
# Returns: {'api_key': '...', 'api_secret': '...', 'testnet': 'false'}

# Get exchange credentials
creds = manager.get_exchange_credentials('gateio')

Backend Configuration¶

1. HashiCorp Vault¶

Recommended for production environments.

Installation¶

# Install hvac library
pip install hvac

Configuration¶

Set environment variables:

export VAULT_ADDR=https://vault.your-domain.com
export VAULT_TOKEN=your-vault-token
export VAULT_NAMESPACE=dsta  # Optional, defaults to 'dsta'

Vault Setup¶

Enable KV v2 secrets engine:

vault secrets enable -path=secret kv-v2

Store secrets:

# Store exchange credentials
vault kv put secret/exchange/binance \
  api_key=your_api_key \
  api_secret=your_api_secret \
  testnet=false

vault kv put secret/exchange/gateio \
  api_key=your_api_key \
  api_secret=your_api_secret

# Store database credentials
vault kv put secret/database/postgres \
  host=localhost \
  port=5432 \
  user=dsta \
  password=secure_password \
  database=dsta

Create access policy:

# dsta-policy.hcl
path "secret/data/exchange/*" {
  capabilities = ["read"]
}

path "secret/data/database/*" {
  capabilities = ["read"]
}

Apply policy:

vault policy write dsta-policy dsta-policy.hcl

Create token with policy:

vault token create -policy=dsta-policy -ttl=720h

Docker Integration¶

Add to docker-compose.prod.yml:

services:
  api-server:
    environment:
      VAULT_ADDR: https://vault.your-domain.com
      VAULT_TOKEN: ${VAULT_TOKEN}
      VAULT_NAMESPACE: dsta

2. AWS Secrets Manager¶

Best for AWS-hosted deployments.

Installation¶

# Install boto3
pip install boto3

Configuration¶

export AWS_REGION=us-east-1
export AWS_ACCESS_KEY_ID=your_access_key
export AWS_SECRET_ACCESS_KEY=your_secret_key

Or use IAM roles (recommended for EC2/ECS):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue",
        "secretsmanager:DescribeSecret"
      ],
      "Resource": "arn:aws:secretsmanager:us-east-1:*:secret:dsta/*"
    }
  ]
}

Create Secrets¶

Using AWS CLI:

# Create exchange credentials
aws secretsmanager create-secret \
  --name exchange/binance \
  --secret-string '{"api_key":"your_key","api_secret":"your_secret"}'

aws secretsmanager create-secret \
  --name exchange/gateio \
  --secret-string '{"api_key":"your_key","api_secret":"your_secret"}'

# Create database password
aws secretsmanager create-secret \
  --name database/postgres/password \
  --secret-string 'your_secure_password'

Terraform Example¶

resource "aws_secretsmanager_secret" "binance_credentials" {
  name = "dsta/exchange/binance"
  description = "Binance API credentials for DSTA"
}

resource "aws_secretsmanager_secret_version" "binance_credentials" {
  secret_id = aws_secretsmanager_secret.binance_credentials.id
  secret_string = jsonencode({
    api_key    = var.binance_api_key
    api_secret = var.binance_api_secret
  })
}

3. Environment Variables¶

For development and fallback.

Configuration¶

Create .env file:

# Exchange API Keys
BINANCE_API_KEY=your_binance_api_key
BINANCE_API_SECRET=your_binance_api_secret
GATEIO_API_KEY=your_gateio_api_key
GATEIO_API_SECRET=your_gateio_api_secret

# Database
POSTGRES_PASSWORD=your_db_password
DATABASE_URL=postgresql://user:pass@localhost/dsta

# Other secrets
SECRET_KEY=django-secret-key
REDIS_PASSWORD=redis_password

Load with:

# Using python-dotenv
pip install python-dotenv

# In code
from dotenv import load_dotenv
load_dotenv()

Secret Organization¶

Recommended Secret Paths¶

secret/
├── exchange/
│   ├── binance/
│   │   ├── api_key
│   │   └── api_secret
│   ├── gateio/
│   │   ├── api_key
│   │   └── api_secret
│   └── huobi/
│       ├── api_key
│       └── api_secret
├── database/
│   ├── postgres/
│   │   ├── host
│   │   ├── port
│   │   ├── user
│   │   ├── password
│   │   └── database
│   └── redis/
│       └── password
├── app/
│   ├── secret_key
│   └── encryption_key
├── monitoring/
│   ├── sentry_dsn
│   └── grafana_api_key
└── notifications/
    ├── slack_webhook_url
    └── email_password

Security Best Practices¶

1. Secret Rotation¶

Rotate secrets regularly (recommended: every 90 days).

from infrastructure.secrets import SecretRotationManager, SecretManager

manager = SecretManager(backend='vault')
rotator = SecretRotationManager(manager)

# Manual rotation
# rotator.rotate_secret('exchange/binance/api_key', new_key)

# Scheduled rotation (future implementation)
# rotator.schedule_rotation('exchange/binance/api_key', days=90)

Rotation Checklist: - [ ] Generate new credentials on exchange - [ ] Update secret in Vault/AWS - [ ] Restart affected services - [ ] Verify functionality - [ ] Revoke old credentials - [ ] Document rotation in audit log

2. Access Control¶

Vault Policies:

# Read-only policy for API servers
path "secret/data/exchange/*" {
  capabilities = ["read"]
}

# Admin policy for rotation
path "secret/data/exchange/*" {
  capabilities = ["create", "read", "update", "delete"]
}

AWS IAM:

{
  "Effect": "Allow",
  "Action": ["secretsmanager:GetSecretValue"],
  "Resource": "arn:aws:secretsmanager:*:*:secret:dsta/exchange/*"
}

3. Encryption at Rest¶

Vault: Encrypted by default
AWS Secrets Manager: Uses KMS encryption
Environment: Use encrypted volumes

4. Audit Logging¶

Enable audit logs:

Vault:

vault audit enable file file_path=/var/log/vault/audit.log

AWS:

aws secretsmanager update-secret \
  --secret-id exchange/binance \
  --kms-key-id alias/dsta-secrets

5. Least Privilege¶

Use read-only tokens for applications
Separate admin tokens for rotation
Use time-limited tokens (TTL)
Revoke unused tokens

Integration Examples¶

Django Settings¶

# settings.py
from infrastructure.secrets import get_secret

# Database
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'HOST': get_secret('database/postgres/host', default='localhost'),
        'PORT': get_secret('database/postgres/port', default='5432'),
        'USER': get_secret('database/postgres/user', default='dsta'),
        'PASSWORD': get_secret('database/postgres/password'),
        'NAME': get_secret('database/postgres/database', default='dsta'),
    }
}

# Secret key
SECRET_KEY = get_secret('app/secret_key')

# Redis
REDIS_PASSWORD = get_secret('database/redis/password', default='')

Exchange Clients¶

# trading/exchanges/binance.py
from infrastructure.secrets import get_exchange_credentials
from binance.client import Client

def get_binance_client():
    creds = get_exchange_credentials('binance')
    return Client(creds['api_key'], creds['api_secret'])

Celery Tasks¶

# tasks.py
from celery import shared_task
from infrastructure.secrets import get_secret

@shared_task
def send_notification():
    webhook_url = get_secret('notifications/slack_webhook_url')
    # Send notification

Monitoring¶

Health Checks¶

from infrastructure.secrets import SecretManager

def check_secret_backend():
    """Health check for secret backend."""
    manager = SecretManager()

    if not manager.backend.is_available():
        return {
            'status': 'unhealthy',
            'backend': manager.backend_name,
            'error': 'Backend not available'
        }

    # Test secret retrieval
    try:
        manager.get_secret('health/check', default='ok')
        return {'status': 'healthy', 'backend': manager.backend_name}
    except Exception as e:
        return {
            'status': 'unhealthy',
            'backend': manager.backend_name,
            'error': str(e)
        }

Metrics¶

Track secret operations:

from prometheus_client import Counter, Histogram

secret_requests = Counter(
    'secret_requests_total',
    'Total secret requests',
    ['backend', 'status']
)

secret_latency = Histogram(
    'secret_request_duration_seconds',
    'Secret request duration',
    ['backend']
)

Troubleshooting¶

Common Issues¶

1. Vault Not Available¶

Error: Vault not available: Connection refused

Solution:

# Check Vault address
echo $VAULT_ADDR

# Verify Vault is running
curl $VAULT_ADDR/v1/sys/health

# Check token
vault token lookup

2. AWS Credentials Not Found¶

Error: Unable to locate credentials

Solution:

# Configure AWS credentials
aws configure

# Or use environment variables
export AWS_ACCESS_KEY_ID=your_key
export AWS_SECRET_ACCESS_KEY=your_secret

# Or use IAM role (EC2/ECS)

3. Secret Not Found¶

Error: Secret not found: exchange/binance/api_key

Solution:

# List secrets in Vault
vault kv list secret/exchange

# Read specific secret
vault kv get secret/exchange/binance

# Create if missing
vault kv put secret/exchange/binance api_key=xxx api_secret=yyy

4. Permission Denied¶

Error: Permission denied

Solution:

# Check Vault token capabilities
vault token capabilities secret/exchange/binance

# Update policy if needed
vault policy write dsta-policy policy.hcl

# Create new token with correct policy
vault token create -policy=dsta-policy

Migration Guide¶

From Environment Variables to Vault¶

Export existing secrets:

# Create migration script
cat > migrate_secrets.sh <<'EOF'
#!/bin/bash
vault kv put secret/exchange/binance \
  api_key=$BINANCE_API_KEY \
  api_secret=$BINANCE_API_SECRET

vault kv put secret/exchange/gateio \
  api_key=$GATEIO_API_KEY \
  api_secret=$GATEIO_API_SECRET
EOF

chmod +x migrate_secrets.sh
./migrate_secrets.sh

Update application config:

# Change backend preference
manager = SecretManager(backend='vault')

Test:

# Verify secrets work
python -c "from infrastructure.secrets import get_secret; print(get_secret('exchange/binance/api_key'))"

Remove old environment variables:

# After verification
unset BINANCE_API_KEY BINANCE_API_SECRET

Testing¶

Unit Tests¶

# tests/test_secrets.py
import pytest
from unittest.mock import patch, MagicMock
from infrastructure.secrets import SecretManager, VaultBackend

def test_environment_backend():
    """Test environment variable backend."""
    with patch.dict('os.environ', {'TEST_KEY': 'test_value'}):
        manager = SecretManager(backend='env')
        assert manager.get_secret('test/path/test_key') == 'test_value'

def test_vault_backend():
    """Test Vault backend."""
    with patch('hvac.Client') as mock_client:
        mock_client.return_value.secrets.kv.v2.read_secret_version.return_value = {
            'data': {'data': {'api_key': 'test_key'}}
        }

        manager = SecretManager(backend='vault')
        key = manager.get_secret('exchange/binance/api_key')
        assert key == 'test_key'

def test_fallback_chain():
    """Test automatic backend fallback."""
    # Vault not available, should fall back to environment
    with patch.dict('os.environ', {'TEST_KEY': 'env_value'}):
        manager = SecretManager()  # Auto-select backend
        # Should use environment as fallback
        value = manager.get_secret('test/path/test_key', default='default')
        assert value in ('env_value', 'default')

Support¶

For issues or questions: - Check troubleshooting section - Review Vault/AWS documentation - Open issue on GitHub - Contact security team

Version History¶

1.0.0 (2025-01-27): Initial secret management implementation