Deployment Guide¶

This guide covers deployment of the DSTA trading system to production environments.

Overview¶

DSTA can be deployed in multiple configurations:

Development: Local development with hot-reload
Staging: Pre-production testing environment
Production: Live trading environment
Backtesting: Dedicated backtesting server

Architecture¶

┌─────────────────────────────────────────────────────────────┐
│                    Load Balancer (Optional)                  │
└─────────────────────────┬───────────────────────────────────┘
                          │
        ┌─────────────────┼─────────────────┐
        │                 │                 │
        ▼                 ▼                 ▼
┌──────────────┐  ┌──────────────┐  ┌──────────────┐
│  API Server  │  │  API Server  │  │  API Server  │
│   (Django)   │  │   (Django)   │  │   (Django)   │
└──────┬───────┘  └──────┬───────┘  └──────┬───────┘
       │                 │                 │
       └─────────────────┼─────────────────┘
                         │
        ┌────────────────┼────────────────┐
        │                │                │
        ▼                ▼                ▼
┌──────────────┐  ┌──────────────┐  ┌──────────────┐
│  PostgreSQL  │  │    Redis     │  │   Celery     │
│   Database   │  │    Cache     │  │   Workers    │
└──────────────┘  └──────────────┘  └──────────────┘

Quick Start¶

Prerequisites¶

Docker 20.10+
Docker Compose 2.0+
Git
4GB RAM minimum (8GB recommended)
50GB disk space

Clone Repository¶

git clone https://github.com/minhdqdev/dsta.git
cd dsta

Configure Environment¶

# Copy example environment file
cp .env_files/base.env .env_files/prod.env

# Edit configuration
nano .env_files/prod.env

Required Environment Variables:

# Database Configuration
POSTGRES_DB=dsta_prod
POSTGRES_USER=dsta_user
POSTGRES_PASSWORD=<strong-password>
POSTGRES_HOST=postgres
POSTGRES_PORT=5432

# Redis Configuration
REDIS_HOST=redis
REDIS_PORT=6379
REDIS_PASSWORD=<redis-password>

# Django Configuration
SECRET_KEY=<generate-with-django>
DEBUG=False
ALLOWED_HOSTS=yourdomain.com,www.yourdomain.com
ENVIRONMENT=production

# Logging
LOG_LEVEL=WARNING
LOG_DIR=/var/log/dsta
USE_JSON_LOGGING=true

# Exchange API Keys (Encrypted)
BINANCE_API_KEY=<your-api-key>
BINANCE_API_SECRET=<your-api-secret>

# Email Configuration (for alerts)
EMAIL_HOST=smtp.gmail.com
EMAIL_PORT=587
EMAIL_HOST_USER=your-email@gmail.com
EMAIL_HOST_PASSWORD=<app-password>
EMAIL_USE_TLS=True

# Monitoring
SENTRY_DSN=<your-sentry-dsn>  # Optional

Generate SECRET_KEY¶

python -c "from django.core.management.utils import get_random_secret_key; print(get_random_secret_key())"

Deploy with Docker Compose¶

# Set environment profile
export PROFILE=prod

# Start services
cd deploy
docker-compose --profile full up -d

# Check status
docker-compose ps

# View logs
docker-compose logs -f api-server

Initialize Database¶

# Run migrations
docker-compose exec api-server python manage.py migrate

# Create superuser
docker-compose exec api-server python manage.py createsuperuser

# Collect static files
docker-compose exec api-server python manage.py collectstatic --noinput

Verify Deployment¶

# Health check
curl http://localhost:8000/health

# Expected response:
# {"status": "healthy", "database": "connected", "redis": "connected"}

# Admin interface
open http://localhost:8000/admin

Docker Profiles¶

DSTA uses Docker Compose profiles for different deployment scenarios:

Minimal Profile¶

Minimum services for development:

docker-compose --profile minimal up -d

Services: - api-server - postgres - redis

Dev Profile¶

Development with debugging:

docker-compose --profile dev up -d

Services: - Minimal profile + - bot-listener - bot-notifier - dsta-sync-login-debug (debugpy enabled)

Full Profile¶

Production deployment:

docker-compose --profile full up -d

Services: - All services - Data collection workers - Telegram bots - Background tasks

Testing Profile¶

Isolated testing environment:

docker-compose --profile testing up -d

Services: - api-server - postgres (test database) - redis

Production Deployment¶

System Requirements¶

Minimum: - 2 CPU cores - 4GB RAM - 50GB SSD - Ubuntu 20.04+ or Debian 11+

Recommended: - 4 CPU cores - 8GB RAM - 100GB SSD - Dedicated server or VPS

Server Setup¶

# Update system
sudo apt update && sudo apt upgrade -y

# Install Docker
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

# Add user to docker group
sudo usermod -aG docker $USER
newgrp docker

# Install Docker Compose
sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

# Verify installation
docker --version
docker-compose --version

Security Hardening¶

1. Firewall Configuration:

# Install UFW
sudo apt install ufw

# Default policies
sudo ufw default deny incoming
sudo ufw default allow outgoing

# Allow SSH
sudo ufw allow 22/tcp

# Allow HTTP/HTTPS (if using reverse proxy)
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

# Enable firewall
sudo ufw enable

2. SSL/TLS Certificate (with Let's Encrypt):

# Install certbot
sudo apt install certbot python3-certbot-nginx

# Obtain certificate
sudo certbot --nginx -d yourdomain.com -d www.yourdomain.com

# Auto-renewal (crontab)
sudo crontab -e
# Add: 0 3 * * * certbot renew --quiet

3. Nginx Reverse Proxy:

# /etc/nginx/sites-available/dsta
upstream dsta_backend {
    server 127.0.0.1:8000;
}

server {
    listen 80;
    server_name yourdomain.com www.yourdomain.com;
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 ssl http2;
    server_name yourdomain.com www.yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;

    # Security headers
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    add_header X-Frame-Options "DENY" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;

    # Rate limiting
    limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;
    limit_req zone=api_limit burst=20 nodelay;

    client_max_body_size 10M;

    location / {
        proxy_pass http://dsta_backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Timeouts
        proxy_connect_timeout 60s;
        proxy_send_timeout 60s;
        proxy_read_timeout 60s;
    }

    location /static/ {
        alias /var/www/dsta/static/;
        expires 1y;
        add_header Cache-Control "public, immutable";
    }

    location /media/ {
        alias /var/www/dsta/media/;
        expires 1y;
    }

    # Health check endpoint (no rate limit)
    location /health {
        proxy_pass http://dsta_backend;
        access_log off;
    }
}

Enable configuration:

sudo ln -s /etc/nginx/sites-available/dsta /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx

Database Configuration¶

PostgreSQL Optimization for production:

# Edit postgresql.conf
docker-compose exec postgres bash
vi /var/lib/postgresql/data/postgresql.conf

# Memory Settings
shared_buffers = 2GB                    # 25% of RAM
effective_cache_size = 6GB              # 75% of RAM
work_mem = 64MB                         # Per operation
maintenance_work_mem = 512MB            # For VACUUM, etc.

# Checkpoint Settings
checkpoint_timeout = 15min
checkpoint_completion_target = 0.9
wal_buffers = 16MB

# Connection Settings
max_connections = 100

Backup Configuration:

# Create backup script
cat > /home/dsta/backup-db.sh << 'EOF'
#!/bin/bash
BACKUP_DIR=/var/backups/dsta
DATE=$(date +%Y%m%d_%H%M%S)
mkdir -p $BACKUP_DIR

# Backup database
docker-compose exec -T postgres pg_dump -U dsta_user dsta_prod | gzip > $BACKUP_DIR/db_backup_$DATE.sql.gz

# Keep only last 7 days
find $BACKUP_DIR -name "db_backup_*.sql.gz" -mtime +7 -delete

echo "Backup completed: $DATE"
EOF

chmod +x /home/dsta/backup-db.sh

# Schedule daily backups
crontab -e
# Add: 0 2 * * * /home/dsta/backup-db.sh >> /var/log/dsta-backup.log 2>&1

Redis Configuration¶

# Redis configuration
# deploy/redis.conf
maxmemory 1gb
maxmemory-policy allkeys-lru
save 900 1
save 300 10
save 60 10000
appendonly yes

Update docker-compose.yaml:

redis:
  image: redis:8
  volumes:
    - ./redis.conf:/usr/local/etc/redis/redis.conf
  command: redis-server /usr/local/etc/redis/redis.conf

Monitoring and Alerting¶

Application Monitoring with Sentry¶

# src/dsta/settings.py
import sentry_sdk
from sentry_sdk.integrations.django import DjangoIntegration

if not DEBUG and SENTRY_DSN:
    sentry_sdk.init(
        dsn=SENTRY_DSN,
        integrations=[DjangoIntegration()],
        environment=ENVIRONMENT,
        traces_sample_rate=0.1,
        send_default_pii=False
    )

System Monitoring¶

Install Prometheus + Grafana:

# Add to docker-compose.yaml
prometheus:
  image: prom/prometheus:latest
  volumes:
    - ./prometheus.yml:/etc/prometheus/prometheus.yml
    - prometheus_data:/prometheus
  ports:
    - "9090:9090"

grafana:
  image: grafana/grafana:latest
  volumes:
    - grafana_data:/var/lib/grafana
  ports:
    - "3000:3000"
  environment:
    - GF_SECURITY_ADMIN_PASSWORD=<strong-password>

Prometheus Configuration:

# deploy/prometheus.yml
global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'dsta'
    static_configs:
      - targets: ['api-server:8000']

  - job_name: 'postgres'
    static_configs:
      - targets: ['postgres-exporter:9187']

  - job_name: 'redis'
    static_configs:
      - targets: ['redis-exporter:9121']

Health Checks¶

# Create health check script
cat > /home/dsta/health-check.sh << 'EOF'
#!/bin/bash
HEALTHCHECK_URL="http://localhost:8000/health"
ALERT_EMAIL="admin@yourdomain.com"

response=$(curl -s -o /dev/null -w "%{http_code}" $HEALTHCHECK_URL)

if [ $response -ne 200 ]; then
    echo "ALERT: DSTA health check failed (HTTP $response)" | mail -s "DSTA Health Alert" $ALERT_EMAIL
    exit 1
fi

echo "Health check passed"
exit 0
EOF

chmod +x /home/dsta/health-check.sh

# Schedule every 5 minutes
crontab -e
# Add: */5 * * * * /home/dsta/health-check.sh

Log Management¶

Centralized Logging:

# Add to docker-compose.yaml
loki:
  image: grafana/loki:latest
  volumes:
    - loki_data:/loki
  ports:
    - "3100:3100"

promtail:
  image: grafana/promtail:latest
  volumes:
    - /var/log:/var/log
    - ./promtail-config.yml:/etc/promtail/config.yml
  command: -config.file=/etc/promtail/config.yml

Log Rotation:

# /etc/logrotate.d/dsta
/var/log/dsta/*.log {
    daily
    missingok
    rotate 14
    compress
    delaycompress
    notifempty
    create 0640 dsta dsta
    sharedscripts
    postrotate
        docker-compose -f /home/dsta/deploy/docker-compose.yaml kill -s HUP api-server
    endscript
}

Backup and Recovery¶

Full System Backup¶

#!/bin/bash
# /home/dsta/full-backup.sh

BACKUP_DIR=/var/backups/dsta
DATE=$(date +%Y%m%d_%H%M%S)

# Create backup directory
mkdir -p $BACKUP_DIR

# Stop services (optional, for consistency)
cd /home/dsta/deploy
# docker-compose stop

# Backup database
docker-compose exec -T postgres pg_dump -U dsta_user dsta_prod | gzip > $BACKUP_DIR/db_$DATE.sql.gz

# Backup volumes
docker run --rm -v dsta_postgres_data:/data -v $BACKUP_DIR:/backup alpine tar czf /backup/postgres_data_$DATE.tar.gz -C /data .
docker run --rm -v dsta_redis_data:/data -v $BACKUP_DIR:/backup alpine tar czf /backup/redis_data_$DATE.tar.gz -C /data .

# Backup configuration
tar czf $BACKUP_DIR/config_$DATE.tar.gz ../.env_files ../deploy

# Restart services (if stopped)
# docker-compose start

# Upload to remote storage (optional)
# aws s3 sync $BACKUP_DIR s3://your-bucket/dsta-backups/

echo "Backup completed: $DATE"

Disaster Recovery¶

Restore from Backup:

#!/bin/bash
# /home/dsta/restore-backup.sh

BACKUP_FILE=$1
BACKUP_DIR=/var/backups/dsta

if [ -z "$BACKUP_FILE" ]; then
    echo "Usage: $0 <backup-date>"
    echo "Example: $0 20240115_020000"
    exit 1
fi

cd /home/dsta/deploy

# Stop services
docker-compose down

# Restore database
gunzip < $BACKUP_DIR/db_$BACKUP_FILE.sql.gz | docker-compose exec -T postgres psql -U dsta_user dsta_prod

# Restore volumes
docker run --rm -v dsta_postgres_data:/data -v $BACKUP_DIR:/backup alpine sh -c "cd /data && tar xzf /backup/postgres_data_$BACKUP_FILE.tar.gz"
docker run --rm -v dsta_redis_data:/data -v $BACKUP_DIR:/backup alpine sh -c "cd /data && tar xzf /backup/redis_data_$BACKUP_FILE.tar.gz"

# Restore configuration
tar xzf $BACKUP_DIR/config_$BACKUP_FILE.tar.gz -C /home/dsta

# Start services
docker-compose up -d

echo "Restore completed from backup: $BACKUP_FILE"

Scaling¶

Horizontal Scaling¶

Multiple API Servers:

# docker-compose.yaml
api-server-1:
  <<: *django
  container_name: dsta-api-1

api-server-2:
  <<: *django
  container_name: dsta-api-2

api-server-3:
  <<: *django
  container_name: dsta-api-3

nginx:
  image: nginx:latest
  volumes:
    - ./nginx-loadbalancer.conf:/etc/nginx/nginx.conf
  ports:
    - "80:80"
  depends_on:
    - api-server-1
    - api-server-2
    - api-server-3

Nginx Load Balancer Configuration:

# deploy/nginx-loadbalancer.conf
upstream dsta_cluster {
    least_conn;
    server api-server-1:8000 weight=1 max_fails=3 fail_timeout=30s;
    server api-server-2:8000 weight=1 max_fails=3 fail_timeout=30s;
    server api-server-3:8000 weight=1 max_fails=3 fail_timeout=30s;
}

server {
    listen 80;

    location / {
        proxy_pass http://dsta_cluster;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

Database Scaling¶

Read Replicas:

postgres-primary:
  image: postgres:17
  environment:
    - POSTGRES_DB=dsta_prod
  volumes:
    - postgres_primary_data:/var/lib/postgresql/data

postgres-replica:
  image: postgres:17
  environment:
    - POSTGRES_DB=dsta_prod
  command: |
    bash -c "
    until pg_basebackup -h postgres-primary -D /var/lib/postgresql/data -U replication -Fp -Xs -R; do
      sleep 5
    done
    postgres
    "
  volumes:
    - postgres_replica_data:/var/lib/postgresql/data
  depends_on:
    - postgres-primary

Troubleshooting¶

Common Issues¶

1. Service Won't Start:

# Check logs
docker-compose logs api-server

# Common causes:
# - Database not ready
# - Missing environment variables
# - Port already in use

# Solution: Ensure postgres is healthy
docker-compose exec postgres pg_isready

2. Database Connection Errors:

# Test connection
docker-compose exec api-server python manage.py dbshell

# Check credentials
docker-compose exec api-server env | grep POSTGRES

3. High Memory Usage:

# Check container stats
docker stats

# Limit memory per service
services:
  api-server:
    mem_limit: 2g
    mem_reservation: 1g

4. Disk Space Full:

# Check disk usage
df -h

# Clean Docker
docker system prune -a --volumes

# Clean logs
docker-compose logs --tail=0 -f

Performance Tuning¶

Database Connection Pooling:

# src/dsta/settings.py
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'CONN_MAX_AGE': 600,  # Connection pooling
        'OPTIONS': {
            'connect_timeout': 10,
            'options': '-c statement_timeout=30000'  # 30 seconds
        }
    }
}

Gunicorn Configuration:

# src/gunicorn_config.py
workers = 4  # 2 * CPU cores + 1
worker_class = 'gevent'
worker_connections = 1000
max_requests = 1000
max_requests_jitter = 50
timeout = 30
keepalive = 2

Resources¶

Docker Documentation: https://docs.docker.com/
Docker Compose: https://docs.docker.com/compose/
PostgreSQL Tuning: https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server
Nginx Best Practices: https://nginx.org/en/docs/
Django Deployment: https://docs.djangoproject.com/en/stable/howto/deployment/

Support¶

For deployment issues:

Check this documentation
Review Docker logs: docker-compose logs
Check health endpoint: curl http://localhost:8000/health
Open an issue with the deployment label