Skip to content

Changelog

All notable changes to the DSTA project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[Unreleased]

Added (2026-03-01) - Makefiles for All Services

  • Makefile Infrastructure: Created standardized Makefiles for all microservices
  • dsta-core-svc: Django-specific commands (migrate, makemigrations, runserver, collectstatic)
  • dsta-cli: CLI-specific commands (cli, cli-help)
  • dsta-qa: Validation commands (validate, validate-full)
  • dsta-data-svc: Data commands with exchange-specific test target
  • dsta-ml-svc: ML commands (train-model, evaluate-model, tensorboard, jupyter)
  • dsta-trading-svc: Trading commands (backtest, paper-trade, live-trade with safety warning)
  • dsta-exchange-svc: Exchange commands (test-connection, get-ticker, get-orderbook, sync-markets)
  • dsta-web: Standard Django commands
  • Root monorepo Makefile: Cross-service commands (install-all, test-all, lint-all, etc.)

Fixed (2026-03-01) - Test Suite Stabilization

  • dsta-ml-svc: Fixed 23 test failures
  • Updated deprecated fillna(method='ffill') to ffill() in feature_engineer.py
  • Removed deprecated verbose parameter from ReduceLROnPlateau
  • Fixed regime detector to always calculate returns for volatility features
  • Added weights_only=False to torch.load calls for PyTorch 2.6 compatibility
  • Fixed CUDA/CPU tensor device mismatch in RL agent

  • Result: 77 tests passing (100%)

  • dsta-trading-svc: Fixed 15 test failures

  • Rewrote test_alerts.py to match actual AlertNotificationSystem API
  • Added filled_at and canceled_at fields to Order dataclass
  • Added rejection_reason field to Order dataclass
  • Fixed logging conflict with reserved 'message' field in alerts.py
  • Fixed multi-strategy veto conflict resolution to properly detect conflicts

  • Result: 103 tests passing (100%)

  • dsta-exchange-svc: Fixed Django configuration errors

  • Added RatelimitsConfig to ratelimits/apps.py
  • Fixed websockets package shadowing issue with lazy imports
  • Cleaned up erroneous test files from other services

  • Result: Django setup works, no tests remain

  • dsta-data-svc: Fixed test infrastructure

  • Updated deprecated fillna(method='ffill') in test_validation.py
  • Fixed OrderSide import to use Direction from backtesting.base
  • Added database availability check in conftest.py
  • Result: 755+ tests passing (integration tests skipped due to database requirements)

Added (2026-03-01) - Comprehensive Test Suite Expansion

  • dsta-core-svc: 162 comprehensive tests (100% passing)
  • Cache management: CacheManager, @cached decorator, cache strategies
  • Prometheus metrics: HTTP, database, trading, backtesting metrics
  • Performance profiling: Memory profiler, CPU profiler, database query profiler

  • Secret management: Environment, Vault, AWS Secrets Manager backends

  • dsta-cli: 75 comprehensive tests (100% passing)

  • Configuration management: APIConfig, AuthConfig, OutputConfig, TradingConfig
  • API client: Authentication, request handling, retry logic

  • Output formatters: Success/error messages, JSON/YAML/table formats

  • dsta-qa: 35 comprehensive tests (100% passing)

  • Created complete analytics modules:
    • SharpeCalculator: Risk-adjusted returns calculation
    • DrawdownAnalyzer: Portfolio drawdown analysis
    • TransactionCostModel: Trading cost modeling by asset class
    • SensitivityAnalyzer: Parameter sensitivity and robustness scoring
    • CapacityScreener: Strategy capacity estimation
    • ValidationPipeline: Complete strategy validation workflow
  • All tests pass with proper module imports

Total Test Count: 272 tests (100% passing)

Fixed (2026-02-28) - Microservices Test Suite

  • Test Infrastructure Updates:
  • Fixed test imports across all microservices to use correct module paths
  • Removed test files that don't belong to each service
  • Created proper test settings for dsta-core-svc using SQLite
  • Fixed import paths from monolithic structure (from src., from dsta., from api.) to microservice structure

  • Updated test fixtures to match actual module APIs

  • Service-Specific Fixes:

  • dsta-core-svc: 40 tests passing (100%) - Created tests for middleware, logging, and config models
  • dsta-cli: 2 tests passing (100%)
  • dsta-qa: 7 tests passing (100%)
  • dsta-data-svc: 755 tests passing - Fixed pandas deprecations, numpy bool types
  • dsta-ml-svc: 54 tests passing - Fixed feature engineering, regime detector tests

  • dsta-trading-svc: 87 tests passing - Fixed module imports, alert system parameters

  • Code Fixes:

  • Pandas fillna method='bfill' deprecated → bfill()
  • Pandas freq='1H' deprecated → freq='1h'
  • Numpy boolean types (np.True_) → Python bool
  • Date comparison datetime64 vs date → pd.Timestamp()
  • AlertManager → AlertNotificationSystem class rename
  • AlertType.STOP_TRIGGERED → STOP_LOSS_HIT enum value

Added (2025-01-27) - Visualization Features

  • TASK-VIZ-002: Portfolio Overview Dashboard
  • Real-time portfolio value and P&L tracking
  • Open positions management with unrealized P&L
  • Portfolio overview and detailed positions views
  • Auto-refresh API endpoint for real-time updates

  • Files: src/api/views/portfolio/, src/api/templates/portfolio/

  • TASK-VIZ-003: Performance Charts

  • Comprehensive analytics charts using Plotly
  • Equity curve with drawdown overlay
  • Returns distribution histogram
  • Monthly returns heatmap
  • Rolling Sharpe ratio visualization
  • Detailed drawdown analysis

  • Files: src/analytics/charts.py, src/api/views/charts/performance.py

  • TASK-VIZ-004: Strategy Comparison

  • Multi-strategy side-by-side comparison
  • Overlay equity curves for visual comparison
  • Export to CSV and JSON formats
  • Comprehensive metrics comparison table

  • Files: src/api/views/strategy/comparison.py, src/api/templates/strategy/

  • TASK-VIZ-005: Interactive Price Charts

  • Candlestick charts with technical indicators
  • Support for SMA, EMA, RSI, Bollinger Bands, MACD
  • Trade entry/exit markers overlay
  • Multiple timeframes (1m to 1d)
  • Volume and indicator subplots

  • Files: src/analytics/price_charts.py, src/api/views/charts/price_chart.py

  • TASK-VIZ-008: Trade Journal

  • Complete trade lifecycle tracking
  • Manual notes and post-trade analysis
  • Tags and categorization system
  • Advanced filtering and search capabilities
  • CSV export functionality
  • Automatic P&L calculation

  • Files: src/api/models.py (TradeJournal), src/api/forms/trade_journal.py, src/api/views/journal/

  • Infrastructure:

  • Base Django template with Bootstrap 5 and responsive design
  • 31 comprehensive tests for all visualization features
  • Complete documentation in docs/VISUALIZATION_FEATURES.md
  • Database migration for TradeJournal model
  • Django admin integration for all models

Technical Details

  • Lines of Code: ~15,000 (views, templates, utilities, tests)
  • Test Coverage: 31 tests covering models, views, forms, and utilities
  • Technologies: Django 4.0, Plotly, Chart.js, Bootstrap 5, pandas, numpy
  • Database: New TradeJournal table with 3 indexes for performance

Added - 2026-01-27

Backtesting Documentation and Tutorials (TASK-BACKTEST-009, TASK-BACKTEST-010)

  • Best Practices Documentation: docs/BACKTESTING_BEST_PRACTICES.md
  • Common pitfalls and solutions (look-ahead bias, survivorship bias, data quality)
  • Comprehensive validation checklist
  • Real-world implementation examples
  • Avoiding overfitting strategies (parameter limits, cross-validation, regularization)
  • Walk-forward analysis guidelines and interpretation
  • Corporate actions handling methodology
  • Bias prevention strategies and detection

  • Academic references and research papers

  • Jupyter Notebooks:

  • notebooks/backtesting_tutorial.ipynb: Basic backtesting tutorial

    • Step-by-step backtest setup and execution
    • Using EnhancedBacktestEngine features
    • Corporate actions handling examples
    • Risk metrics calculation (Sharpe, Sortino, Calmar, VaR, CVaR)
    • Result visualization and analysis
    • Performance report generation

  • notebooks/advanced_backtesting.ipynb: Advanced techniques

    • Parameter optimization (Grid Search and Genetic Algorithms)
    • Walk-forward analysis implementation
    • Bias detection and prevention demonstrations
    • Multi-strategy backtesting and comparison
    • Performance ranking and composite scoring
    • Visualization of optimization results
    • Strategy selection recommendations

Added - 2026-01-26

Backtesting Engine (TASK-BT-003 to BT-007)

  • DataHandler Implementation: HistoricDataHandler for loading and serving historical data
  • Bar-by-bar iteration to prevent lookahead bias
  • Multi-symbol support with timestamp synchronization
  • Date range filtering

  • Efficient in-memory data storage

  • Strategy Framework: BaseStrategy base class with technical indicators

  • Built-in indicators: SMA, EMA, RSI, Bollinger Bands, MACD, ATR
  • Signal emission framework
  • Indicator caching per symbol
  • SMACrossoverStrategy: Moving average crossover example

  • RSIMeanReversionStrategy: RSI-based mean reversion example

  • Portfolio Manager: BacktestPortfolio for position and risk management

  • Position tracking and cash management
  • Multiple position sizing methods (Fixed, Percent Capital, Percent Available)
  • Order generation from signals
  • Performance metrics calculation (return, drawdown, win rate)

  • Equity curve generation

  • Execution Handler: SimulatedExecutionHandler for realistic order simulation

  • Configurable slippage (default 0.05%)
  • Configurable commission (default 0.1%)
  • Support for market, limit, and stop orders

  • Execution statistics tracking

  • Backtesting Engine: BacktestEngine coordinator

  • Event-driven architecture orchestration
  • Progress tracking and logging
  • Comprehensive results generation
  • JSON export functionality

  • Example usage script

  • Test Suite: Comprehensive tests for all backtesting components

  • 35+ test cases covering unit and integration tests

  • Tests for DataHandler, Strategy, Portfolio, Execution, and Engine

  • Documentation:

  • Implementation summary (docs/BACKTESTING_IMPLEMENTATION.md)
  • Module README (src/backtesting/README.md)
  • Usage examples (src/backtesting/example.py)

Added - 2025-11-21

Market Data Collection Module

  • BTC Dominance Collector (TASK-DATA-009): Implemented BTC dominance tracking
  • MarketMetricsCollector class with CoinGecko API integration
  • Fetch BTC dominance percentage with metadata (total market cap, active cryptocurrencies)
  • Rate limiting with configurable delays between requests
  • Storage in MarketMetric model with timestamp tracking
  • CLI command collect_market_metrics with flexible options
  • REST API endpoint for querying historical data

  • Comprehensive error handling and logging

  • Total Market Cap Collector (TASK-DATA-010): Implemented total market cap tracking

  • Fetch total cryptocurrency market capitalization
  • 24-hour volume and market cap change tracking
  • Metadata includes active cryptocurrencies, markets, ICO stats
  • Shared infrastructure with BTC dominance collector

  • CLI and API support for data access

  • Enhanced Fear & Greed Index (TASK-DATA-011): Major enhancements to Fear & Greed Index

  • Historical data storage (up to 365 days)
  • Trend analysis with analyze_fear_greed_trend() method
  • Identifies improving, declining, or stable trends
  • Extreme value detection with configurable thresholds
  • Tracks extreme fear days (0-24) and extreme greed days (76-100)
  • Alert generation for extreme market conditions
  • Backward compatibility with existing implementation

  • CLI flags for trend analysis and extreme value checking

  • Order Book Data Collection (TASK-DATA-012): Implemented comprehensive order book analysis

  • OrderBookCollector class for fetching L2 order book snapshots
  • OrderBookAnalyzer class for market microstructure analysis
  • Support/resistance level detection using volume percentiles
  • Order book imbalance calculation (buy vs sell pressure)
  • Spread metrics (bid-ask spread, mid-price)
  • Order book aggregation for depth visualization
  • Integration with BinanceClient for real-time data
  • CLI command collect_orderbooks with analysis display
  • Storage in OrderBook model with JSON bid/ask arrays
  • REST API endpoint for historical order book queries

Data Collection Module

  • Historical Data Collection (TASK-DATA-001): Implemented bulk historical data downloader
  • CLI command download_historical for downloading OHLCV candlestick data
  • Support for single or multiple symbols (--symbol, --symbols)
  • Date range specification (--start, --end, or --days)
  • Progress tracking with detailed output
  • Resume capability from last downloaded timestamp
  • Overwrite mode for updating existing data
  • Batch processing with configurable batch size
  • JSON config file support for complex configurations
  • Integration with existing BinanceClient

  • Automatic storage in Candlestick model

  • Real-Time Data Collection (TASK-DATA-005): Implemented WebSocket client for Binance

  • BinanceWebSocketClient class for connecting to Binance WebSocket streams
  • Support for kline (candlestick) streams
  • Support for depth (order book) streams
  • Support for trade streams
  • Support for ticker streams
  • Support for book ticker (best bid/ask) streams
  • Automatic reconnection with exponential backoff
  • Ping/pong heartbeat monitoring
  • Proper message parsing and data transformation

  • Stream type enumeration for type safety

  • WebSocket Manager (TASK-DATA-008): Implemented WebSocket manager service

  • WebSocketManager class for coordinating multiple WebSocket connections
  • Automatic data persistence to database
  • Health monitoring and auto-restart capabilities
  • Custom callback support for stream data
  • Subscription tracking across symbols
  • Client status monitoring

  • Support for multiple stream types per symbol

  • Real-Time Streaming CLI: CLI command stream_realtime for real-time data collection

  • Stream data for multiple symbols simultaneously
  • Support for multiple stream types (klines, trades, depth)
  • Configurable intervals for kline streams
  • Configurable update speed for order book depth
  • Optional database storage (--no-save flag)
  • JSON config file support
  • Graceful shutdown handling with signal handlers
  • Detailed logging of received data

Testing

  • Comprehensive test suite for historical data download command
  • Date parsing tests
  • Symbol extraction tests
  • Date range calculation tests
  • Database persistence tests
  • Duplicate handling tests

  • Overwrite functionality tests

  • Comprehensive test suite for WebSocket functionality

  • Client initialization tests
  • Subscription tests for all stream types
  • Message parsing tests for all data formats
  • WebSocketManager tests
  • Subscription tracking tests
  • Client status monitoring tests

Changed

  • None

Deprecated

  • None

Removed

  • None

Fixed

  • None

Security

  • None

[0.1.0] - 2025-11-21

Added

  • Initial project setup
  • Django application structure
  • Database models (Candlestick, Trade, OrderBook, MarketMetric)
  • Exchange client abstraction layer
  • BinanceClient implementation
  • Health check endpoints
  • Configuration management system
  • Logging framework with JSON formatting
  • Error handling middleware
  • CI/CD pipeline with GitHub Actions
  • Comprehensive test suite structure

For detailed task tracking, see TASKS.md. For product requirements, see PRD.md.