Price Prediction with LSTM Neural Networks¶

This document describes the LSTM-based price prediction system for cryptocurrency trading in the DSTA project.

Overview¶

The price prediction module uses Long Short-Term Memory (LSTM) neural networks to forecast future price movements based on historical market data and technical indicators. It supports multiple prediction modes and provides a flexible training framework.

Architecture¶

Model Components¶

1. LSTM Layers: Multi-layer LSTM for temporal pattern recognition
2. Attention Mechanism: Optional attention for improved long-term dependencies
3. Batch Normalization: Stabilizes training and improves convergence
4. Fully Connected Layers: Maps LSTM output to predictions

Prediction Modes¶

• Price Prediction: Direct future price forecasting
• Return Prediction: Percentage return forecasting
• Direction Prediction: Binary up/down classification

Quick Start¶

Basic Usage¶

from src.ml.models.price_predictor import PricePredictor
from src.ml.trainers.price_trainer import PriceTrainer
import pandas as pd

# Load your data
data = pd.read_csv('btc_hourly.csv')

# Initialize predictor
predictor = PricePredictor(
    input_size=50,              # Number of features
    hidden_size=128,            # LSTM hidden size
    num_layers=2,               # Number of LSTM layers
    prediction_horizon=24,      # Predict 24 hours ahead
    sequence_length=60,         # Use 60 hours of history
    prediction_mode='price'     # Predict actual prices
)

# Train the model
history = predictor.fit(
    data,
    epochs=100,
    validation_split=0.2,
    early_stopping_patience=10
)

# Make predictions
predictions = predictor.predict(data)

# Save trained model
predictor.save('models/price_predictor_btc.pt')

Advanced Training with Cross-Validation¶

from src.ml.trainers.price_trainer import PriceTrainer
from src.ml.features import FeatureEngineer

# Set up feature engineering
feature_engineer = FeatureEngineer(normalize=True)

# Initialize trainer
trainer = PriceTrainer(
    model_class=PricePredictor,
    model_params={
        'input_size': 50,
        'hidden_size': 256,
        'num_layers': 3,
        'dropout': 0.3,
        'use_attention': True
    },
    feature_engineer=feature_engineer
)

# Train with time-series cross-validation
results = trainer.train_with_cv(
    data,
    n_splits=5,
    epochs=100
)

# Get best model
best_model = trainer.get_best_model()

# Evaluate performance
for i, result in enumerate(results):
    print(f"Fold {i+1}:")
    print(f"  RMSE: {result.metrics['rmse']:.4f}")
    print(f"  R²: {result.metrics['r2']:.4f}")
    print(f"  MAPE: {result.metrics['mape']:.2f}%")

Hyperparameter Tuning¶

# Define parameter grid
param_grid = {
    'hidden_size': [128, 256, 512],
    'num_layers': [2, 3, 4],
    'dropout': [0.2, 0.3, 0.4],
    'learning_rate': [1e-4, 3e-4, 1e-3]
}

# Search for best parameters
best_params, all_results = trainer.hyperparameter_search(
    data,
    param_grid=param_grid,
    n_splits=3,
    epochs=50
)

print(f"Best parameters: {best_params}")

Walk-Forward Validation¶

# Perform walk-forward validation
wf_results = trainer.walk_forward_validation(
    data,
    window_size=1000,       # Training window size
    step_size=100,          # Step forward by 100 samples
    retrain_frequency=5,    # Retrain every 5 steps
    epochs=50
)

print(f"Walk-forward RMSE: {wf_results['metrics']['rmse']:.4f}")
print(f"Walk-forward R²: {wf_results['metrics']['r2']:.4f}")

Features and Data Preparation¶

Feature Engineering¶

The model works best with engineered features:

from src.ml.features import FeatureEngineer

# Initialize feature engineer
fe = FeatureEngineer(normalize=True, handle_missing='ffill')

# Generate features
data_with_features = fe.fit_transform(data)

# Available feature categories:
# - Technical indicators (RSI, MACD, Bollinger Bands, etc.)
# - Price patterns (candlestick patterns)
# - Volume features (OBV, VWAP)
# - Time features (hour, day, seasonality)

Data Requirements¶

Minimum required columns: - close: Closing price - open: Opening price - high: High price - low: Low price - volume: Trading volume

Optional but recommended: - Technical indicators - Market sentiment features - Order book features - External market data

Model Parameters¶

Core Parameters¶

• input_size (int): Number of input features
• hidden_size (int): Size of LSTM hidden state (default: 128)
• num_layers (int): Number of LSTM layers (default: 2)
• prediction_horizon (int): Steps ahead to predict (default: 1)
• sequence_length (int): Length of input sequences (default: 60)
• dropout (float): Dropout rate for regularization (default: 0.2)

Training Parameters¶

• learning_rate (float): Adam optimizer learning rate (default: 0.001)
• batch_size (int): Training batch size (default: 32)
• epochs (int): Maximum training epochs
• early_stopping_patience (int): Patience for early stopping (default: 10)
• validation_split (float): Fraction for validation (default: 0.2)

Advanced Parameters¶

• use_attention (bool): Enable attention mechanism (default: True)
• prediction_mode (str): Type of prediction ('price', 'return', 'direction')

Performance Metrics¶

The model provides comprehensive evaluation metrics:

Regression Metrics (price/return modes)¶

• MAE: Mean Absolute Error
• MSE: Mean Squared Error
• RMSE: Root Mean Squared Error
• R²: Coefficient of determination
• MAPE: Mean Absolute Percentage Error

Classification Metrics (direction mode)¶

• Accuracy: Prediction accuracy
• Precision: Positive predictive value
• Recall: True positive rate
• F1-Score: Harmonic mean of precision and recall

Integration with Backtesting¶

from src.backtesting.engine import EnhancedBacktestEngine
from src.ml.models.price_predictor import PricePredictor

# Load trained model
predictor = PricePredictor(input_size=50)
predictor.load('models/price_predictor_btc.pt')

# Create prediction-based strategy
class MLPredictionStrategy(BaseStrategy):
    def __init__(self, bars, events_queue, symbol_list, predictor):
        super().__init__(bars, events_queue, symbol_list, 'ml_prediction')
        self.predictor = predictor

    def calculate_signals(self, event):
        # Get recent data
        recent_data = self.get_recent_bars(100)

        # Make prediction
        prediction = self.predictor.predict(recent_data)

        # Generate signal based on prediction
        if prediction > current_price * 1.01:  # Expect 1% rise
            self.emit_signal(symbol, SignalType.LONG)
        elif prediction < current_price * 0.99:  # Expect 1% fall
            self.emit_signal(symbol, SignalType.SHORT)

# Run backtest with ML strategy
engine = EnhancedBacktestEngine(
    symbol_list=['BTCUSDT'],
    strategy_class=MLPredictionStrategy,
    strategy_params={'predictor': predictor},
    # ... other params
)

results = engine.run()

Best Practices¶

1. Data Quality¶

• Use clean, validated data
• Handle missing values appropriately
• Remove outliers or handle them explicitly
• Ensure sufficient data (minimum 1000+ samples)

2. Feature Engineering¶

• Use domain knowledge to create features
• Include lagged features for temporal context
• Normalize/standardize features
• Remove highly correlated features

3. Model Selection¶

• Start with simple models (2 layers, 128 hidden size)
• Use cross-validation to prevent overfitting
• Monitor validation loss carefully
• Consider ensemble methods for production

4. Hyperparameter Tuning¶

• Use grid search or random search
• Optimize on validation set, not training set
• Consider computational cost vs. performance gain
• Use early stopping to prevent overfitting

5. Production Deployment¶

• Retrain periodically with new data
• Monitor prediction performance in production
• Have fallback strategies
• Log predictions for analysis

Troubleshooting¶

Poor Performance¶

1. Check data quality: Ensure no data leakage
2. Increase model capacity: More layers or hidden size
3. Adjust sequence length: Try longer/shorter sequences
4. Add more features: Include additional technical indicators
5. Try different prediction modes: Direction might work better than price

Overfitting¶

1. Increase dropout: Try 0.3-0.5 dropout rate
2. Use regularization: Add L2 regularization
3. Reduce model complexity: Fewer layers or smaller hidden size
4. More training data: Collect additional historical data
5. Early stopping: Stop when validation loss plateaus

Training Issues¶

1. Gradient explosion: Reduce learning rate, use gradient clipping
2. Slow convergence: Increase learning rate, check feature scaling
3. Memory issues: Reduce batch size or sequence length
4. NaN losses: Check for invalid data, reduce learning rate

API Reference¶

PricePredictor Class¶

class PricePredictor:
    def __init__(self, input_size, hidden_size=128, ...)
    def fit(self, data, features=None, epochs=100, ...)
    def predict(self, data, features=None, return_confidence=False)
    def save(self, path)
    def load(self, path)

PriceTrainer Class¶

class PriceTrainer:
    def __init__(self, model_class, model_params=None, ...)
    def train_with_cv(self, data, n_splits=5, ...)
    def train_single(self, data, ...)
    def hyperparameter_search(self, data, param_grid, ...)
    def walk_forward_validation(self, data, ...)
    def get_best_model(self)

Examples¶

See notebooks/price_prediction_model.ipynb for detailed examples including: - Data preparation and feature engineering - Model training and evaluation - Hyperparameter tuning - Walk-forward validation - Integration with backtesting - Production deployment example

References¶

• LSTM Networks
• Attention Mechanisms
• Time Series Forecasting Best Practices
• Financial Time Series Prediction

Support¶

For issues or questions: 1. Check the troubleshooting section 2. Review example notebooks 3. Check unit tests for usage examples 4. Open an issue on GitHub