Performance Profiling Guide¶

Overview¶

DSTA provides comprehensive profiling tools for identifying performance bottlenecks.

Profiling Tools¶

1. Memory Profiling¶

Track memory allocations and usage:

from infrastructure.profiler import profile_memory

@profile_memory
def my_function():
    # Code to profile
    data = [i for i in range(1000000)]
    return data

# Output:
# my_function Memory Usage:
#   Current: 45.2 MB
#   Peak: 76.8 MB

2. CPU Profiling¶

Profile CPU time and function calls:

from infrastructure.profiler import profile_cpu

@profile_cpu
def expensive_calculation():
    # CPU-intensive code
    result = sum(i**2 for i in range(1000000))
    return result

# Output: cProfile statistics sorted by cumulative time

3. Database Query Profiling¶

Track database queries:

from infrastructure.profiler import DatabaseProfiler

with DatabaseProfiler() as profiler:
    candlesticks = Candlestick.objects.filter(
        symbol='BTCUSDT'
    ).order_by('-timestamp')[:100]

# Output:
# Database Profile:
#   Total Queries: 1
#   Total Time: 0.0234s
#   Top 5 Slowest Queries: ...

4. Comprehensive Profiling¶

Profile everything at once:

from infrastructure.profiler import profile_function

@profile_function(memory=True, cpu=True, database=True)
def complex_operation():
    # Multi-aspect profiling
    data = fetch_data()
    results = process_data(data)
    save_results(results)
    return results

5. Line Profiling¶

Line-by-line profiling (requires line_profiler):

pip install line_profiler

from infrastructure.profiler import profile_lines

@profile_lines
def optimizable_function():
    # Each line will be profiled
    a = expensive_operation_1()
    b = expensive_operation_2()
    return a + b

Context Managers¶

For profiling specific code blocks:

from infrastructure.profiler import profile_cpu_context, APICallProfiler

# CPU profiling
with profile_cpu_context():
    expensive_calculation()

# API call profiling  
with APICallProfiler("Binance API"):
    ticker = client.get_ticker(symbol='BTCUSDT')

Performance Reports¶

Generate and save reports:

from infrastructure.profiler import PerformanceReport

report = PerformanceReport.generate_report(
    function_name='backtest_strategy',
    execution_time=45.2,
    memory_usage=128.5,
    query_count=234,
    query_time=3.4
)

PerformanceReport.save_report(report, 'performance_report.json')

Jupyter Notebook¶

Use profiling in notebooks: notebooks/performance_profiling.ipynb

Best Practices¶

Profile in production-like environment
Use representative data sets
Profile both cold and warm caches
Compare before/after optimization
Focus on biggest bottlenecks first

Common Issues¶

High Memory Usage¶

Check for memory leaks
Reduce data structure sizes
Use generators instead of lists
Clear caches periodically

Slow Database Queries¶

Add indexes
Optimize query structure
Use select_related/prefetch_related
Cache frequently accessed data

CPU Bottlenecks¶

Optimize algorithms
Use vectorization (NumPy)
Parallel processing
Cache expensive calculations

Continuous Monitoring¶

Integrate with Prometheus metrics for ongoing monitoring:

from infrastructure.metrics import indicator_calculation_duration_seconds

with indicator_calculation_duration_seconds.labels(indicator='rsi').time():
    result = calculate_rsi(data)

Version History¶

1.0.0 (2025-01-27): Initial profiling implementation