Home/Product/Performance & Backtesting
Deep Dive

Performance tracking as a first-class feature

How do you know if forecasts are any good? We built performance tracking into the core of Nordict, not as an afterthought, but as the foundation of trust.

Walk-forward validation mirrors real deployment
Every forecast is tracked against actual outcomes
Performance is versioned alongside models
No cherry-picking, all results visible
See metrics

Performance Dashboard

Last 30 days • Model v2.4.1

Live

Directional Accuracy

68%

+2.1%

Calibration Score

0.92

+0.04

Avg Confidence

71%

-1.3%

Forecasts Made

12,847

+847

Rolling Accuracy

7-day MA

12 weeks agoNow
All systems operational
Updated 2 min ago

Evaluation metrics

The metrics that actually matter.

We track multiple metrics because no single number tells the whole story. Each measures a different aspect of forecast quality.

Walk-forward validation

Testing that mirrors reality.

Walk-forward validation is the gold standard for evaluating time-series forecasts. It simulates how the model would have performed if deployed in the past.

Walk-Forward Process

PastPresent
Training
Test
1
Training
Test
2
Training
Test
3
Training data (model learns)
Test data (never seen before)
Windows roll forward in time

No future data leakage

The model never sees validation data during training. This mimics real conditions where you can't peek at tomorrow's prices.

Multiple test periods

Instead of one backtest, we run many across different market conditions bull, bear, sideways, volatile.

Realistic constraints

We include transaction costs, slippage estimates, and data delays. Results aren't idealized.

Regime-aware evaluation

Performance is broken down by market regime so you know where the model excels and where it struggles.

Historical performance views

Inspect performance across time and regimes.

Performance isn't static. See how forecasts have performed over different time periods and market conditions.

Directional Accuracy Over Time

Rolling 30-day accuracy vs 50% baseline

80%70%60%50%40%
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
Nov
Dec
Above target (≥65%)
Below target (<65%)

By Market Regime

Performance varies by conditions

Trending Up

3,247 forecasts

74%

Trending Down

2,891 forecasts

71%

Ranging

4,102 forecasts

62%

High Volatility

1,847 forecasts

58%

Performance is strongest in trending markets and weaker during high volatility. This is expected—and honestly reported.

Live vs backtest tracking

Separating simulation from reality.

Backtest results and live performance are tracked separately, and clearly labeled. No mixing, no confusion about what's real.

Performance Comparison

Model v2.4.1

Directional Accuracy

Backtest

69%

Live

68%

Delta

-1%

Calibration Score

Backtest

0.91

Live

0.92

Delta

+0.01

Brier Score

Backtest

0.17

Live

0.18

Delta

+0.01

75% Band Coverage

Backtest

74%

Live

76%

Delta

+2%

Live performance aligned with backtest

All metrics within expected variance

Why results can differ

Data availability

Backtest

Complete historical data, no gaps

Live

Real-time feeds with occasional delays

Market conditions

Backtest

Known regimes, can stratify analysis

Live

Unknown future, regime shifts possible

Execution reality

Backtest

Assumed perfect execution

Live

Actual latency and slippage

Data Timeline

Jan 2023
Live Start
Now
Backtest period (18 months)
Live tracking (6 months)