Build a Forecast Accuracy Tracker
This guide explains how to build a forecast accuracy tracker using multiple Model Reef models over time and actuals imported from your accounting system or historical data sources.
Forecast accuracy is not a single model feature. It is an analytical process that compares what the model predicted in the past with what actually happened.
Before you start
You should have:
A process of saving periodic snapshots of your model, for example monthly or quarterly.
An Actuals model that reflects realised financial performance.
The ability to export key metrics from your models.
If you do not yet snapshot models, establish a simple naming convention first, such as Company - Forecast - Jan, Company - Forecast - Feb and so on.
What you will build
A time series of forecasts for selected metrics from prior model versions.
A set of realised actual values for the same metrics.
A view of forecast error over time and by horizon.
Define which metrics and horizons to track
Decide which metrics you care about, for example:
Revenue
Gross profit
EBITDA
Cash balance
Capex
For each metric, decide which forecast horizons matter, for example:
One month ahead
Three months ahead
Twelve months ahead
Clarity on metrics and horizons keeps the tracker focused.
Snapshot your forecast models periodically
On a regular cadence (monthly or quarterly):
Save a copy of your current forecast model with a date specific name.
Do not modify historical snapshots once created. They represent your state of knowledge at that time.
Continue this over time so you accumulate a sequence of forecast models.
Each snapshot contains what the model predicted at that point.
Extract forecast and actual values
For each snapshot model and the Actuals model:
Export selected metrics for relevant periods.
Store them in an external table or analysis file with columns such as:
Snapshot date
Target period date
Forecast value
Actual value
This creates the raw data for accuracy analysis.
Calculate forecast error
In your analysis file, calculate error measures, for example:
Absolute error: Forecast minus Actual
Percentage error: (Forecast minus Actual) divided by Actual
Absolute percentage error where sign is not important
You can then summarise these by:
Horizon (for example average twelve month ahead error)
Metric
Snapshot date (to see if forecasting is improving)
Interpret and act on accuracy insights
Use the forecast accuracy tracker to:
Identify systematic bias, for example:
Persistent over estimation of revenue
Underestimation of costs
Improve driver design and assumptions in future models
Communicate uncertainty ranges rather than precise point forecasts where appropriate
Model Reef provides the modelling engine. The accuracy tracker is a layer on top that improves how you use that engine over time.
Check your work
Forecast snapshots are clearly labelled and not edited retrospectively.
Actuals are mapped consistently with the metrics being tracked.
Error measures are computed reliably and interpreted with care.
Insights from accuracy analysis feed back into assumption and model design.
Troubleshooting
Difficult to align forecast and actual periods
Ensure that model calendars and accounting periods align, or adjust exported data to a common period framework before comparison.
Large percentage errors in low volume periods
Focus on periods and metrics where the absolute level is meaningful or use appropriate thresholds.
Accuracy appears to worsen after major changes
This can indicate that new product lines or regimes need separate modelling approaches. Adjust your forecasting design, not just your expectations.
Related guides
Last updated
Was this helpful?