Annex B - Value for money analysis

The methodology for assessing the Value for Money (VfM) of the policy change was developed specifically for the Pilot. Normally, appraisal of transport interventions is undertaken before they are in place, using standard tools and techniques to predict the impact of options and following delivery. This is followed up by evaluation of outputs and outcomes – did the appraisal accurately capture the actual outcomes of the intervention.

For the Pilot of the removal of peak fares, an almost unique situation was faced in that a “natural experiment” is being undertaken i.e. the purpose of the Pilot is to specifically test the impact it has on behaviour. This allowed a different approach to be taken. As is standard practice, the predicted outcomes of the Pilot were assessed but the trial means that these predicted outcomes can now be adjusted to see what the actual impact is (see below). This is particularly useful in the context of the changing patterns of demand post pandemic and the fact that such a significant change in fares is unusual.

The approach was as follows:

• Use the Transport Model for Scotland (TMfS) to assess the impact of the Pilot with the expectation that the model would not necessarily capture the actual impact.
• Calibrate the results of the model to the actual impacts of the Pilot, check that the detail of the results matched what was seen in reality and use these calibrated results to estimate the Value for Money in the usual way.

Simplistically, whist existing rail users benefit from reduced fares, the full impact of the Pilot is measured by capturing the value to those who switch to rail from other modes or undertake additional journeys. The approach captures the wellbeing (or welfare) gained by those who switch to rail as well as the impact on existing users, the loss of revenue from existing users (and the gain from new users) to ScotRail, the impact on bus patronage, as well as an estimate of CO2 emissions impacts from changes in car use.

The model runs on an annual basis, so the inputs are adjusted to reflect this. For example, the initial additional costs incurred by ScotRail are annualised.

As such, the assessment of VfM for the Pilot combines the appraisal and evaluation methodologies within Scottish Transport Appraisal Guidance (STAG) to give the most accurate possible assessment of the impact.

Of particular importance was the establishment of a robust “counterfactual” – what would have happened if the Pilot had not taken place. This was difficult over the interim period due to significant weather disruptions in October and November and the impact of the Christmas and New Year holiday period in December and early January especially when combined with changing patterns of travel demand – more leisure and fewer commuting trips, post-pandemic. Significant further work was undertaken to ensure that the counterfactual used over the full extent of the Pilot (Scenario M2) is as robust as possible.