Rail 2014 Consultation Analysis

4 Achieving reliability, performance and service quality

Q9 - Under the franchise, should we incentivise good performance or only penalise poor performance

Popular themes

1. - Reward good performance and penalise bad performance
2. - Only reward good performance
3. - Only penalise poor performance

Key themes

4.1 More than half of the respondents thought it important to use both 'carrots and sticks', rewarding good performance whilst penalising poor performance. A greater number of respondents indicated 'both' rather than one or the other.

4.2 However, a number of responses pointed out that incentivisation is current 'best practice' and is frequently used throughout the business world. Comments on this included the assertion that franchise holders would be tempted to 'raise their game' when there was a financial prize available, rather than 'slip into mediocrity'. Essentially, these views suggested incentives would help provide a better rail service to the public.

4.3 It should also be noted, however, that some respondents thought good performance should be regarded as a prerequisite of the contract. As such, there should be no requirement to pay extra public money as an incentive and therefore Transport Scotland should only penalise instances of poor performance. Related to this opinion was the notion that good performance will generate revenue for the franchise holder, this is an incentive in itself. Some respondents who stated this opinion thought that if incentives were to be offered, they should only be awarded for examples of superior performance far beyond the norm expected.

"The Scottish Government, Transport Scotland and the franchise holder have a responsibility to ensure that Scotland has an effective, reliable and efficient rail network. Where performance standards are set, these must be met and failure to attain these standards should be penalised where that failure is attributable to the franchisee. Incentives should be available for exceeding minimum standards of performance, not for merely meeting them."
Scottish Chambers of Commerce

"Good performance should not be rewarded by the taxpayer/government. The increasing attraction of passenger numbers and hence revenue should be sufficient reward in itself."
Public response

4.4 As an alternative to straight financial rewards, there were suggestions that performance targets should be written into the contract, services and targets should be reviewed on an annual basis, and incentives should be provided for continuous improvements rather than passing a threshold. Another alternative respondents suggested was to provide incentives and penalties which were not overtly financial, for example extensions and early curtailments of contracts.

"The franchise could be made for a certain length but with a level of flexibility attached to it, where Transport Scotland can reduce or increase the franchise length by for example 20%, depending on poor or good performance."
Fife Council

4.5 Members of the public tended to provide more wide ranging views in terms of penalties to impose upon operators. Suggestions included financial penalties which should be repaid to passengers rather than repaid as a fine to Transport Scotland. Numerous respondents thought that operators should be made to refund passengers for service cancellations or part refund those which were excessively late.

4.6 Whilst not prescribing the incentivisation methodology, a number of comments suggested that any incentives and penalties should be documented within the contract and include a system for meeting and failing targets. These should be, made more transparent with the public kept aware of performance targets and how regularly these are met.

4.7 Another theme which arose was that incentives should only be given to franchise operators for improvements arising as a direct result of the service the franchisee provided. For example, respondents thought that infrastructure improvements such as electrification will lead to improvements in services, however the franchise holder should not profit from improvements paid from the public purse.

4.8 Similarly, when commenting on penalties there were a number of respondents who noted that penalties should only be used in a fair way and not for instances which are out-with the franchise operator's control, such as poorly maintained infrastructure or extreme weather delays.

4.9 Whilst individuals often simply stated their view, stakeholder organisations tended to provide a justified argument for their position whether it be incentives, penalties or both, considering the legal and financial impacts upon the public purse and operator.

"Ideally, there should be both a mechanism to reward good performance and penalise poor performance. This could take the form of flexibility built into the length of the franchise which could be lengthened or shortened by a specific amount depending on overall performance levels."
East Lothian Council

"The current performance regime is relatively complicated and requires a high level of resource. We are working with The Office of Rail Regulation to consider the appropriateness of the current regime and how we can reduce the cost of operating the regime while maintaining the incentives for the industry to deliver the reliable and punctual network required by passengers."
Network Rail

Organisational and individual responses

4.10 Responses were broadly similar between both organisations and individuals across each of the key themes identified. The top three themes were consistent in terms of content and order between both respondent groups. A significantly larger proportion of individuals than organisations reported that a system of both rewards and penalties should be implemented. It is important to note that this system of rewards and penalties was still the largest single theme to emerge from all organisations. In particular it was strongly supported by local authorities, economic and business groups and tourism and recreation groups.

4.11 Outside the three most significant themes, other differences were observed which suggested that organisations were less inclined to support further financial reward.

4.12 Whilst there were small numbers of responses which warned that penalties can often be unfair or unjust, these submissions were only made by organisations rather than members of the public.

Q10 - Should the performance regime be aligned with actual routes or service groups, or should there be one system for the whole of Scotland?

Popular themes

1. - Performance regime should be aligned with actual routes
2. - There should be one system for the whole of Scotland

Only two significant popular themes

Key themes

4.13 This question seemed to split the majority of respondents into two main groups which when combined accounted for almost 75% of all responses to this question. Slightly more than half of these respondents thought that the performance regime should be aligned with actual routes, whilst the remaining number thought that there should be one system for the whole of Scotland.

4.14 Respondents who thought that the performance regime should be aligned with actual routes did so for a number of reasons. The main reason identified was that it would become easier to identify the poorly performing routes allowing services on those routes to be improved. Respondents also suggested that urban and rural services are incomparable so performance regimes would need to be tailored to their environment.

4.15 Respondents thought that using this system, operators could be penalised more easily for poorly performing routes as opposed to the statistics and reports being lost in the overall results. Respondents were concerned that good performances could be obscured by poorly performing routes. They also thought that one size does not fit all and each route must be judged on its own criteria.

"It needs to be locally aligned. Bad performance on one or two routes should be penalised, not lost in an overall national average."
Public Response

4.16 The conflicting argument presented by those respondents who thought that the performance regime should be for the whole of Scotland was mainly two-fold.

4.17 Firstly respondents thought with one system there is a greater chance of obtaining service consistency and equality across Scotland without the possibility of standards differing based on the type of route.

"Overall it is important that there is a universal performance regime for the whole of Scotland to ensure equity and consistency in the access to and delivery of services."
Campaign to Open Blackford Railway-station Again

4.18 Secondly respondents thought that with a Scotland wide performance regime it would be easier to fully integrate rail with all transport modes and achieve a modal shift from cars to public transport.

"Scotland needs a fully integrated and efficient rail service and network that can provide a viable alternative to both air and car travel (internal and cross boundary) and contribute to Scotland's environmental targets. A consistent standard of service across the whole rail service is essential. It is important that rural services are reliable and punctual as they provide a vital commuting route for residents and support tourism and other industries. Rail users may have other modal connections at the end of their rail journeys; therefore, punctuality and reliability are important across the service."
Scottish Environment Protection Agency (SEPA)

Organisational and individual responses

4.19 Responses were broadly similar between both organisations and individuals across the two key themes identified. Proportionately, slightly more individuals than organisations commented that the performance regime should be aligned with actual routes. Slightly more organisations, however, noted that services should be aligned by service group.

4.20 Trade unions recommended that one system should be provided for the whole of Scotland; this recommendation was echoed by a significant proportion of local government groups, passenger groups and tourism groups.

4.21 Economic and business groups, local authorities and community councils tended to favour aligning the performance regime with actual routes.

Q11 - How can we make the performance regime more aligned with passenger issues?

Popular themes

1. - Through passenger consultation/surveys. This would include publishing the results of these consultations/surveys, which could make use of a national performance website to do both
2. - Making the regime punctuality based. Would use a reviewed system that re-defines late running, counting running time to all stations and remove the practice of skipping stations
3. - Making the regime based on a number of factors such as punctuality, train facilities, train condition and connectivity

Key themes

4.22 The vast majority of respondents thought that the best method to align the performance regime with passenger issues was through a series of passenger surveys and consultations. Respondents thought that as customers are essential to the success of services then they should be consulted in order to be able to identify and prioritise the issues that concern them the most. Similarly, there was a feeling that passenger groups should also be involved in this.

"Passengers should be more involved in setting the performance regime. This can be done through random sampling on-journey surveys identifying the key issues facing passengers in their day to day travel."
The Highland Council

"more involvement with consumer and campaign groups such as Passenger Focus, RAGES and CRAG, etc."
Campaign for Rail Action Group

4.23 A significant number of respondents expanded upon this position by insisting that these passenger views/issues should be published, thus ensuring that the operator takes these views into account and are acted upon as opposed to 'brushing under the carpet'.

4.24 A sizeable number of passengers thought that using an interactive survey online could make this process easier and allow for comments to be more readily collated and processed.

4.25 Around a fifth of all respondents thought that in order to align the performance regime with passenger issues, the regime should be punctuality based. This would involve a reviewed system which changes the definition of a late service to that of a service being late if it does not arrive on time and not within a ten-minute window. Many respondents were aggrieved to learn that services are not classed as being late if they arrive within five or ten minutes of the scheduled time.

4.26 Respondents also thought that the late running of a service should be accounted for at each of the scheduled stops and not just registered upon arrival at the final destination. In addition to this point, respondents also wanted to raise concerns over services skipping stations if they are running late to make up time. Respondents thought that reliability of a service was more important, thus trains should stop at the stations instead of missing them in order to forgo any punishment for arriving late at the final destination.

4.27 Some respondents thought that the performance regime should also focus on more factors than just punctuality, including for example connectivity, train facilities such as the provision and condition of toilets, and the capacity of services.

4.28 Further suggestions for aligning the performance regime with passenger issues included, involving feedback from local communities, passenger groups and passenger representatives on boards; customer compensation for late running/cancelled services and also for missing connecting services; and improved customer service to ensure information and questions are provided and answered with reasons provided for certain issues.

Organisational and individual responses

4.29 Individuals felt very strongly that public opinion is fundamental to aligning the performance regime to passenger issues, and that use should be made of surveys and consultations. Trade unions and passenger groups tended to agree with individuals on this point.

4.30 Rail groups, local government groups and community councils, however, were more inclined to believe that a number of factors should be looked at to align performance with passenger issues.

4.31 Local authorities and tourism groups tended to favour a punctuality based system, which reviews the methodology for classing trains as late running only after 10 minutes.

Q12 - What should the balance be between journey times and performance?

Popular themes

1. - System for measuring journey times/timetabling needs to be carefully refined. Do away with in-built recovery time (timetable padding)
2. - Reliability is the key factor
3. - As both are essential and closely linked, there needs to be a balance

Key themes

4.32 Opinion was very much divided on this question. Responses varied between the importance of journey times, performance and reliability.

4.33 Where journey times were concerned, respondents thought that the system for measuring these should be investigated, especially with respect to timetabling and building in recovery times. Respondents thought that services should run to published times with timetables created by being realistic over achievable journey times.

4.34 Respondents also stated that journey times should be reduced to as short a time as possible. A proposal to extend journey times should not be allowed just so that the operator could improve its performance statistics.

4.35 A significant number of respondents thought that performance is the more important factor and should take precedence over journey time. This includes issues such as time-keeping of services, condition of rolling stock and facilities. Respondents stated that the majority of customers assume that journey times already take into account the possibility of delay and that the performance of the service is thus more important.

"Performance should dominate. People would rather arrive on time, than shave a few minutes off a journey which then might run late."
Public Response

4.36 Some respondents also thought that performance and journey times should have a balance between the two, as they are essentially closely linked. In fact some responses stated that they thought they were one and the same, in that journey times should be part of the overall performance.

4.37 A considerable number of respondents thought that the key to gaining a balance between performance and journey times is reliability. Respondents stated that they do not want to see the provision of some services to the detriment of other services. The example often given was that the Edinburgh Glasgow Improvement Programme (EGIP) should not lead to a reduction in services in the north of Glasgow. They also stated that reliability includes frequency of cancellations, delays and late-running services on routes. Respondents thought that the reliability of services is more important to passengers as they expect a high standard of service. Therefore in order for performance to be measured, services first of all have to be reliable.

"Punctuality/Reliability to be key indicator of performance and most important issue for consideration."
Public Response

4.38 There was also some opposition to implementing further changes in the rail timetable to improve flexibility. It would appear that both the public and organisations are wary of the appropriateness of 'increased flexibility'.

"We strongly oppose making timetable adjustments increasing journey times to increase flexibility. There is already pathing and recovery time in the system. Furthermore, inserting additional time would facilitate a less disciplined approach to punctuality, and quite possibly make performance worse rather than better. Increased journey times conflicts with the Scottish Government's strategic objectives."
City of Edinburgh Council

4.39 Individuals and members of the public also commented on the current system for measuring performance, specifically that journey time performance is based on the destination station. A number of respondents thought that intermediate legs of the journey should also be subjected to performance monitoring.

"There needs to be greater regulation of journey times and I would like some mechanism to control them within the franchise contract. I am concerned that franchisees are able to extend the advertised journey times in order to meet their performance targets. This is a practice which I understand is common in the airline industry but obviously given the nature of rail travel it is more important that trains run quickly and on time. Passengers expect train times to be precise."
Public Response

"Timetables must be set realistically so that they can be relied upon but the current practice of building in 'excessive' slack for the last leg of the journey before arriving at the terminus station must be modified.

"SEStran strongly supports the idea that there should be journey-time performance measures at key stations along the route and not only at the end station."
South East Scotland Transport Partnership

Organisational and individual responses

4.40 Responses were broadly similar between both organisations and individuals across each of the key themes identified. Indeed the top three themes were consistent in terms of content and order between both user groups.

4.41 Whilst most groups supported reliability to some extent, local government groups, passenger groups and Regional Transport Partnerships were amongst the strongest supporters of reliability. These three groups also championed the notion that the current system used for measuring journey time requires refinement and possible removal of the 'recovery time' aspect.

4.42 Economic and business groups, local authorities, trade unions and equality groups to some extent supported a performance regime which incorporates factors other than journey time, such as punctuality, into the calculation of performance.

Geographical analysis

4.43 Looking at the responses to this question across the RTP areas, it became apparent that there are two main themes significantly more important than all others, regardless of the type of area, whether it be urban or rural.

4.44 In general the majority of individuals spread across the different RTP areas agreed:

• Firstly, that the current method used to determine journey times and timetabling needs to be addressed, with services keeping to published times and doing away with timetable padding. They felt strongly that services should depart and arrive at the published time and that in-built recovery time should be removed from the journey time. For this key theme, a higher than expected response was observed from the SPT region illustrating the importance of the topic to those based in the area.
• Secondly most were keen to stress that reliability is key and is the most important performance indicator. As such they believe there should be less emphasis on journey times and performance as they feel they are one and the same, and instead the main focus should be on the reliability of the service. An argument was made that there is no point in a service completing its journey in the scheduled journey time, if in doing so it has skipped stations etc. Responses from the SEStran region in particular were keen to stress the importance of reliability.

Q13 - Is a Service Quality Incentive Regime required? And if so should it cover all aspects of stations and service delivery, or just those being managed through the franchise?

Popular themes

1. - Yes it is required and should cover all aspects
2. - SQUIRE may be too prescriptive
   (only two significant popular themes)

Key themes

4.45 The overwhelming majority of respondents that answered this question thought that SQUIRE is required and that it should cover all aspects of stations and service delivery. Indeed, this was the only real key theme to emerge from the question. Respondents thought that the current scheme is working and that it should be further amended to incorporate all aspects to ensure that everything matches up to the standards expected.

"A service quality assessment process is essential to the improvement of rail services within Scotland. Our belief is that any assessment regime should cover all aspects of station and services delivery, within agreed parameters. We also believe that the stations not covered in the franchise should also be working towards the same standards."
Scottish Borders Council

4.46 Some respondents, however, thought that SQUIRE is currently too prescriptive and could be re-formatted in order to prioritise different factors and views, which can then be rolled out to all aspects of delivery.

4.47 A significant number of respondents thought that SQUIRE is required although to what level is up for discussion. Respondents thought the regime was working well and that certain standards are being met. Without this system it was feared that standards would drop and that some stations would suffer by becoming neglected. There were also fears around staffing levels dropping both on the train and at stations.

4.48 The regime is an important system to be retained in order to provide a good quality service to passengers.

"A service quality regime is desirable but it must be seen to be fair and encourage rewards."
Public Response

4.49 A sizeable number of respondents also stated that although SQUIRE is important, it should only be incorporated into areas that are controlled by the franchise and are managed by the franchisee. They stated that otherwise it could become very difficult to enforce the standards and quality underlined by the regime.

"It would seem to go without saying that such a regime be part of any franchise. It would also seem unreasonable to expect the franchisee to be responsible for factors outwith their control. Other mechanisms would, surely, kick-in to deal with, for example, failures on the part of Network Rail."
Public Response

4.50 Others noted both the positives and negatives of franchise holder responsibilities compared with one organisation being responsible for all aspects of station management.

"The complexities of station ownership/leasing/operation must go a long way to explaining the bleak character of so many of our stations. One solution would be to place stations clearly in the SQUIRE criteria and lay the onus on the franchisee/s as a more significant contribution to assessment and evaluation of performance (PPM). On the other hand, in terms of simplification, it might be more appropriate to have ownership and responsibilities allocated to one single body - Network Rail, probably - and so bring all stations under that organisation while setting high, common standards."
Seniors Together

4.51 The remainder of respondents thought that there should be some sort of charter or framework in existence that both rewards and penalises the franchisee, and also one that takes into account the views held by passengers.

Organisational and individual responses

4.52 Both organisations and individuals identified the same key theme from this question: SQUIRE was important. Local authorities, government groups, equality groups, passenger groups and leisure/tourism groups were particularly likely to support SQUIRE including all aspects of station and service delivery.

Q14 - What other mechanisms could be used for assessing train and station quality?

Popular themes

1. - Ask passengers. Use passenger surveys/responses
2. - Train reliability, standard of facilities including disabled access and quality of service
3. - Make use of an independent group/panel to carry out random checks, audits and handle customer complaints

Key themes

4.53 The majority of respondents thought that the 'next' best mechanism for assessing train and station quality was to ask passengers, taking account of passenger responses and surveys. Respondents feel that passengers are the ones who use the service everyday and that their views are most important. They feel these views can be gained through regular surveys and through the national passenger survey. Respondents also considered the role of new media in gathering public opinion as passengers are more likely to answer online surveys whilst actually travelling.

"There could be a website allowing passengers and potential passengers to feed back directly to the relevant Government department. (Some potential passengers may wish to say why they do not or cannot travel by train.)"
Public Response

4.54 The second popular theme was that respondents believed factors such as train reliability, standards of facilities and quality of service should be used as a means of assessing train and station quality (perhaps due to uncertainty as to what SQUIRE encompasses). Opinions on these specific factors though would also need to be gathered through interaction with passengers, which then links back with the first theme of consultation with rail users.

"Additional station quality measures that should be included in assessing station and train quality should include (but not be restricted to) accessibility for people with mobility issues, amenities for wheel chair users, prams buggies and cyclists. Stations should also consider perceived safety of access for all users."
Cycling Scotland

4.55 An option favoured by a significant number of respondents was to involve a form of external independent auditors. Responses included suggestions such as using an independent panel to carry out spot checks and to collate and process customer complaints. Respondents thought that an independent panel would be best in improving the quality of service and for highlighting the issues and concerns of passengers which the operator would need to address.

"Regular independent inspections of stations, trains, provision of catering services where advertised, and provision of operating toilet facilities should be made by the body awarding the franchise, to ensure that provision meets the standard agreed when the franchise is awarded."
Public Response

4.56 Opinion on further mechanisms was mixed with numerous suggestions presented. A number of respondents thought that incorporating the current SQUIRE regime into the franchise would be beneficial but that this should be an amended version including some enhancements and adjustments. The regime should not to be too prescriptive and detailed in some areas, but also more visible in areas of concern to passengers.

4.57 Another suggestion highlighted by a group of respondents was to use a mystery passenger system to establish what areas need to be addressed. Some respondents also suggested including local involvement through community councils, rail interest groups and action groups.

Organisational and individual responses

4.58 Responses from both organisations and individuals were proportionally similar for this question.

4.59 Local authorities and Regional Transport Partnerships stressed the importance of consultation with passengers, through passenger surveys and passenger interaction.

Equality issues

4.60 Equality groups thought that train reliability, standard of facilities and disabled access and quality of service could all be used for assessing the overall train and station quality.

4.61 Indeed, many of the equality groups who answered this question would also like to see the needs of disabled people taken into account when considerations are made over quality of service of trains and stations. One organisation thought that it may be beneficial to include people with impairments in the inspection and audit of trains and stations.

"We strongly recommend that disabled people with a range of impairments (including physical, sensory, cognitive and learning disabilities) are employed to assess train and station quality and accessibility."
Capability Scotland

Geographical analysis

4.62 As explained within section 2.26, a geographical analysis was undertaken of all the responses that included location information. In terms of responses to Question 14, the SPT area generated a sizeable proportion of returns with 27% of all mappable responses originating from this area.

4.63 Within the SPT area, the City of Glasgow's share of responses was around 38%, with East Dunbartonshire (14%), North Lanarkshire (11%) and South Lanarkshire (11%) providing the next highest proportions of returns.

4.64 In terms of the key themes, SPT provided a considerable number of responses suggesting theme 1 - 'asking passengers through a series of customer satisfaction surveys'. Whilst both SEStran and SPT generated high proportions of responses for theme 2 - 'using train reliability, the standard of facilities and quality of service' as a measure for assessing quality.

4.65 Respondents from SEStran and Nestrans were both supporters of theme 3 - 'the use of independent groups to carry out checks', although the response from the SEStran region was particularly significant with around 40% of all responses to this theme originating within the area.