Monitoring and Review

The consultation paper reported that previous feedback has highlighted that communities would like greater clarity and accuracy on performance reporting to be embedded within the next contract. It also indicates that the true passenger experience is not reflected in the way the Operator reports performance.

Responses to Question 18 by respondent type are set out in Table 20 below.

Table 20: Responses to Question 18 by respondent type

Organisations	Yes	No	Total
Community Council or Development Trust	5	1	6
Farming or land management organisation	3	0	3
Ferry Board, Committee or Group	6	0	6
Haulage company or representative body	2	0	2
Local Authority or HSCP	4	0	4
Public Body	2	0	2
Tourism business	3	0	3
Trade Union	1	0	1
Transport Partnership	2	0	2
Voluntary sector organisation	2	0	2
Other business or representative body	3	0	3
Total organisations	33	1	34
% of organisations	97%	3%	Not Applicable
Individuals	308	46	354
% of individuals	87%	13%	Not Applicable
All respondents	341	47	388
% of all respondents	88%	12%	Not Applicable

A majority of respondents – 88% of those who answered the question – would like to be able to give feedback to improve services. This rose to 97% of the organisations who answered.

Around 325 respondents made a comment at Question 18.

How often should feedback be given

Among respondents who would like to be able to give feedback to improve services, the most common suggestions as to how often this should happen, were annually/at least annually or quarterly, with these choices made at approximately equal frequencies. The next most frequent preferences in terms of a fixed period of time for feedback were six monthly and then monthly, with the latter sometimes associated with a view that CalMac should publish monthly performance statistics and then provide ferry users with the opportunity to discuss these. Others suggested regular or ongoing/continuous feedback, or feedback as and when required. It was also argued that feedback must be representative of the whole year, and not focus on the summer season.

Rather than at any particular time intervals, some respondents suggested that opportunities to provide feedback should be triggered by events, most frequently each booking or each journey. New timetables or fare changes were also suggested as possible opportunities to provide feedback.

How and where should feedback be given

With respect to how feedback from individual customers should be collected, the most frequent suggestion was that this should be online via a website or using an app, with email, telephone, and paper-based options also proposed. Opportunities to leave feedback on board vessels or at ferry terminals were also suggested – potentially by completing a questionnaire, scanning a QR code, or using a simple touch screen indicator of the overall quality of the experience. Improved use of social media, and monitoring of third-party feedback sites were also suggested. However collected, it was suggested that feedback should be sought with respect to a particular route or island and not simply across the network as a whole.

The importance of providing opportunities to leave feedback in Gaelic was highlighted.

While some respondents suggested that feedback should be provided via locally recognised groups such as Community Councils or Ferry Committees there were also calls for public meetings in communities or for face-to-face engagement events at ports. A Ferry Board, Committee or Group respondent argued that the Operator’s staff should be visible within the communities served, meeting stakeholders on a regular basis and championing improvement on behalf of the community.

Some organisational respondents indicated that they already provide feedback via Ferry Stakeholder Groups which, it was suggested, provide a useful forum but could meet more frequently or could provide more in-depth analysis. It was also suggested that greater weight should be given to information gathered through consultation with groups such as the Convention of the Highlands and Islands, the Highlands and the Islands Regional Economic Partnership and local ferry user groups.

Designing surveys

Many respondents who referenced the format that should be used for collecting feedback talked about surveys or questionnaires. Comments included that current questionnaires should be improved, should avoid leading questions, or should better reflect the key issues that are important for service users. It was also argued that questionnaires should differentiate residents who are regular service users from occasional visitors such as tourists, giving more weight to the views of those who depend on lifeline services throughout the year. A Ferry Board, Committee or Group respondent reported frustration that local concerns may be discounted, on the grounds that general passenger surveys come to a different conclusion.

As well as suggesting responses on a five-or-six-point scale there were requests for a free-text opportunity to explain a problem or complaint.

What should be done with the feedback

While a small number of Individual respondents noted that they already provide feedback, others were sceptical about the value of feedback exercises, expressing the view that they are a waste of time if no action is taken in response. Some suggested that any feedback should be submitted via an organisation independent of CalMac, should be made public, or should be subject to external scrutiny. A Local Authority respondent argued that the new contract should set out how feedback will be analysed and what/how improvements are made as a result.

It was also suggested that the SQUIRE mechanism (Service Quality Inspection Regime) already used by Transport Scotland to monitor the rail industry could be adapted for use with respect to ferry services.

Other issues raised

One Trade Union respondent argued that, while the views of service users are a key part of monitoring and review, other metrics including staff satisfaction, environmental performance, and business confidence should also be evaluated. Another cautioned that a customer feedback mechanism with contractual implications for the Operator should not result in ‘an open season on CalMac staff’.

Around 290 respondents answered Question 19, although some simply responded to note that they had no further suggestions or to refer back to answers at earlier questions.

Reflecting the issues raised at Question 18, many respondents highlighted the importance of:

• Engaging with communities and asking for feedback.
• Listening to what passengers have to report and using the feedback to drive improvements.
• Publishing the feedback gathered, whether positive or negative.
• Differentiating the views of different groups of passengers (particularly residents and tourists) who may want or need different things. In view of the subsidy from government for running a lifeline service, it was argued the views of islanders should take precedence.

There were also calls to relocate members of CalMac senior management to island communities so they can better understand the problems created by unreliable ferry services, and for greater representation of island communities on the board of CalMac.

A small number of respondents commented on the extent to which CalMac can be held responsible for issues that are beyond its control with one suggestion that it would be helpful to separate complaints in relation to procurement problems from those, such as the e-ticketing system, that are the direct responsibility of the Operator.

While many responses focused on the problems being experienced by service users, it was also suggested that attempts to reflect more positive aspects of passenger experience may be difficult to gauge, as elements of a journey that do meet expected standards – for example in relation to comfort or onboard facilities – may not attract comments.

Returning to issues raised at Questions 1 and 2 with respect to the difference between CalMac’s published performance statistics and the lived experience of many who depend on ferry services there were calls for:

• Transparency around performance reporting, publishing details of all cancellations rather than presenting figures after relief events.
• Collecting and reporting data on a wider range of performance measures, for example with respect to availability, and the booking system
• Reporting impacts of cancellations or reduced capacity.
• Providing performance statistics by route and making customer satisfaction on individual routes a key metric.
• Improving communication around cancellations.