Publication - Report

Smart City : How do Walloon municipalities monitor their projects? (Study)

February 2020

13 February 2020 : The Smart City Institute (SCI) publishes a new report on the evaluation of Smart City projects in Wallonia. To develop this qualitative study, 25 municipalities have answered the questions of our researcher, Audrey Lebas. Discover the main lessons learned from these interviews.

For several years now, the Smart City dynamic has been gaining ground in Wallonia. This is notably reflected in the Smart Region strategy of Digital Wallonia or in the launch of the "Smart Territory" call for projects in 2019. The latest encouraging figures from the 2019 Walloon Barometer have moreover demonstrated this: 62.5% of Walloon municipalities are already truly committed to a Smart City approach.

While these new projects are being launched in Wallonia, a question is also emerging: How are these projects evolving? Do they ultimately have a positive impact for the inhabitants? And therefore, more generally : Are there processes in place in Wallonia to evaluate these projects? Since the majority of these are initiated by the public authorities (61% - see Walloon Barometer 2018), our team decided to look into the following question: Do Walloon municipalities evaluate their Smart City projects?

In Wallonia, although monitoring and evaluation are generally popular, there is still no publication dealing with monitoring and evaluation specific to Smart City projects. With this report, the SCI is laying the foundations for this reflection.

Download the complete rEport

The main lessons of this report

Through the 25 interviews conducted, this report provides a better understanding of the situation:

the perception that Walloon municipalities have of the monitoring* and evaluation of their projects (all types of projects taken together)
their state of progress in monitoring and evaluating Smart City projects

*process of "systematic collection of data on specified indicators to provide (...) indications of the extent of progress and achievement of targets".

Monitoring and evaluation in general: structural obstacles identified

"Although there is an interest in project monitoring and evaluation, there is generally no culture of evaluation in Walloon municipalities.”

In fact, out of the 25 municipalities surveyed, 15 of them replied that monitoring/evaluation was important, while specifying that they do not use it. Why not? Several structural obstacles were mentioned:

Lack of time or human resources (12/25)
Lack of technical knowledge and/or tools (11/25)
Political will (10/25)
Bias and subjectivity potentials (4/25)
Lack of corporate culture (2/25)
Problem related to access to data (2/25)

Illu RR3 Image large min

Monitoring and evaluation of Smart City projects: a mixed picture

"In our Walloon municipalities, few monitoring and evaluation mechanisms for Smart City projects are put in place and, when they are, the project is rarely analysed as a whole.”

Indeed, a little less than half of the municipalities surveyed (12/25) consider that they have set up a process to monitor and/or evaluate their Smart City projects. Given that less than half of this number evaluate on a daily basis, a contradiction can be pointed out here.

As for the 13 municipalities that do not evaluate their Smart City projects, they consider :

Not being advanced enough in their projects or ;
Not having the methods and/or tools to monitor and evaluate their projects.
Nevertheless, 6 of them indicate that they would like to remedy this situation.

It is also interesting to specify that 10 municipalities out of the 25 interviewed declare that they do not have a clear idea of what the Smart City represents at their level.

Smart City projects: Whose job to evaluate them?

About the potential evaluators for their Smart City projects, the municipalities are divided. Among them :

15 consider that monitoring and evaluation can or should be carried out by a single actor.
Almost half of the municipalities surveyed (12/25) consider that an external entity can or should take care of this.
9 municipalities consider that the municipal administration should be responsible for monitoring and evaluation.
5 communes consider that this should be done by a specific project steering committee (COMAC).
4 municipalities consider that the Walloon Region can or should take charge of the monitoring and evaluation of projects resulting from the "smart territory" call of projects.

Some recommendations of the Smart City Institute

In order to help municipalities in their efforts, but also to get a clearer picture of the situation, the SCI proposes a few recommendations at the end of its report:

#1 : Moving from a logic of "control" to a logic of "results".
#2 : Clarify the purpose of the evaluation and choose its indicators well
#3 : Knowing why we evaluate
#4 : Raising awareness and training the administration
#5 : Partially or fully delegate evaluation

DOWNLOAD THE FULL REPORT

Evaluation process: How useful is it?

Monitoring - or evaluation - is an important step in the development of a Smart City strategy/project, and occurs at various stages of its progress. Indeed, as indicated in our Smart City Practical Guide - Volume 1, monitoring the evolution, measuring the results and impacts of the strategy and projects implemented is crucial. This evaluation phase can not only contribute to the legitimacy of Smart Cities projects and policies, but also help to ensure their sustainable performance. Evaluation is therefore a real tool for public managers in their decision-making and implementation tasks.

Who are the respondents of this study?

The selection of municipalities was made on the basis of the call for projects "Smart Region - Smart Territory" organised by the“Agence du Numérique” (AdN). The 78 municipalities that submitted a project were contacted in order to participate in the study (we identified the submission of a project as a significant indicator of interest in the Smart City theme). 25 of them responded favourably to our request (detailed list available in the report).

And to go further

See also our report “Performance Measurement in Smart Cities”. The aim of this report is :

to introduce cities and municipalities to the concept of performance measurement in the context of Smart Cities;
to help them establish their own method of performance measurement through an integrative method in the form of a structure/canvas, based on their own specificities and strategies.