Evaluation and monitoring of territorial projects in Wallonia

To mark the release of our 5th Practical Handbook dedicated to municipalities and cities engaged in a process of sustainable and smart transition, our team interviewed three experts in the field in Wallonia, with a dual objective in mind: firstly, to obtain their feedback, as Smart Region operational referents, on this latest volume, and secondly, to discuss the realities faced by Walloon municipalities when it comes to evaluating and monitoring the performance of their sustainable and intelligent projects.

Here is what we learned from our interviews with Mariléna Cassotti – SPI, Blanche Flémal – in BW and Marine Keresztes – IGRETEC :

WATCH THE WEBINAR WITH OUR 3 GUESTS FOR THE RELEASE OF THE PRACTICAL HANDBOOK #5 (IN FRENCH)  

Monitoring and evaluation: still many barriers to overcome

This had not escaped our team in February 2020 in our study "Monitoring and evaluation of Smart City projects - Exploratory study of the perceptions of Walloon municipalities": monitoring and evaluation remain underdeveloped in our Walloon municipalities. A reality that our experts also confirm:

« We have to admit it, monitoring and evaluation are still the poor cousins of projects (whether they are Smart City or not)..." admits Mariléna Cassotti. Our three interlocutors mention some of the reasons for this.

A lack of resources

The expert from Liège explains: "Municipalities lack global and standardised references and tools to facilitate decision-making, the creation of indicators and the collection of data."

Furthermore, according to Marine Keresztes, "most of the municipalities we met mentioned the lack of time, the fear of seeing the budget evolve (and especially explode) during the process. There is also a lack of knowledge of the aid and resources available. Despite the support that we offer as intermunicipal organisations, in particular via the Smart Region operational referents, a certain number of small and medium-sized Walloon municipalities are still struggling to define a longer-term strategy for sustainable and smart transition. As a result, it is sometimes complicated to be able to evaluate initiatives in a global context and logic."

READ ALSO "SMART CITY: WHAT FOLLOW-UP FOR WALLOON PROJECTS (STUDY)? " 

Persistent reluctance or fears

Certain perceptions and preconceived ideas about evaluation and monitoring also seem, in some cases, to hinder the process. According to Marine Keresztes: "Evaluation is often perceived as a constraint imposed by external factors such as quality certification, an external authority, a subsidising power, etc. and not as a real opportunity." 

In addition, there may be a "fear of external judgement and of critics, both from the opposition and from citizens, etc." regarding the results obtained, whether in the context of support committees or communication to the citizen, for example.

Accepting and dealing with failures in a constructive way

Another observation made by Marine Keresztes: "When it is actually carried out, the evaluation is done ex post and often focuses on the positive aspects that can be drawn out, enhanced and reproduced (but above all communicated) in the process and not so much on the elements that did not work as well."

Indeed, it is tempting to want to put more emphasis on the successes in order to demonstrate that the investments have paid off and that the project is working, and less on what has not worked so well. However, the monitoring and evaluation process should above all be seen as an opportunity: to learn from both successes and failures. A negative evaluation can certainly be very frustrating, but it should above all allow us to learn from it and to be able to adapt projects on an ongoing basis, and thus develop more sustainable solutions that are more in line with the realities of the territory.

As we have clearly understood, there are still many issues to be addressed in order to facilitate and encourage the implementation of evaluation and monitoring processes within Wallonia's municipalities. Concerned about fulfilling its role as Smart Region academic referent for Wallonia, our team wanted to bring its contribution through its new Practical Handbook dedicated to the theme. A book that could be a first step towards a response.

A clear knowledge and methodological frame

In designing this thematic handbook, our team sought to provide a clear and detailed reference for municipalities. More than a methodological manual offering a methodical path in different steps, this book also allows to reconsider the evaluation and monitoring, since it proposes :

• to better understand the stakes and the importance of such a process by setting out a clear theoretical framework
• to demonstrate the usefulness and real added value of evaluation and monitoring
• to demystify the evaluation process, which is sometimes seen as a cumbersome and/or restrictive process

Blanche Flémal gave us her first impressions after reading the handbook : "The handbook provides a fairly practical and pragmatic methodology. The main asset, in my opinion, are all the practical cases and examples. It allows you to put into practice and better understand the purpose of the theoretical concepts mentioned, but also to have a concrete view of the process, particularly in terms of KPIs (indicators). It provides a clear, pragmatic and easy-to-use framework for public operators." 

Taking stock and adopting a different approach

We also sought to find out the initial benefits that each of our experts could draw from the handbook after reading it.

In addition to their role as Smart Region operational referent, all three are involved in a project conducted in the framework of the Walloon Region's "Intelligent Territory" call for proposals, for which a first evaluation phase is soon to be completed. The publication of this handbook comes at the right time, as Blanche Flémal explains: "The handbook will allow us to check whether all the phases have been completed and, if necessary, to correct the situation, but also to go further in the development of our indicators."

For her part, Mariléna Cassotti adds: "It is clear that if we have established evaluation indicators for our projects, we could improve the way we document our project sheets, their follow-up and the way we draw lessons once the project is over". Marine Keresztes confirms: "The handbook will be very useful for analysing ongoing and completed projects in order to apply a logic of openness and replicability of these projects in the future and thus to draw on the experience gained for future projects."

Towards a standardisation of indicators for Wallonia?

Finally, to conclude this article, it seems interesting to quote Mariléna Cassotti, who opens an interesting reflection on the definition of a common framework in Wallonia:

A reflection that echoes the Smart Region dynamic developed in Wallonia and the need for its ecosystem to continue to exchange and collaborate in order to create a framework that favours the development of sustainable and intelligent initiatives throughout the Walloon territory.