43 KPIs to track to evaluate the impact
of a MaaS application

DashBoard2

Although more and more MaaS applications are appearing on the market, it is still very difficult to assess their impact on the environment, economy and society. The problem? An underestimation of the costs and resources for such an evaluation campaign. Or an obvious lack of hindsight and data, or even difficulty in proving causal links. However, it is important to address these issues. That's why we have compiled the results of in-depth studies conducted by researchers from Chalmers University of Technology, MaaSLab, Cerema and the Smart City Institute. An assessment at all levels, including more than forty KPIs which we will detail in this article...

Level of analysis "Micro"
Related to the end user

With the goal of facilitating the experience, assessing impact and value generation at the end-user level is critical. As you can imagine, there is nothing better than conducting qualitative surveys. For this, we strongly advise you to set up interviews. A more in-depth approach addressing the following different points. To know :

  • Its use of the MaaS application
    • KPI 1: The total number of trips searched per month
    • KPI 2: The total number of trips booked on the application / month
    • KPI 3: The share for each type of trip made (work, shopping, doctor, activities, etc.)
  • The financial aspect
    • KPI 4: Total expenses per month
    • KPI 5: Share of expenditure by mode of transport

  • Its perception of accessibility
    • KPI 6: Level of satisfaction with each feature
    • KPI 7: The level of satisfaction with the accessibility of mobility offers on the application
    • KPI 8: The walking time between the different mobility services (in minutes)
    • KPI 9: The level of satisfaction with the accessibility of commuting (home-work, school,...)
    • KPI 10: Level of satisfaction with accessibility for exceptional trips (home to work, school, etc.)

  •  Its change in modal shift behavior
    • KPI 11: Modal share of trips per month
    • KPI 12: Percentage of intermodal trips per month
    • KPI 13: Average number of trips taken per trip
    • KPI 14: Change in car use.

Various performance indicators that can be correlated with the user's profile (age, gender, social class, location, resources, etc.). An in-depth analysis that could eventually lead to predictive models, which could then be integrated into the journey planner for example.

 

Case study : The sucess story of Jakarta's MaaS app


Level of analysis "Meso"
Related to the territory and the stakeholders


In this second level, we focus on your MaaS application and the stakeholders in the project. By stakeholders, we mean the owner(s) of the application, the public and private mobility service partners and the potential MaaS providers.

Complementary collaboration is crucial for the success of any MaaS application. It is therefore essential to establish a list of associated KPIs for each of these stakeholders. Thus:

  • The interests for the MaaS application
    • KPI 15: The total number and evolution of downloads
    • KPI 16: Le nombre total et évolution des clients (+segmentation)
    • KPI 17: Le nombre total et évolution des réservations effectuées sur l’application
    • KPI 18: La rentabilité de votre solution
    • KPI 19: Le taux de rétention
    • KPI 16: The total number and evolution of customers (+segmentation)
    • KPI 17: The total number and evolution of bookings made on the application
    • KPI 18: The profitability of your solution
    • KPI 19: Retention rate

  • The Interest for the mobility partner(s)
    • KPI 20: The total number and evolution of bookings per month
    • KPI 21: Revenue
    • KPI 22: Share of all bookings in total
    • KPI 23: The number of packages booked including its mobility service

  • The Interest for the MaaS provider(s)
    • KPI 24: The number of integrated mobility services
    • KPI 25: The percentage of mobility services that can be booked on the application (no redirection, no deep-link)
    • KPI 26: The percentage of mobility offers present on the app versus the mobility offers present on the whole territory
    • KPI 27: The total number of calls to the APIs
    • KPI 28: Technical scalability
    • KPI 29: Innovation potential

Level of analysis "Macro"
Related to the environment


Finally, this third layer gives an overview of the landscape in which you evolve. In other words, all the elements over which you have no control. From the information you might think, it is impossible to make exact predictions.

However, certain upheavals at the macroeconomic level can strongly influence the activity of your application, or even the whole sector. This was the case with the government's decree this summer, requiring transportation applications to put forward route proposals that have the lowest impact in terms of greenhouse gas emissions.

A finding that emphasizes the importance of taking this last level of analysis into account. And to do so, nothing better than to draw inspiration from the PESTEL model.

A matrix analysis that aims to give a global vision of the environment, encompassing six criteria that correspond to the six letters of the acronym "PESTEL". Namely:

  • Policy
    • KPI 30: The implementation of global measures to encourage data sharing by mobility providers (Yes/No)
    • KPI 31: The implementation of global regulations at the level of mobility platforms in general
  • Economic 
    • KPI 32: Number of new MaaS solutions
    • KPI 33: The cost and financial evolution for each of the transport providers
    • KPI 34: The cost of the physical infrastructure put in place to provide a "physigital" experience
  • Socio-cultural
    • KPI 35: The evolution of the purchasing power of users
    • KPI 36: The level of satisfaction with the accessibility of transportation
  • Tecnological
    • KPI 37: The evolution of ticketing systems
    • KPI 38: The evolution of innovations in the transport offer, such as autonomous vehicles
  • Ecological
    • KPI 39: Greenhouse gas emissions per transport
    • KPI 40: Collective awareness of more environmentally friendly modes of transport
  • Legal
    • KPI 43: The implementation of a framework for redistribution
    • KPI 42:The implementation of a competition law with an obligation of equity and transparency
    • KPI 43:The number of regulations or measures implemented to promote alternative, more environmentally friendly modes of transport
Evaluation Cerema

TO GO FURTHER 
The complete Cerema's study (fr)
Evaluation of MaaS :
How to measure its effects?