The review of evaluation practices, focusing on challenges in the Hungarian small and medium cities

Tamás Kaiser, Institute of State Theory and Governance, National University of Public Service
András Bojtor, Office of Research Methodology and Measurement, National University of Public Service


There is an emphasized attention in the policies of the Hungarian Government on the smart city developments. Ten cities, including Tata, are selected as host settlements for smart city pilot developmental projects. The international evaluation systems can provide a good basis for comparative analysis however there is a need for a well-structured efficient local evaluation system on smart intervention. Smart governance needs to be put through for achieving the best available citizen-centred results. It is desired from the local and central governmental institutions to provide enabling environment which can function as an intermediate engine of digital developments for the social and smart interventions. As a complex digitization developmental program focusing not just on the infrastructural development but also on the improvement of the human abilities, the Hungarian Government launched the comprehensive Digital Welfare Program. The monitoring of the executed programs and the reasonable, effective measurement of the impacts of the developmental programs are inadequate for policy and decision makers to evaluate the results and to disclose the basis of the required further interventions. We intend to reveal the relevant literature review of the international practices, the convenient possibilities of social and economic evaluation and reporting. We analyse these processes in detail in Hungarian context based on a case study about the Hungarian city of Tata, however these smart city strategies – programmes - projects are in the preparation phases or at the beginning of implementations. Following these, we formulate the possible future sequences of the research.


[1] UNITED NATIONS. World Urbanization Prospects: The 2018 Revision 2018.

[2] KOURTIT, K., NIJKAMP, P. & ARRIBAS, D., Smart Cities in perspective. in: Innovation: The European Journal of Social Sciences, 25(2), p. 229–246 2012

[3] SHELTON, T., ZOOK, M., & WIIG, A., The ‘actually existing smart city’. in: Cambridge Journal of Regions, Economy and Society, 8(1), p. 13–25 2015.

[4] GIFFINGER, R. & FERTNER, C., City-ranking of European medium-sized cities. in: Centre of Regional …, p. 1–12 2007.

[5] SHI, H., TSAI, S., B., LIN, X. & ZHANG, T., How to evaluate smart cities’ construction? A comparison of Chinese smart City evaluation methods based on PSF. in: Sustainability (Switzerland), 10(1) 2017.

[6] VAN BASTELAER, B., Digital cities and transferability of results. In in: the 4th EDC conference on digital cities (pp. 61–70) 1998. Salzburg.

[7] COCCHIA, A., Smart and Digital City: A Systematic Literature Review (pp. 13–43) 2014. Springer, Cham.

[8] HARRISON, C., ECKMAN, B., HAMILTON, R., HARTSWICK, P., KALAGNANAM, J., PARASZCZAK, J. & WILLIAMS, P., Foundations for Smarter Cities. in: IBM Journal of Research and Development, 54(4), p. 1–16 2010.

[9] HUNGARIAN GOVERNMENT. 56/2017. (III. 20.) Korm. rendelet egyes kormányrendeleteknek az „okos város”, „okos város módszertan” fogalom meghatározásával összefüggő módosításáról 2017.

[10] FEKETE, D., A Modern Városok Program elemzési lehetõségei. in: Polgári Szemle, 13(1–3), p. 94–105 2017.

[11] OECD. Toolkit for the preparation, implementation, monitoring, reporting and evaluation of administration reform and sector strategies 2018. OECD.

[12] CAIRD, S. P. & HALLETT, S. H., Towards evaluation design for smart city development, p.1–22 2018.

[13] WU, Z. (n.d.). Intelligent city evaluation system.

[14] UNITED NATIONS. Transforming Our World: The 2030 Agenda for Sustainable Development 2015.

[15] KLOPP, J. M. & PETRETTA, D. L., The urban sustainable development goal: Indicators, complexity and the politics of measuring cities. in: Cities, 63, p. 92–97 2017.

[16] HORVÁTHNÉ BARSI, B., Evaluating and modelling smart city performance. In in: Central and Eastern European e|Dem and e|Gov Days 2016: Multi-Level (e)Governance : is ICT a means to enhance transparency and democracy? (pp. 275–282) 2016. Austrian Computer Society.

[17] SCHÖNERT, M., Städteranking und Imagebildung 2003.

[18] GIFFINGER, R., HAINDLMAIER, G. & KRAMAR, H., The role of rankings in growing city competition. in: Urban Research and Practice, 3(3), p. 299–312 2010

[19] CENTRE OF REGIONAL SCIENCE VIENNA. Smart cities Ranking of European medium-sized cities 2007.

[20] SHEN, L., HUANG, Z., WONG, S. W., LIAO, S. & LOU, Y., A holistic evaluation of smart city performance in the context of China. in: Journal of Cleaner Production, 200, p. 667–679 2018.

[21] HUOVILA, A., AIRAKSINEN, M., PINTO-SEPPÄ, I., PIIRA, K. & PENTTINEN, T., Smart city performance measurement system. In in: 41st IAHS WORLD CONGRESS Sustainability and Innovation for the Future (p. 10) 2016.

[22] LOMBARDI, P., GIORDANO, S., FAROUH, H. & YOUSEF, W., Modelling the smart city performance, Innovation. in: The European Journal of Social Science Research, 25(2), p.137–149, 2012.

[23] AHVENNIEMI, H., HUOVILA, A., PINTO-SEPPÄ, I. & AIRAKSINEN, M., What are the differences between sustainable and smart cities? in: Cities, 60, p. 234–245, 2017.

[24] AKANDE, A., CABRAL, P., GOMES, P. & CASTELEYN, S., The Lisbon Ranking for Smart Sustainable Cities in Europe. in: Sustainable Cities and Society, 44(August 2018), p. 475–487, 2018.

[25] JOUILI, K., FURJANI, A. AL, SHAHROUR, I. & WASHINGTON, K., The Smart City: How to Evaluate Performance ? in: International Conference, Responsible organizations in the Global Context, (June), p. 1–16, 2017.

[26] CAIRD, S. P. & HALLETT, S. H., Towards evaluation design for smart city development. in: Journal of Urban Design, 4809(May), p. 1–22 2018.

[27] EUROPEAN COMMISSION. The Cultural and Creative Cities Monitor 2017.

[28] RODRIGUES, M. & FRANCO, M., Measuring the performance in creative cities: Proposal of a multidimensional model. in: Sustainability (Switzerland), 10(11), p. 1–21, 2018.

[29] SIKORA-FERNANDEZ, D., Smarter cities in post-socialist country: Example of Poland. in: Cities, 78(June 2017), p. 52–59, 2018.

[30] ROMAN, K., Analysis and Evaluation of the Implementation Level of the Smart City Concept in Selected Polish Cities. in: BRAIN – Broad Research in Artificial Intelligence and Neuroscience, 9(1), p. 138–145, 2018.

[31] HORVÁTHNÉ BARSI, B. & LADOS, M., ‘ Smart cities’ tanulmány 2011. Győr.


[33] NEMZETI INNOVÁCIÓS HIVATAL. National Smart Specialisation Strategy, 2014.

[34] MCCANN, P., The regional and urban policy of the European Union: Cohesion, results-orientation and smart specialisation 2015. Edward Elgar Publishing.

[35] KAISER, T., Good State and Governance Report 2016. Budapest.

[36] NORTH, D., Institutions. in: Journal of Economic Perspectives, 5(1), p. 97–112, 1991.


Central and Eastern European e|Dem and e|Gov Days 2019

Cyber Security and eGovernment
Proceedings of the Central and Eastern European E|Dem and E|Gov Days, May 2-3, 2019, Budapest
Facultas, 1. Ed., 536 p.
ISBN: 978-3-7089-1898-3,
ISBN: 978-3-903035-24-9

Editors: András Nemeslaki, Alexander Prosser, Dona Scola, Tamás Szádeczky