The Problems of Assessing Validity and Political Potential of Cross-National Comparative Ratings

The author carries out a correlation analysis of positions of 145 countries in 16 comparative rating studies highly cited in social and political science and calculates the coefficient of their correlation with the positions of the Russian Federation. Additionally, the author compares the scores of 144 countries in the array of 16 ratings with the relevant indicators of the Russian Federation, which allowed to distinguish a group of countries showing maximum institutional identity with Russia, according to the compilers of the ratings. The author states that discovered correlations lack empirical evidence and convincing interpretation. At the same time, they are largely affected by political and ideological factors, as well as the effect of «image evaluation». The author concludes that the analyzed cross-country indices have considerable heuristic potential, but the patterns and correlations based solely on statistical analysis of their data, and, particularly, qualitative assessments, do not fully meet the criteria of obtaining new scientific knowledge and should be considered mainly as hypotheses that require additional causal substantiation and empirical verification. The article shows that mandatory hierarchy of representation inherent in the cross-country ratings creates the basis for their politicization and contradicts the contemporary reality that strives towards polymodality of the world order.

1. Bartenev, V.I. 2017. From ‘Failed States’ to ‘States of Fragility’: Logic of Conceptual Acrobatics // Polis. Political Studies, 2017, No. 2, pp. 26-41.

2. Broome, A.; Homolar, A.; Kranke, M. Bad Science: International Organizations and the Indirect Power of Global Benchmarking // European Journal of International Relations, 2018, Vol. 24, Issue: 3, pp. 514-539.

3. Indeksy razvitija gosudarstv mira: spravochnik. Pod red. Yu.A. Nisnevich (The Development Indices of the Countries of the World: A Handbook. Ed. By Yu.A. Nisnevich). Мoscow: The Higher School of Economics Publishing House. 2014.

4. Kelley, B.A.; Simmons, J.G. Politics by Number: Indicators as Social Pressure in International Relations // American Journal of Political Science, 2015, Vol. 59, No. 1, pp. 57-58.

5. Mennillo, G. Credit Rating Agencies in Asia: A Battle of Ideas / IPSA Conference Proceedings Library. 2018 Brisbane 25th World Congress (Borders and Margins).

6. Shapiro, I. The Flight from Reality in the Human Sciences. (Russ. ed.: Shapiro, I. Begstvo ot real’nosti v gumanitarnyh naukah. Мoscow: The Higher School of Economics Publishing House. 2011. 368 p.)

7. Sravnitel'naja politologija. Pod red. O.V. GamanGolutvina (Comparative Politics. Ed. by O.V. GamanGolutvina). Мoscow: Aspect-Press. 2015.