nep-sog New Economics Papers
on Sociology of Economics
Issue of 2011‒07‒21
two papers chosen by
Jonas Holmström
Swedish School of Economics and Business Administration
  1. Improving quality assessment of composite indicators in university rankings: a case study of French and German universities of excellence By Monica Benito; Rosario Romera
  2. Citations and Impact of ISI Tourism and Hospitality Journals By Chia-Lin Chang; Michael McAleer
  1. By: Monica Benito; Rosario Romera
    Abstract: Composite indicators play an essential role for benchmarking higher education institutions. One of the main sources of uncertainty building composite indicators and, undoubtedly, the most debated problem in building composite indicators is the weighting schemes (assigning weights to the simple indicators or subindicators) together with the aggregation schemes (final composite indicator formula). Except the ideal situation where weights are provided by the theory, there clearly is a need for improving quality assessment of the final rank linked with a fixed vector of weights. We propose to use simulation techniques to generate random perturbations around any initial vector of weights to obtain robust and reliable ranks allowing to rank universities in a range bracket. The proposed methodology is general enough to be applied no matter the weighting scheme used for the composite indicator. The immediate benefit achieved is a reduction of the uncertainty associated with the assessment of a specific rank which is not representative of the real performance of the university, and an improvement of the quality assessment of composite indicators used to rank. To illustrate the proposed methodology we rank the French and the German universities involved in their respective 2008 Excellence Initiatives.
    Keywords: Composite indicators, Rankings, Benchmarking, Higher education institutions, Weighting schemes, Simulation techniques
    Date: 2011–07
    URL: http://d.repec.org/n?u=RePEc:cte:wsrepe:ws112015&r=sog
  2. By: Chia-Lin Chang (Department of Applied Economics, Department of Finance, National Chung Hsing University Taichung, Taiwan); Michael McAleer (Econometrisch Instituut (Econometric Institute), Faculteit der Economische Wetenschappen (Erasmus School of Economics), Erasmus Universiteit, Tinbergen Instituut (Tinbergen Institute).)
    Abstract: The paper analyses the leading international journals in Tourism and Hospitality Research using quantifiable Research Assessment Measures (RAMs), highlights the similarities and differences in alternative RAMs, shows that several RAMs capture similar performance characteristics of highly cited journals, and shows that some other RAMs have low correlations with each other, and hence add significant informational value. Several RAMs are discussed for the Thomson Reuters ISI Web of Science database (hereafter ISI). Alternative RAMs may be calculated annually or updated daily to answer the questions as to When, Where and How (frequently) published papers are cited. The RAMs include the most widely used RAM, namely the classic 2-year impact factor including journal self citations (2YIF), 2-year impact factor excluding journal self citations (2YIF*), 5-year impact factor including journal self citations (5YIF), Immediacy (or zero-year impact factor (0YIF)), Eigenfactor, Article Influence, C3PO (Citation Performance Per Paper Online), h-index, PI-BETA (Papers Ignored - By Even The Authors), 2-year Self-citation Threshold Approval Ratings (2Y-STAR), Historical Self-citation Threshold Approval Ratings (H-STAR), Impact Factor Inflation (IFI), and Cited Article Influence (CAI). As data are not available for 5YIF, Article Influence and CAI for 11 of the 14 journals considered, 10 RAMs are analysed for 14 highly-cited journals in Tourism and Hospitality in the ISI category of Hospitality, Leisure, Sports & Tourism. Harmonic mean rankings of the 10 RAMs for the 14 highly-cited journals are also presented. It is shown that emphasizing the 2-year impact factor of a journal, which partly answers the question as to When published papers are cited, to the exclusion of other informative RAMs, which answer Where and How (frequently) published papers are cited, can lead to a distorted evaluation of journal impact.
    Keywords: Research assessment measures, Impact factor, IFI, C3PO, PI-BETA, STAR, Eigenfactor, Article Influence, h-index.
    JEL: L83 Y10
    Date: 2011
    URL: http://d.repec.org/n?u=RePEc:ucm:doicae:1126&r=sog
This nep-sog issue is ©2011 by Jonas Holmström. It is provided as is without any express or implied warranty. It may be freely redistributed in whole or in part for any purpose. If distributed in part, please include this notice.
General information on the NEP project can be found at http://nep.repec.org. For comments please write to the director of NEP, Marco Novarese at <director@nep.repec.org>. Put “NEP” in the subject, otherwise your mail may be rejected.
NEP’s infrastructure is sponsored by the School of Economics and Finance of Massey University in New Zealand.
This page is being maintained by Thomas Krichel. It was last updated on 2023‒07‒08 at 15:11.