|
on Sociology of Economics |
|
Issue of 2012‒07‒08
two papers chosen by Jonas Holmström Swedish School of Economics and Business Administration |
| By: | Juan A. Crespo; Li Yunrong; Javier Ruiz-Castillo |
| Abstract: | This paper has two aims: (i) to introduce a novel method for measuring which part of overall citation inequality can be attributed to differences in citation practices across scientific fields, and (ii) to implement an empirical strategy for making meaningful comparisons between the number of citations received by articles in the 22 broad fields distinguished by Thomson Scientific. The paper is based on a model in which the number of citations received by any article is a function of the article’s scientific influence, and the field to which it belongs. The model includes a key assumption according to which articles in the same quantile of any field citation distribution have the same degree of citation impact in their respective field. Using a dataset of 4.4 million articles published in 1998-2003 with a five-year citation window, we find that differences in citation practices between the 22 fields account for about 14% of overall citation inequality. Our empirical strategy for making comparisons of citation counts across fields is based on the strong similarities found in the behavior of citation distributions over a large quantile interval. We obtain three main results. Firstly, we provide a set of exchange rates to express citations in any field into citations in the all-fields case. (This can be done for articles in the interval between, approximately, the 71st and the 99th percentiles of their citation distributions). The answer is very satisfactory for 20 out of 22 fields. Secondly, when we normalize the raw citation data with our exchange rates, the effect of differences in citation practices is reduced to, approximately, 2% of overall citation inequality in the normalized citation distributions. Thirdly, we provide an empirical explanation of why the usual normalization procedure based on the fields' mean citation rates is found to be equally successfully. |
| Date: | 2012–06 |
| URL: | http://d.repec.org/n?u=RePEc:cte:werepe:we1206&r=sog |
| By: | Abel Brodeur (PSE - Paris-Jourdan Sciences Economiques - CNRS : UMR8545 - Ecole des Hautes Etudes en Sciences Sociales (EHESS) - Ecole des Ponts ParisTech - Ecole Normale Supérieure de Paris - ENS Paris - INRA, EEP-PSE - Ecole d'Économie de Paris - Paris School of Economics - Ecole d'Économie de Paris); Mathias Lé (PSE - Paris-Jourdan Sciences Economiques - CNRS : UMR8545 - Ecole des Hautes Etudes en Sciences Sociales (EHESS) - Ecole des Ponts ParisTech - Ecole Normale Supérieure de Paris - ENS Paris - INRA, EEP-PSE - Ecole d'Économie de Paris - Paris School of Economics - Ecole d'Économie de Paris); Marc Sangnier (PSE - Paris-Jourdan Sciences Economiques - CNRS : UMR8545 - Ecole des Hautes Etudes en Sciences Sociales (EHESS) - Ecole des Ponts ParisTech - Ecole Normale Supérieure de Paris - ENS Paris - INRA, EEP-PSE - Ecole d'Économie de Paris - Paris School of Economics - Ecole d'Économie de Paris, Sciences Po - Sciences Po); Yanos Zylberberg (CREI - Centre de Recerca en Economia Internacional - Universitat Pompeu Fabra) |
| Abstract: | Journals favor rejections of the null hypothesis. This selection upon results may distort the behavior of researchers. Using 50,000 tests published between 2005 and 2011 in the AER, JPE and QJE, we identify a residual in the distribution of tests that cannot be explained by selection. The distribution of p-values exhibits a camel shape with abundant p-values above :25, a valley between :25 and :10 and a bump slightly under :05. Missing tests are those which would have been accepted but close to being rejected (p-values between :25 and :10). We show that this pattern corresponds to a shift in the distribution of p-values: between 10% and 20% of marginally rejected tests are misallocated. Our interpretation is that researchers might be tempted to inflate the value of their tests by choosing the specification that provides the highest statistics. Note that Inflation is larger in articles where stars are used in order to highlight statistical significance and lower in articles with theoretical models. |
| Keywords: | Hypothesis testing ; Distorting incentives ; Selection bias ; Research in economics |
| Date: | 2012–06 |
| URL: | http://d.repec.org/n?u=RePEc:hal:psewpa:halshs-00710122&r=sog |