|
on Sociology of Economics |
| By: | Brodeur, Abel (University of Ottawa); Cook, Nikolai (Wilfrid Laurier University); Neisser, Carina (University of Cologne) |
| Abstract: | In this paper, we examine the relationship between p-hacking and data-sharing policies for published articles. We collect 38,876 test statistics from 1,106 articles published in leading economic journals between 2002–2020. While a data-sharing policy increases the provision of research data to the community, we find a well-estimated null effect that requiring authors to share their data at the time of publication does not alter the presence of p-hacking. Similarly, articles that use hard-to-access administrative data or third-party surveys, as compared to those that use easier-to-access (e.g., own-collected) data are not different in their p-hacking extent. Voluntary provision of data by authors on their homepages offers no evidence of reduced p-hacking. |
| Keywords: | p-hacking, publication bias, data and code availability, data sharing policy, administrative data, survey data |
| JEL: | A11 B41 C13 C40 I23 |
| Date: | 2022–09 |
| URL: | http://d.repec.org/n?u=RePEc:iza:izadps:dp15586&r= |
| By: | Olivier Chanel (AMSE - Aix-Marseille Sciences Economiques - EHESS - École des hautes études en sciences sociales - AMU - Aix Marseille Université - ECM - École Centrale de Marseille - CNRS - Centre National de la Recherche Scientifique); Alberto Prati (The University of Mississippi [Oxford], LSE - London School of Economics and Political Science); Morgan Raux (University of Luxembourg [Luxembourg]) |
| Abstract: | We provide an estimate of the environmental impact of the recruitment system in the economics profession, known as the "international job market for economists". Each year, most graduating PhDs seeking jobs in academia, government, or companies participate in this job market. The market follows a standardized process, where candidates are pre-screened in a short interview which takes place at an annual meeting in Europe or in the United States. Most interviews are arranged via a non-profit online platform, econjobmarket.org, which kindly agreed to share its anonymized data with us. Using this dataset, we estimate the individual environmental impact of 1057 candidates and one hundred recruitment committees who attended the EEA and AEA meetings in December 2019 and January 2020. We calculate that this pre-screening system generated the equivalent of about 4800 tons of avoidable CO2-eq and a comprehensive economic cost over €4.4 million. We contrast this overall assessment against three counterfactual scenarios: an alternative in-person system, a hybrid system (where videoconference is used for some candidates) and a fully online system (as it happened in 2020–21 due to the COVID-19 pandemic). Overall, the study can offer useful information to shape future recruitment standards in a more sustainable way. |
| Keywords: | Job market for economists,International job market,Carbon footprint,Environmental impact,Comprehensive economic cost |
| Date: | 2022–11 |
| URL: | http://d.repec.org/n?u=RePEc:hal:journl:hal-03778777&r= |
| By: | ONISHI Koichiro; OWAN Hideo |
| Abstract: | We examine how reviewer–applicant social ties (department and university affiliation, co-applicant relationships, research field similarity) influence reviewer evaluations, based on Japanese research grant application data (2005–2016). All relationships between social ties and scores are positively correlated, even after accounting for unobservable applicant characteristics and proposal quality. Regarding bias and information advantage effects, upward deviation from department match negatively correlates with applicants’ future research outputs, implying bias. Upward deviation from research field or university match positively correlates with future productivity, indicating that information advantage predicts applicants’ future productivity. Information advantage through social ties is stronger for younger applicants. |
| Date: | 2022–09 |
| URL: | http://d.repec.org/n?u=RePEc:eti:dpaper:22096&r= |
| By: | Vu, Patrick |
| Abstract: | Selective publication is among the most-cited reasons for widespread replication failures. I show in a simple model of the publication process that the replication rate is completely unresponsive to the suppression of insignificant results. I then show that the expected replication rate falls below its intended target owing to issues with common power calculations in replication studies, even in the absence of other factors such as p-hacking or heterogeneous treatment effects. I estimate an empirical model to evaluate if issues with power calculations alone are sufficient to explain the low replication rates observed in large-scale replication studies. The model produces replication rate predictions (using only data from original studies) that are almost identical to observed replication rates in experimental economics and social science. In psychology, the model explains two-thirds of the gap between the replication rate and its intended target. I conclude by discussing alternative measures of replication that are more responsive to selective publication. |
| Date: | 2022 |
| URL: | http://d.repec.org/n?u=RePEc:zbw:i4rdps:3&r= |