|
on Sociology of Economics |
| By: | Scoggins, Bermond; Robertson, Matthew P. |
| Abstract: | The scientific method is predicated on transparency - yet the pace at which transparent research practices are being adopted by the scientific community is slow. The replication crisis in psychology showed that published findings employing statistical inference are threatened by undetected errors, data manipulation, and data falsification. To mitigate these problems and bolster research credibility, open data and preregistration practices have gained traction in the natural and social sciences. However, the extent of their adoption in different disciplines are unknown. We introduce procedures to identify the transparency of a research field using large-scale text analysis and machine learning classifiers. Using political science and international relations as an illustrative case, we examine 93, 931 articles across the top 160 political science and international relations journals between 2010 and 2021. We find that approximately 21% of all statistical inference papers have open data and 5% of all experiments are preregistered. Despite this shortfall, the example of leading journals in the field shows that change is feasible and can be effected quickly. |
| Date: | 2023 |
| URL: | http://d.repec.org/n?u=RePEc:zbw:i4rdps:14&r=sog |
| By: | Alberto Corsini (UCA - Université Côte d'Azur); Michele Pezzoni (UCA - Université Côte d'Azur, Observatoire des Sciences et Techniques - HCERES - Haut Conseil de l'Evaluation de la Recherche et de l'Enseignement Supérieur, OFCE - Observatoire français des conjonctures économiques (Sciences Po) - Sciences Po - Sciences Po, ICRIOS - ICRIOS, Bocconi University, Italy - Bocconi University [Milan, Italy]) |
| Abstract: | Over the last fifteen years, European countries have increasingly relied on competitive grants to allocate research funding, replacing the more traditional block funding model. Policymakers are interested in assessing the effectiveness of the grant funding model in producing impactful research. However, the literature aiming to quantify the effect of grants on the resulting research's impact is scant. In the French context, we compare the impact of scientific articles resulting from the support of competitive grants from the main national funding agency with the impact of articles not supported by grants. We rely on publication acknowledgments to retrieve funding information and on citation data to assess the articles' impact. We find that articles supported by competitive grants receive more citations than articles not supported by grants in the long run, while the difference is not statistically significant in the short run. We find heterogeneity in the effect of grant funding on citations across fields. |
| Keywords: | Competitive funding, Research impact, French funding agency |
| Date: | 2022–12–24 |
| URL: | http://d.repec.org/n?u=RePEc:hal:spmain:hal-03912647&r=sog |
| By: | Ilan Noy; Shakked Noy |
| Abstract: | “Longtermism” is the view that the impacts of our actions on the very long-term future deserve prominent consideration in decision-making. We discuss the primary barrier that prevents academic economists from contributing to longtermist research: an overly rigid preference for methodological “hardness” (Akerlof, 2020). Hardness bias prevents economists from engaging in methodologically pluralistic, interdisciplinary, qualitative, and other kinds of research, including most potential longtermist research. We unpack hardness bias, discuss its roots, illustrate how it prevents economists from engaging in longtermist research, and try to present a positive vision of the kinds of longtermist research economists could engage in if hardness norms were relaxed. |
| Keywords: | economic methodology, longtermism, academic economics, methodological hardness |
| JEL: | B40 |
| Date: | 2022 |
| URL: | http://d.repec.org/n?u=RePEc:ces:ceswps:_10160&r=sog |
| By: | Augusteijn, Hilde Elisabeth Maria (Tilburg University); Wicherts, Jelte M. (Tilburg University); Sijtsma, Klaas; van Assen, Marcel A. L. M. |
| Abstract: | We report a vignette study and a survey to investigate which study characteristics influence quality ratings academics give of articles submitted for publication, and academics and students give of students’ theses. In the vignette study, 800 respondents evaluated the quality of an abstract of studies with small or large sample sizes, showing statistically significant or non-significant results, and containing statistical reporting errors or no errors. In the survey, the same participants rated the importance of 29 manuscript characteristics related to the study’s theory, design, conduct, data analyses, and presentation for assessing either the quality of a manuscript or its publishability (article) or grade (thesis). Results showed that quality ratings were affected by sample sizes but not by statistical significance or the presence of statistical reporting errors in the rated research vignette. These results suggest that researchers’ assessments of manuscript quality are not responsible for publication bias. Furthermore, academics and students provided highly similar ratings of the importance of different aspects relevant to quality assessment of articles and theses. These results suggest that quality criteria for scientific manuscripts are already adopted by students and are similar for submitted manuscripts and theses. |
| Date: | 2023–01–03 |
| URL: | http://d.repec.org/n?u=RePEc:osf:osfxxx:7dc6a&r=sog |