| By: |
Brodeur, Abel;
Mikola, Derek;
Cook, Nikolai;
Brailey, Thomas;
Briggs, Ryan;
de Gendre, Alexandra;
Dupraz, Yannick;
Fiala, Lenka;
Gabani, Jacopo;
Gauriot, Romain;
Haddad, Joanne;
Lima, Goncalo;
Ankel-Peters, Jörg;
Dreber, Anna;
Campbell, Douglas;
Kattan, Lamis;
Marino Fages, Diego;
Mierisch, Fabian;
Sun, Pu;
Wright, Taylor;
Connolly, Marie;
Hoces de la Guardia, Fernando;
Johannesson, Magnus;
Miguel, Edward;
Vilhuber, Lars;
Abarca, Alejandro;
Acharya, Mahesh;
Adjisse, Sossou Simplice;
Akhtar, Ahwaz;
Ramirez Lizardi, Eduardo Alberto;
Albrecht, Sabina;
Andersen, Synøve Nygaard;
Andlib, Zubaria;
Arrora, Falak;
Ash, Thomas;
Bacher, Etienne;
Bachler, Sebastian;
Bacon, Félix;
Bagues, Manuel;
Balogh, Timea;
Batmanov, Alisher;
Barschkett, Mara;
Basdil, B. Kaan;
Baxa, Jaromír;
Becker, Sascha;
Beeder, Monica;
Beland, Louis-Philippe;
Bello, Abdel Hamid;
Markovits, Daniel Benenson;
Benjamin, Grant;
Bergeron, Thomas;
Blimpo, Moussa P.;
Binetti, Marco;
Bonander, Carl;
Bonneau, Joseph;
Borbáth, Endre;
Topstad Borgen, Nicolai;
Topstad Borgen, Solveig;
Borowsky, Jonathan;
Brini, Elisa;
Brown, Myriam;
Brun, Martín;
Bruns, Stephan;
Buliskeria, Nino;
Calef, Andrea;
Cameron, Alistair;
Campa, Pamela;
Campos-Rodríguez, Santiago;
Cantone, Giulio Giacomo;
Carpena, Fenella;
Carter, Perry;
Castañeda Dower, Paul;
Castek, Ondrej;
Caviglia-Harris, Jill;
Strand, Gabriella Chauca;
Chen, Shi;
Chzhen, Asya;
Chung, Jong;
Collins, Jason;
Coppock, Alexander;
Cordeau, Hugo;
Couillard, Ben;
Crechet, Jonathan;
Crippa, Lorenzo;
Cui, Jeanne;
Czymara, Christian;
Daarstad, Haley;
Dao, Danh Chi;
Dao, Dong;
Schmandt, Marco David;
de Linde, Astrid;
De Melo, Lucas;
Deer, Lachlan;
De Vera, Micole;
Dimitrova, Velichka;
Dollbaum, Jan Fabian;
Dollbaum, Jan Matti;
Donnelly, Michael;
Huynh, Luu Duc Toan;
Dumbalska, Tsvetomira;
Duncan, Jamie;
Duong, Kiet Tuan;
Duprey, Thibaut;
Dworschak, Christoph;
Ellingsrud, Sigmund;
Elminejad, Ali;
Eissa, Yasmine;
Erhart, Andrea;
Etingin-Frati, Giulian;
Fatemi-Pour, Elaheh;
Federice, Alexa;
Feld, Jan;
Fenig, Guidon;
Firouzjaeiangalougah, Mojtaba;
Fleisje, Erlend;
Fortier-Chouinard, Alexandre;
Engel, Julia Francesca;
Fries, Tilman;
Fortier, Reid;
Fréchet, Nadjim;
Galipeau, Thomas;
Gallegos, Sebastián;
Gangji, Areez;
Gao, Xiaoying;
Garnache, Cloé;
Gáspár, Attila;
Gavrilova, Evelina;
Ghosh, Arijit;
Gibney, Garreth;
Gibson, Grant;
Godager, Geir;
Goff, Leonard;
Gong, Da;
González, Javier;
Gretton, Jeremy;
Griffa, Cristina;
Grigoryeva, Idaliya;
Grøtting, Maja;
Guntermann, Eric;
Guo, Jiaqi;
Gugushvili, Alexi;
Habibnia, Hooman;
Häffner, Sonja;
Hall, Jonathan D.;
Hammar, Olle;
Kordt, Amund Hanson;
Hashimoto, Barry;
Hartley, Jonathan S.;
Hausladen, Carina I.;
Havránek, Tomáš;
Hazen, Jacob;
He, Harry;
Hepplewhite, Matthew;
Herrera-Rodriguez, Mario;
Heuer, Felix;
Heyes, Anthony;
Ho, Anson T. Y.;
Holmes, Jonathan;
Holzknecht, Armando;
Hsu, Yu-Hsiang Dexter;
Hu, Shiang-Hung;
Huang, Yu-Shiuan;
Huebener, Mathias;
Huber, Christoph;
Huynh, Kim P.;
Irsova, Zuzana;
Isler, Ozan;
Jakobsson, Niklas;
Frith, Michael James;
Jananji, Raphaël;
Jayalath, Tharaka A.;
Jetter, Michael;
John, Jenny;
Forshaw, Rachel Joy;
Juan, Felipe;
Kadriu, Valon;
Karim, Sunny;
Kelly, Edmund;
Dang, Duy Khanh Hoang;
Khushboo, Tazia;
Kim, Jin;
Kjellsson, Gustav;
Kjelsrud, Anders;
Kotsadam, Andreas;
Korpershoek, Jori;
Krashinsky, Lewis;
Kundu, Suranjana;
Kustov, Alexander;
Lalayev, Nurlan;
Langlois, Audrée;
Laufer, Jill;
Lee-Whiting, Blake;
Leibing, Andreas;
Lenz, Gabriel;
Levin, Joel;
Li, Peng;
Li, Tongzhe;
Lin, Yuchen;
Listo, Ariel;
Liu, Dan;
Lu, Xuewen;
Lukmanova, Elvina;
Luscombe, Alex;
Lusher, Lester R.;
Lyu, Ke;
Ma, Hai;
Mäder, Nicolas;
Makate, Clifton;
Malmberg, Alice;
Maitra, Adit;
Mandas, Marco;
Marcus, Jan;
Margaryan, Shushanik;
Márk, Lili;
Martignano, Andres;
Marsh, Abigail;
Masetto, Isabella;
McCanny, Anthony;
McManus, Emma;
McWay, Ryan;
Metson, Lennard;
Kinge, Jonas Minet;
Mishra, Sumit;
Mohnen, Myra;
Möller, Jakob;
Montambeault, Rosalie;
Montpetit, Sébastien;
Morin, Louis-Philippe;
Morris, Todd;
Moser, Scott;
Motoki, Fabio;
Muehlenbachs, Lucija;
Musulan, Andreea;
Musumeci, Marco;
Nabin, Munirul;
Nchare, Karim;
Neubauer, Florian;
Nguyen, Quan M. P.;
Nguyen, Tuan;
Nguyen-Tien, Viet;
Niazi, Ali;
Nikolaishvili, Giorgi;
Nordstrom, Ardyn;
Nüß, Patrick;
Odermatt, Angela;
Olson, Matt;
Øien, Henning;
Ölkers, Tim;
Oliver i Vert, Miquel;
Oral, Emre;
Oswald, Christian;
Ousman, Ali;
Özak, Ömer;
Pandey, Shubham;
Pavlov, Alexandre;
Pelli, Martino;
Penheiro, Romeo;
Park, RyuGyung;
Pérez Martel, Eva;
Petrovičová, Tereza;
Phan, Linh;
Prettyman, Alexa;
Procházka, Jakub;
Putri, Aqila;
Quandt, Julian;
Qiu, Kangyu;
Nguyen, Loan Quynh Thi;
Rahman, Andaleeb;
Rea, Carson H.;
Reiremo, Adam;
Renée, Laëtitia;
Richardson, Joseph;
Rivers, Nicholas;
Rodrigues, Bruno;
Roelofs, William;
Roemer, Tobias;
Rogeberg, Ole;
Rose, Julian;
Roskos-Ewoldsen, Andrew;
Rosmer, Paul;
Sabada, Barbara;
Saberian, Soodeh;
Salamanca, Nicolas;
Sator, Georg;
Sawyer, Antoine;
Scates, Daniel;
Schlüter, Elmar;
Sells, Cameron;
Sen, Sharmi;
Sethi, Ritika;
Shcherbiak, Anna;
Sogaolu, Moyosore;
Soosalu, Matt;
Sørensen, Erik Ø.;
Sovani, Manali;
Spencer, Noah;
Staubli, Stefan;
Stans, Renske;
Stewart, Anya;
Stips, Felix;
Stockley, Kieran;
Strobel, Stephenson;
Struby, Ethan;
Tang, John;
Tanrisever, Idil;
Yang, Thomas Tao;
Tastan, Ipek;
Tatić, Dejan;
Tatlow, Benjamin;
Seuyong, Féraud Tchuisseu;
Thériault, Rémi;
Thivierge, Vincent;
Tian, Wenjie;
Toma, Filip-Mihai;
Totarelli, Maddalena;
Tran, Van-Anh;
Truong, Hung;
Tsoy, Nikita;
Tuzcuoglu, Kerem;
Ubfal, Diego;
Villalobos, Laura;
Walterskirchen, Julian;
Wang, Joseph Taoyi;
Wattal, Vasudha;
Webb, Matthew D.;
Weber, Bryan;
Weisser, Reinhard;
Weng, Wei-Chien;
Westheide, Christian;
White, Kimberly;
Winter, Jacob;
Wochner, Timo;
Woerman, Matt;
Wong, Jared;
Woodard, Ritchie;
Wroński, Marcin;
Yazbeck, Myra;
Yang, Gustav Chung;
Yap, Luther;
Yassin, Kareman;
Ye, Hao;
Yoon, Jin Young;
Yurris, Chris;
Zahra, Tahreen;
Zaneva, Mirela;
Zayat, Aline;
Zhang, Jonathan;
Zhao, Ziwei;
Zhong Yaolang |
| Abstract: |
This study pushes our understanding of research reliability by reproducing and
replicating claims from 110 papers in leading economic and political science
journals. The analysis involves computational reproducibility checks and
robustness assessments. It reveals several patterns. First, we uncover a high
rate of fully computationally reproducible results (over 85%). Second,
excluding minor issues like missing packages or broken pathways, we uncover
coding errors for about 25% of studies, with some studies containing multiple
errors. Third, we test the robustness of the results to 5, 511 re-analyses. We
find a robustness reproducibility of about 70%. Robustness reproducibility
rates are relatively higher for re-analyses that introduce new data and lower
for re-analyses that change the sample or the definition of the dependent
variable. Fourth, 52% of re-analysis effect size estimates are smaller than
the original published estimates and the average statistical significance of a
re-analysis is 77% of the original. Lastly, we rely on six teams of
researchers working independently to answer eight additional research
questions on the determinants of robustness reproducibility. Most teams find a
negative relationship between replicators' experience and reproducibility,
while finding no relationship between reproducibility and the provision of
intermediate or even raw data combined with the necessary cleaning codes. |
| Keywords: |
Reproduction, Replication, Research Transparency, Open Science, Economics, Political Science |
| JEL: |
B41 C10 C81 |
| Date: |
2024 |
| URL: |
http://d.repec.org/n?u=RePEc:zbw:i4rdps:107&r=sog |