|
on Artificial Intelligence |
| By: | Bohren, Noah (University of Lausanne); Hakimov, Rustamdjan (University of Lausanne); Lalive, Rafael (University of Lausanne) |
| Abstract: | Generative artificial intelligence (AI) has made substantial progress, but some capabilities of AI are not well understood. This study compares the ability of AI to a representative population of US adults in creative and strategic tasks. The creative ideas produced by AI chatbots are rated more creative than those created by humans. Moreover, ChatGPT is substantially more creative than humans, while Bard lags behind. Augmenting humans with AI improves human creativity, albeit not as much as ideas created by ChatGPT alone. Competition from AI does not significantly reduce the creativity of men, but it decreases the creativity of women. Humans who rate the text cannot discriminate well between ideas created by AI or other humans but assign lower scores to the responses they believe to be AI-generated. As for strategic capabilities, while ChatGPT shows a clear ability to adjust its moves in a strategic game to the play of the opponent, humans are, on average, more successful in this adaptation. |
| Keywords: | artificial intelligence, ChatGPT, Bard, creativity, experiment |
| JEL: | I24 J24 D91 C90 |
| Date: | 2024–09 |
| URL: | https://d.repec.org/n?u=RePEc:iza:izadps:dp17302 |
| By: | Ziyan Cui; Ning Li; Huaikang Zhou |
| Abstract: | Artificial Intelligence (AI) is increasingly being integrated into scientific research, particularly in the social sciences, where understanding human behavior is critical. Large Language Models (LLMs) like GPT-4 have shown promise in replicating human-like responses in various psychological experiments. However, the extent to which LLMs can effectively replace human subjects across diverse experimental contexts remains unclear. Here, we conduct a large-scale study replicating 154 psychological experiments from top social science journals with 618 main effects and 138 interaction effects using GPT-4 as a simulated participant. We find that GPT-4 successfully replicates 76.0 percent of main effects and 47.0 percent of interaction effects observed in the original studies, closely mirroring human responses in both direction and significance. However, only 19.44 percent of GPT-4's replicated confidence intervals contain the original effect sizes, with the majority of replicated effect sizes exceeding the 95 percent confidence interval of the original studies. Additionally, there is a 71.6 percent rate of unexpected significant results where the original studies reported null findings, suggesting potential overestimation or false positives. Our results demonstrate the potential of LLMs as powerful tools in psychological research but also emphasize the need for caution in interpreting AI-driven findings. While LLMs can complement human studies, they cannot yet fully replace the nuanced insights provided by human subjects. |
| Date: | 2024–08 |
| URL: | https://d.repec.org/n?u=RePEc:arx:papers:2409.00128 |
| By: | Jingru Jia; Zehua Yuan |
| Abstract: | This study explores the potential of large language models (LLMs) to conduct market experiments, aiming to understand their capability to comprehend competitive market dynamics. We model the behavior of market agents in a controlled experimental setting, assessing their ability to converge toward competitive equilibria. The results reveal the challenges current LLMs face in replicating the dynamic decision-making processes characteristic of human trading behavior. Unlike humans, LLMs lacked the capacity to achieve market equilibrium. The research demonstrates that while LLMs provide a valuable tool for scalable and reproducible market simulations, their current limitations necessitate further advancements to fully capture the complexities of market behavior. Future work that enhances dynamic learning capabilities and incorporates elements of behavioral economics could improve the effectiveness of LLMs in the economic domain, providing new insights into market dynamics and aiding in the refinement of economic policies. |
| Date: | 2024–09 |
| URL: | https://d.repec.org/n?u=RePEc:arx:papers:2409.08357 |
| By: | Vikram Krishnaveti; Saannidhya Rawat |
| Abstract: | Scholastic Aptitude Test (SAT) is crucial for college admissions but its effectiveness and relevance are increasingly questioned. This paper enhances Synthetic Control methods by introducing "Transformed Control", a novel method that employs Large Language Models (LLMs) powered by Artificial Intelligence to generate control groups. We utilize OpenAI's API to generate a control group where GPT-4, or ChatGPT, takes multiple SATs annually from 2008 to 2023. This control group helps analyze shifts in SAT math difficulty over time, starting from the baseline year of 2008. Using parallel trends, we calculate the Average Difference in Scores (ADS) to assess changes in high school students' math performance. Our results indicate a significant decrease in the difficulty of the SAT math section over time, alongside a decline in students' math performance. The analysis shows a 71-point drop in the rigor of SAT math from 2008 to 2023, with student performance decreasing by 36 points, resulting in a 107-point total divergence in average student math performance. We investigate possible mechanisms for this decline in math proficiency, such as changing university selection criteria, increased screen time, grade inflation, and worsening adolescent mental health. Disparities among demographic groups show a 104-point drop for White students, 84 points for Black students, and 53 points for Asian students. Male students saw a 117-point reduction, while female students had a 100-point decrease. |
| Date: | 2024–09 |
| URL: | https://d.repec.org/n?u=RePEc:arx:papers:2409.10750 |
| By: | Leonardo Gambacorta; Han Qiu; Shuo Shan; Daniel M Rees |
| Abstract: | In this paper we examine the effects of generative artificial intelligence (gen AI) on labour productivity. In September 2023, Ant Group introduced CodeFuse, a large language model (LLM) designed to assist programmer teams with coding. While one group of programmers used it, other programmer teams were not informed about this LLM. Leveraging this event, we conducted a field experiment on these two groups of programmers. We identified employees who used CodeFuse as the treatment group and paired them with comparable employees in the control group, to assess the impact of AI on their productivity. Our findings indicate that the use of gen AI increased code output by more than 50%. However, productivity gains are statistically significant only among entry-level or junior staff, while the impact on more senior employees is less pronounced. |
| Keywords: | artificial intelligence, productivity, field experiment, big tech |
| JEL: | D22 G31 R30 |
| Date: | 2024–09 |
| URL: | https://d.repec.org/n?u=RePEc:bis:biswps:1208 |
| By: | Lee, Jaehyun; Lim, Jihye; Hwang, Junseok; Lee, Junmin |
| Abstract: | As artificial intelligence (AI) advances, it has become involved in decision-making that can significantly affect people's lives. Organizations have adopted AI in their human resource planning which makes decisions over workers and impacts their livelihoods. For the impact of decisions, it can potentially lead to conflicting opinions within organizations. When workers form their opinions on AI decision-making, how do the counterparts surrounding them, namely the organization and the AI, affect their opinions? To answer this question, we analyze how workers' trust created by the organization and their perceptions arising from the AI influence their opinions on allowing AI to make decisions of recruitment and dismissal. We find that workers' trust in organizational management of AI significantly affects their permission for AI to decide who is recruited and dismissed. Also, we confirm that workers' perceptions on AI are significant factors influencing their trust in organizational management of AI. Our findings suggest that in organizations where workers' trust in organizational management of AI is established, AI can be applied to make critical decisions about individuals. These results imply that as AI becomes more commonly involved in organizations' decision-making, the role of managing trust within organizations will become increasingly significant. |
| Keywords: | artificial intelligence, worker trust, organizational decision, human resource, decision making |
| Date: | 2024 |
| URL: | https://d.repec.org/n?u=RePEc:zbw:itsb24:302513 |
| By: | Ali Merali |
| Abstract: | This paper derives 'scaling laws' -- empirical relationships between the amount of training compute used for a Large Language Model (LLM) and its performance -- for economic outcomes. In a preregistered experiment, 300 professional translators completed 1800 tasks with access to one of thirteen LLMs with differing model training compute sizes (or a control). Our results show that model scaling substantially raises productivity: for every 10x increase in model compute, translators completed tasks 12.3% quicker, received 0.18 s.d. higher grades, and earned 16.1% more per minute (including bonus payments). Further, the gains from model scaling are much higher for lower-skilled workers who gain a 4x larger improvement in task completion speed. These results imply further frontier model scaling -- which is currently estimated at 4x increase per year -- may have significant economic implications. |
| Date: | 2024–09 |
| URL: | https://d.repec.org/n?u=RePEc:arx:papers:2409.02391 |
| By: | Yueling Huang |
| Abstract: | This paper empirically investigates the impact of Artificial Intelligence (AI) on employment. Exploiting variation in AI adoption across US commuting zones using a shift-share approach, I find that during 2010-2021, commuting zones with higher AI adoption have experienced a stronger decline in the employment-to-population ratio. Moreover, this negative employment effect is primarily borne by the manufacturing and lowskill services sectors, middle-skill workers, non-STEM occupations, and individuals at the two ends of the age distribution. The adverse impact is also more pronounced on men than women. |
| Keywords: | Artificial intelligence; technology; labor; local labor markets; shift share |
| Date: | 2024–09–13 |
| URL: | https://d.repec.org/n?u=RePEc:imf:imfwpa:2024/199 |
| By: | Alexander Bick; Adam Blandin; David Deming |
| Abstract: | Generative Artificial Intelligence (AI) is a potentially important new technology, but its impact on the economy depends on the speed and intensity of adoption. This paper reports results from the first nationally representative U.S. survey of generative AI adoption at work and at home. In August 2024, 39 percent of the U.S. population age 18-64 used generative AI. More than 24 percent of workers used it at least once in the week prior to being surveyed, and nearly one in nine used it every workday. Historical data on usage and mass-market product launches suggest that U.S. adoption of generative AI has been faster than adoption of the personal computer and the internet. Generative AI is a general purpose technology, in the sense that it is used in a wide range of occupations and job tasks at work and at home. |
| Keywords: | generative artificial intelligence (AI); technology adoption; employment |
| JEL: | J24 O33 |
| Date: | 2024–09–20 |
| URL: | https://d.repec.org/n?u=RePEc:fip:fedlwp:98805 |
| By: | Doron Yeverechyahu; Raveesh Mayya; Gal Oestreicher-Singer |
| Abstract: | Generative AI (GenAI) has been shown to enhance individual productivity in a guided setting. While it is also likely to transform processes in a collaborative work setting, it is unclear what trajectory this transformation will follow. Collaborative environment is characterized by a blend of origination tasks that involve building something from scratch and iteration tasks that involve refining on others' work. Whether GenAI affects these two aspects of collaborative work and to what extent is an open empirical question. We study this question within the open-source development landscape, a prime example of collaborative innovation, where contributions are voluntary and unguided. Specifically, we focus on the launch of GitHub Copilot in October 2021 and leverage a natural experiment in which GitHub Copilot (a programming-focused LLM) selectively rolled out support for Python, but not for R. We observe a significant jump in overall contributions, suggesting that GenAI effectively augments collaborative innovation in an unguided setting. Interestingly, Copilot's launch increased maintenance-related contributions, which are mostly iterative tasks involving building on others' work, significantly more than code-development contributions, which are mostly origination tasks involving standalone contributions. This disparity was exacerbated in active projects with extensive coding activity, raising concerns that, as GenAI models improve to accommodate richer context, the gap between origination and iterative solutions may widen. We discuss practical and policy implications to incentivize high-value innovative solutions. |
| Date: | 2024–09 |
| URL: | https://d.repec.org/n?u=RePEc:arx:papers:2409.08379 |
| By: | Francesco D'Alessandro (Dipartimento di Politica Economica, DISCE, Università Cattolica del Sacro Cuore, Milano, Italy); Enrico Santarelli (, Department of Economics, University of Bologna, Italy - Global Labor Organization (GLO), Essen, Germany); Marco Vivarelli (Dipartimento di Politica Economica, DISCE, Università Cattolica del Sacro Cuore, Milano, Italy – UNU-MERIT, Maastricht, The Netherlands – IZA, Bonn, Germany) |
| Abstract: | In this paper we integrate the insights of the Knowledge Spillover Theory of Entrepreneurship and Innovation (KSTE+I) with Schumpeter's idea that innovative entrepreneurs creatively apply available local knowledge, possibly mediated by Marshallian, Jacobian and Porter spillovers. In more detail, in this study we assess the degree of pervasiveness and the level of opportunities brought about by AI technologies by testing the possible correlation between the regional AI knowledge stock and the number of new innovative ventures (that is startups patenting in any technological field in the year of their foundation). Empirically, by focusing on 287 Nuts-2 European regions, we test whether the local AI stock of knowledge exerts an enabling role in fostering innovative entry within AI-related local industries (AI technologies as focused enablers) and within non AI-related local industries, as well (AI technologies as generalised enablers). Results from Negative Binomial fixed-effect and Poisson fixed-effect regressions (controlled for a variety of concurrent drivers of entrepreneurship) reveal that the local AI knowledge stock does promote the spread of innovative startups, so supporting both the KSTE+I approach and the enabling role of AI technologies; however, this relationship is confirmed only with regard to the sole high-tech/AI-related industries. |
| Keywords: | KSTE+I, Artificial Intelligence, innovative entry, enabling technologies |
| JEL: | O33 L26 |
| Date: | 2024–09 |
| URL: | https://d.repec.org/n?u=RePEc:ctc:serie5:dipe0039 |
| By: | Martin Obschonka; Moren Levesque |
| Abstract: | The rapid expansion of AI adoption (e.g., using machine learning, deep learning, or large language models as research methods) and the increasing availability of big data have the potential to bring about the most significant transformation in entrepreneurship scholarship the field has ever witnessed. This article makes a pressing meta-contribution by highlighting a significant risk of unproductive knowledge exchanges in entrepreneurship research amid the AI revolution. It offers strategies to mitigate this risk and provides guidance for future AI-based studies to enhance their collective impact and relevance. Drawing on Akerlof's renowned market-for-lemons concept, we identify the potential for significant knowledge asymmetries emerging from the field's evolution into its current landscape (e.g., complexities around construct validity, theory building, and research relevance). Such asymmetries are particularly deeply ingrained due to what we term the double-black-box puzzle, where the widely recognized black box nature of AI methods intersects with the black box nature of the entrepreneurship phenomenon driven by inherent uncertainty. As a result, these asymmetries could lead to an increase in suboptimal research products that go undetected, collectively creating a market for lemons that undermines the field's well-being, reputation, and impact. However, importantly, if these risks can be mitigated, the AI revolution could herald a new golden era for entrepreneurship research. We discuss the necessary actions to elevate the field to a higher level of AI resilience while steadfastly maintaining its foundational principles and core values. |
| Date: | 2024–09 |
| URL: | https://d.repec.org/n?u=RePEc:arx:papers:2409.08890 |
| By: | Anthony Harding; Juan Moreno-Cruz |
| Abstract: | With the rapid expansion of Artificial Intelligence, there are expectations for a proportional expansion of economic activity due to increased productivity, and with it energy consumption and its associated environmental consequences like carbon dioxide emissions. Here, we combine data on economic activity, with early estimates of likely adoption of AI across occupations and industries, to estimate the increase in energy use and carbon dioxide emissions at the industry level and in aggregate for the US economy. At the industry level, energy use can increase between 0 and 12 PJ per year, while emissions increase between 47 tCO$_2$ and 272 ktCO$_2$. Aggregating across industries in the US economy, this totals an increase in energy consumption of 28 PJ per year, or around 0.03% of energy use per year in the US. We find this translates to an increase in carbon dioxide emissions of 896 ktCO$_2$ per year, or around 0.02% of the CO$_2$ emissions per year in the US. |
| Date: | 2024–09 |
| URL: | https://d.repec.org/n?u=RePEc:arx:papers:2409.06626 |
| By: | Cristian Trout |
| Abstract: | Many experts believe that AI systems will sooner or later pose uninsurable risks, including existential risks. This creates an extreme judgment-proof problem: few if any parties can be held accountable ex post in the event of such a catastrophe. This paper proposes a novel solution: a government-provided, mandatory indemnification program for AI developers. The program uses risk-priced indemnity fees to induce socially optimal levels of care. Risk-estimates are determined by surveying experts, including indemnified developers. The Bayesian Truth Serum mechanism is employed to incent honest and effortful responses. Compared to alternatives, this approach arguably better leverages all private information, and provides a clearer signal to indemnified developers regarding what risks they must mitigate to lower their fees. It's recommended that collected fees be used to help fund the safety research developers need, employing a fund matching mechanism (Quadratic Financing) to induce an optimal supply of this public good. Under Quadratic Financing, safety research projects would compete for private contributions from developers, signaling how much each is to be supplemented with public funds. |
| Date: | 2024–09 |
| URL: | https://d.repec.org/n?u=RePEc:arx:papers:2409.06672 |
| By: | OECD |
| Abstract: | Competition authorities have developed various tools to detect cartels and substantiate the basis for opening investigations. Ex officio investigations, meaning investigations initiated by the authorities themselves, are derived from detection tools that require a higher level of proactivity from the agency, for instance, industry monitoring and cartel screenings. New technologies such as artificial intelligence also provide competition authorities with greater opportunities to improve their detection tools. This paper provides an overview of detection tools to launch ex officio cartel investigations, including recent trends and experiences from Latin America and the Caribbean. It concludes by highlighting the need for competition authorities to implement a variety of approaches to complement one another and enhance cartel detection. |
| Date: | 2024–09–19 |
| URL: | https://d.repec.org/n?u=RePEc:oec:dafaac:311-en |
| By: | Sandy Chen; Leqi Zeng; Abhinav Raghunathan; Flora Huang; Terrence C. Kim |
| Abstract: | Large Language Models (LLMs) research in the financial domain is particularly complex due to the sheer number of approaches proposed in literature. Retrieval-Augmented Generation (RAG) has emerged as one of the leading methods in the sector due to its inherent groundedness and data source variability. In this work, we introduce a RAG framework called Mixture of Agents (MoA) and demonstrate its viability as a practical, customizable, and highly effective approach for scaling RAG applications. MoA is essentially a layered network of individually customized small language models (Hoffmann et al., 2022) collaborating to answer questions and extract information. While there are many theoretical propositions for such an architecture and even a few libraries for generally applying the structure in practice, there are limited documented studies evaluating the potential of this framework considering real business constraints such as cost and speed. We find that the MoA framework, consisting of small language models (Hoffmann et al., 2022), produces higher quality and more grounded responses across various financial domains that are core to Vanguard's business while simultaneously maintaining low costs. |
| Date: | 2024–09 |
| URL: | https://d.repec.org/n?u=RePEc:arx:papers:2409.07487 |
| By: | Shengkun Wang; Taoran Ji; Linhan Wang; Yanshen Sun; Shang-Ching Liu; Amit Kumar; Chang-Tien Lu |
| Abstract: | The stock price prediction task holds a significant role in the financial domain and has been studied for a long time. Recently, large language models (LLMs) have brought new ways to improve these predictions. While recent financial large language models (FinLLMs) have shown considerable progress in financial NLP tasks compared to smaller pre-trained language models (PLMs), challenges persist in stock price forecasting. Firstly, effectively integrating the modalities of time series data and natural language to fully leverage these capabilities remains complex. Secondly, FinLLMs focus more on analysis and interpretability, which can overlook the essential features of time series data. Moreover, due to the abundance of false and redundant information in financial markets, models often produce less accurate predictions when faced with such input data. In this paper, we introduce StockTime, a novel LLM-based architecture designed specifically for stock price data. Unlike recent FinLLMs, StockTime is specifically designed for stock price time series data. It leverages the natural ability of LLMs to predict the next token by treating stock prices as consecutive tokens, extracting textual information such as stock correlations, statistical trends and timestamps directly from these stock prices. StockTime then integrates both textual and time series data into the embedding space. By fusing this multimodal data, StockTime effectively predicts stock prices across arbitrary look-back periods. Our experiments demonstrate that StockTime outperforms recent LLMs, as it gives more accurate predictions while reducing memory usage and runtime costs. |
| Date: | 2024–08 |
| URL: | https://d.repec.org/n?u=RePEc:arx:papers:2409.08281 |
| By: | Junjie Li; Yang Liu; Weiqing Liu; Shikai Fang; Lewen Wang; Chang Xu; Jiang Bian |
| Abstract: | Generative models aim to simulate realistic effects of various actions across different contexts, from text generation to visual effects. Despite efforts to build real-world simulators, leveraging generative models for virtual worlds, like financial markets, remains underexplored. In financial markets, generative models can simulate market effects of various behaviors, enabling interaction with market scenes and players, and training strategies without financial risk. This simulation relies on the finest structured data in financial market like orders thus building the finest realistic simulation. We propose Large Market Model (LMM), an order-level generative foundation model, for financial market simulation, akin to language modeling in the digital world. Our financial Market Simulation engine (MarS), powered by LMM, addresses the need for realistic, interactive and controllable order generation. Key objectives of this paper include evaluating LMM's scaling law in financial markets, assessing MarS's realism, balancing controlled generation with market impact, and demonstrating MarS's potential applications. We showcase MarS as a forecast tool, detection system, analysis platform, and agent training environment. Our contributions include pioneering a generative model for financial markets, designing MarS to meet domain-specific needs, and demonstrating MarS-based applications' industry potential. |
| Date: | 2024–09 |
| URL: | https://d.repec.org/n?u=RePEc:arx:papers:2409.07486 |
| By: | Samuel Chang; Andrew Kennedy; Aaron Leonard; John List |
| Abstract: | We provide twelve best practices and discuss how each practice can help researchers accurately, credibly, and ethically use Generative AI (GenAI) to enhance experimental research. We split the twelve practices into four areas. First, in the pre-treatment stage, we discuss how GenAI can aid in pre-registration procedures, data privacy concerns, and ethical considerations specific to GenAI usage. Second, in the design and implementation stage, we focus on GenAI's role in identifying new channels of variation, piloting and documentation, and upholding the four exclusion restrictions. Third, in the analysis stage, we explore how prompting and training set bias can impact results as well as necessary steps to ensure replicability. Finally, we discuss forward-looking best practices that are likely to gain importance as GenAI evolves. |
| Date: | 2024 |
| URL: | https://d.repec.org/n?u=RePEc:feb:artefa:00796 |