Human versus Machine Creativity: A Large-Scale Comparison of The Divergent Thinking Ability in Humans and Large Language Models

Artificial intelligence (AI) research traditionally focuses on knowledge-based problem-solving. Yet, as Einstein observed, “Imagination is more important than knowledge. Knowledge is limited. Imagination encircles the world.” As human-machine partnerships play an increasing role in addressing grand societal challenges, a deeper understanding of the comparative strengths of human and machine creativity is needed. This research examines human and large language model (LLM) creativity using the divergent association task, which assesses the ability to generate novel ideas. We compared the creativity test results of approximately 10,000 human participants with over 200,000 observations from various LLMs. Our findings reveal three key results. First, the average creativity level of humans is slightly higher than that of LLMs. However, differences in creativity are most pronounced at the extremes of the distribution, with human creativity exhibiting greater variability and a higher level of originality in the upper end of the distribution. Second, increasing the "creativity" parameter of LLMs enhanced performance up to a threshold, beyond which outputs became nonsensical. Third, attempts to improve LLM performance through prompt engineering yielded mixed to negative results. LLMs performed best when guided to focus on specific task components but performed significantly worse when instructed to assume the personas of creative geniuses in science, business, or the arts. Similarly, when LLMs were directed to take the creativity test from the perspective of different demographic groups, their performance declined and reversed the patterns observed in corresponding human demographic groups. We discuss the implications of these fundamental differences in human and LLM creativity for human-machine collaboration.
主讲人简介：
黄棣芳，中国科学院数学与系统科学研究院助理研究员。他的主要研究领域涵盖金融科技和人工智能，研究成果发表在自然科学和金融管理领域期刊，如PNAS、PNAS Nexus、Nature Human Behaviour、Management Science、Journal of Accounting Research、Journal of Financial and Quantitative Analysis、Econometric Reviews、Financial Management、Journal of Corporate Finance、Journal of Financial Intermediation、Journal of Economic Behavior and Organization、Journal of Empirical Finance等。