Evaluating the ability of large language models to predict human social decisions

Two studies compared GPT-3.5, GPT-4, and GPT-4o against human decisions across social scenarios, revealing LLMs differ in risk framing and social sensitivity, often misaligning with human patterns.

Researchers evaluated GPT-3.5, GPT-4, and GPT-4o on their ability to predict human social decisions across 51 scenarios (9,600 responses) and additional social-group contexts (1,600 responses).

Results showed notable discrepancies: LLMs were less sensitive to kinship and group size, displayed risk preferences differing from human patterns e.g., GPT-4 was consistently risk-averse and framed decisions in ways humans do not.

These findings highlight both the predictive power and limitations of LLMs in modeling human social behavior.

No items found.