01
Reliable human-agent interaction
How can we train and evaluate LLM agents for reliable multi-turn interactions with people, including conversation and multi-step tool use?
AI research & engineering
Yun-Shiuan
(Sean) Chuang
Staff AI Research Scientist · PayPal AI Research
My research spans training, evaluation, and alignment of LLM agents with human preferences, beliefs, and opinions. My goal is to build reliable AI systems that interact well with people.
Research
LLM agents and human behavior
I study multi-turn conversation and tool use, how LLM agents represent human views, and how opinion dynamics emerge when LLM agents influence one another. These questions guide the LLM alignment and evaluation methods I develop for interactive systems.
02
Pluralistic alignment
How can we develop alignment algorithms that account for diverse human preferences, beliefs, and opinions?
03
Collective agent behavior
How can we measure and benchmark social influence and opinion dynamics in groups of LLM agents, and develop methods to align their collective behavior with human group behavior?
Selected work
Publications
2026 · ACL (Industry Track)
Toward Scalable Verifiable Reward: Proxy State-Based Evaluation for Multi-turn Tool-Calling LLM Agents
Chuang, Y.S., Kulkarni, C., Chiu, A. M., Thangali, A., Pan, Z., Shekhar, S., Ge, Y., Li, Y., Kona, U., Pang, L., & Mehrotra, P. (2026). In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track), 1251–1264.
2026 · NeurIPS (Evaluations and Datasets Track)
DEBATE: A Large-Scale Benchmark for Evaluating Opinion Dynamics in Role-Playing LLM Agents
Chuang, Y.S., Tu, R., Dai, C., Li, Y., Vasani, S., Yao, B., Tessler, M. H., Yang, S., Shah, D., Hawkins, R., Hu, J., & Rogers, T. T. (2026). arXiv:2510.25110. Accepted to NeurIPS 2026 (Evaluations and Datasets Track).
2025 · EMNLP (Main Track)
Probing LLM World Models: Enhancing Guesstimation with Wisdom of Crowds Decoding
Chuang, Y.S., Narendran, S., Harlalka, N., Cheung, A., Gao, S., Suresh, S., Hu, J., & Rogers, T. T. (2025). In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 4699–4713.
2025 · ICLR (Main Conference)
No Preference Left Behind: Group Distributional Preference Optimization
Yao, B., Cai, Z., Chuang, Y.S., Yang, S., Jiang, M., Yang, D., & Hu, J. (2025). In The Thirteenth International Conference on Learning Representations.
2024 · EMNLP (Findings)
Beyond Demographics: Aligning Role-playing LLM-based Agents Using Human Belief Networks
Chuang, Y.S., Nirunwiroj, K., Studdiford, Z., Goyal, A., Frigo, V. V., Yang, S., Shah, D. V., Hu, J., & Rogers, T. T. (2024). In Findings of the Association for Computational Linguistics: EMNLP 2024, 14010–14026.
2024 · NAACL (Findings)
Simulating Opinion Dynamics with Networks of LLM-based Agents
Chuang, Y.S., Goyal, A., Harlalka, N., Suresh, S., Hawkins, R., Yang, S., Shah, D., Hu, J., & Rogers, T. (2024). In Findings of the Association for Computational Linguistics: NAACL 2024, 3326–3346.
2024 · CogSci (Best Paper Award; Computational Modeling Prize; 4 of 1,200+ submissions)
The Delusional Hedge Algorithm as a Model of Human Learning from Diverse Opinions
Chuang, Y.S., Zhu, X., & Rogers, T. T. (2024). In Proceedings of the 46th Annual Conference of the Cognitive Science Society, 306–312. Computational Modeling Prize for Higher-Level Cognition.
Background
Professional Experience and Education
Now
PayPal AI Research
Staff AI Research Scientist
Post-training and evaluation for interactive LLM systems, including tool-calling agents and conversational recommendation.
2019–2024
University of Wisconsin–Madison
Ph.D. Student Researcher, Human-AI Lab
Research on LLM agents, social simulation, and pluralistic preference alignment.
Education
University of Wisconsin–Madison
Ph.D., Cognitive Science · M.S., Computer Science
2019–2024 · GPA: 4.0 / 4.0