Jiri Gesi

Research

My research centers on post-training for LLMs.

Selected papers

Reasoning & Training

1. 2025
   Gödel-Prover-V2: Scaling Formal Theorem Proving with Scaffolded Data Synthesis and Self-Correction

   Yong Lin, …, Jiri Gesi, …, Chi Jin · 20 authors

   Yong Lin, Shange Tang, Bohan Lyu, Ziran Yang, Jui-Hui Chung, Haoyu Zhao, Lai Jiang, Yihan Geng, Jiawei Ge, Jingruo Sun, Jiayun Wu, Jiri Gesi, Ximing Lu, David Acuna, Kaiyu Yang, Hongzhou Lin, Yejin Choi, Danqi Chen, Sanjeev Arora, Chi Jin

   Introduces a theorem-proving LLM stack combining scaffolded data synthesis, verifier-guided self-correction, and model averaging. Achieves strong MiniF2F and PutnamBench results while remaining much smaller than prior SOTA.

2. 2025
   SFT Doesn’t Always Hurt General Capabilities: Revisiting Domain-Specific Fine-Tuning in LLMs

   Jiacheng Lin, …, Jiri Gesi, …, Lihong Li · 17 authors

   Jiacheng Lin, Zhongruo Wang, Kun Qian, Tian Wang, Arvind Srinivasan, Hansi Zeng, Ruochen Jiao, Xie Zhou, Jiri Gesi, Dakuo Wang, Yufan Guo, Kai Zhong, Weiqi Zhang, Sujay Sanghavi, Changyou Chen, Hyokun Yun, Lihong Li

   Re-examines the “SFT hurts generality” assumption. Shows small learning rates largely mitigate degradation while preserving target-domain gains; introduces TALR (Token-Adaptive Loss Reweighting) to further improve the trade-off.

3. 2024
   Beyond Self-Learned Attention: Mitigating Attention Bias in Transformer-Based Models Using Attention Guidance

   Jiri Gesi, Iftekhar Ahmed

   Introduces SyntaGuid, which guides attention toward syntax/AST signals to counter bias toward special tokens. Improves multiple software-engineering LLM tasks without extra data, correcting a notable share of prior errors.

Agents & Simulation

1. 2025
   Shop-R1: Rewarding LLMs to Simulate Human Behavior in Online Shopping via Reinforcement Learning

   Yimeng Zhang, …, Jiri Gesi, …, Dakuo Wang · 17 authors

   Yimeng Zhang, Tian Wang, Jiri Gesi, Ziyi Wang, Yuxuan Lu, Jiacheng Lin, Sinong Zhan, Vianne Gao, Ruochen Jiao, Junze Liu, Kun Qian, Yuxin Tang, Ran Xue, Houyu Zhang, Qingjun Cui, Yufan Guo, Dakuo Wang

   Frames shopper simulation as two-stage RL—first generate rationales, then predict actions—with tailored rewards. Improves realism and controllability versus task-completion baselines and enables downstream evaluation/policy analysis.

2. 2025
   OPeRA: A Dataset of Observation, Persona, Rationale, and Action for Evaluating LLMs on Human Online Shopping Behavior Simulation

   Ziyi Wang, …, Jiri Gesi, …, Dakuo Wang · 16 authors

   Ziyi Wang, Yuxuan Lu, Wenbo Li, Amirali Amini, Bo Sun, Yakov Bart, Weimin Lyu, Jiri Gesi, Tian Wang, Jing Huang, Yu Su, Upol Ehsan, Malihe Alikhani, Toby Jia-Jun Li, Lydia Chilton, Dakuo Wang

   A public dataset pairing user personas, browser observations, fine-grained web actions, and just-in-time rationales. Establishes benchmarks for next-action/rationale prediction and digital-twin evaluation.

3. 2025
   Prompting is Not All You Need! Evaluating LLM Agent Simulation Methodologies with Real-World Online Customer Behavior Data

   Yuxuan Lu, …, Jiri Gesi, …, Dakuo Wang · 9 authors

   Yuxuan Lu, Jing Huang, Yan Han, Bingsheng Yao, Sisong Bei, Jiri Gesi, Yaochen Xie, Qi He, Dakuo Wang

   Evaluates LLMs on web action generation using real shopping behavior data. Shows fine-tuning with real trajectories significantly outperforms prompting alone, and adding synthesized reasoning traces further improves accuracy.

4. 2025
   LLM Agent Meets Agentic AI: Can LLM Agents Simulate Customers to Evaluate Agentic-AI-Based Shopping Assistants?

   Lu Sun, …, Jiri Gesi, …, Dakuo Wang · 10 authors

   Lu Sun, Shihan Fu, Bingsheng Yao, Yuxuan Lu, Wenbo Li, Hansu Gu, Jiri Gesi, Jing Huang, Chen Luo, Dakuo Wang

   Combines a large-scale human study with persona-grounded “digital-twin” simulation. Shows simulations can mirror real-world task outcomes and UX dimensions, enabling faster and cheaper iteration on shopping assistants.

5. 2025
   Multi-Agent-as-Judge: Aligning LLM-Agent-Based Automated Evaluation with Multi-Dimensional Human Evaluation

   Jiaju Chen, …, Jiri Gesi, …, Dakuo Wang · 9 authors

   Jiaju Chen, Yuxuan Lu, Xiaojie Wang, Huimin Zeng, Jing Huang, Jiri Gesi, Ying Xu, Bingsheng Yao, Dakuo Wang

   Proposes MAJ-Eval—multi-persona evaluator agents that debate and aggregate judgments. Produces evaluations more aligned with expert ratings than typical automatic metrics in domains like education and medicine.

6. 2025
   UXAgent: A System for Simulating Usability Testing of Web Design with LLM Agents

   Yuxuan Lu, …, Jiri Gesi, …, Dakuo Wang · 10 authors

   Yuxuan Lu, Bingsheng Yao, Hansu Gu, Jing Huang, Jessie Wang, Yang Li, Jiri Gesi, Qi He, Toby Jia-Jun Li, Dakuo Wang

   A practical framework to simulate UX studies before recruiting participants. Generates personas and browsing agents, logs quantitative traces/interviews, and provides replays to help teams iterate rapidly on study design.