{"page":"/blog/reinforcement-learning-bidding","modified":"2026-11-10","license":"https://creativecommons.org/licenses/by/4.0/","columns":["Year","Paper and venue","Authors","Idea","Deployment evidence (paper's claim)"],"rows":[["2017","RTB by RL in Display Advertising, WSDM","Han Cai, Kan Ren, Weinan Zhang, Kleanthis Malialis, Jun Wang, Yong Yu, Defeng Guo","Bidding as an MDP; neural value approximation for scale","Experiments on real-world auction data"],["2018","Budget Constrained Bidding by Model-free RL, CIKM","Di Wu, Xiujun Chen, Xun Yang, Hao Wang, Qing Tan, Xiaoxun Zhang, Jian Xu, Kun Gai","Learned reward design for constrained bidding","\"Easy to be deployed in large-scale industrial applications\""],["2018","RTB with Multi-Agent RL, CIKM","Junqi Jin, Chengru Song, Han Li, Kun Gai, Jun Wang, Weinan Zhang","Clustered advertisers, coordinated agents","Industry-scale real data"],["2018","Deep RL for Sponsored Search RTB, KDD","Jun Zhao, Guang Qiu, Ziyu Guan, Wei Zhao, Xiaofei He","Robust MDP at hourly level; control-by-model","Deployed in Alibaba's e-commerce search auction"],["2021","A Unified Solution to Constrained Bidding, KDD","Yue He, Xiujun Chen, Di Wu, Junwei Pan, Qing Tan, Chuan Yu and others","One framework for several constraint types (USCB)","n/d in sources read"],["2022","Sustainable Online RL for Auto-bidding, NeurIPS","Zhiyu Mou, Yusen Huo, Rongquan Bai, Mingzhou Xie, Chuan Yu, Jian Xu, Bo Zheng","Train against the live system with safe exploration","Simulated and real-world experiments"],["2024","AIGB: Generative Auto-bidding, KDD","Jiayan Guo, Yusen Huo, Zhilin Zhang, Tianyu Wang, Chuan Yu, Jian Xu, Yan Zhang, Bo Zheng","Diffusion model generates whole bid trajectories","Online A/B test at Alibaba: GMV +2.81%, ROI +3.36%"],["2024","AuctionNet, arXiv","Kefan Su, Yusen Huo, Zhilin Zhang, Shuai Dou, Chuan Yu, Jian Xu, Zongqing Lu, Bo Zheng","Public benchmark: environment, dataset, baselines","Derived from a real advertising platform"],["2025","Model-based Offline RL for Auto-bidding, arXiv","Zhiyu Mou, Miao Xu, Wei Chen, Rongquan Bai, Chuan Yu, Jian Xu","Learn an environment model from real data, penalise model errors","\"Real-world experiments\""]],"sourceUrls":[["https://arxiv.org/abs/1701.02490"],["https://arxiv.org/abs/1802.08365"],["https://arxiv.org/abs/1802.09756"],["https://arxiv.org/abs/1803.00259"],["https://doi.org/10.1145/3447548.3467199"],["https://arxiv.org/abs/2210.07006"],["https://arxiv.org/abs/2405.16141"],["https://arxiv.org/abs/2412.10798"],["https://arxiv.org/abs/2506.17919"]]}