{"page":"/blog/reinforcement-learning-bidding","modified":"2026-11-10","license":"https://creativecommons.org/licenses/by/4.0/","columns":["Risk","What happens","Containment"],"rows":[["Non-stationary market","A policy trained on last month's prices misjudges today's","Short retraining cycles; drift alerts on clearing prices"],["Reward misspecification","The policy hits the metric and misses the goal, for example cheap conversions of low value","Reward reviewed by the business owner; secondary metrics in every test"],["Interaction between bidders","Many learning bidders in one auction can behave in unexpected ways; see Paes Leme et al., \"Complex Dynamics in Autobidding Systems\" (EC 2024)","Damped updates; caps on how fast the multiplier can move"],["Overspend","The policy spends ahead of plan on a traffic spike","Hard per-interval spend caps outside the model"],["Opaque decisions","Nobody can say why spend shifted","Log state, action and reward per interval; see explainable AI in bidding"],["Distribution shift in training data","Offline data covers only past behaviour","Conservative updates; staged exploration"]],"sourceUrls":[[],[],["https://doi.org/10.1145/3670865.3673551"],[],[],[]]}