Algorithmic models of human decision making in Gaussian multi-armed bandit problems

P. Reverdy, V. Srivastava, and N.E. Leonard
Proc. of the European Control Conference, 2210-2215, 2014
Recipient of the Best Student Paper Award, ECC 2014.

(pdf)
We consider a heuristic Bayesian algorithm as a model of human decision making in multi-armed bandit problems with Gaussian rewards. We derive a novel upper bound on the Gaussian inverse cumulative distribution function and use it to show that the algorithm achieves logarithmic regret. We extend the algorithm to allow for stochastic decision making using Boltzmann action selection with a dynamic temperature parameter and provide a feedback rule for tuning the temperature parameter such that the stochastic algorithm achieves logarithmic regret. The stochastic algorithm encodes many of the observed features of human decision making.