Modeling Value-Based Decision-Making Policies Using Genetic Programming
A Proof-of-Concept Study
Abstract
Abstract. An important way to develop models in psychology and cognitive science is to express them as computer programs. However, computational modeling is not an easy task. To address this issue, some have proposed using artificial-intelligence (AI) techniques, such as genetic programming (GP) to semiautomatically generate models. In this paper, we establish whether models used to generate data can be recovered when GP evolves models accounting for such data. As an example, we use an experiment from decision-making which addresses a central question in decision-making research, namely, to understand what strategy, or “policy,” agents adopt in order to make a choice. In decision-making, this often means understanding the policy that best explains the distribution of choices and/or reaction times of two-alternative forced-choice decisions. We generated data from three models using different psychologically plausible policies and then evaluated the ability and extent of GP to correctly identify the true generating model among the class of virtually infinite candidate models. Our results show that, regardless of the complexity of the policy, GP can correctly identify the true generating process. Given these results, we discuss implications for cognitive science research and computational scientific discovery as well as possible future applications.
References
2020). Introduction to machine learning, MIT.
(1998). Genetic programming, Springer.
(2010). The mechanism of the Einstellung (set) effect: A pervasive source of cognitive bias. Current Directions in Psychological Science, 19(2), 111–115.
(2006). The physics of optimal decision-making: A formal analysis of models of performance in two-alternative forced-choice tasks. Psychological Review, 113(4), 700–765.
(2020). Comparison of magnitude-sensitive sequential sampling models in a simulation-based study. Journal of Mathematical Psychology, 94, 102298.
(2003). Bayesian strategy assessment in multi-attribute decision-making. Journal of Behavioral Decision Making, 16(3), 193–213.
(2012). New advances in understanding decisions among multiple alternatives. Current Opinion in Neurobiology, 22(6), 920–926.
(1995). Molecular geometry optimization with a genetic algorithm. Physical Review Letters, 75(2), 288–291.
(2007). Automatic generation of cognitive theories using genetic programming. Minds and Machines, 17(3), 287–309.
(2006). Simplifying decision trees learned by genetic programming. 2006 IEEE International Conference on Evolutionary Computation, 2142–2148.
(2009). Investigating intuitive and deliberate processes statistically: The multiple-measure maximum likelihood strategy classification method. Judgment and Decision Making, 4(3), 186.
(2020). Value-based attention but not divisive normalization influences decisions with multiple alternatives. Nature Human Behaviour, 4, 634–645. https://doi.org/10.1038/s41562-020-0822-0
(2011). Foundations of cognitive psychology, McGraw Hill.
(2005). Evolving structure-function mappings in cognitive neuroscience using genetic programming. Swiss Journal of Psychology, 64, 231–239.
(2014). Generalized outcome-based strategy classification: Comparing deterministic and probabilistic choice models. Psychonomic Bulletin & Review, 21(6), 1431–1443.
(2006). Nuance 3.0: Using genetic programming to model variable relationships. Behavior Research Methods, 38(2), 218–228.
(2010). Implementation of the multiple-measure maximum likelihood strategy classification method in R: Addendum to Glöckner (2009) and practical guide for application. Judgment and Decision Making, 5(1), 54.
(2011). Darwin’s “tug-of-war” vs. starlings’ “horse-racing”: How adaptations for sequential encounters drive simultaneous choice. Behavioral Ecology and Sociobiology, 65(3), 547–558.
(1992). Genetic Programming: On the programming of computers by means of natural selection, MIT.
(2004). Routine automated synthesis of five patented analog circuits using genetic programming. Soft Computing, 8(5), 318–324.
(2010). Visual fixations and the computation and comparison of value in simple choice. Nature Neuroscience, 13(10), 1292–1298.
(2012). The attentional drift-diffusion model extends to simple purchasing decisions. Frontiers in Psychology, 3, 193.
(2016).
(Analysing psychological data by evolving computational models . In A. WilhelmH. KestlerEds., Analysis of large and complex data (pp. 587–597). Springer.2013). Foundations of genetic programming, Springer Science & Business Media.
(2016). Bayesian outcome-based strategy classification. Behavior Research Methods, 48(1), 29–41.
(2004). Evolutionary antenna design for a NASA spacecraft. Genetic Programming Theory and Practice, II, 301–315.
(2013). A mechanism for value-sensitive decision-making. PLoS One, 8(9).
(2018). Evidence for the speed–value trade-off: Human and monkey decision-making is magnitude sensitive. Decision, 5(2), 129.
(2014). When natural selection should optimize speed-accuracy trade-offs. Frontiers in Neuroscience, 8, 73.
(2008). A field guide to genetic programming. https://www.Lulu.com
(2008). The diffusion decision model: Theory and data for two-choice decision tasks. Neural Computation, 20(4), 873–922.
(2016). Diffusion decision model: Current issues and history. Trends in Cognitive Sciences, 20(4), 260–281.
(2020). Pruning of genetic programming trees using permutation tests. Evolutionary Intelligence. Retrieved at http://eprints.whiterose.ac.uk/134313/1/PruningReport.pdf
(2019). Are reaction time transformations really beneficial? PsyArXiv. Retrieved at https://psyarxiv.com/9ksa6/
(2003). GPLAB – A genetic programming toolbox for MATLAB. Proceedings of the Nordic MATLAB Conference, 273–278.
(1959). Theories of decision-making in economics and behavioral science. The American Economic Review, 49(3), 253–283.
(1977). Models of discovery: And other topics in the methods of science, Reidel.
(2018). Attention and choice across domains. Journal of Experimental Psychology: General, 147(12), 1810–1826.
(2019). Optimal policy for multi-alternative decisions. Nature Neuroscience, 22(9), 1503–1511.
(2016). Optimal policy for value-based decision-making. Nature Communications, 7(1), 1–12.
(2003). Using genetic programming to discover nonlinear variable interactions. Behavior Research Methods, Instruments & Computers, 35(2), 202–216.
(