Skip to main content Accessibility help
×
Home
  • Print publication year: 2011
  • Online publication date: November 2011

5 - Compromise strategies for action selection

from Part I - Rational and optimal decision making

Summary

Summary

Among many properties suggested for action selection mechanisms, a prominent one is the ability to select compromise actions, i.e., actions that are not the best to satisfy any active goal in isolation, but rather compromise between the multiple goals. This chapter briefly reviews the history of compromise behaviour and performs experimental analyses of it in an attempt to determine how much compromise behaviour aids an agent. It concludes that optimal compromise behaviour has a surprisingly small benefit over non-compromise behaviour in the experiments performed, it presents some reasons why this may be true, and hypothesises cases where compromise behaviour is truly useful. In particular, it hypothesises that a crucial factor is the level at which an action is taken (low level actions are specific, such as ‘move left leg’; high level actions are vague, such as ‘forage for food’). The chapter hypothesises that compromise behaviour is more beneficial for high-level actions than low-level actions.

Introduction

Agents act. An agent, be it a robot, animal, or piece of software, must repeatedly select actions from a set of candidates. A controller is the mechanism within an agent that selects the action. The question of how to design controllers for such agents is the action selection problem. Researchers who consider the action selection problem have identified potential properties of these controllers. One such property is the ability to exhibit compromise behaviour. A controller exhibits compromise behaviour when the agent has multiple conflicting goals, yet the action selected is not the optimal action for achieving any single one of those goals, but is good for achieving several of those goals in conjunction. For example, a predator stalking two prey might not move directly toward one of the prey, but in between the two, in case one flees (Hutchinson, 1999). The action would not be optimal for individual goals to catch either prey, instead being a compromise between them.

Related content

Powered by UNSILO
References
Antonelli, GArrichiello, FChiaverini, S 2008 The null-space-based behavioral control for autonomous robotic systemsIntell. Serv. Rob 1 27
Arkin, R 1998 Behavior-based RoboticsCambridge, MAMIT Press
Avila-Garcia, OCanamero, L 2004 Using hormonal feedback to modulate action selection in a competitive scenarioFrom Animals to Animats 8: Proceedings of the Eighth International Conference on Simulation of Adaptive BehaviorSchaal, SIjspeert, A. JBillard, ACambridge, MAMIT Press,243
Bailey, W. JCunningham, R. JLebel, L 1990 Song power, spectral distribution and female phonotaxis in the bushcricket (Tettigoniiddae: Orthoptera): active female choice or passive attractionAnim. Behav 40 33
Bartumeus, FCatalan, J 2009 Optimal search behavior and classic foraging theoryJ. Phys. A-Math. Theor 42 434002
Bertsekas, D. P 2005 Dynamic Programming and Optimal ControlBelmont, MAAthena Scientific
Blumberg, B. M 1994 Action-selection in Hamsterdam: lessons from ethologyFrom Animals to Animats 3: Proceedings of the Third International Conference on the Simulation of Adaptive BehaviorCliff, DHusbands, PMeyer, J. AWilson, SCambridge, MAMIT Press108
Blumberg, B. MTodd, P. MMaes, P 1996 No bad dogs: ethological lessons for learning in HamsterdamFrom Animals to Animats 4: Proceedings of the Fourth International Conference on Simulation of Adaptive BehaviorMaes, PMataric, MMeyer, J. APollack, JWilson, WCambridge, MAMIT Press295
Bonasso, R. PFirby, R. JGat, EKortenkamp, DMiller, D 1997 Experiences with an architecture for intelligent, reactive agentsJ. Exp. Theor. Artif. In 9 237
Bridle, S. LLahav, OOstriker, J. PSteinhardt, P. J 2003 Precision cosmology? Not just yetScience 299 1532
Brigant, I 2005 The instinct concept of the early Konrad LorenzJ. Hist. Biol 38 571
Brooks, R 1986 A robust layered control system for a mobile robotIEEE J. Robotic. Autom RA-2 14
Brooks, R 1997 From earwigs to humansRobot. Auton. Syst 20 291
Brown, J. SKotler, B 2004 Hazardous duty pay and the foraging cost of predationEcol. Lett 7 999
Bryson, J 2000 Hierarchy and sequence vs. full parallelism in action selectionFrom Animals to Animats 6: Proceedings of the Sixth International Conference on the Simulation of Adaptive BehaviorMeyer, J. ABerthoz, AFloreano, DRoitblat, HWilson, SCambridge, MAMIT Press,147
Burkhardt, R. W 2004 Patterns of Behavior: Konrad Lorenz, Niko Tinbergen, and the Founding of EthologyChicagoChicago University Press
Caldwell, G 1986 Predation as a selective force on foraging herons: effects of plumage color and flockingAuk 103 494
Cannings, CCruz Orive, L. M 1975 On the adjustment of the sex ratio and the gregarious behavior of animal populationsJ. Theor. Biol 55 115
Carruthers, P 2004 Practical reasoning in a modular mindMind Lang 19 259
Choset, HLynch, K. MHutchinson, S 2005 Principles of Robot Motion: Theory, Algorithms, and ImplementationsCambridge, MAMIT Press
Clemen, R 1996 Making Hard Decisions: An Introduction to Decision AnalysisBelmont, CADuxbury Press
Crabbe, F. L 2002 Compromise Candidates in Positive Goal ScenariosFrom Animals to Animats 7: Proceedings of the Seventh International Conference on the Simulation of Adaptive BehaviorHallam, BFloreano, DHallam, JHeyes, GMeyer, J. ACambridge, MAMIT Press105
Crabbe, F. L 2004 Optimal and non-optimal compromise strategies in action selectionFrom Animals to Animats 8: Proceedings of the Eighth International Conference on Simulation of Adaptive BehaviorSchaal, SIjspeert, A. JBillard, ACambridge, MAMIT Press,pp233
Crabbe, F. L 2007 Compromise strategies for action selectionPhil. Trans. R. Soc. Lond. B Biol. Sci 362 1559
Crabbe, F. LDyer, M. G 1999 Second-order networks for wall-building agentsProceedings of the International Joint Conference on Neural NetworksBrown, DWashington, DC, USAIEEE
de Sevin, EKallmann, MThalmann, D 2001 Towards real time virtual human life simulationsComputer Graphics InternationalHong KongIEEE Computer Society Press, pp31
Decugis, VFerber, J 1998 An extension of Maes’ action selection mechanism for animatsFrom Animals to Animats 5: Proceedings of the Fifth International Conference on Simulation of Adaptive BehaviorPfeifer, RBlumberg, BMeyer, J-AWilson, S. WCambridge, MAMIT Press153
Dewsbury, D. A 1978 fixed action pattern?Anim. Behav 26 310
Dewsbury, D. A 1992 Comparative psychology and ethology: a reassessmentAm. Psychol 47 208
Evans, JPatron, PSmith, BLane, M 2008 Design and evaluation of a reactive and deliberative collision avoidance and escape architecture for autonomous robotsAuton. Rob 24 247
Favilla, M 2002 Reaching movements: mode of motor programming influences programming by time itselfExp Brain Res 144 414
Fikes, R. ENilsson, N. J 1971 STRIPS: a new approach to the application of theorem proving to problem solvingArtif. Intell 2 189
Fraenkel, G. SGunn, D. L 1961 The Orientation of AnimalsNew YorkDover
Fraser, D. FCerri, R. D 1982 Experimental evaluation of predator–prey relationships in a patchy environment: consequences for habitat use patterns in minnowsEcology 63 307
Gat, E 1991 Reliable goal-directed reactive control for real-world autonomous mobile robotsVirginia Polytechnic Institute and State UniversityBlacksburg, VA
Gerevini, A. ESaetti, ASerina, I 2008 An approach to efficient planning with numerical fluents and multi-criteria plan qualityArtif. Intell 172 8
Ghallab, MNau, DTraverso, P 2004 Automated Planning: Theory and PracticeSan Francisco, CAMorgan Kaufmann Publishers Inc
Ghez, CFavilla, MGhilardi, M. F 1997 Discrete and continuous planning of hand movements and isometric force trajectoriesExp. Brain Res 115 217
Girard, BCuzin, VGuillot, AGurney, K. NPrescott, T. J 2002 Comparing a brain-inspired robot action selection mechanism with ‘winner-takes-all’From Animals to Animats 7: Proceedings of the Seventh International Conference on the Simulation of Adaptive BehaviorHallam, BFloreano, DHallam, JHeyes, GMeyer, J. ACambridge, MAMIT Press75
Grubb, T. CGreenwald, L 1982 Sparrows and a brushpile: foraging responses to different combinations of predation risk and energy costAnim. Behav 30 637
Hillier, F. SLieberman, G. J 2002 Introduction to Operations ResearchNew YorkMcGraw-Hill
Hinde, R. A 1966 Animal BehaviourNew YorkMcGraw-Hill
Hoffmann, J 2003 The Metric-FF planning system: translating ignoring delete lists to numeric state variablesJ. Artif. Intell. Res 20 291
Houston, A. IMcNamara, J. MSteer, S 2007 Do we expect natural selection to produce rational behaviorPhil. Trans. Roy. Soc. B 362 1531
Howard, R. A 1977 Risk preferenceReadings in Decision AnalysisMenlo Park, CASRI International
Humphrys, M 1996 Action Selection Methods Using Reinforcement LearningUniversity of CambridgeCambridge, UK
Hurdus, J. GHong, D. W 2009 Behavioral programming with hierarchy and parallelism in the DARPA Urban Challenge and RoboCupMultisensor Fusion and Integration for Intelligent SystemsHahn, HKo, HLee, SBerlinSpringer255
Hutchinson, J. M. C 1999 Bet-hedging when targets may disappear: optimal mate-seeking or prey-catching trajectories and the stability of leks and herdsJ. Theor. Biol 196 33
Hutchinson, J. M. CGigerenzer, G 2005 Simple heuristics and rules of thumb: where psychologists and behavioural biologists might meetBehav. Process 69 97
Iglesias, ALuengo, F 2005 New goal selection scheme for behavioral animation of intelligent virtual agentsIEICE Trans. Inf. and Syst E88-D 865
Jaafar, JMcKenzie, E 2008 A fuzzy action selection method for virtual agent navigation in unknown virtual environmentsJ. Uncertain Syst 2 144
Jones, R. MLaird, J. ENielsen, P. E 1999 Automated intelligent pilots for combat flight simulationAI Magazine 20 27
Krebs, J. RDavies, N 1997 Behavioural Ecology: An Evolutionary ApproachOxfordBlackwell Publishers
Krebs, J. RErichsen, J. TWebber, M. ICharnov, E. L 1977 Optimal prey selection in the great tit ()Anim. Behav 25 30
Latimer, WSippel, M 1987 Acoustic cues for female choice and male competition in Anim. Behav 35 887
Lima, S. L 1998 Stress and decision making under the risk of predation: recent developments from behavioral, reproductive, and ecological perspectivesAdv. Stud. Behav 27 215
Lorenz, K. Z 1981 The Foundations of EthologyNew YorkSpringer-Verlag
Lund, H. HWebb, BHallam, J 1997 A robot attracted to the cricket species Fourth European Conference on Artificial LifeHusbands, PHarvey, IBrighton, UKMIT Press/Bradford Books,246
Luo, XJennings, N. R 2007 A spectrum of compromise aggregation operators for multi-attribute decision makingArtif. Intell 171 161
Maes, P 1990 How to do the right thingConnect. Sci., Special Issue on Hybrid Systems 1 291
McNamara, J. MHouston, A. I 1980 The application of statistical decision theory to animal behaviourJ. Theor. Biol 85 673
McNamara, J. MHouston, A. I 1994 The effect of a change in foraging options on intake rate and predation rateAm. Nat 144 978
Mesterton-Gibbons, M 1989 On compromise in foraging and an experiment by Krebs . (1977)J. Math. Biol 27 273
Milinksi, M 1986 Constraints places by predators on feeding behaviorThe behavior of teleost fishesPitcher, T. JLondonCroom Helm, pp.236
Mitchell, T. M 1997 Machine LearningBostonMcGraw-Hill
Montes-Gonzales, FPrescott, T. JGurney, KHumphrys, MRedgrave, P 2000 An embodied model of action selection mechanisms in the vertebrate brainFrom Animals to Animats 6: Proceedings of the Sixth International Conference on the Simulation of Adaptive BehaviorMeyer, J. ABerthoz, AFloreano, DRoitblat, HWilson, SCambridge, MAMIT Press157
Morris, G. KKerr, G. EFullard, J. H 1978 Phonotactic preferences of female meadow katydidsCan. J. Zool 56 1479
Müller, A 1925 Über Lichtreaktionen von LandasselnZ. vergl. Physiol 3 113
Newell, ASimon, H. A 1976 Computer science as empirical inquiry: symbols and searchComm. ACM 19 113
Pfeifer, RScheier, C 1999 Understanding IntelligenceCambridge, MAMIT Press
Pirjanian, P 2000 Multiple objective behavior-based controlJ. Robot. Auton. Syst 31 53
Pirjanian, PChristensen, H. IFayman, J. A 1998 Application of voting to fusion of purposive modules: an experimental investigationJ. Robot. Auton. Syst 23 253
Quinlan, J. R 1983 Learning efficient classification procedures and their application to chess and gamesMachine Learning: an artificial intelligence approachMichalski, R. SCarbonell, J. GMitchell, T. MSan Mateo, CAMorgan Kaufmann, pp. 463–82
Quinlan, J. R 1993 C4.5: Programs for Machine LearningSan Mateo, CAMorgan Kaufmann
Refanidis, IVlahavas, I 2003 Multiobjective heuristic state-space planningArtif. Intell 145 1
Römer, H 1993 Environmental and biological constraints for the evolution of long-range signalling and hearing in acoustic insectsPhil. Trans. Roy. Soc. B 340 179
Russell, SNorvig, P 2010 Artificial Intelligence. A Modern ApproachCambridge, MAMIT Press
Seth, A. K 2007 The ecology of action selectionPhil. Trans. Roy. Soc. B 362 1545
Stephens, WKrebs, J. R 1986 Foraging TheoryPrinceton, NJPrinceton University Press
Thorisson, K. R 1996 Communicative humanoids: a computational model of psychosocial dialogue skillsMassachusetts Institute of TechnologyCambridge, MA
Thorpe, W. H 1979 The Origins and Rise of EthologyNew YorkPreager
Tinbergen, N 1950 The hierarchical organisation of nervous mechanisms underlying instinctive behaviorSympos. Soc. Exper. Biol 4 305
Todd, P. M 1992 Machine intelligence – the animat path to intelligent adaptive behaviourIEEE Computer 25 78
Tu, X 1996 Artificial animals for computer animation: biomechanics, locomotion, perception, and behaviorUniversity of TorontoDepartment of Computer Science
Tyrrell, T 1993 Computational mechanism for action selectionUniversity of Edinburgh
von Holst, E 1935 Über den Lichtruchenreflex bei FischenPubl. Staz. Zool 15 143
Werner, G. M 1994 Using second order neural connection for motivation of behavioral choicesProceedings of the Third International Conference on Simulation of Adaptive BehaviorCliff, DHusbands, PMeyer, J-AWilson, S. WBrighton, UKMIT Press,154