Hostname: page-component-8448b6f56d-mp689 Total loading time: 0 Render date: 2024-04-23T07:02:11.172Z Has data issue: false hasContentIssue false
  • English
  • Français

Self-reproduction for articulated behaviors with dual humanoid robots using on-line decision tree classification

Published online by Cambridge University Press:  24 June 2011

Jane Brooks Zurn ,
Yuichi Motai  and
Scott Vento
Jane Brooks Zurn
Affiliation:
School of Engineering, Virginia Commonwealth University, Richmond, VA 23284, USA. E-mail: brooks@med-associates.com
Yuichi Motai*
Affiliation:
School of Engineering, Virginia Commonwealth University, Richmond, VA 23284, USA. E-mail: brooks@med-associates.com Med Associates, Inc. P.O. Box 319, St. Albans, VT 05478, USA.
Scott Vento
Affiliation:
Chelsio Communications Inc. Sunnyvale, CA, USA. E-mail: svento@gmail.com
*
*Corresponding author. E-mail: ymotai@vcu.edu
Rights & Permissions [Opens in a new window]

Summary

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

We have proposed a new repetition framework for vision-based behavior imitation by a sequence of multiple humanoid robots, introducing an on-line method for delimiting a time-varying context. This novel approach investigates the ability of a robot “student” to observe and imitate a behavior from a “teacher” robot; the student later changes roles to become the “teacher” for a naïve robot. For the many robots that already use video acquisition systems for their real-world tasks, this method eliminates the need for additional communication capabilities and complicated interfaces. This can reduce human intervention requirements and thus enhance the robots' practical usefulness outside the laboratory. Articulated motions are modeled in a three-layer method and registered as learned behaviors using color-based landmarks. Behaviors were identified on-line after each iteration by inducing a decision tree from the visually acquired data. Error accumulated over time, creating a context drift for behavior identification. In addition, identification and transmission of behaviors can occur between robots with differing, dynamically changing configurations. ITI, an on-line decision tree inducer in the C4.5 family, performed well for data that were similar in time and configuration to the training data but the greedily chosen attributes were not optimized for resistance to accumulating error or configuration changes. Our novel algorithm, OLDEX identified context changes on-line, as well as the amount of drift that could be tolerated before compensation was required. OLDEX can thus identify time and configuration contexts for the behavior data. This improved on previous methods, which either separated contexts off-line, or could not separate the slowly time-varying context into distinct regions at all. The results demonstrated the feasibility, usefulness, and potential of our unique idea for behavioral repetition and a propagating learning scheme.

Keywords

Humanoid robotsMultirobot systemsRobot dynamicsRobotic self-replicationOn-line pattern recognition
Type
Articles
Information
Robotica , Volume 30 , Issue 2 , March 2012 , pp. 315 - 332
Creative Commons
The online version of this article is published within an Open Access environment subject to the conditions of the Creative Commons Attribution-NonCommercial-ShareAlike licence . The written permission of Cambridge University Press must be obtained for commercial re-use.
Copyright
Copyright © Cambridge University Press 2011. The online version of this article is published within an Open Access environment subject to the conditions of the Creative Commons Attribution-NonCommercial-ShareAlike licence <http://creativecommons.org/licenses/by-nc-sa/2.5/>. The written permission of Cambridge University Press must be obtained for commercial re-use.

References

1.Amit, R. and Matarić, M., “Learning Movement Sequences from Demonstration,” Proceedings of the International Conference on Development and Learning (ICDL '02), Cambridge, Massachusetts (2002) pp. 203–208.Google Scholar
2.Arsenio, A., “Children, Humanoid Robots and Caregivers,” Proceedings of the 4th International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems Children, Genoa, Italy (2004) vol. 117, pp. 19–26.Google Scholar
3.Atkeson, C. G., Moore, A. W. and Schaal, S., “Locally weighted learning,” Artif. Intell. Rev. 11 (1), 11–73 (1997).Google Scholar
4.Bentivegna, D. and Atkeson, C. G., “Using Primitives in Learning from Observation,” Proceedings of the 1st IEEE-RAS International Conference on Humanoid Robots, Boston, MA (2000).Google Scholar
5.Bentivegna, D. C. and Atkeson, C. G., “Learning from Observation Using Primitives,” Proceedings of the IEEE International Conference on Robotics and Automation (ICRA'01) (IEEE, Piscataway, NJ, USA, 2001), Seoul, Korea, vol. 2, pp. 1988–1993.Google Scholar
6.Billard, A. and Hayes, G., “DRAMA, a connectionist architecture for control and learning in autonomous robots,” Adapt. Behav. 7 (1), 35–63 (1999).CrossRefGoogle Scholar
7.Billard, A. and Matarić, M. J., “Learning human arm movements by imitation: Evaluation of a biologically inspired connectionist architecture,” Robot. Auton. Syst. 37 (2–3), 145–160 (2001).Google Scholar
8.Bongard, J. and Pfeifer, R., “Evolving Complete Agents Using Artificial Ontogeny,”. In: Morpho-functional Machines: The New Species (Designing Embodied Intelligence) (Springer-Verlag, Berlin, 2003) pp. 237–258.CrossRefGoogle Scholar
9.Bongard, J., Zykov, V. and Lipson, H., “Resilient machines through continuous self-modeling,” Science 314 (5802), 1118–1121 (2006).CrossRefGoogle ScholarPubMed
10.Boyd, R. S., 2009, “Robots are narrowing the gap with humans,” http://www.mcclatchydc.com/226/story/66530.htmlGoogle Scholar
11.Breiman, L., Friedman, J. H., Olshen, R. A. and Stone, C. J., Classification and Regression Trees (Wadsworth, Belmont, CA, 1984).Google Scholar
12.Calinon, S. and Billard, A., “Recognition and Reproduction of Gestures using a Probabilistic Framework combining PCA, ICA and HMM,” Proceedings of the International Conference on Machine Learning (ICML), Bonn, Germany, August 2005 (2005) pp. 105–112.Google Scholar
13.Calinon, S., Guenter, F., and Billard, A., “On learning, representing, and generalizing a task in a humanoid robot,” IEEE Trans. Syst. Man Cybern. B, 37 (2), 286–298 (2007).CrossRefGoogle Scholar
14.Cao, F. and Shepherd, B., “MIMIC: A Robot Planning Environment Integrating Real and Simulated Worlds,” Proceedings, IEEE International Symposium on Intelligent Control (IEEE, Piscataway, NJ, USA, Sep. 25–26, 1989), Albany, NY, USA, pp. 459–464.Google Scholar
15.Cole, E., “AMARSi project could see robots learn from co-workers,” Retrieved Mar. 17, 2010, http://www.wired.co.uk/news/archive/2010-03/12/amarsi-project-could-see-robots-learn-from-co-workers.aspxGoogle Scholar
16.Dey, A. K. and Abowd, G. D., “Towards a Better Understanding of Context and Context-Awareness,” Technical Report, GIT-GVU-99-22. Georgia Institute of Technology (1999).Google Scholar
17.Drumwright, E. and Matarić, M. J., “Generating and Recognizing Free-Space Movements in Humanoid Robots,” Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE, Piscataway, NJ, USA, 2003), Las Vegas, NV, USA, vol. 2, pp. 1672–1678.Google Scholar
18.Drury, J. L., Scholtz, J. and Yanco, H. A., “Awareness in Human-Robot Interactions,” Proceedings of the IEEE International Conference on Systems, Man and Cybernetics— (IEEE, Piscataway, NJ, USA, 2003), Washington, DC, USA, vol. 1, pp. 912–918.Google Scholar
19.Fong, T., Nourbakhsh, I. and Dautenhahn, K., “A Survey of Socially Interactive Robots,” Robotics and Autonomous Systems 42 (3–4), 143–166 (2003).Google Scholar
20.Friedman, J. H., “A recursive partitioning decision rule for nonparametric classification,” IEEE Trans. Comput. 26 (4), 404–408 (1977).CrossRefGoogle Scholar
21.Hamner, E., Gockley, R., Porter, E. and Nourbakhsh, I., “The personal rover project: The comprehensive design of a domestic personal robot,” Robot. Auton. Syst. Special Issue on Socially Interact. Robots 42 (3–4), 245–258 (2003).Google Scholar
22.Haritaoglu, I., Harwood, D. and Davis, L. S., “W4: Real-time surveillance of people and their activities,” IEEE Trans. Pattern Anal. Mach. Intell. 22 (8), 809–830 (2000).Google Scholar
23.Harries, M. B., Sammut, C. and Horn, K., “Extracting hidden context,” Mach. Learn. 32 (2), 101–126 (1998).CrossRefGoogle Scholar
24.Hulten, G., Spencer, L. and Domingos, P., “Mining Time-Changing Data Streams,” Paper Presented at the Proceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA (2001) pp. 97–106.Google Scholar
25.Hunt, E. B., Marin, J. and Stone, P. J., Experiments in Induction (Academic Press, New York, NY, USA, 1966).Google Scholar
26.Ijspeert, A. J., Nakanishi, J., Shibata, T. and Schaal, S., “Nonlinear Dynamical Systems for Imitation with Humanoid Robots,” Proceedings of the 2nd IEEE-RAS International Conference on Humanoid Robots, Tokyo, Japan, (IEEE, Piscataway, NJ, USA, 2001) pp. 219–226.Google Scholar
27.Inoue, Y., Tohge, T. and Iba, H., “Object Transportation by Two Humanoid Robots Using Cooperative Learning,” Proceedings of the 2004 Congress Evolutionary Computation, Portland, OR, USA (IEEE, Piscataway, NJ, USA, 2004) vol. 1, pp. 1201–1208.Google Scholar
28.Khalid, O., “A unified approach for motion and force control of robot manipulators: the operational space formulation,” IEEE J. Robot. Autom. 3 (1), 43–53 (1987).Google Scholar
29.Kim, B. and Lee, G., “Decision-Tree Based Error Correction for Statistical Phrase Break Prediction in Korean,” Paper presented at the Proceedings of the 18th Conference on Computational linguistics (COLING) (Morgan Kaufmann Publishers, Saarbrücken, Germany, San Francisco, CA, USA, 2000) vol. 2, pp. 1051–1055.Google Scholar
30.Klingspor, V., Demiris, J., and Kaiser, M., “Human-robot-communication and machine learning,” Applied Artificial Intelligence Journal 11 (7/8), 719–746 (1997).Google Scholar
31.Kondo Kagaku Co., Ltd., Jun. 30, 2008, Retrieved Nov. 7, 2008, http://www.kondo-robot.com/Google Scholar
32.Kosuge, K. and Oosumi, T., “Decentralized Control of Multiple Robots Handling an Object,” Proceedings of the IEEE Int. Conf. Intelligent Robots and Systems (IROS '96), Osaka, Japan (IEEE, Piscataway, NJ, USA, Nov. 4–8, 1996).Google Scholar
33.Kozima, H. and Yano, H., “A Robot that Learns to Communicate with Human Caregivers,” Proceedings of the 1st International Workshop on Epigenetic Robotics (Lund University Cognitive Studies, Lund, Sweden, Lund, Sweden, 2001).Google Scholar
34.Kruger, V., Herzog, D., Baby, S., Ude, A. and Kragic, D., “Learning actions from observations,” IEEE Robot. Autom. Mag. 17 (2), 30–43 (2010).CrossRefGoogle Scholar
35.Liu, J.-S., Liang, T.-C. and Lin, Y.-A., “Realization of a ball passing strategy for a robot soccer game: A case study of integrated planning and control,” Robotica 22 (3), 329–338 (2004).CrossRefGoogle Scholar
36.Loh, W.-Y. and Shih, Y.-S., “Split selection methods for classification trees,” Statistica Sinica 7, 815–840 (1997).Google Scholar
37.Martinoli, A., Ijspeert, A. J. and Gambardella, L. M., “A Probabilistic Model for Understanding and Comparing Collective Aggregation Mechanisms,” Proceedings of the 5th European Conference on Advances in Artificial Life (Springer-Verlag, Berlin/Heidelberg, 1999), Lausanne, Switzerland, vol. 1674, pp. 575–584.CrossRefGoogle Scholar
38.Matarić, M. J., “Reinforcement learning in the multi-robot domain,” Auton. Robots 4 (1), 73–83 (1997).Google Scholar
39.Matarić, M. J., “Sensory-Motor Primitives as a Basis for Imitation: Linking Perception to Action and Biology to Robotics,” In: Imitation in Animals and Artifacts (Dautenhahn, K. and Nehaniv, C. L., eds.), (MIT Press, Cambridge, MA, 2002) pp. 391–422.CrossRefGoogle Scholar
40.McCallum, R. A., “Hidden state and reinforcement learning with instance-based state identification,” IEEE Trans. Syst. Man Cybern. 26 (3), 464–473 (1996).Google Scholar
41.Motion Analysis, Inc., Retrieved Nov. 7, 2008, http://www.motionanalysis.comGoogle Scholar
42.Nicolescu, M. N. and Matarić, M. J., “Natural Methods for Robot Task Learning: Instructive Demonstrations, Generalization and Practice,” in Proceedings Second International Joint Conference on Autonomous Agents and Multi-Agent Systems pages 241–248, Melbourne, Australia, July 14–18, 2003.Google Scholar
43.A.P.A.S, Ariel Dynamics, Jun. 30, 2008, Retrieved Nov. 7, 2008, http://www.arielnet.com/Google Scholar
44.Pereira, G. A. S., Kumar, V., Spletzer, J. R., Taylor, C. J. and Campos, M. F. M., “Cooperative Transport of Planar Objects by Multiple Mobile Robots Using Object Closure,” In: Experimental Robotics VIII (Springer, Berlin/Heidelberg, 2003) vol. 5, pp. 287–296.CrossRefGoogle Scholar
45.Peters, J. and Schaal, S., “Policy Gradient Methods for Robotics,” Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Beijing, China, (2006) pp. 2219–2225.Google Scholar
46.Perzanowski, D., Schultz, A. C., Adams, W., Marsh, E. and Bugajska, M., “Building a multimodal human-robot interface,” Intell. Syst. 16 (1), 16–21 (2001).Google Scholar
47.Phillips, N., 2010, “The robots are cutting in on our dance moves,” http://www.smh.com.au/technology/technology-news/the-robots-are-cutting-in-on-our-dance-moves-20100827-13vzn.htmlGoogle Scholar
48.Piaget, J., Play, Dreams, and Imitation in Childhood (Gattegno, C. and Hodgson, F. M., Trans.) (Norton, New York, NY, USA, 1962 (translation), 1945 (French)).Google Scholar
49.Pollard, N. S., Hodgins, J. K., Riley, M. J. and Atkeson, C. G., “Adapting Human Motion for the Control of a Humanoid Robot,” Proceedings of the IEEE International Conference on Robotics and Automation (IEEE, Piscataway, NJ, USA, 2002) Washington, DC, USA, vol. 2, pp. 1390–1397.Google Scholar
50.Quinlan, J. R., “Discovering Rules by Induction from Large Collections of Examples,” In: Expert Systems in the Micro-electronic Age (Michie, D., ed.) (Edinburgh University Press, Edinburgh, UK, 1979).Google Scholar
51.Quinlan, J. R., “Induction of decision trees,” Mach. Learn. 1 (1), 81–106 (1986).CrossRefGoogle Scholar
52.Quinlan, J. R., C4.5: Programs for Machine Learning (Morgan Kaufmann Publishers, Inc., San Francisco, CA, USA, 1993).Google Scholar
53.Rescorla, R. A., “Probability of shock in the presence and absence of CS in fear conditioning,” J. Comp. Physiol. Psychol. 66, 1–5 (1968).CrossRefGoogle ScholarPubMed
54.Schaal, S., “Is imitation learning the route to humanoid robots?,” Trends Cogn. Sci. 3 (6), 233–242 (1999).Google Scholar
55.Schlimmer, J. C. and Fisher, D., “A Case Study of Incremental Concept Induction,” Proceedings of the 5th National Conference on Artificial Intelligence (Morgan Kaufmann, Philadelphia, PA, USA, 1986) Philadelphia, PA, USA, Vol. 1, pp. 495–501.Google Scholar
56.Schlimmer, J. C. and Granger, R. H., “Incremental learning from noisy data,” Mach. Learn. 1 (3), 317–354 (1986).CrossRefGoogle Scholar
57.Shah Hamzei, G. H., Mulvaney, D. J. and Sillitoe, I. P. W., “Batch-Mode Decision Tree Learning Applied to Intelligent Reactive Robot Control,” Proceedings of the 6th International Conference on Emerging Technologies and Factory Automation (ETFA '97) Los Angeles, CA, USA, (IEEE, Piscataway, NJ, USA, 1997) pp. 416–420.Google Scholar
58.Stauffer, C. and Grimson, W. E. L., “Learning patterns of activity using real-time tracking,” IEEE Trans. Pattern Anal. Mach. Intell. 22 (8), 747–757 (2000).Google Scholar
59.Steels, L. and Vogt, P., “Grounding Adaptive Language Games in Robotic Agents,” In Proceedings of the 4th European Conference on Artificial Life Brighton, UK, (MIT Press, Cambridge, MA, USA/London, 1997) pp. 474–482.Google Scholar
60.Tani, J., Nishimoto, R., Namikawa, J., and Ito, M., “Codevelopmental learning between human and humanoid robot using a dynamic neural-network model”, IEEE Trans. on Syst. Man and Cybern. Part B-Cybernetics 38 (1), pp. 43–59, 2008.Google Scholar
61.Tan, K. C., Chen, Y. J., Tan, K. K. and Lee, T. H., “Task-oriented developmental learning for humanoid robots,” IEEE Trans. Ind. Electron. 52 (3), 906–914 (2005).Google Scholar
62.Utgoff, P., Mar. 23, 2001, “Incremental tree induction,” Retrieved Nov. 7, 2008, http://www-lrn.cs.umass.edu/iti/index.htmlGoogle Scholar
63.Utgoff, P. E., “Incremental induction of decision trees,” Mach. Learn. 4 (2), 161–186 (1989).Google Scholar
64.Utgoff, P. E., Berkman, N. C. and Clouse, J. A., “Decision tree induction based on efficient tree restructuring,” Mach. Learn. 29 (1), 5–44 (1997).Google Scholar
65.Vicon Motion Systems, Oxford Metrics Ltd., Retrieved Nov. 7, 2008, http://www.vicon.comGoogle Scholar
66.Vijayakumar, S., D'Souza, A., Shibata, T., Conradt, J. and Schaal, S., “Statistical learning for humanoid robots,” Auton. Robots 12 (1), 55–69 (2002).CrossRefGoogle Scholar
67.Widmer, G. and Kubat, M., “Learning in the presence of concept drift and hidden contexts,” Mach. Learn. 23 (1), 69–101 (1996).CrossRefGoogle Scholar
68.Wren, C. R., Azarbayejani, A., Darrell, T. and Pentland, A. P., “Pfinder: real-time tracking of the human body,” IEEE Trans. Pattern Anal. Mach. Intell. 19 (7), 780–785 (1997).Google Scholar
69.Yanco, H. A., “Synthetic Robot Language Development,” In Proceedings of the Twelfth National Conference on Artificial Intelligence. Seattle, Washington, USA. AAAI Press/The MIT Press, 1994. p. 1500.Google Scholar
70.Yokokohji, Y., Kitaoka, Y. and Yoshikawa, T., “Motion capture from demonstrator's viewpoint and its application to robot teaching,” J. Robot. Syst. 22 (2), 87–97 (2005).CrossRefGoogle Scholar