[PDF] Intrinsically Motivated Reinforcement Learning A Promising Framework For Developmental Robot Learning eBook
Intrinsically Motivated Reinforcement Learning A Promising Framework For Developmental Robot Learning Book in PDF, ePub and Kindle version is available to download in english. Read online anytime anywhere directly from your device. Click on the download button below to get a free pdf file of Intrinsically Motivated Reinforcement Learning A Promising Framework For Developmental Robot Learning book. This book definitely worth reading, it is an incredibly well-written.
One of the primary challenges of developmental robotics is the question of how to learn and represent increasingly complex behavior in a self-motivated, open-ended way Barto, Singh, and Chentanez (Barto, Singh, & Chentanez 2004; Singh, Barto, & Chentanez 2004) have recently presented an algorithm for intrinsically motivated reinforcement learning that strives to achieve broad competence in an environment in a task-nonspecific manner by incorporating internal reward to build a hierarchical collection of skills. This paper suggests that with its emphasis on task-general, self-motivated, and hierarchical learning, intrinsically motivated reinforcement learning is an obvious choice for organizing behavior in developmental robotics. We present additional preliminary results from a gridworld abstraction of a robot environment and advocate a layered learning architecture for applying the algorithm on a physically embodied system.
It has become clear to researchers in robotics and adaptive behaviour that current approaches are yielding systems with limited autonomy and capacity for self-improvement. To learn autonomously and in a cumulative fashion is one of the hallmarks of intelligence, and we know that higher mammals engage in exploratory activities that are not directed to pursue goals of immediate relevance for survival and reproduction but are instead driven by intrinsic motivations such as curiosity, interest in novel stimuli or surprising events, and interest in learning new behaviours. The adaptive value of such intrinsically motivated activities lies in the fact that they allow the cumulative acquisition of knowledge and skills that can be used later to accomplish fitness-enhancing goals. Intrinsic motivations continue during adulthood, and in humans they underlie lifelong learning, artistic creativity, and scientific discovery, while they are also the basis for processes that strongly affect human well-being, such as the sense of competence, self-determination, and self-esteem. This book has two aims: to present the state of the art in research on intrinsically motivated learning, and to identify the related scientific and technological open challenges and most promising research directions. The book introduces the concept of intrinsic motivation in artificial systems, reviews the relevant literature, offers insights from the neural and behavioural sciences, and presents novel tools for research. The book is organized into six parts: the chapters in Part I give general overviews on the concept of intrinsic motivations, their function, and possible mechanisms for implementing them; Parts II, III, and IV focus on three classes of intrinsic motivation mechanisms, those based on predictors, on novelty, and on competence; Part V discusses mechanisms that are complementary to intrinsic motivations; and Part VI introduces tools and experimental frameworks for investigating intrinsic motivations. The contributing authors are among the pioneers carrying out fundamental work on this topic, drawn from related disciplines such as artificial intelligence, robotics, artificial life, evolution, machine learning, developmental psychology, cognitive science, and neuroscience. The book will be of value to graduate students and academic researchers in these domains, and to engineers engaged with the design of autonomous, adaptive robots. The contributing authors are among the pioneers carrying out fundamental work on this topic, drawn from related disciplines such as artificial intelligence, robotics, artificial life, evolution, machine learning, developmental psychology, cognitive science, and neuroscience. The book will be of value to graduate students and academic researchers in these domains, and to engineers engaged with the design of autonomous, adaptive robots.
Neuromorphic and brain-based robotics have enormous potential for furthering our understanding of the brain. By embodying models of the brain on robotic platforms, researchers can investigate the roots of biological intelligence and work towards the development of truly intelligent machines. This book provides a broad introduction to this groundbreaking area for researchers from a wide range of fields, from engineering to neuroscience. Case studies explore how robots are being used in current research, including a whisker system that allows a robot to sense its environment and neurally inspired navigation systems that show impressive mapping results. Looking to the future, several chapters consider the development of cognitive, or even conscious robots that display the adaptability and intelligence of biological organisms. Finally, the ethical implications of intelligent robots are explored, from morality and Asimov's three laws to the question of whether robots have rights.
This book constitutes the refereed proceedings of the 9th European Conference on Artificial Life, ECAL 2007, held in Lisbon, Portugal. The 125 revised full papers cover morphogenesis and development, robotics and autonomous agents, evolutionary computation and theory, cellular automata, models of biological systems and their applications, ant colony and swarm systems, evolution of communication, simulation of social interactions, self-replication, artificial chemistry.
This book constitutes the proceedings of the International Conference on Adaptive and Intelligent Systems, ICAIS 2011, held in Klagenfurt, Austria, in September 2011. The 36 full papers included in these proceedings together with the abstracts of 4 invited talks, were carefully reviewed and selected from 72 submissions. The contributions are organized under the following topical sections: incremental learning; adaptive system architecture; intelligent system engineering; data mining and pattern recognition; intelligent agents; and computational intelligence.
This book constitutes the refereed proceedings of the 9th International Conference on Simulation of Adaptive Behavior, SAB 2006. The 35 revised full papers and 35 revised poster papers presented are organized in topical sections on the animat approach to adaptive behaviour, perception and motor control, action selection and behavioral sequences, navigation and internal world models, learning and adaptation, evolution, collective and social behaviours, applied adaptive behavior and more.
"Brain-inspired information technology" is one of key concepts for the development of information technology in the next generation. Explosive progress of computer technology has been continuing based on a simple principle called "if-then rule". This means that the programmer of software have to direct every action of the computer programs in response to various inputs. There inherently is a limitation of complexity because we human have a limited capacity for managing complex systems. Actually, many bugs, mistakes of programming, exist in computer software, and it is quite difficult to extinguish them. The parts of computer programs where computer viruses attack are also a kind of programming mistakes, called security hole. Of course, human body or nervous system is not perfect. No creator or director, however, exists for us. The function of our brain is equipped by learning, self-organization, natural selection, and etc, resulting in adaptive and flexible information system. Brain-inspired information technology is aiming to realize such nature-made information processing system by using present computer system or specific hardware. To do so, researchers in various research fields are getting together to inspire each other and challenge cooperatively for the same goal.
Even since computers were invented, many researchers have been trying to understand how human beings learn and many interesting paradigms and approaches towards emulating human learning abilities have been proposed. The ability of learning is one of the central features of human intelligence, which makes it an important ingredient in both traditional Artificial Intelligence (AI) and emerging Cognitive Science. Machine Learning (ML) draws upon ideas from a diverse set of disciplines, including AI, Probability and Statistics, Computational Complexity, Information Theory, Psychology and Neurobiology, Control Theory and Philosophy. ML involves broad topics including Fuzzy Logic, Neural Networks (NNs), Evolutionary Algorithms (EAs), Probability and Statistics, Decision Trees, etc. Real-world applications of ML are widespread such as Pattern Recognition, Data Mining, Gaming, Bio-science, Telecommunications, Control and Robotics applications. This books reports the latest developments and futuristic trends in ML.
Building autonomous machines that can explore large environments, discover interesting interactions and learn open-ended repertoires of skills is a long-standing goal in artificial intelligence. Inspired by the remarkable lifelong learning of humans, the field of developmental machine learning aims at studying the mechanisms enabling autonomous machines to self-organize their own developmental trajectories and grow their own repertoires of skills. This research makes steps towards that goal.Reinforcement learning methods (RL) train learning agents to control their environment by maximizing future rewards and, thus, seem adapted to our purpose. Although it achieved impressive results in the last decade--beating humans at video games, chess, go or controlling robotic agents--it falls short of solving our goal. Indeed, RL agents demonstrate low autonomy and open-endedness because they usually target a (small) set of pre-defined tasks characterized by hand-defined reward functions. In this research, we transfer, adapt and extend ideas from a developmental framework called intrinsically motivated goal exploration process (IMGEP) to the RL setting. The resulting framework builds on goal-conditioned RL techniques to design autotelic RL agents: agents that are intrinsically motivated to represent, generate, pursue and master their own goals as a way to grow repertoires of skills.The efficient acquisition of open-ended repertoires of skills further requires agents to creatively generate novel goals out of the domain of known effects (creative exploration), to readily generalize their understanding of known skills to similar ones (systematic generalization), and to compose known skills to form new ones (composition). Inspired by developmental psychology, we propose to use language as a cognitive tool to support such properties.We organize the manuscript around these two notions: goals and language. The first part focuses on goals. It covers foundational concepts and related work on intrinsic motivations, reinforcement learning and developmental robotics before introducing our framework, goal-conditioned intrinsically motivated goal exploration process (GC-IMGEP), the intersection of RL and IMGEPs. Building on this framework, we present three computational studies of the properties of autotelic agents. We first show that we can use autotelic exploration to solve external hard-exploration tasks (study 1: GEP-PG and 2: ME-ES). We then move on to reward-free environments and propose CURIOUS, an autotelic agent that targets a diversity of goals, transfers knowledge across skills and organizes its own learning trajectory by pursuing goals associated with high learning progress (study 3).The second part focuses on language. Inspired by the pioneering work of Vygotsky and others, we first discuss existing communicative and cognitive uses of language for goal-directed artificial agents. Language facilitates human-agent communications, abstraction, systematic generalization, long-horizon control, but also creativity and mental simulations. In two subsequent computational studies, we propose to implement these two last cognitive uses of language. IMAGINE uses language both to learn goal representations from social interactions (communicative use) and to imagine out-of-distribution goals used to drive its creative exploration and enhance systematic generalization (cognitive use). In our last study, LGB trains a language-conditioned world model to generate a diversity of possible futures conditioned on linguistic descriptions. This leads to behavioral diversity and strategy-switching behaviors.