Humans (and many non-human animals) are apt at learning cumulatively and exploiting experiences prospectively to generate goal-directed behaviors with flexibility and creativity. Understanding the computational processes in brains that give rise to cognitive behaviors and endowing artificial agents (robots) with such brain-like mechanisms to enable them act cognitively like (and alongside) natural agents, form the long-term goals of my research. I explore these core problems by blending studies from cognitive neurosciences and experiments from developmental psychology into cognitive robotics (see Figure 1) to investigate the organization of learning, memory and reasoning in cumulatively developing systems (animals and robots). Imbibing emerging trends from neurosciences I endeavor to design brain-guided cognitive architectures, validate them by reenacting infant and animal behaviors on machines and finally port the ensuing developments into application domains. The research is of significance in understanding the brain-behavior relationship better and in designing cognitive artifacts that assist us in environments we inhabit and create, ultimately enhancing the quality of life. Looking at cognition from a holistic perspective and cognitive agents as integrated systems has led me contribute to many subdomains over the last few years of my career, particularly the following ones:

How are object concepts represented in our brain? Mounting evidence from functional imaging and connectomics suggests a) semantic knowledge is grounded in a distributed fashion in modality/property-specific cortical networks that directly support perception and action and that were active during learning (1); and b) brain exhibits small world topology to support segregated/specialized and distributed/integrated information processing (2). With my supervisor, I worked on a model imbibing these key principles that allow functional segregation of property features (e.g. shape, color, size, weight etc.) and their global integration at multi-modal hubs into object concepts (3). Through network dynamics for cross-modal, top-down and bottom-up communication, the proposed small-world of self-organizing maps facilitates learning of object concepts and both real perception and imagination of leant as well as novel objects.

How do infants acquire vocabulary progressively despite substantial referential uncertainty in what they hear and see? And if this learning happens through statistical co-occurrence tracking as suggested in literature (19), what neural mechanisms underlie these forms of cross-situational word learning? In this context, I with my colleagues propose a neurally grounded process account (WOLVES) of early vocabulary acquisition that integrates Word-Object Learning with Visual Exploration in Space across multiple timescales (20). Grounded in dynamical systems framework, the model addresses how top-down & bottom-up attention, working memory and long-term memory traces influences word learning and moment-to-moment looking behaviors; and how these systems co-evolve over development. Focusing on cross-situational word learning, WOLVES model successfully captures the visual dynamics and learning behavior of infants (see Figure 2) as reported in (19) such as the number of words learned by strong and weak learners, looking times to targets vs distractors, fixation counts and duration of fixations etc. The model replicates empirical studies that suggest competitive mechanisms involved in cross-situational learning (21) and explains how such mechanisms arise through dynamic interactions between memory, attention and incoming information. The model also confirms the direct role of contextual diversity in improving word-object binding in cross-situational learning tasks (22) while unveiling the underlying mechanisms involved. I have modelled eight more empirical studies on infants, young children and adults investigating how learning is shaped by partial knowledge, novelty bias, mutual exclusivity, selective attention and memory. Extensive simulations on these studies are going on and will be key to isolating different component processes that drive the vocabulary development.

What are the neural representations of sensory-motor experiences, how are they encoded, recalled and reconsolidated? Converging neuro-scientific studies reveal that episodic memory circuit involves higher-cortical areas to process sensorimotor information, interfacing para-hippocampal hubs and the hippocampus to integrate, encode and recall an experience (4). Additionally, evidence suggests that a network of cortical hubs (including para-hippocampal areas and hippocampus) known as Default Mode Network (DMN) is shared while recalling the past and simulation of the future, goal-directed planning and creative thinking (5). Imbibing trends on conceptual representation (1) and small world networks (2) in brain, I and my supervisor modelled a semantic-episodic memory system (3,6) using a excitatory-inhibitory auto-associative neural network that emulates brain-like organization of concepts and episodes; and mimics DMN by engaging a network of hubs (representing actions on various objects, the ensuing consequences, internal state of body, and rewards received) during encoding, recall and goal-directed reuse of experiences.

How do cognitive agents exploit their past experiences in the context of present to realize goals and reason into future to solve non-trivial problems creatively? A classic experiment from psychology investigating novel behavior was on Betty (7), that when faced with a non-trivial problem to pull out her dinner basket trapped in a vertical tube, recalled her year-old experience of bending flexible pipe cleaners to shape a hook out of a wire lying nearby and thereby was able to pull out the basket. With the hypothesis that cognition is a constructive manipulation of memory, I designed two experiments that mimicked the Betty task on iCub (6,8) investigating the reasoning and creativity capabilities of the proposed memory architecture. Through Results, I showed that the robot can recall context relevant past experiences, merge past experiences with explorative actions to learn new things and anticipate through episodic simulation the consequences of a recalled experience if relived in the present context. Then in novel situations where goals were directly unrealizable, the robot can creatively connect with the currently unfolding experience, the past experiences that are concussive towards realization of a goal at hand, thus executing a novel sequence of actions to realize goals in settings never experienced before.

How over cumulating experiences, causal relations between goals, actions and the objects in the environment are both abstracted and then exploited in novel contexts? Several experiments from psychology like the crow and Pitcher (9) task (from Aesop's Fables) show that corvids, apes as well as children are able to learn and infer causal-effect relations. Reenacting this Aesop's fable task on iCub humanoid in the context of open-ended learning-prediction-abstraction loop, I investigated the problem by employing the memory architecture and proposing four task-agnostic learning rules (elimination, growth, uncertainty, status-quo) that correlate predictions from remembered past experiences in the context of present to abstract underlying causal relations (10). The ensuing robot behaviors were found strikingly similar to children. We further demonstrated how by cumulatively interacting with few objects, the predictions of the robot in case of novel objects converges close to the physical law i.e. the Archimedes principle. We applied a simpler version of this architecture in another task of pushing where the iCub robot was able to successfully abstract causally dominant properties that influence motion of various objects when forces are exerted on them (11). An implicit advantage of this form of learning is that learnt ‘property-action’ relations can be effortlessly generalized to a domain of objects for which the robot need not have any experience but nevertheless share the ‘property’.

How does the brain coordinate action in redundant bodies as well as anticipate and understand actions of self and others? Emerging trends in motor neurosciences provide converging evidence that cortical networks in predominantly motor areas are also activated in imagination and observation of actions without causing any overt movement. Revisiting older ideas from motor control like Equilibrium Point Hypothesis and synergy formation in light of these emerging trends, I worked on a framework (12,13) proposing that actions are consequences of a simulation process that animates a plastic, expandable, configurable internal model of the body (namely the body schema), under the force-fields of intended goals/constraints. The framework provides a unified computational basis for actions with and without movements through emergence of muscle-less motor synergies thus facilitating a seamless continuum between motor control and motor imagery. The body schema (a) can be learnt any robotic embodiment and extended to coordinated tools; (b) can be composed into diverse forward/inverse models at runtime by coupling/decoupling different body (body + tool) chains with task relevant goals and constraints represented as multi-referential force fields; and (c) is computationally cheap operating through well-posed computations without kinematic inversions. I demonstrated the performance of the neural architecture through a range of motor tasks on both a 53-DoFs robot iCub and two industrial robots performing real world assembly emphasizing dexterity, accuracy, speed, obstacle avoidance, multiple task-specific constraints and task-based configurability (12,13).

How do humans imitate, learn and exhibit and recycle skilled actions including the use of tools? Building on a framework that proposes an abstract representation of movement i.e. its shape as the core component to skill learning, I showed through experiments how the motor knowledge acquired by a robot during learning a skill like drawing can be recycled in a completely different skill like the use of a tool (14). Mental imagery is employed for fine-tuning the force-fields and their timing of application on the body schema to shape the recycled skill to the new context. Using this architecture, I recently conducted experiments with both humans and humanoids to show how a range of common tool-use actions, hand-written scripts (Devanagari, Latin, Arabic etc.) and architectural designs can be learnt and generated by combining three basic shape primitives: line, bump and cusp (15).

How can multiple agents in spatially unstructured and temporally evolving shared environments plan their actions and cooperate to realize common goals? In a real-world industrial setting where multiple robots are jointly operating in a shared workspace to perform a joint assembly, I worked on the design of a biomimetic architecture that operates through coupled interactions between the robots’ body model (body schema) and an internal model of its peripersonal space (16). Based on reward field dynamics, the models engage in a range of inferences related to the feasibility and consequence of potential actions of oneself and other to either jointly or individually complete as many assemblies as possible and maximize their individual rewards.

What neural computations occur in our brains when we start to think what someone else is thinking about something; and how does this ability to take others’ perspective lead to social behaviors? Evidence suggests that there is extensive overlap in DMN-related cortical networks while remembering the past and those engaged during simulation of the future and adopting the perspective of the other (17). Using our DMN-like architecture, my supervisor & I designed a task for iCub in which the robot had to cooperate with a human in a tower-building task (18). By observing its counterpart human stacking objects, the robot recalls its own similar episodic experiences and infer the goal of the human i.e. to build the tallest stack. Thereafter, the robot stacks the rest of the objects one-top of other completing the goal. In extension to this work, I plan further to explore the role of episodic buffers in forming theories of mind through recursive episodic simulations.

1. Kiefer M, Pulvermüller F. Conceptual representations in mind and brain: Theoretical developments, current evidence and future directions. Cortex. 2012. p. 805–25. 2. van den Heuvel MP, Sporns O. Network hubs in the human brain. Trends in Cognitive Sciences. 2013. p. 683–96. 3. Bhat AA, Mohan V, Rea F, Sandini G, Morasso P. “Connecting experiences”: Towards a biologically inspired memory for developmental robots. IEEE ICDL-EPIROB 2014 - 4th Jt IEEE Int Conf Dev Learn Epigenetic Robot. 2014;359–65. 4. Allen T a, Fortin NJ. The evolution of episodic memory. Proc Natl Acad Sci U S A. 2013;110(2):10379–86. 5. Raichle ME. The brain’s default mode network. Annu Rev Neurosci. 2015;0. 6. Bhat AA. Towards a Brain-like “MEMORY” for Cognitive Robots.PhD Thesis, University of Genoa, Italy; 2016. 7. Weir A a S, Chappell J, Kacelnik A. Shaping of hooks in New Caledonian crows. Science. 2002;297(5583):981. 8. Bhat AA, Mohan V. Bending it like Betty: Connecting Memories to be Creative. Submitted. 9. Jelbert SA, Taylor AH, Cheke LG, Clayton NS, Gray RD. Using the aesop’s fable paradigm to investigate causal understanding of water displacement by new caledonian crows. PLoS One. Public Library of Science; 2014 Jan 26;9(3):e92895. 10. Bhat AA, Mohan V, Sandini G, Morasso P. Humanoid infers Archimedes’ principle: understanding physical relations and object affordances through cumulative learning experiences. J R Soc Interface. 2016;13(120):20160310. 11. Mohan V, Bhat AA, Sandini G, Morasso P. From object-action to property-action: Learning causally dominant properties through cumulative explorative interactions. Biol Inspired Cogn Archit. 2014;10(C):42–50. 12. Bhat AA, Akkaladevi SC, Mohan V, Eitzinger C, Morasso P. Towards a learnt neural body schema for dexterous coordination of action in humanoid and industrial robots. Auton Robots. Springer US; 2017;41(4):945–66. 13. Mohan V, Bhat AA, Morasso P. Muscleless Motor Synergies and Actions without Movements: From Motor Neuroscience to Cognitive Robotics. in Press, Phys Life Rev. 2018; 14. Bhat AA, Mohan V. How iCub Learns to Imitate Use of a Tool Quickly by Recycling the Past Knowledge Learnt During Drawing. In: Biomimetic and Biohybrid Systems. Springer International Publishing; 2015. p. 339–47. 15. Mohan V, Bhat AA, Morasso P. Why Human Skills, Scripts and Tools are the Way They are? Insights from Teaching Motor Skills to Humanoid Robots. Submitted. 16. Bhat AA, Mohan V. Goal-Directed Reasoning and Cooperation in Robots in Shared Workspaces: an Internal Simulation Based Neural Framework. Cognit Comput. Springer US; 2018 Apr 14;1–19. 17. Andrews-Hanna JR, Saxe R, Yarkoni T. Contributions of episodic retrieval and mentalizing to autobiographical thought: Evidence from functional neuroimaging, resting-state connectivity, and fMRI meta-analyses. Neuroimage. 2014;91:324–35. 18. Mohan V, Bhat AA. Joint Goal Human Robot collaboration-From Remembering to Inferring. Procedia Comput Sci. Elsevier; 2018 Jan 1;123:579–84. 19. Yu C, Smith LB. What you learn is what you see: Using eye movements to study infant cross-situational word learning. Dev Sci. 2011;14(2):165–80. 20. Bhat AA, Spencer JP, Samuelson LK. A dynamic neural field model of memory , attention and cross-situational word learning: The Model WOLVES. In: to appear in Proceedings of the 40th Annual Conference of the Cognitive Science Society. 2018. 21. Yurovsky D, Yu C, Smith LB. Competitive processes in cross-situational word learning. Cogn Sci. 2013;37(5):891–921. 22. Suanda SH, Mugwanya N, Namy LL. Cross-situational statistical word learning in young children. J Exp Child Psychol. 2014;126:395–411.