MC4 – Learning Mappings via Symbolic, Probabilistic, and Connectionist Modeling

Lecturer: Afsaneh Fazly
Fields: Machine Learning, Cognitive Modelling, Language Acquisition


In session 1, we cover the basics of several mapping (association) problems, including theoretically important challenges such as the acquisition of word meanings in young children, as well as applied settings such as learning multimodal or multilingual representations.

Session 2 focuses on the early approaches applied to a mapping problem, including symbolic and probabilistic methods.

Session 3 covers the more recent techniques (linear transformations and deep learning), in the context of several mapping problems, such as learning multimodal and multilingual mappings.


The objective is to cover three different approaches applied to the same problem of learning mappings across modalities (e.g., learning the meanings of words, learning mappings between audio/words and image/video segments, learning multilingual representations, etc.).


J.M. Siskind (1995). Grounding Language in PerceptionArtificial Intelligence Review, 8:371-391, 1995. [LINK]

J.M. Siskind (1996). A Computational Study of Cross-Situational Techniques for Learning Word-to-Meaning MappingsCognition, 61(1-2):39-91, October/November 1996. Also appeared in Computational Approaches to Language Acquisition, M.R. Brent, ed., Elsevier, pp. 39-91, 1996. [LINK]

Frank, M. C., Goodman, N. D., & Tenenbaum, J. B. (2009). Using speakers’ referential intentions to model early cross-situational word learningPsychological Science, 20, 579-585. [LINK]

Fazly, A., Alishahi A., Stevenson, S. (2010). A probabilistic computational model of cross-situational word learning. Cognitive Science: A Multidisciplinary Journal, 34(6): 1017—1063. [LINK]

Tadas Baltrusaitis, Chaitanya Ahuja, and Louis-Philippe Morency (2017). Multimodal Machine Learning: A Survey and Taxonomy. [LINK]

Zhang, Y., Chen, C.H., & Yu, C. (2019). Mechanisms of Cross-situational Learning: Behavioral and Computational Evidence. Advances in child development and behavior. [LINK]

Sebastian Ruder, Ivan Vulić, Anders Søgaard (2019). A Survey of Cross-lingual Word Embedding Models. Journal of Artificial Intelligence Research 65: 569-631. [LINK]


Dr. Afsaneh Fazly

Afsaneh Fazly is a Research Director at Samsung Toronto AI Centre, and an Adjunct Professor at the Computer Science Department of University of Toronto in Canada. Afsaneh has extensive experience in both academia and the industry, publishing award-winning papers, and building strong teams solving real-world problems. Afsaneh’s research draws on many subfields of AI, including Computational Linguistics, Cognitive Science, Computational Vision, and Machine Learning. Afsaneh strongly believes that solving many of today’s real-world problems requires an interdisciplinary approach that can bridge the gap between machine intelligence and human cognition.

Before joining Samsung Research, Afsaneh worked at several Canadian companies as Research Director, where she helped build and lead teams of outstanding scientists and engineers solving a diverse set of AI problems. Prior to that, Afsaneh was a Research Scientist and Course Instructor at the University of Toronto, where she also received her PhD from. Afsaneh lives in Toronto, with her husband and two young children. Afsaneh’s main hobby these days is reading and spending time with her family.

Affiliation: Samsung Toronto AI Centre

PC3 – Juggling – experience your brain at work

Lecturer: Susan Wache & Julia Wache
Fields: Neurobiology


In this course we will teach you how to juggle. Juggling is a motor activity that requires a lot of different skills. 

The activity of juggling requires a lot of different abilities. Obviously, you need to learn the movement pattern and practice a lot to get the reward – being able to juggle! To learn such specific movement patterns requires a highly complex electrical and chemical circuitry in the brain, which becomes a more and more important field of neuroscience. Juggling seems to encourage nerve fiber growth and therefore scientist believe it not only promotes brain fitness in general but could also help with debilitating illnesses.

Nevertheless, learning to juggle requires attention, focus, concentration and persistence. As every juggler would agree, the key for success is repetition. We will teach juggling mainly practical. While training you can feel constant progress independently of your previous skill level. 

In the last session you will also get an introduction to site swap, a mathematical description of juggling patterns you can notate, calculate and e.g. feed into a juggling simulator.

  1. Session: Basic introduction to juggling and the neuroscience behind it
  2. Session: How to learn juggling most effectively
  3. Session: Common mistakes and how to avoid them
  4. Session: Site swap – a mathematical description of juggling patterns


In this course you will learn to juggle with 3 balls, you will learn how to avoid common mistakes when practicing, how to improve effectively also when practicing on your own. Apart from the basic 3-ball-cascade you will learn additional simple patterns and get an introduction to advanced tricks and techniques.

All sessions are mainly practical training of juggling.


Susan Wache studied Cognitive Science at the University of Osnabrück. She worked in the Research Group feelSpace that investigates human senses and co-founded in 2015 the startup feelSpace that develops and sells naviBelts, tactile navigation devices especially for the visually impaired.

Julia Wache studied Cognitive Science in Vienna and Potsdam. She finished her PhD in Trento working on the Emotion Recognition via physiological signals and mental effort in the context of using tactile belts for orientation. In parallel she participated in the EIT Digital doctoral program to learn entrepreneurial skills. In 2016 she joined the feelSpace GmbH.

Together the sisters started juggling and performing over 20 years ago and gave courses for different audiences in various occasions.

Affiliation: feelSpace GmbH

FC11 – Artificial curiosity for robot learning in interaction with its environment and teachers

Lecturer: Sao Mai Nguyen
Fields: Machine learning, robot learning, reinforcement learning, goal babbling, active imitation learning


This course will provide an overview of research in machine learning and robotics of artificial curiosity. Also referred to as intrinsic motivation, this stream of algorithms inspired by theories of developmental psychology allow artificial agents to learn more autonomously, especially in stochastic high-dimensional environments, for redundant tasks, for multi-task, life-long or curriculum learning. The course will cover the following topics:

  • Basis of reinforcement learning
  • Curiosity-driven exploration
  • Goal babbling
  • Intrinsic motivation for imitation learning


The students will learn about the different uses of intrinsic motivation for motor control, and see several illustrations of application and implementation of intrinsically motivated exploration algorithms for motor control by embodied agents. They will also understand the importance of data sampling, exploration and source of information selection for robot learning. They will also have a practical experience on a simple robotic simulation setup.


  1. J. Schmidhuber. Formal theory of creativity, fun, and intrinsic motivation (1990-2010). IEEE Transactions on Autonomous Mental Development, 2(3):230–247, 2010.
  2. G. Baldassarre. What are intrinsic motivations? a biological perspective. In Development and Learning (ICDL), 2011 IEEE International Conference on, volume 2, pages 1–8. IEEE, 2011.
  3. J. Gottlieb and P.-Y. Oudeyer. Towards a neuroscience of active sampling and curiosity. Nature Reviews Neuroscience, 19(12):758–770, 2018.
  4. P.-Y. Oudeyer. The New Science of Curiosity, chapter Computational Theories of Curiosity-Driven Learning. NOVA, 02 2018.


Nguyen Sao Mai

Nguyen Sao Mai specialises in robotic learning, especially cognitive developmental learning. She is currently an associate professor at the UI2S Lab at Ensta Paris, France, after a few years in IMT Atlantique. She received a PhD in 2013 in computer science, for her studies on how to combine curiosity-driven exploration and socially guided exploration for multi-task learning and curriculum learning. She holds a master’s degree in computer science from Ecole Polytechnique and a master’s degree in adaptive machine systems from Osaka University. She has coordinated of the experiment KERAAL, funded by the European Union through project ECHORD++, which proposes an intelligent tutoring humanoid robot for physical rehabilitation. She is currently associate editor of IEEE TCDS and co-chair of the Task force “Action and Perception” du IEEE Technical Committee on Cognitive and Developmental Systems.

Affiliation: Ensta Paris