RC1 – Homeostatically-driven behavioral architectures: How to model biological organisms throughout their life-cycle

Lecturer: Panagiotis Sakagiannis
Fields: Behavioral modeling, Systems Neuroscience, Robotics, Computational Ecology


Why do organisms behave? When do they take risks and when do rewards matter to them? What is the nervous system’s role in a successful life cycle and how does it relate to its evolutionary origins? In this course, we adopt a behavioral modeler’s view integrating insights from systems neuroscience, ecological energetics, and layered robotic architectures in order to sketch a framework for dynamic mechanistic models of biological behavior. We address the advantages and shortcomings of region-specific biologically realistic neurocomputational models, of agent-based ecological simulations and of optimality-driven intelligent artificial agents and discuss ways of combining these powerful computational tools with a focus on the persisting individual homeostasis. Nested behaviors, recurrent neural networks, and entangled spatiotemporal scales are our main modeling challenges. An intensively studied organism, the drosophila fruit fly larva, will serve as our model agent for the whole course.


Participants will benefit from an introduction to diverse scientific fields studying behavior or homeostasis, along with their computational tools. Philosophical debate on the normativity of behavior and mechanistic explanation will be touched upon, in the face of pressing modeling decisions. The valuable interaction between modelers and experimentalists will be highlighted. Finally, the delicate balance between detail and abstraction in behavioral modeling will be interactively discussed.


[1] S. a. L. M. Kooijman, “Dynamic Energy Budget theory for metabolic organisation : third edition,” Water, vol. 365, p. 68, 2010.
[2] T. J. Prescott, P. Redgrave, and K. Gurney, “Layered Control Architectures in Robots and Vertebrates,” Adaptive Behavior, pp. 99–127, 1999.
[3] M. J. Almeida-Carvalho et al., “The Ol1mpiad: Concordance of behavioural faculties of stage 1 and stage 3 Drosophila larvae,” J. Exp. Biol., vol. 220, no. 13, pp. 2452–2475, 2017.
[4] A. Campos-Candela, M. Palmer, S. Balle, A. Álvarez, and J. Alós, “A mechanistic theory of personality-dependent movement behaviour based on dynamic energy budgets,” Ecol. Lett., vol. 22, no. 2, pp. 213–232, 2018.
[5] W. Bechtel and A. Abrahamsen, “Dynamic mechanistic explanation: Computational modeling of circadian rhythms as an exemplar for cognitive science,” Stud. Hist. Philos. Sci. Part A, vol. 41, no. 3, pp. 321–333, 2010.


Panagiotis Sakagiannis:
Transitions across scientific fields are signs of both uneasiness and curiosity. In my case, a dual path can be traced, having medicine and clinical neurology on one side, mathematics and computational neuroscience on the other, while lately a PhD on insect behavioral modeling. Always seeking the broad picture when confronted with biological detail and the operationally useful formalization when attending philosophical debates, I still remain agnostic to my true inclination.

RC2 – Can patterns of word usage tell us what lemon and moon have in common? Analyzing the semantic content of distributional semantic models

Lecturer: Pia Sommerauer
Fields: Computational linguistics, cognitive linguistics


Can patterns of textual contexts in which words appear tell you (or your model) that both, a lemon and the moon are described as yellow and round but differ with respect to (almost) everything else? In other words: How much information about concepts is encoded in patterns of word usage (i.e. distributional data)?

In this course, I will take stock of what we know about the semantic content encoded in data-derrived meaning representations (e.g Word2Vec), which are commonly used in Natural Language Processing and cognitive modelling (e.g. metaphor interpretation).

I will focus on how we can find out whether (and what) semantic knowledge they represent (beyond a general sense of semantic word similarity and relatedness). Drawing on methods in the area of neural network interpretability, I will discuss how we can “diagnose” semantic knowledge to find out whether a model can in fact distinguish flying from non-flying birds or tell you what lemons and the moon have in common.


  • Become familiar with linguistic theories of the semantic encoded in linguistic context and what we could expect from it
  • Understand how distributional word representations are created, evaluated and used (with practical examples)
  • Understand why distributional word representations provide rich information for machine learning systems, but at the same time do not allow for straight-forward semantic interpretation
  • Understand the challenges of diagnostic methods and how they can be dealt with



Pia Sommerauer is a PhD student at the Computational Lexicology and Terminology Lab at Vrije Universiteit Amsterdam. Her research focuses on the type of semantic information captured by distributional representations of word meaning and whether they could be used for semantic reasoning. She has authored papers on this topic at venues specialized in lexical semantics and model interpretability together with her supervisors Antske Fokkens and Piek Vossen.

Website: https://piasommerauer.github.io/

RC3 – Representing Uncertainties in Artificial Neural Networks

Lecturer: Kai Standvoss
Fields: Computational Neuroscience, Artificial Intelligence


Tracking uncertainties is a key capacity of an intelligent system for successfully interacting within a changing environment. Representations of uncertainty are relevant to optimally weigh sensory evidence against prior expectations, to adjust learning rates accordingly, and importantly to trade-off exploitation and exploration. Thus, uncertainty is a crucial component of curiosity and reward driven behavior. Additionally, calibrated uncertainty estimates are relevant for human interaction as well as reliable artificial systems. However, it is not yet well understood how uncertainties are tracked in the brain.
Bayesian views on Deep Learning offer a way to specify distributions over model parameters and to learn generative models of data generation processes. Thereby, different levels and kind of uncertainties can be represented. In this course, we will discuss different Bayesian methods to track uncertainties in neural networks and speculate about possible links to neuroscience.


The objective of this course is to discuss the relevance of uncertainty for intelligent systems and its relationship to neural information processing. Participants will get an overview of Bayesian methods to calculate uncertainties in Deep Neural Networks.
After the course, participants should have the resources to choose the right tools for specific research questions or applications requiring explicit uncertainty estimates.
Open questions in the literature will be discussed.


  • Knill, D. C., & Pouget, A. (2004). The Bayesian brain: the role of uncertainty in neural coding and computation. TRENDS in Neurosciences, 27(12), 712- 719
  • Kingma, D. P., & Welling, M. (2013). Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114
  • Kendall, A., & Gal, Y. (2017). What uncertainties do we need in bayesian deep learning for computer vision?. In Advances in neural information processing systems (pp. 5574-5584).
  • Standvoss, K., & Grossberger, L. (2019). Uncertainty through Sampling: The Correspondence of Monte Carlo Dropout and Spiking in Artificial Neural Networks. In 2019 Conference on Cognitive Computational Neuroscience


Kai Standvoss

Kai obtained his bachelor’s degree in Cognitive Science from the University of Osnabrück. Later he studied Cognitive Neuroscience and Artificial Intelligence at the Donders Institute for Brain, Cognition, and Behavior. There he got interested in the representation of uncertainty and worked on a deep learning model of visual attention guided by uncertainty minimization. Currently he pursues a PhD at the Einstein Center for Neurosciences Berlin where he investigates visual metacognition.

ET4 – Biosignal Processing for Human-Machine Interaction

Lecturer: Tanja Schultz


Human interaction is a complex process involving modalities such as speech,
gestures, motion, and brain activities emitting a wide range of biosignals, which can be captured by a broad panoply of sensors. The processing and interpretation of these biosignals offer an inside perspective on human physical and mental activities and thus complement the traditional way of observing human interaction from the outside. As recent years have seen major advances in sensor technologies integrated into ubiquitous devices, and in machine learning methods to process and learn from the resulting data, the time is right to use of the full range of biosignals to gain further insights into the process of human-machine interaction.

In my talk I will present ongoing research at the Cognitive Systems Lab (CSL), where we
explore interaction-related biosignals with the goal of advancing machine-mediated human
communication and human-machine interaction. Several applications will be described such as Silent Speech Interfaces that rely on articulatory muscle movement captured by
electromyography to recognize and synthesize silently produced speech, as well as Brain
Computer Interfaces that use brain activity captured by electrocorticography to recognize
speech (brain-to-text) and directly convert electrocortical signals into audible speech (brain-to-speech). I will also describe the recording, processing and automatic structuring of human everyday activities based on multimodal high-dimensional biosignals within the framework of EASE, a collaborative research center on cognition-enabled robotics. This work aims to establish an open-source biosignals corpus for investigations on how humans plan and execute interactions with the aim of facilitating robotic mastery of everyday activities.






Dr. Tanja Schultz

Tanja Schultz received her diploma (1995) and doctoral degree (2000) in Informatics from University of Karlsruhe and completed her Masters degree (1989) in Mathematics, Sports, and Educational Science from Heidelberg University, Germany.
Dr. Schultz is the Professor for Cognitive Systems at the University of Bremen, Germany and adjunct Research Professor at the Language Technologies Institute of Carnegie Mellon, PA USA. Since 2007, she directs the Cognitive Systems Lab, where her research activities include multilingual speech recognition and the processing of biosignals for human-centered technologies and applications. Since 2019 she is the spokesperson of Bremen’s high-profile area “Minds, Media, Machines”. Dr. Schultz is an Associate Editor of ACM Transactions on Asian Language Information Processing and serves on the Editorial Board of Speech Communication. She was President and elected Board Member of ISCA, and a General Co-Chair of Interspeech 2006. She is a Fellow of ISCA and member of the European Academy of Sciences and Arts. Dr. Schultz was the recipient of several awards including the Alcatel Lucent Award for Technical Communication, the PLUX Wireless Biosignals Award, the Allen Newell Medal for Research Excellence, and received the Speech Communication Best paper awards in 2001 and 2015.  

Affiliation: University of Bremen

PC4 – Curious Making, Taking Fabrication Risks and Crafting Rewards

Lecturer: Janis Meißner
Fields: Design, Human Computer Interaction


This course is about getting hands-on curious with electronics and different crafts materials. Maker toolkits are a great way to get started with designing your own interactive sensor systems – but what if these designs could also integrate other (potentially more aesthetic) materials? E-textiles and paper circuits are good examples for how functional electronic systems can be recrafted with rewarding results. In principle, any every-day materials could be used with a bit of thinking outside the (tool)box. Let’s see what you will use to hack for your ideas!

Course Outline:

After a brief intro to microcontrollers and programming them with the Arduino IDE, participants will design their own simple input-output systems and gradually re-craft the hardware in innovative ways by using crafting materials such as for example paper, fabric and paperclips. Participants who seek a little extra-challenge are invited to work in small teams (2-4) to design an interactive artefact in this way that combines their respective research interests.

The course is structured as follows:

Session 1: Introduction to microcontrollers, off-the-shelf components and self-paced experimenting with the help of tutorials

Session 2: Designing an input-output system with off-the-shelf components. Starting to explore how ready-made components can be re-made with crafts materials.

Session 3-4: Recrafting your system design with craft materials of your choice. Don’t forget to present your inventions to your course mates so that everyone can applaud your creative hacking genius! 🙂


  • Learning the basics of programming electronics with microcontrollers
  • Learning the basics of how a selection of sensors and actuators work
  • Exploring alternative approaches to electronics than using o
  • Unleashing your creative hacking skills


Perner-Wilson, H., Buechley, L. & Satomi, M. (2011) ‘Handcrafting textile interfaces from a kit-of-no-parts’, in Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction – TEI ’11. New York, USA: ACM Press. p. 61. https://doi.org/10.1145/1935701.1935715

Posch, I. & Fitzpatrick, G. (2018) Integrating Textile Materials with Electronic Making. Proceedings of the Twelfth International Conference on Tangible, Embedded, and Embodied Interaction – TEI ’18. 158–165. https://doi.org/10.1145/3173225.3173255

Meissner, J.L., Strohmayer, A., Wright, P. & Fitzpatrick, G. (2018) ‘A Schnittmuster for Crafting Context-Sensitive Toolkits’, in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems – CHI ’18. New York, New York, USA: ACM Press. https://doi.org/10.1145/3173574.3173725


Janis Lena Meißner

Janis Lena Meißner is a doctoral trainee in Digital Civics at Open Lab, Newcastle University, and co-founder of fempower.tech, a group of intersectional feminists who aim to raise awareness of feminist issues in Human Computer Interaction. As maker technologies give individuals an opportunity to develop their own objects and tools, Janis is interested in exploring ways that these technologies can empower different non-technical communities who lack access to infrastructures such as fablabs or makerspaces. In her research she has collaborated with groups as diverse as urban knitters, glass artists, quilting sex workers, makers with disabilities and members of a Men Shed interested in combining their woodworking skills with 3D-printing. Using a Participatory Action Research methodology and a portable makerspace for adapting tool(kit)s to the specific contexts of making, her aim is to develop a community-driven approach to Making that allows people to weave in pre-existing crafting skills into their use of digital maker technologies.

Affiliation: Newcastle University
Websites: https://fempower.tech/ https://openlab.ncl.ac.uk/people/janis-lena-meissner/ https://twitter.com/janislena

IC4 – Introduction to Ethics in AI

Lecturer: Heike Felzmann
Fields: Ethics, AI


The last few years have seen an explosion of societal uses of AI technologies, but at the same time widespread public scepticism and fear about their use have emerged. In response to these concerns, a wide range of guidance documents for good practice in AI have been published by professional and societal actors recently. Both as researchers in AI and as consumers of AI it is helpful to understand ethical concepts and concerns associated with the use of AI and to be familiar with some of these guidance documents, in order to be able to reflect carefully on their ethical and social meaning and the balance of their benefits and risks and adapt one’s practices accordingly.

This course provides a general introduction to emergent ethical issues in the field of AI. It will be suitable for anyone with an interest in reflecting on how AI impacts on contemporary life and society. Over the four sessions of the course we will introduce and reflect on ideas and practical applications related to the following topics:

  • Understanding privacy, consent and transparency
  • Automated decision-making, algorithmic biases, autonomous artificial agents and accountability for decisions by artificial agents
  • Assistance, surveillance, persuasion, and human replacement
  • Responsible design and implementation, trustworthiness, and AI for good


The goal of the course is for participants to gain familiarity with core ethical concepts and concerns arising in the development and societal uses of AI, allowing participants to engage in a differentiated and informed manner with the societal debates on AI.


Eubanks, V. (2018). Automating inequality: How high-tech tools profile, police, and punish the poor. St. Martin’s Press. (on Google Books)

HLEG on AI (2019) Ethics Guidelines for Trustworthy AI, https://ec.europa.eu/futurium/en/ai-alliance-consultation 

Nissenbaum, H. (2019). Contextual Integrity Up and Down the Data Food Chain. Theoretical Inquiries in Law, 20(1), 221-256. http://www7.tau.ac.il/ojs/index.php/til/article/download/1614/1715 (ignore the abstract, which is much more obscure than the rest of the article! Contextual integrity is a useful theory of privacy.)

Zuboff, S. (2019) The Age of Surveillance Capitalism: The fight for a human future at the frontier of power. (Youtube interviews with Zuboff might be a good introduction.)


Heike Felzmann is a lecturer in Ethics in the School of History and Philosophy at NUI Galway, Ireland. She works on ethics in information technologies (especially on healthcare robots and AI), research ethics, and general health care ethics. She has been part of several European projects, including H2020 MARIO on a care robot for patients with dementia, H2020 ROCSAFE on robot supported incident response, COST 16116 on robotic exoskeletons, COST RANCARE on rationing in nursing care, ITN DISTINCT on technology use in dementia care, ERASMUS PROSPERO on education on social robots for social care, and was the chair of the COST Action CHIPME on innovations in genomics for health. She has also had extensive experience with research ethics governance and research ethics training. She teaches ethics widely across disciplines and is looking forward to meeting the interdisciplinary audience at the IK.

Website: http://www.nuigalway.ie/our-research/people/humanities/heikefelzmann/

Affiliation: NUI Galway

FC13 – Hominum-ex-Machina: About Artificial and Real Intelligence

Lecturer: Markus Krause
Fields: Human Computer Interaction, Artificial Intelligence (actually: Advanced Statistical Analysis and Pattern Recognition), Human Computation


Modern computational systems have amazing capabilities. They can detect a face or fingerprint in millions of samples, find a search term in a sea of billions of documents, and control the flow of trillions of dollars. Some of these abilities seem almost supernatural and even frightening. Yet, our brains are still the architects of invention and might remain to be so for aeons to come. Understanding and utilising the difference between machine und human intelligence is one of the new frontiers of computer science. With the advent of the next AI winter integrating human intervention into almost autonomous systems will gain crucial importance in the near future.

In this course we aim at lifting a bit of the mystic shroud that surrounds artificial intelligence. We will uncover its abilities, unveil short comings, and even conjure a deep neural network from (almost) thin air. You do not need to be an experienced coder or mathematical genius. Basic python understanding, and 8 grade math skills are enough to follow the course and build your own “AI”. After this hopefully disillusionary exercise we take a refreshing dive into reality. We will investigate real intelligence and how our brains talent for strategic problem solving can fuse with the sheer calculation power of machines. We will explore how these socio-technical systems will shape the future and the risks and pitfalls of the Hominum-ex-Machina.


Understanding the limitations of machine-based decision capabilities, the abilities setting humans apart from computers, and how human and machine abilities can fuse to form large scale computational systems.


Interesting AI Papers:
David Saxton: https://arxiv.org/pdf/1904.01557.pdf
Rumelhart et al: Learning internal representations by error-propagation
Krizhevsky et al: Imagenet classification with deep convolutional neural networks
Hochreiter, Schmidhuber: Long short-term memory
A Vaswan et al: Attention is all you need

HComp Papers:

First Paper about Human Computation and the inverse Turing test: http://www.wisdom.weizmann.ac.il/~naor/PAPERS/human.pdf

Book by Luis von Ahn and Edith Law basics about the inverted touring test at work: https://www.google.com/books/edition/Human_Computation/bF7ePcj-cUMC?hl=en&gbpv=1&printsec=frontcover

A set of interesting papers to take crowdsourcing to a higher complexity level:

Brian Christian’s account of participating in the Turing test yearly competition https://www.amazon.com/Most-Human-Artificial-Intelligence-Teaches/dp/0307476707


Dr. Markus Krause
Dr. Markus Krause

Dr. Markus Krause is a computer scientist, professional game designer, and serial entrepreneur. He co-founded Mooqita a Berkeley based Non-Profit supporting students in finding the job they love. Mooqita uses a novel approach combining human and machine intelligence. Dr. Krause also co-founded Brainworks.ai. Brainworks develops a new neural cortex to use smartphones as diagnostic tools for online health care applications. He is also the primary investigator for the Mooqita project at the International Computer Science Institute at UC Berkeley and part of the advisory committee to the DAAD IFI. Dr. Krause earned is doctoral degree in computer science from the University of Bremen, Germany and the Carnegie Mellon University in Pittsburgh, USA.

Websites: https://www.mooqita.org/

MC4 – Learning Mappings via Symbolic, Probabilistic, and Connectionist Modeling

Lecturer: Afsaneh Fazly
Fields: Machine Learning, Cognitive Modelling, Language Acquisition


In session 1, we cover the basics of several mapping (association) problems, including theoretically important challenges such as the acquisition of word meanings in young children, as well as applied settings such as learning multimodal or multilingual representations.

Session 2 focuses on the early approaches applied to a mapping problem, including symbolic and probabilistic methods.

Session 3 covers the more recent techniques (linear transformations and deep learning), in the context of several mapping problems, such as learning multimodal and multilingual mappings.


The objective is to cover three different approaches applied to the same problem of learning mappings across modalities (e.g., learning the meanings of words, learning mappings between audio/words and image/video segments, learning multilingual representations, etc.).


J.M. Siskind (1995). Grounding Language in PerceptionArtificial Intelligence Review, 8:371-391, 1995. [LINK]

J.M. Siskind (1996). A Computational Study of Cross-Situational Techniques for Learning Word-to-Meaning MappingsCognition, 61(1-2):39-91, October/November 1996. Also appeared in Computational Approaches to Language Acquisition, M.R. Brent, ed., Elsevier, pp. 39-91, 1996. [LINK]

Frank, M. C., Goodman, N. D., & Tenenbaum, J. B. (2009). Using speakers’ referential intentions to model early cross-situational word learningPsychological Science, 20, 579-585. [LINK]

Fazly, A., Alishahi A., Stevenson, S. (2010). A probabilistic computational model of cross-situational word learning. Cognitive Science: A Multidisciplinary Journal, 34(6): 1017—1063. [LINK]

Tadas Baltrusaitis, Chaitanya Ahuja, and Louis-Philippe Morency (2017). Multimodal Machine Learning: A Survey and Taxonomy. [LINK]

Zhang, Y., Chen, C.H., & Yu, C. (2019). Mechanisms of Cross-situational Learning: Behavioral and Computational Evidence. Advances in child development and behavior. [LINK]

Sebastian Ruder, Ivan Vulić, Anders Søgaard (2019). A Survey of Cross-lingual Word Embedding Models. Journal of Artificial Intelligence Research 65: 569-631. [LINK]


Dr. Afsaneh Fazly

Afsaneh Fazly is a Research Director at Samsung Toronto AI Centre, and an Adjunct Professor at the Computer Science Department of University of Toronto in Canada. Afsaneh has extensive experience in both academia and the industry, publishing award-winning papers, and building strong teams solving real-world problems. Afsaneh’s research draws on many subfields of AI, including Computational Linguistics, Cognitive Science, Computational Vision, and Machine Learning. Afsaneh strongly believes that solving many of today’s real-world problems requires an interdisciplinary approach that can bridge the gap between machine intelligence and human cognition.

Before joining Samsung Research, Afsaneh worked at several Canadian companies as Research Director, where she helped build and lead teams of outstanding scientists and engineers solving a diverse set of AI problems. Prior to that, Afsaneh was a Research Scientist and Course Instructor at the University of Toronto, where she also received her PhD from. Afsaneh lives in Toronto, with her husband and two young children. Afsaneh’s main hobby these days is reading and spending time with her family.

Affiliation: Samsung Toronto AI Centre