ET4 – Biosignal Processing for Human-Machine Interaction

Lecturer: Tanja Schultz
Fields:

Content

Human interaction is a complex process involving modalities such as speech,
gestures, motion, and brain activities emitting a wide range of biosignals, which can be captured by a broad panoply of sensors. The processing and interpretation of these biosignals offer an inside perspective on human physical and mental activities and thus complement the traditional way of observing human interaction from the outside. As recent years have seen major advances in sensor technologies integrated into ubiquitous devices, and in machine learning methods to process and learn from the resulting data, the time is right to use of the full range of biosignals to gain further insights into the process of human-machine interaction.

In my talk I will present ongoing research at the Cognitive Systems Lab (CSL), where we
explore interaction-related biosignals with the goal of advancing machine-mediated human
communication and human-machine interaction. Several applications will be described such as Silent Speech Interfaces that rely on articulatory muscle movement captured by
electromyography to recognize and synthesize silently produced speech, as well as Brain
Computer Interfaces that use brain activity captured by electrocorticography to recognize
speech (brain-to-text) and directly convert electrocortical signals into audible speech (brain-to-speech). I will also describe the recording, processing and automatic structuring of human everyday activities based on multimodal high-dimensional biosignals within the framework of EASE, a collaborative research center on cognition-enabled robotics. This work aims to establish an open-source biosignals corpus for investigations on how humans plan and execute interactions with the aim of facilitating robotic mastery of everyday activities.

Objectives

None

Literature

None

Lecturer

Dr. Tanja Schultz

Tanja Schultz received her diploma (1995) and doctoral degree (2000) in Informatics from University of Karlsruhe and completed her Masters degree (1989) in Mathematics, Sports, and Educational Science from Heidelberg University, Germany.
Dr. Schultz is the Professor for Cognitive Systems at the University of Bremen, Germany and adjunct Research Professor at the Language Technologies Institute of Carnegie Mellon, PA USA. Since 2007, she directs the Cognitive Systems Lab, where her research activities include multilingual speech recognition and the processing of biosignals for human-centered technologies and applications. Since 2019 she is the spokesperson of Bremen’s high-profile area “Minds, Media, Machines”. Dr. Schultz is an Associate Editor of ACM Transactions on Asian Language Information Processing and serves on the Editorial Board of Speech Communication. She was President and elected Board Member of ISCA, and a General Co-Chair of Interspeech 2006. She is a Fellow of ISCA and member of the European Academy of Sciences and Arts. Dr. Schultz was the recipient of several awards including the Alcatel Lucent Award for Technical Communication, the PLUX Wireless Biosignals Award, the Allen Newell Medal for Research Excellence, and received the Speech Communication Best paper awards in 2001 and 2015.  

Affiliation: University of Bremen

ET1 – How to Know

Lecturer: Celeste Kidd
Fields: Developmental psychology, cognitive science, (and a tiny bit of neuroscience)

Content

This evening lecture will discuss Kidd’s research about how people come to know what they know. The world is a sea of information too vast for any one person to acquire entirely. How then do people navigate the information overload, and how do their decisions shape their knowledge and beliefs? In this talk, Kidd will discuss research from her lab about the core cognitive systems people use to guide their learning about the world—including attention, curiosity, and metacognition (thinking about thinking). The talk will discuss the evidence that people play an active role in their own learning, starting in infancy and continuing through adulthood. Kidd will explain why we are curious about some things but not others, and how our past experiences and existing knowledge shape our future interests. She will also discuss why people sometimes hold beliefs that are inconsistent with evidence available in the world, and how we might leverage our knowledge of human curiosity and learning to design systems that better support access to truth and reality.

Objectives

I hope to introduce students to the approach of combining computational models with behavioural experiments in order to develop robust theories of the systems that govern human cognition, especially attention, curiosity, and learning. We will take a very high-level conceptual approach to these topics, and I also hope students will leave understanding something useful about how people solve the problem of sampling from the world in order to understand something profound about it. I hope students will leave with a better understanding about how a person’s past experiences and expectations combine in a way that influences their subsequent sampling decisions and beliefs.    

Literature

Optional to read: Kidd, C., & Hayden, B. Y. (2015). The psychology and neuroscience of curiosity. Neuron88(3), 449-460.

https://www.cell.com/neuron/fulltext/S0896-6273(15)00767-9

Lecturer

Celeste Kidd is an Assistant Professor of Psychology at the University of California, Berkeley. Her lab investigates learning and belief formation using a combination of computational models and behavioural experiments. She earned her PhD in Brain and Cognitive Sciences at the University of Rochester, then held brief visiting fellow positions at Stanford’s Center for the Study of Language and Information and MIT’s Department of Brain and Cognitive Sciences before starting up her own lab. Her research has been funded by the National Science Foundation, the Jacobs Foundation, the Templeton Foundation, the Human Frontiers Science Program, Google, and the Berkeley Center for New Media. Kidd also advocates for equity in educational opportunities, work for which she was made one of TIME Magazines 2017 Persons of the Year as one of the “Silence Breakers”. 

Affiliation: University of California, Berkeley
Website: www.kiddlab.com

 

ET2 – Data-Driven Dynamical Models for Neuroscience and Neuroengineering

Lecturer: Bing W. Brunton
Fields: Computational neuroscience, Neuoengineering, Data Science

Content

Discoveries in modern neuroscience are increasingly driven by quantitative understanding of complex data. The work in my lab lies at an emerging, fertile intersection of computation and biology. I develop data-driven analytic methods that are applied to, and are inspired by, neuroscience questions. Projects in my lab explore neural computations in diverse organisms.  We work with theoretical collaborators on developing methods, and with experimental collaborators studying insects, rodents, and primates. The common theme in our work is the development of methods that leverage the escalating scale and complexity of neural and behavioural data to find interpretable patterns.

Lecturer

Bing Brunton is the Washington Research Foundation Innovation Associate Professor of Neuroengineering in the Department of Biology. She joined the University of Washington in 2014 as part of the Provost’s Initiative in Data-Intensive Discovery to build an interdisciplinary research program at the intersection of biology and data science. She also holds appointments in the Paul G. Allen School of Computer Science & Engineering and the Department of Applied Mathematics. Her training spans biology, biophysics, molecular biology, neuroscience, and applied mathematics (B.S. in Biology from Caltech in 2006, Ph.D. in Neuroscience from Princeton in 2012). Her group develops data-driven analytic methods that are applied to, and are inspired by, neuroscience questions. The common thread in this work is the development of methods that leverage the escalating scale and complexity of neural and behavioural data to find interpretable patterns. She has received the Alfred P. Sloan Research Fellowship in Neuroscience (2016), the UW Innovation Award (2017), and the AFOSR Young Investigator Program award (2018) for her work on sparse sensing with wing mechanosensory neurons.

Affiliation: University of Washington
Website: www.bingbrunton.com
Twitter: @bingbrunton

 

ET3 – Information as a Resource: How Organisms Deal with Uncertainty

Lecturer: Alex Kacelnik
Fields: Comparative cognition / decision-making, learning, problem solving, intelligence.

Content

Organisms nearly always act with incomplete information about the outcome of possible actions. They can include unpredictability into their decision process (risk sensitivity), or allocate effort to reduce uncertainty (learning, sampling). In all cases, the consequences of uncertainty, and the cost of reducing it, affect the expected payoffs, and hence can be expected to play a role in the decision mechanisms. Similarly, designers of synthetic intelligences are starting to include information-seeking (i.e. curiosity) in the behaviour of autonomous artificial systems, including problem-solving robots. I will present several lines of behavioural research in this area.

Objectives

This evening lecture is a research seminar introducing current and past but relevant research. Attendants should leave with at least a sense of what the problems are, and where some of the solutions are being sought.

Literature

  • Krebs, J.R., Kacelnik, A., Taylor, P., 1978. Test of optimal sampling by foraging great tits. Nature 275, 27–31. http://dx.doi.org/10.1038/275027a0.
  • Alex Kacelnik & Claire El Mouden. Triumphs and trials of the risk paradigm. Animal Behaviour 86 (2013) 1117-1129; http://dx.doi.org/10.1016/j.anbehav.2013.09.034.
  • Andrés Ojeda, Robin A. Murphy, & Alex Kacelnik. Paradoxical choice in rats: Subjective valuation and mechanism of choice. Behavioural Processes (2018) 152 73–80; https://doi.org/10.1016/j.beproc.2018.03.024.
  • Vasconcelos, M., Monteiro, T., Kacelnik, A., 2015. Irrational choice and the value of information. Sci. Rep. 5, 13874. http://dx.doi.org/10.1038/srep13874.

Lecturer

Alex Kacelnik studied biology in Buenos Aires, Argentina, and completed a PhD on decision making in Oxford, UK in 1979. He has been professor of Behavioural Ecology at Oxford since 1990 (emeritus since 2017). Alex has worked (and continues to work) on diverse topics, including decision-making, comparative cognition, brood parasitism, tool use, learning, and communication. His work bridges across behavioural ecology, behavioural economics, experimental psychology and, more recently, Artificial Intelligence. He is presently an external Principal Investigator in the cluster of excellence ‘Science of Intelligence’ (www.scienceofintelligence.de). 

Website: http://users.ox.ac.uk/~kgroup