Reducing Complex Dynamics to Low-Dimensional Mechanisms

My current research focus lies at the intersection of Reinforcement Learning and complex swarm dynamics. Collective behavior is puzzling and fascinating at the same time. Seemingly complex dynamics from low-dimensional generating processes. That is what I love. Ultimately, I want to understand the mechanisms that underlie the development of such processes. My hypothesis is that reward and gradient-based learning greatly enhances the efficacy of evolutionary processes (see e.g. AlphaStar). Furthermore, language, communication and emergent protocols provide the key to coordinated learning in highly non-stationary multi-agent environments. In summary, I want to solve the inter-temporal as well as inter-agent credit assignment problem.

So why is this important to all of us?

The mechanisms underlying adaptation, learning and communication in large-scale collectives lies is the core of our social existence. Furthermore, the fundamental mechanisms provide a promising interface for shepherding and agent-based nudging. By spawning an artificial shepherd which acts within the agents environment, one is able to guide the behavior in a normative fashion. The applications are endless and include support for collectives of traveling animals, risk reduction for large pilgrimages and festivals as well as macroeconomic guidance of household behavior. Let’s make the world a better place!

How did I get to where I am at right now?

My decision to study Economics was driven by the desire to understand and ultimately solve social inequality and poverty. During this time I worked as a student research assistant for Prof. Alex Ludwig (Goethe University Frankfurt) and Prof. Helge Braun (Ruhr Graduate School in Economics). The projects mainly focused on public policy evaluation and the intersection of retirement and unemployment insurance systems. I developed a fascination for data wrangling and the computational aspects of Economics, Cognitive Sciences and Decision Making. This led to an amazing deep dive into Data Science and computational statistics in Barcelona. Intrigued by Gabor Lugosi’s and Omiros Papaspiliopoulos’ courses on statistical learning theory and Approximate Bayesian Computation, I decided to further deepen my Machine Learning skills in London. At Imperial College London I got to meet Aldo Faisal who introduced me to the beauty of Reinforcement Learning and helped me to explore and exploit the unknown unknowns. My work at the intersection of Computational Linguistics and Hierarchical Reinforcement Learning made me realize the need for a deeper understanding of the mechanisms underlying intelligence. Psychology, cognitive science and neuroscience will provide the insights needed to achieve the next milestones in the quest for general artificial intelligence.