About

I’m a Computer Science student at the Faculty of Electrical Engineering and Computing (FER), Zagreb. At the moment (April 2019) I’m on my last year of Master studies and I’m writing my thesis titled Compositional Deep Learning.

I’m interested in studying nature’s machinery using the most universal language possible.

I’m especially interested in understanding the theoretical and practical considerations of computation performed by systems we classify as intelligent.

I’m a strong advocate of a compositional, rigorous, category-theoretic view of the world.

Research philosophy

One of my main long-term goals is understanding the fundamental principles of intelligence.

This encompasses understanding both how the brain works and the general properties that the behavior of a machine or any sort of a system needs to exhibit in order to be classified as intelligent. It is important to note that it does not seem like we have yet reached consensus on how to define intelligence. Nevertheless, there still exist plenty of subtle clues that tell us how the general shape of the definition might look like. I roughly describe these clues below as I believe it is important to keep them in mind while searching for the definition.

Intelligence is an abstract term. It encompasses pattern recognition, natural langauge processing, knowledge representation, metacognition, planning, perception, analogical reasoning, creativity and many other aspects listed here. In order to describe a system whose behaviors exhibit properties of intelligence - the description can’t be specific to creativity, pattern recognition, planning or any of the other mentioned properties. We need to work on a higher level of abstraction whose special cases are those properties.

Furthermore, all intelligent systems seem to have one thing in common: posession of an internal model of their environment, whose accuracy seems to be correlated with the degree of their intelligence. Intelligent systems tend to organize and layer various hierarchical representations of the world in order to be more efficient at its internal simulation and thus prediction of its future states. Being tightly related to the notion of autopoiesis, these systems show a tendency not to only maintain themselves, but also to maintain their environment in a state which is compatible with their existence. One could consider these systems to be good regulators which strive to become isomorphic to the environment - whatever isomorphic might mean in this context.

Another hint which tells us how to approach reasoning about intelligence comes from deep learning. An increasing number of components of a modern deep learning system can be learned. For example, Generative Adversarial Networks learn the cost function. The paper Learning to Learn by gradient descent by gradient descent specifies networks that learn the optimization function. The paper Decoupled Neural Interfaces using Synthetic Gradients specifies how gradients themselves can be learned. These are just rough examples, but they give a sense of things to come. As more and more components of these systems stop being fixed throughout training, there is an increasingly larger need for precise formal specification of the things that do stay fixed. This is not an easy task; the invariants across all these networks seem to be rather abstract and hard to describe.

My research explores the hypothesis that the language of category theory could be well suited to describe these systems in a precise manner. By doing so, I hope to describe and quantify high-level intelligent behavior.

Category theory is becoming a central hub for all of mathematics - but is slowly finding applications in chemistry, game theory, neuroscience, causality and database theory to name a few. It is unmatched in organizing and layering abstractions across seemingly disparate disciplines.

Category theory, however, is much more than that. I believe we have just begun scratching the surface of general reasoning using category theory, as it can be extremely potent in guiding, structuring, and compressing thought. Category theory and intelligence seem to be deeply linked - both are guided by the goal of organizing and layering abstractions. Automation of exactly that - organizing and layering abstractions - seems to be the quest of machine learning.

With the above in mind, I make my goals more precise. I’m interested in understanding the theoretical and practical considerations of computation performed by systems we classify as intelligent, with a focus on those that utilize gradient information during learning. My interests are both at a low level (automatic differentiation, composition of differentiable maps) and at a high level of abstraction (categorically modeling network architectures, training methods, as well as concepts such as ‘generalization’ and ‘analogy’).

This is, of course, just the tip of the iceberg; understanding and implementing this type of computation implies an understanding of plenty of other things things that are complex enough by themselves already. This is why I’m also a strong advocate of purely functional programming languages and type-driven development. I believe we should outsource a part of our cognitive load to the compiler and use it as a guide while writing programs.

One important thing to note is that it is not the case that I’m trying to impose a special “category-theoretic way” of modeling these systems; what I’m doing is trying to find the most natural, leaky-abstraction-free and composable way there is to understand these systems. It turns out that I end up doing category theory.

This is a vast unexplored area and I’m happy to talk about any of this stuff, feel free to get in touch!