Ward Gauderis

Hello! I’m an FWO PhD Fellow in Brussels, supervised by Prof. Geraint Wiggins.

I refuse to treat neural networks as unstructured black boxes. Instead, I study compositionality as a mathematical foundation for deep learning. Using tensor networks and category theory, I ground the analysis of neural representations in the weight geometry of the architecture itself.

By viewing the model’s weights as a formal compositional system, emergent behaviour becomes a direct function of its algebraic wiring. This enables a divide-and-conquer approach that naturally links local mechanisms to global properties.

To this end, my work proposes tensor models as a unified framework for tractable neuro-symbolic integration and mechanistic analysis, ultimately aiming for truly compositional generalisation and interpretability.

Q-CHARM

How can compositional design improve compositional behaviour?

My FWO-funded PhD project, Q-CHARM, explores this question by distinguishing between a model’s architecture (its compositional design) and the structure that emerges during learning (its compositional behaviour).

I bridge two complementary paths: imposing explicit structure before training (neuro-symbolic design) and exposing implicit structure after training (mechanistic interpretability). Since deep networks cannot efficiently learn compositional functions from data alone, embedding domain structure is essential to guide learning toward representations that align with human understanding and generalise well.

To formalise this, I use string diagrams (life is too short for indices) to cast models as rigorous mathematical objects, cleanly separating high-level Syntax (symbolic rules and structure) from low-level Semantics (subsymbolic representations). As a devout Yoneda disciple, I believe this categorical grounding is the only way to formally reason about behaviour beyond the training set.

The practical blueprint for this vision lies in tensor models. They unify the expressivity of neural networks with the tractability of tensor networks. Because their weights possess a well-understood geometry, we can perform tractable analysis both pre- and post-training. While central to applied category theory and neuro-symbolic AI, they are rarely combined with interpretability; my work unites these perspectives.

Compositional Interpretability

Current mechanistic interpretability lacks formal foundations, relying on post-hoc activation heuristics, effectively reading tea leaves in activation space. Because these methods often assume a layer-by-layer stratification, it is nearly impossible to tell if a feature is causally useful globally or just a local artifact.

The CompInterp framework shifts the focus from isolated features to their interactions as first-class citizens. To achieve measurable interpretability, we ground our analysis in formal decompositions rather than data-dependent heuristics:

• Unified Algebra: By formulating weights, data, and subcircuit interactions via tensor contraction, standard matrix decompositions (SVD, ICA, etc.) naturally lift to complex architectures. Since the result remains a tensor model, we can trace discovered mechanisms back to the full architecture.
• Weight-Based Analysis: Tensor networks capture higher-order relations between representation spaces. By analysing their polynomial coefficients directly within the weight geometry, dependence on input data becomes an explicit, optional choice, ensuring conclusions remain valid beyond the training distribution.
• Disentangling Interactions: To filter out spurious correlations, decompositions must balance complexity and faithfulness. Because these properties are compositional, they can be traced throughout the model

We are currently scaling CompInterp methods to transformers and CNNs by leveraging their low-rank structure.

Research Interests

If you want my full attention, just mention any of these…

• Compositionality in AI: Category theory, string diagrams, geometric deep learning
• Mechanistic Interpretability: Weight space analysis, parameter decompositions
• Neuro-symbolic Architectures: Probabilistic circuits, tensor logic
• Quantum-ish Mathematics: Tensor networks, Hilbert spaces, information geometry
• Effective Theories of DL: Renormalisation, algebraic geometry, stochastic complexity
• Models of Cognition & Creativity: Active inference, conceptual spaces

Hobbies

When I’m not agonising about model structure, I’m probably skating through the city, singing and playing piano, or falling down a philomathematical rabbit hole. I also love building FOSS, playing chess or other (board) games, and conversations that stretch the brain a little.