TabPFN-3: Technical Report
arXivpreprint
I am an AI researcher at Prior Labs in Berlin, where I build and train foundation models for tabular data.
My background is in electrical engineering and mathematics. I studied Electrical Engineering and Information Technology at ETH Zürich, then took Part III of the Mathematical Tripos at Cambridge, concentrating on statistics and probability. Part III is what set my direction: after a year spent on probability theory, I wanted to find out how far Bayesian and probabilistic methods could actually be pushed inside modern deep learning systems.
I went looking for the answer at Helmholtz AI, working with Vincent Fortuin on curvature-aware particle methods for approximate Bayesian inference, uncertainty quantification for low-rank adapted language models, and multi-modal protein models. Deep networks are far too large to treat exactly, and the question that kept me busy was how much principled uncertainty survives at that scale, and what it costs to keep.
As much as I loved that work, in the end my engineer’s heart got the better of me. I care most about making things actually work. Defining an explicit probability distribution over the parameter space of a large network is elegant, but deriving it and performing inference under it is largely impractical. What excites me far more is specifying the prior implicitly, in function space: asking what properties the data itself ought to have, modelling that distribution directly and generating datasets synthetically from it, rather than routing everything through parameter space. The network then amortises the inference, and what once demanded a costly posterior approximation happens in a single forward pass. That is what brought me to Prior Labs.
Outside of research you will usually find me doing calisthenics, travelling, or arguing about films.
arXivpreprint
ICML 2024 Workshopworkshop
AABI 2024 Workshopworkshop
Berlin, Germany
Building and training foundation models for tabular data.
Munich, Germany
Enhanced Bayesian particle-based inference through Hessian computations, incorporated topological priors into diffusion models, built multi-modal protein transformers, and worked on uncertainty quantification for low-rank adapted LLMs.
Schwarzheide, Germany
Worked in the Digitalisation service unit on sensor data analysis and malfunction prediction for a chemical adhesives plant, using autoencoders, CNNs, RNNs and LSTMs in Python, Pandas and TensorFlow.
Zürich, Switzerland
Taught Digital Circuits Laboratory, Real Analysis, Engineering Mechanics and Multivariable Calculus, preparing and delivering example classes and marking exercises.
Berlin, Germany
Supported internal operations, tracking product delivery and customer procedure progress, and helped set up the first retail location.
Part III of the Mathematical Tripos, focused on statistics and probability.
Specialisation in quantum photonics and control systems.
Master's thesis (Part III of the Mathematical Tripos) placing meta-learning on PAC-Bayesian foundations, analysing the PACOH algorithm as a class of meta-learners that come with probabilistic performance guarantees.
Bachelor's thesis on how hexagonal boron nitride (hBN) freeform lenses reshape the electric field inside Fabry-Pérot micro-cavities, and how transverse confinement depends on lens curvature.
A real-time acquisition pipeline built on a Spectrum M4i.4420-x8 digitizer and the SCAPP framework, moving high-resolution spectroscopy and pump-probe data processing onto the GPU.
A German-language exam preparation course and accompanying script for Prof. Tristan Rivière's Analysis II class, covering multivariable calculus from topology through to the major integral theorems.
The fastest way to reach me is email. I read everything, and reply to most of it.
klemens.floege@gmail.com