Winfried van den Dool

About

While deep learning’s capabilities are becoming more and more astonishing, what fascinates me most is actually one of the things it still lags behind on: efficiency. For contrast, on the hardware side a human brain runs on roughly the power of a light bulb; on the algorithmic side, classical mathematics for example can be beautifully elegant. Modern AI, by comparison, is relatively brute force. Regardless of the “is scale all we need?” debate, I enjoy the efficiency problem for its own sake, and think working on it could well lead to new insights for either camp.

I’m a PhD candidate at the QUVA Lab, University of Amsterdam, supervised by Max Welling. My work focuses on computational efficiency in deep learning: hardware-aware learning, (vector-)quantization, adaptive inference, and neural PDE solvers. Of course, when talking efficiency, the carbon-footprint angle is tempting to mention, but I think it’s hard to tell whether efficiency gains actually reduce it or just mean people train bigger models.

My background is in mathematics and physics, which led me to appreciate analytic understanding over pure empirical observation. I find efficiency problems often rewarding in this respect: one can reason analytically about clever tricks, and with enough understanding of the hardware, calculate and understand results rather than only measuring and observing them. Earlier in my PhD I explored alternative hardware ideas like memristors and spiking neural networks, before moving to neural PDE solvers. My current focus is more on the algorithmic side, working on vector quantization and linear attention.

Before the PhD I worked as a data scientist at PwC, and have also done research internships at Rabobank and, more recently, at Fermioniq on quantum algorithms for solving PDEs.