Associate Professor (Reader) in Machine Learning
School of Mathematics, The University of Edinburgh
The Maxwell Institute for Mathematical Sciences
Alan Turing Institute | Bayes Centre | Gen AI Lab
School of Mathematics, Gran Sasso Science Institute
email: f dot tudisco at ed.ac.uk
Our paper Low-Rank Compression of Language Models via Differentiable Rank Selection has been accepted at LREC 2026!
Joint work with Sidhant Sundrani and Pasquale Minervini. We propose LLRC, a gradient-based method for optimally selecting per-layer ranks when compressing large language models via low-rank decomposition, consistently outperforming competing rank-selection methods across various compression rates on reasoning and QA benchmarks.