Low-Rank Compression of Language Models via Differentiable Rank Selection
Language Resources and Evaluation Conference (LREC), 2026
Approaches for compressing large-language models using low-rank decomposition have made strides, particularly with the introduction of activation and loss-aware SVD, which improves the trade-off between decomposition rank and downstream task performance. Despite these advancements, a persistent challenge …