Geometry-aware training of factorized layers in tensor Tucker format
Advances in Neural Information Processing Systems (NeurIPS), 2024
Reducing parameter redundancies in neural network architectures is crucial for achieving feasible computational and memory requirements during training and inference phases. Given its easy implementation and flexibility, one promising approach is layer factorization, which reshapes weight tensors into a matrix format …