Seminars and Events at automatic control
All seminars are held at the Department of Automatic Control, in the seminar room M 3170-73 on the third floor in the M-building, unless stated otherwise.
Seminar by Alp Yurtsever: Implicit Bias in Matrix Factorization and a New Neural Network Architecture
Seminar
From:
2025-12-02 11:00
to
12:00
Place: Seminar Room M 3170-73 in the M-building, LTH
Contact: pontus [dot] giselsson [at] control [dot] lth [dot] se
Date & Time: December 2nd, 11:15-12:00
Location: Seminar Room M 3170-73 in the M-building, LTH
Speaker: Alp Yurtsever
Title: Implicit Bias in Matrix Factorization and a New Neural Network
Architecture
Abstract:
Gradient descent for matrix factorization exhibits an implicit bias toward approximately low-rank solutions. While existing theories often assume the boundedness of iterates, empirically the bias persists even with unbounded sequences. This reflects a dynamic where factors develop low-rank structure while their magnitudes increase, tending to align with certain directions. To capture this behavior in a stable way, we introduce a new factorization model: X ≈ UDV, where U and V are constrained within norm balls, while D is a diagonal factor allowing the model to span the entire search space. Experiments show that this model consistently exhibits a strong implicit bias, yielding truly (rather than approximately) low-rank solutions. Extending the idea to neural networks, we introduce a new model featuring constrained layers and diagonal components that achieves competitive performance on various regression and classification tasks while producing lightweight, low-rank representations.