Associate Professor (Reader) in Machine Learning
School of Mathematics, The University of Edinburgh
The Maxwell Institute for Mathematical Sciences
School of Mathematics, Gran Sasso Science Institute
JCMB, King’s Buildings, Edinburgh EH93FD UK
email: f dot tudisco at ed.ac.uk
Excited that our paper on Robust low-rank training has been accepted on NeurIPS 2023! We propose a method to train networks with low-rank weights while reducing the network condition number and thus increasing its robustness with respect to adversarial attacks. Congrats to my two PhD students Dayana Savostianova and Emanuele Zangrando!
Abstract: Dynamical systems on hypergraphs can display a rich set of behaviours not observable for systems with pairwise interactions. Given a distributed dynamical system with a putative hypergraph structure, an interesting question is thus how much of this hypergraph structure is actually necessary to faithfully replicate the observed dynamical behaviour. To answer this question, we propose a method to determine the minimum order of a hypergraph necessary to approximate the corresponding dynamics accurately. ... Read more
Abstract: With the growth of model and data sizes, a broad effort has been made to design pruning techniques that reduce the resource demand of deep learning pipelines, while retaining model performance. In order to reduce both inference and training costs, a prominent line of work uses low-rank matrix factorizations to represent the network weights. Although able to retain accuracy, we observe that low-rank methods tend to compromise model robustness against adversarial perturbations. ... Read more
Abstract: The computing cost and memory demand of deep learning pipelines have grown fast in recent years and thus a variety of pruning techniques have been developed to reduce model parameters. The majority of these techniques focus on reducing inference costs by pruning the network after a pass of full training. A smaller number of methods address the reduction of training costs, mostly based on compressing the network via low-rank layer factorizations. ... Read more
Traveling today to EPF Lausanne for the DLRA “New Horizon” workshop. I will present our recent work on spectral pruning of deep learning models. Thanks Gianluca Ceruti and Jonas Kusch for the kind invitation!
Our SIAM Review’s SIGEST paper Nonlinear Perron-Frobenius theorem for nonnegative tensors has been published on Vol 65 Iss 2 of SIAM Review. Thanks to the Editors for the support and for the nice presentation!
The Numerical Linear Algebra Days (“La due giorni”) is about to start at GSSI!
This is the 18th workshop in a series dedicated to Numerical Linear Algebra and Applications, aiming at gathering the (mostly Italian) Numerical Linear Algebra scientific community to discuss recent advances in the area and to promote the exchange of novel ideas and the collaboration among researchers.
Here are the slides of my lecture on Nonlinear Perron-Frobenius Theory
Abstract: We present a unifying Perron–Frobenius theory for nonlinear spectral problems defined in terms of nonnegative tensors. By using the concept of tensor shape partition, our results include, as a special case, a wide variety of particular tensor spectral problems considered in the literature and can be applied to a broad set of problems involving tensors (and matrices), including the computation of operator norms, graph and hypergraph matching in computer vision, hypergraph spectral theory, higher-order network analysis, and multimarginal optimal transport. ... Read more
I am very happy that our paper Learning the right layers: a data-driven layer-aggregation strategy for semi-supervised learning on multilayer graphs has been accepted on the proceedings of this year’s ICML conference.
Congrats to Sara Venturini on one more important achievement!
I just accepted an invite to join the editorial board of the Springer’s journal Computational Mathematics and Modeling, a journal estabilished and run by the department of Computational Mathematics and Cybernetics of the Lomonosov Moscow State University, a place that is very important to me.
Abstract: The modeling of opinion dynamics has seen much study in varying academic disciplines. Understanding the complex ways information can be disseminated is a complicated problem for mathematicians as well as social scientists. Inspired by the Cucker-Smale system of flocking dynamics, we present a nonlinear model of opinion dynamics that utilizes an environmental averaging protocol similar to the DeGroot and Freidkin-Johnsen models. Indeed, the way opinions evolve is complex and nonlinear effects ought to be considered when modelling. ... Read more
Abstract: Collaboration is a key driver of science and innovation. Mainly motivated by the need to leverage different capacities and expertise to solve a scientific problem, collaboration is also an excellent source of information about the future behavior of scholars. In particular, it allows us to infer the likelihood that scientists choose future research directions via the intertwined mechanisms of selection and social influence. Here we thoroughly investigate the interplay between collaboration and topic switches. ... Read more
Excited that our work on Social Contagion in Science has been selected as one of the 16 plenary talks at the next International Conference on Computational Social Science in Copenhagen, out of 900+ submissions. I’m very proud of the great multidisciplinary team that has worked on this project, in particular Sara and Satyaki.
Additionally, our work on The COVID-19 research outbreak: how the pandemic culminated in a surge of new researchers has been accepted as oral presentation at the same conference. This is based on a fanstastic ongoing collaboration with Maia Majumder’s team at Harvard Medical School.
Three talks has been accepted at the next NetSci 2023 conference in Vienna:
for our paper
I am honored to receive the SIGEST Award of the Society for Industrial and Applied Mathematics (SIAM) for our paper A unifying Perron-Frobenius thoeorem for nonnegative tensors via mutlihomogeneous mappings.
SIGEST highlights a recent outstanding paper from one of SIAM’s specialized research journals, chosen on the basis of exceptional interest to the entire SIAM community. The winning paper is reprinted in the SIGEST section of SIAM Review, the flagship journal of the society. The SIGEST version of the paper has a new title
Nonlinear Perron-Frobenius theorems for nonnegative tensors
and contains two major additions:
A widely extended introduction, with many non-trivial examples of tensor eigenvalue problems in applications, including problems from computer vision and optimal transport. Here we detail how the nonlinear Perron-Frobenius theorems for tensors that we introduced can be of great help to tackle these problems.
A new nonlinear Perron-Frobenius theorem that significantly improves (parts of) the previous Perron-Frobenius theorems for tensors and allows us to address more general problems (such as the optimal transport problem discussed in the introduction)
Abstract: We consider the problem of attaining either the maximal increase or reduction of the robustness of a complex network by means of a bounded modification of a subset of the edge weights. We propose two novel strategies combining Krylov subspace approximations with a greedy scheme and with the limited-memory BFGS. The paper discuss the computational and modeling aspects of our methodology and illustrates the various optimization problems on networks that can be addressed within the proposed framework. ... Read more
I am presenting today my work on low-rank training of deep neural networks at the Rome Centre on Mathematics for Modelling and Data Science at the Department of Mathematics, University of Rome Tor Vergata (Italy).
Abstract: Graph Semi-Supervised learning is an important data analysis tool, where given a graph and a set of labeled nodes, the aim is to infer the labels to the remaining unlabeled nodes. In this paper, we start by considering an optimization-based formulation of the problem for an undirected graph, and then we extend this formulation to multilayer hypergraphs. We solve the problem using different coordinate descent approaches and compare the results with the ones obtained by the classic gradient descent method. ... Read more
Abstract: The homology groups of a simplicial complex reveal fundamental properties of the topology of the data or the system and the notion of topological stability naturally poses an important yet not fully investigated question. In the current work, we study the stability in terms of the smallest perturbation sufficient to change the dimensionality of the corresponding homology group. Such definition requires an appropriate weighting and normalizing procedure for the boundary operators acting on the Hodge algebra’s homology groups. ... Read more
Excited that our paper Nodal domain count for the generalized p-Laplacian – with Piero Deidda and Mario Putti – has been accepted for publication on Applied and Computational Harmonic Analysis. Among the main results, we prove that the eigenvalues of the p-Laplacian on a tree are all variational (and thus they are exactly n), we show that the number of nodal domains of the p-Laplacian for general graphs can be bound both from above and from below, and we deduce that the higher-order Cheeger inequality is tight on trees.
Arturo De Marinis from our team is attending XMaths workshop at University of Bari this week, presenting preliminary results on our work on stability of neural dynamical systems.
I am giving an invited lecture today on Low-parametric deep learning, at the Data Science in Action day organized by the University of Padua. You can find here the slides of my talk.
I am presenting today my work on fast and efficient neural networks' training via low-rank gradient flows at the Research Seminars in Mathematics at the School of Science and Technology, Örebro University (Sweden). Thanks Andrii Dmytryshyn for the kind invitation!
Steffen Schotthöfer and Emanuele Zangrando from our lab are attending NeurIPS Conference this week in person and will present our work on lowrank training and pruning of neural networks. In this work we developed a framework to perform stable and efficient training on low-rank manifolds, resulting in an order of magnitude less memory cost and training time! Tested successfully on Imagenet1K, transformers and several other benchmarks.
If you are there too, swing by our poster session in HallJ#604 on Wed 30 Nov 9:30 am PST
Emanuele Zangrando is presenting today our work on dynamical low-rank training of artificial neural networks at the SCDM seminar at Karlsruhe Institute of Technology.
I am presenting today my work on generalized $p$-Laplacian on graphs at the Mathematical Physics and Harmonic Analysis Seminar at Texas A&M Univeristy. Thanks Gregory Berkolaiko for the kind invitation!
Thrilled to hear that our paper on Low-rank lottery tickets has been accepted on NeurIPS 2022! We propose a method to speed up and reduce the memory footprint of the training phase (as well as the inference phase) of fully-connected and convolutional NNs by interpreting the training process as a gradient flow and integrating the corresponding ODE directly on the manifold of low-rank matrices.
It has been a wonderful collaboration among a fantastic team of collaborators, and it would have not been possible without the excellent work of the two PhD students Emanuele Zangrando and Steffen Schotthöfer.
Abstract: Nonlinear eigenvalue problems for pairs of homogeneous convex functions are particular nonlinear constrained optimization problems that arise in a variety of settings, including graph mining, machine learning, and network science. By considering different notions of duality transforms from both classical and recent convex geometry theory, in this work we show that one can move from the primal to the dual nonlinear eigenvalue formulation maintaining the spectrum, the variational spectrum as well as the corresponding multiplicities unchanged. ... Read more
Our report on the XXI Householder Symposium on Numerical Linear Algebra appeared today on SIAM News. It has been a great meeting which I really enjoyed!
Excited that our paper A Variance-aware Multiobjective Louvain-like Method for Community Detection in Multiplex Networks – with Sara Venturini, Andrea Cristofari, Francesco Rinaldi – has been accepted for publication on the Journal of Complex Networks, Oxford academic press.
Happy that our paper Hitting times for second-order random walks, joint work with Arianna Tonetto and Dario Fasino (Univ of Udine), has been accepted for publication on the European Journal of Applied Mathematics, Cambridge University Press.
We are looking for a
Abstract: Neural networks have achieved tremendous success in a large variety of applications. However, their memory footprint and computational demand can render them impractical in application settings with limited hardware or energy resources. In this work, we propose a novel algorithm to find efficient low-rank subnetworks. Remarkably, these subnetworks are determined and adapted already during the training phase and the overall time and memory resources required by both training and evaluating them is significantly reduced. ... Read more
I am presenting today my work on generalized $p$-Laplacian on graphs at the Spectral Geomtry Seminar at University of Science and Technology of China in Hefei. Thanks Shiping Liu for the kind invitation!
One more great news! Our paper Core-periphery partitioning and quantum annealing – with Catherine Higham and Desmond Higham – has been accepted on the proceedings of this year’s ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
I am very happy that our paper Nonlinear Feature Diffusion on Hypergraphs – with Austin Benson and Konstantin Prokopchik – has been accepted on the proceedings of this year’s ICML. Congrats to my student Konstantin for one more important achievement!
Nicola Guglielmi and I are orginzing this week the workhop Numerical Methods for Compression and Learning at GSSI. The workshop will take place in the Main Lecture Hall in the Orange Building and will feature lectures from invited speakers as well as poster sessions open to all participants.
Excited to host great colleagues and looking forward to exciting talks!
Online participation will be possible via the zoom link: https://us02web.zoom.us/j/83830006962?pwd=SmI1MTVKRTllU3dBR01Ybko5bzBJdz09
As the amount of available data is growing very fast, the importance of being able to handle and exploit very-large-scale data in an efficient and robust manner is becoming increasingly more relevant. This workshop aims at bringing together experts from signal processing, compressed sensing, low rank methods and machine learning with the goal of highlighting modern approaches as well as challenges in computational mathematics arising in all these areas and at their intersection.
Abstract: In this paper, we focus on the community detection problem in multiplex networks, i.e., networks with multiple layers having same node sets and no inter-layer connections. In particular, we look for groups of nodes that can be recognized as communities consistently across the layers. To this end, we propose a new approach that generalizes the Louvain method by (a) simultaneously updating average and variance of the modularity scores across the layers, and (b) reformulating the greedy search procedure in terms of a filter-based multiobjective optimization scheme. ... Read more
Abstract: Clustering (or community detection) on multilayer graphs poses several additional complications with respect to standard graphs as different layers may be characterized by different structures and types of information. One of the major challenges is to establish the extent to which each layer contributes to the cluster assignment in order to effectively take advantage of the multilayer structure and improve upon the classification obtained using the individual layers or their union. ... Read more
Presenting today in Rome at the third workshop “Calcolo Scientifico e Modelli Matematici”. The streaming of my talk (in Italian) is available here and here you can find the pdf of my slides.
Abstract: Core-periphery detection is a key task in exploratory network analysis where one aims to find a core, a set of nodes well-connected internally and with the periphery, and a periphery, a set of nodes connected only (or mostly) with the core. In this work we propose a model of core-periphery for higher-order networks modeled as hypergraphs and we propose a method for computing a core-score vector that quantifies how close each node is to the core. ... Read more
Gianluca Ceruti from EPFL is visiting our group until Feb 18. Looking forward to some inspiring discussion on dynamical low rank!
I traveling today to Naples to take part to the Italian annual workshop on Numerical Linear Algebra (2ggALN), one of the nicest events of the year. Looking forward to see colleagues and friends and to meet new people!
I am presenting today my work on nonlinear Laplacians for hypergraphs with applications to centrality and core-periphery detection at the Networks Seminar at MPI for Mathematics in the Sciences Leipzig. Thanks Raffella Mulas and Leo Torres for the kind invitation!
I am honored I have been awarded a MUR-Pro3 grant on Stability of Neural Dynamical Systems (STANDS), in collaboration with Michele Benzi (SNS) and Nicola Guglielmi (GSSI). I am looking for a postdoc to join our group on this project. Please reach out if you are interested!
Abstract: We propose a new kernel that quantifies success for the task of computing a core-periphery partition for an undirected network. Finding the associated optimal partitioning may be expressed in the form of a quadratic unconstrained binary optimization (QUBO) problem, to which a state-of-the-art quantum annealer may be applied. We therefore make use of the new objective function to (a) judge the performance of a quantum annealer, and (b) compare this approach with existing heuristic core-periphery partitioning methods. ... Read more
Abstract: nspired by the linear Schrödinger operator, we consider a generalized $p$-Laplacian operator on discrete graphs and present new results that characterize several spectral properties of this operator with particular attention to the nodal domain count of its eigenfunctions. Just like the one-dimensional continuous $p$-Laplacian, we prove that the variational spectrum of the discrete generalized $p$-Laplacian on forests is the entire spectrum. Moreover, we show how to transfer Weyl’s inequalities for the Laplacian operator to the nonlinear case and prove new upper and lower bounds on the number of nodal domains of every eigenfunction of the generalized $p$-Laplacian on generic graphs, including variational eigenpairs. ... Read more
We are starting this week a reading group on Computational Learning, which will concentrate on papers in the general area of Numerics/Optimization/Matrix Theory/Graph Theory with particular emphasis on their application to / connection with Machine Learning, Data Mining and Network Science. The reading group is organized by PhD students and postdocs from the Numerics and Data Sciene group at GSSI. The organizer that is kicking off the initiative is Dayana Savostianova, who will also present the first paper. If you are interested in participating and/or presenting, please fill out this form (available to GSSI users only) to be notified about the meetings and to be kept in the loop.
Abstract: The self-consistent field (SCF) iteration, combined with its variants, is one of the most widely used algorithms in quantum chemistry. We propose a procedure to adapt the SCF iteration for the p-Laplacian eigenproblem, which is an important problem in the field of unsupervised learning. We formulate the p-Laplacian eigenproblem as a type of nonlinear eigenproblem with one eigenvector nonlinearity , which then allows us to adapt the SCF iteration for its solution after the application of suitable regularization techniques. ... Read more