Speaker
Description
Energy-based learning is a biologically plausible alternative to the widely used backpropagation method for training artificial neural networks. It considers models governed by an energy function and learns by shaping this function such that its minima coincide with observed data. This paradigm is particularly promising for training analog circuits in an energy-efficient manner, since learning can be implemented through relatively simple, local parameter updates.
Despite its potential, a general theoretical understanding of energy-based learning is still largely missing. This talk presents steps toward such a theory. After a brief introduction, we focus on nonlinear resistive circuits and propose energy-based learning methods inspired by the literature on contrastive learning. Specifically, we define a contrastive function that compares two energy levels: the free energy, where the circuit’s output potentials are unconstrained, and the clamped energy, where the output potentials are fixed to match the target data. The learning objective is to minimize this contrastive function.
As our main results, we introduce energy-based learning algorithms, and we establish conditions under which these algorithms converge. We will also discuss how the framework can be extended to more general classes of dynamical networks composed of passive subsystems.