Avatar 🍥

TLE

My page where I post about topics I find interesting!!!

  1. Home
  2. About
  3. Archives
  4. Search
  5. Dark Mode
Deep Learning Theory

Training invariances in deep nets and its consequences I: Set up and Problem description

In this first post, we will take things slow and set things up for subsequent discussions, first for the simple case of linear networks; then more practical architectures including ReLU layers, convolutional layers and a certain ResNet variant for subsequent posts.

Nov 20, 2021
9 minute read
Deep Learning Theory

Training invariances in deep nets and its consequences II: Linear network

In this second post of the series, we go through the one-line proof of a certain training invariance in linear neural networks and its consequences, in term of alignment, weight matrix rank, and margin.

Nov 20, 2021
6 minute read
© 2020 - 2023 TLE
Built with Hugo
Theme Stack designed by Jimmy

Archives

2021 2

Tags

dlt