LINKED LIST [txt mode] ▸ @stats385 ▸ Theory & analysis papers
home explore | log in

Theory & analysis papers

by @stats385 · last added 2026-09-24 · 18 items [theory]

High-signal theory from guest-lecture readings: NTK, mean-field, robustness, implicit bias, Hessians.

* Critical Learning Periods in Deep Neural Networks
Achille/Soatto — early training phases matter disproportionately (like critical periods).
arxiv.org · node›
* Data-dependent Sample Complexity via Lipschitz Augmentation
Generalization bound reading from Tengyu Ma’s lecture set.
arxiv.org · node›
* Regularization Matters: Nets vs Induced Kernel
Tengyu Ma lecture reading — when finite-width nets differ from their NTK.
arxiv.org · node›
* Identifying and Attacking the Saddle Point Problem
Dauphin et al. — saddle points as the hard part of nonconvex deep optimization.
arxiv.org · node›
* Full Spectrum of Deepnet Hessians at Scale
Papyan — Hessian spectrum under SGD; ties to the final guest lecture.
arxiv.org · node›
* Exact Solutions to Nonlinear Dynamics of Deep Linear Nets
Saxe et al. — closed-form learning dynamics; Andrew Saxe lecture core.
arxiv.org · node›
* Implicit Bias of Gradient Descent on Separable Data
Characterizes how GD converges to max-margin solutions.
arxiv.org · node›
* On the Role of Implicit Regularization in Deep Learning
Srebro lineage — seeking the real inductive bias of gradient methods.
arxiv.org · node›
* Explaining and Harnessing Adversarial Examples (FGSM)
Goodfellow et al. — simple linear explanation + fast gradient sign method.
arxiv.org · node›
* Intriguing Properties of Neural Networks
Early adversarial examples paper — started the robustness literature.
arxiv.org · node›
* Robustness May Be at Odds with Accuracy
Fundamental tradeoff between clean accuracy and adversarial robustness.
arxiv.org · node›
* Towards Deep Learning Models Resistant to Adversarial Attacks
Madry et al. PGD adversarial training — robustness baseline.
arxiv.org · node›
* Deep Learning and the Information Bottleneck Principle
Tishby & Zaslavsky — IB lens on what deep nets compress and preserve.
arxiv.org · node›
* Emergence of Invariance and Disentanglement (Achille & Soatto)
Information-theoretic view of representations — Soatto guest lecture core.
arxiv.org · node›
* Mean Field View of Two-Layer Neural Networks
Mean-field landscape analysis — Song Mei lecture lineage.
arxiv.org · node›
* Wide Neural Networks Evolve as Linear Models (Lee et al.)
Shows wide nets under GD behave like linear models in function space.
arxiv.org · node›
* Deep Neural Networks as Gaussian Processes
NNGP correspondence — infinitely wide nets as GPs. Pair with Pennington lecture.
arxiv.org · node›
* Neural Tangent Kernel (Jacot et al.)
Defines the NTK; explains lazy-training convergence. Anchor paper for Lecture 6.
arxiv.org · node›
privacy | terms | regular mode