Posts

A collection of thoughts, notes, and small projects, updated sporadically.

Not 24

Unnecessary proofs

Normal Gradients of Kernel Interpolants

Why models can overfit on noisy data and still generalize optimally

\(\mu\)P as Optimal Transport in a Vanilla MLP

Deriving \(\mu\)P as the unique scaling maximizing Wasserstein transport under stability constraints

Room Auctions

Designing an auction system for rooms in our house

Deep Learning Training Dynamics are not Fisher Geodesics

A cool bit of math that falls apart empirically

A Category Theoretic View of Machine Learning

Using symmetries to better understand how to design attention kernels

Every Model is Kernel Ridge Regression

Some notes on Domingos' paper: Every Model Learned by Gradient Descent Is Approximately a Kernel Machine

A Markov Picture of Heat

A derivation of the heat equation from a stochastic model of energy exchange

Eigenlearning

Some notes on the Eigenlearning paper by Simon et al.

Reproducing Kernel Hilbert Spaces

Building up RKHSs from the Riesz Representation Theorem

Cookie Magic

How I got ~$500 worth of cookies for free

Neural Tangent Kernels

A rough derivation of the neural tangent kernel

Hopfield Networks

A quick implementation of a Hopfield network in Python