Blog

November 18, 2025
15 min read

Gaussian Error Linear Units (GELUs)

The GELU is an activation function introduced in the paper Gaussian Error Linear Units. If you're unfamiliar with activation functions or want a refresher, you can check out this blog where I explain them in detail.

September 14, 2025
7 min read

Activation Functions in Deep Learning (in progress)

An intuitive journey into how they shape learning, bend gradients, and sometimes break them.

November 6, 2024
5 min read

Self-Attention: The Magic Behind Transformers and the AI Revolution (old, redundant writing new)

Today, we're diving into something that's at the core of Transformers, and Transformers themselves are at the core of this AI revolution!