The Inner Universe: A Mechanistic Inquiry into the Representations and Reasoning of Transformer Architectures

Introduction: The Opaque Mind of the Machine: From Black Boxes to Mechanistic Understanding The advent of large language models (LLMs) built upon the transformer architecture represents a watershed moment in Read More …

Deconstructing the Transformer: A Neuron-Level Analysis of a Modern Neural Circuit

Section 1: Foundational Principles: From Recurrence to Parallel Attention The advent of the Transformer architecture in 2017 marked a watershed moment in the field of deep learning, particularly for sequence Read More …