In little more than a decade, neural networks have moved from an academic curiosity to the engine behind image search, machine translation, protein-structure prediction, and the conversational assistants millions now use every day. Yet there is no single “neural network” behind all of this — there is a sprawling family tree of architectures, dozens of distinct families, each invented to fix a problem the previous generation could not solve. This article walks that tree from the ground up: what a neural network is, and then, family by family, where each design came from, what it is good at, and where it falls short.
What is a neural network?
A neural network is a function built by stacking many small, simple units into layers. Each unit takes a set of numbers, multiplies them by learned weights, adds the results, and passes the sum through a non-linear activation such as ReLU or tanh. Stacked deep enough and trained on enough examples, that arrangement can approximate astonishingly complex relationships — from pixels to object labels, from audio to text. The network in Fig 1 shows the smallest interesting example: an input layer, one hidden layer, and an output layer, every unit in one layer connected to every unit in the next.
Learning means adjusting those weights so the network's output moves closer to the desired answer. The workhorse algorithm is backpropagation: it measures the error at the output and pushes it backwards through the layers, telling each weight how to change. Almost every architecture in this article — however exotic — is trained by some form of this idea, and the whole field is often called deep learning once the layers grow numerous.[1]
From brains to artificial neurons
The metaphor is old. In 1943, McCulloch and Pitts proposed that a biological neuron — which gathers signals through its dendrites, sums them in the cell body, and fires down its axon when the total crosses a threshold — could be modelled as a simple logical unit.[2] The artificial neuron keeps exactly that shape, as Fig 2 makes explicit: inputs scaled by weights stand in for dendrites, the weighted sum plays the role of the soma, and the activation function is the firing decision. The analogy is loose — real neurons are far richer — but it gave the field its name and its first designs.
What distinguishes modern networks from this early model is not just depth but the differentiability of every component. The choice of activation function, for instance, dictates how gradients flow; ReLU (Rectified Linear Unit) passes positive values unchanged and zeros out negatives, which avoids the saturation that plagued earlier sigmoid-based networks. This seemingly minor change, popularised around 2010, was one of the unsung enablers of the deep-learning revolution, because it kept error signals alive across dozens of layers.
Their role in modern systems
What changed between the 1943 metaphor and today's systems is scale. Three ingredients arrived together: large labelled datasets, graphics processors fast enough to train big models, and a set of architectural ideas — convolution, gating, attention — that let networks exploit the structure of images, sequences, and graphs. Together they turned neural networks into the default tool for perception and generation, displacing hand-engineered features across vision, speech, and language.[1]
What they are good at — and where they struggle
Neural networks excel whenever the mapping from input to output is complex, noisy, and backed by plenty of examples but hard to write down as rules: recognising a cat, transcribing speech, translating a sentence. Their weaknesses are the mirror image. They are data-hungry and compute-hungry; they can be confidently wrong on inputs unlike anything they were trained on; and their reasoning is opaque, which matters in medicine, law, and finance. Much of the family tree that follows is a history of trading one of these weaknesses away — usually at the cost of adding structure that suits a particular kind of data.
The rest of this article is organised as a tour of that tree. Each of the following pages takes one family of architectures: what it is, when and why it was introduced, a diagram of its characteristic shape, what it is used for, and where it stumbles. Where one family was invented to cure another's disease, the text links the two.