Neural Networks Taxonomy - 21. Neuroevolution and Evolutionary Networks

21. Neuroevolution & Evolutionary Networks

Neuroevolution searches for networks without gradient descent. As Fig 23 shows, an evolutionary loop scores a population of networks by a fitness measure, keeps the best, and breeds the next generation by mutation and crossover — of the weights, and sometimes of the network topology itself.

Origins and rise

NEAT evolved both weights and structure, growing networks from minimal seeds.^[298] HyperNEAT evolved a pattern that generates the weights of a much larger network.^[299] For deep reinforcement learning, Evolution Strategies showed a simple population-based optimiser can rival gradient methods while parallelising across thousands of workers.^[300]

Evolving topology and indirect encodings

NEAT's idea of evolving structure scales up in several directions. CoDeepNEAT coevolves reusable modules and the blueprints that wire them together, assembling deep networks in an evolutionary counterpart to architecture search.^[301] ES-HyperNEAT extends HyperNEAT's indirect encoding so evolution also decides where neurons sit and how densely they connect, rather than assuming a fixed substrate.^[302] EANT2 grows topologies incrementally while optimising each structure's weights with an evolution strategy, a combination well suited to control.^[303] Taking the idea to its limit, Weight Agnostic Neural Networks search for architectures that already perform a task when every weight shares a single random value, showing how much a topology alone can encode.^[304]

Rewarding novelty and scaling neuroevolution

A recurring failure of fitness-driven search is convergence to deceptive local optima. Novelty Search confronts this by abandoning the objective entirely and rewarding behaviours simply for being different, often reaching goals that direct optimisation cannot.^[305] On the scaling front, Deep Neuroevolution showed a plain genetic algorithm can train networks with millions of parameters for reinforcement learning, competitive with gradient-based methods and evolution strategies.^[306] Population Based Training blends the two paradigms, evolving a population of networks while they train by gradient descent so that hyperparameters are tuned online.^[307]

Applications

• Reinforcement-learning control without gradients.
• Evolving architectures, akin to architecture search.
• Black-box optimisation of non-differentiable objectives.

Strengths and limitations