AI learning

Chapter 8 · Shaping models: tuning and alignment

Mixture of experts

A router sends each token to a few expert sub-networks, so a huge model only uses part of itself per token.

Preparing mixture of experts: a router picks the specialists
YOUR BROWSER, YOUR CHOICE