Tag: DeepMind
Scaling Language Models with Millions of Tiny Experts: DeepMind’s PEER Approach
DeepMind has introduced a new approach called Parameter Efficient Expert Retrieval (PEER) to address the limitations of current Mixture-of-Experts (MoE) techniques used in scaling...
Interpreting LLMs with Sparse Autoencoders: DeepMind’s Breakthrough
Large language models (LLMs) have been advancing rapidly in recent years, but the challenge of understanding how they work remains. Researchers at artificial intelligence...