Mixture-of-Experts
Mixture of experts (MoE) is a machine learning technique where multiple expert networks (learners) are used to divide a problem space into homogeneous regions. MoE represents a form of ensemble learning. They were also called committee machines.
Timeline
1- Research MilestoneMar 11, 2026
New research reveals structural inference disadvantage via 'qs inequality', showing MoE models can be 4.5x slower than dense models
View source
Relationships
13Uses
Recent Articles
9Google Releases Gemma 4 Family Under Apache 2.0, Featuring 2B to 31B Models with MoE and Multimodal Capabilities
~Google has released the Gemma 4 family of open-weight models, derived from Gemini 3 technology. The four models, ranging from 2B to 31B parameters and
100 relevanceKimi 2.5's 1T Parameter MoE Model Runs on 96GB Mac Hardware via SSD Streaming
+Developers have demonstrated that Kimi 2.5's 1 trillion parameter Mixture-of-Experts model can run on Mac hardware with just 96GB RAM by streaming exp
85 relevanceFine-Tuning OpenAI's GPT-OSS 20B: A Practitioner's Guide to LoRA on MoE Models
~A technical guide details the practical challenges and solutions for fine-tuning OpenAI's 20-billion parameter GPT-OSS model using LoRA. This is cruci
100 relevancellmfit Tool Scans System Specs to Match 497 LLMs from 133 Providers to Local Hardware
~llmfit analyzes RAM, CPU, and GPU to recommend which of 497 LLMs will run locally without OOM crashes. It scores models on quality, speed, fit, and co
85 relevanceNVIDIA Releases Nemotron-Cascade 2: A 30B MoE Model with 3B Active Parameters
~NVIDIA has open-sourced Nemotron-Cascade 2, a 30B parameter Mixture-of-Experts model that activates only 3B parameters per token. It claims 'gold meda
100 relevanceQwen 3.5 397B-A17B MoE Model Runs on M3 Mac at 5.7 TPS with 5.5GB Active Memory via SSD Streaming
+Developer Dan reportedly runs the 209GB Qwen 3.5 397B-A17B MoE model on an M3 Mac at ~5.7 tokens per second using only 5.5GB of active memory by quant
85 relevanceThe Hidden Cost of Mixture-of-Experts: New Research Reveals Why MoE Models Struggle at Inference
-A groundbreaking paper introduces the 'qs inequality,' revealing how Mixture-of-Experts architectures suffer a 'double penalty' during inference that
75 relevanceAlibaba's Qwen3.5: The Efficiency Breakthrough That Could Democratize Multimodal AI
+Alibaba has open-sourced Qwen3.5, a multimodal AI model that combines linear attention with sparse Mixture of Experts architecture to deliver high per
85 relevanceQwen's 9B Base Model Breaks Language Barriers with 1M Context Window
~Alibaba's Qwen team has released Qwen3.5-9B-Base, a multimodal foundation model supporting 201 languages with a massive 1 million token context window
95 relevance
Predictions
No predictions linked to this entity.
AI Discoveries
2- observationactive6d ago
Lifecycle: Mixture-of-Experts
Mixture-of-Experts is in 'active' phase (0 mentions/3d, 5/14d, 9 total)
90% confidence - observationactiveMar 22, 2026
Velocity spike: Mixture-of-Experts
Mixture-of-Experts (technology) surged from 1 to 3 mentions in 3 days (velocity_spike).
80% confidence
Sentiment History
| Week | Avg Sentiment | Mentions |
|---|---|---|
| 2026-W10 | 0.10 | 2 |
| 2026-W11 | 0.15 | 2 |
| 2026-W12 | 0.15 | 4 |
| 2026-W13 | 0.60 | 1 |
| 2026-W14 | 0.10 | 1 |