AI / Language Models Market: Small Language Models: When Smaller Is Better

This MLPills newsletter issue explains small language models (SLMs) — compact AI models designed to run under resource constraints such as limited memory, power, and latency budgets. It clarifies that 'small' is a comparative concept rather than a specific parameter count, and distinguishes SLMs from quantized frontier models and distillation. The article covers four main routes to building SLMs: curated data training, distillation, pruning, and quantization, and highlights examples including Microsoft's Phi-4 family, Google's Gemma 3n and Gemma 4 edge models, Hugging Face's SmolLM3, and Cisco's Antares vulnerability-localization models. It describes ideal use cases like classification, entity extraction, and tool selection, and recommends a layered architecture using deterministic code, small models, large models, and human oversight. The piece also cautions about evaluation, over-pruning, and privacy limitations of local inference.