Liquid AI vs vLLM

Liquid AI Enters German Market Via Vago Solutions Partnership

US-based AI startup Liquid AI, an MIT spin-off valued at about $2.35 billion, is entering the German market through a development partnership with Vago Solutions GmbH from Hennef. The collaboration aims to build efficient AI models that companies and public authorities can operate on their own infrastructure instead of in the cloud. Liquid AI, founded in 2023, develops Liquid Foundation Models based on Liquid Neural Networks, which require less computing power than conventional large language models. Vago Solutions, known for the open-source German-language model SauerkrautLM, will adapt Liquid AI's models to specific industries via fine-tuning. The move addresses growing demand for compact, specialized on-premise AI solutions in Germany, driven by data protection, compliance, cost, and energy concerns. Financial details of the partnership were not disclosed.

• Liquid AI has formed a development partnership with Vago Solutions GmbH to enter the German market.
• Liquid AI was founded in 2023 as a spin-off from MIT's CSAIL.
• Liquid AI raised a $250 million Series A led by AMD Ventures, reaching a $2.35 billion valuation.

Liquid AI Enters German Market with Vago Solutions Partnership

US AI company Liquid AI, a 2023 MIT CSAIL spin-off valued at around $2.35 billion, has entered the German market through a development partnership with Vago Solutions GmbH, a specialized AI firm based in Hennef. The collaboration aims to build efficient AI models that companies and public authorities can run on their own infrastructure, on-premise instead of in the cloud. Liquid AI uses Liquid Neural Networks instead of transformer architectures, with open-source LFM2 and LFM2.5 models and the LEAP deployment platform. Vago, known for the German-language open-source model SauerkrautLM, will fine-tune and adapt the models to specific industries and domains. The companies did not disclose financial details or the scope of the cooperation. The partnership reflects growing demand for compact, specialized AI models for data-protection, compliance, cost and energy reasons in German-speaking markets.

• Liquid AI was founded in 2023 as a spin-off from MIT CSAIL and is based in Brookline, Massachusetts, with around 120 employees.
• Liquid AI raised a $250 million Series A led by AMD Ventures in late 2024, reaching a valuation of approximately $2.35 billion.
• Liquid AI and Vago Solutions GmbH announced a development partnership to deploy efficient, on-premise AI models in Germany.

MiniMax H3 on vLLM-Omni: From System-Wide Optimization to Real-Time Serving with FastVideo’s FastH3

How vLLM-Omni optimizes and scales the complete MiniMax H3 stack, then integrates FastVideo’s four-step FastH3 for generation faster than playback.

Qwen3-8B inference benchmark and FP8 on Blackwell

Independent benchmarks compare Qwen3-8B inference on an RTX PRO 6000 Blackwell (96 GB) across three serving stacks (vLLM 0.27.1, SGLang 0.5.9, and llama.cpp CUDA). At concurrency 32 using BF16, vLLM achieved 1,725 aggregate tokens/s (TTFT p50 39 ms), SGLang 1,327 tok/s (TTFT p50 42 ms), and llama.cpp 428 tok/s (TTFT p50 316 ms). Applying an FP8 checkpoint to vLLM increased throughput by ~1.5x (aggregate 1,725 -> 2,597 tok/s; single-stream 86 -> 130 tok/s) with lower latency and no detected regressions on a fixed factual check. The author documents methodology, reproductions, and an sm_120-specific kernel workaround required to run FP8 on workstation Blackwell hardware.

• GPU used: RTX PRO 6000 Blackwell, 96 GB (workstation Blackwell, sm_120).
• Model benchmarked: Qwen3-8B across vLLM 0.27.1, SGLang 0.5.9, and llama.cpp (CUDA).
• BF16, concurrency 32 aggregate throughput: vLLM 1,725 tok/s; SGLang 1,327 tok/s; llama.cpp 428 tok/s.

Tokens-per-Second Benchmarks Explained

This technical guide explains what "tokens per second" (tok/s) actually measures for local LLM inference, why single-user tok/s numbers can be misleading, and how concurrency, batching, and prompt processing change the observed speed. It contrasts single-user latency with server throughput, highlights vLLM's continuous-batching advantage versus Ollama under high concurrency, defines related metrics (P99 latency, time to first token / TTFT), and provides practical measurement advice using tools like Ollama and vLLM and calculators from notAcalculator. The article also gives realistic tok/s expectations for different model sizes on consumer hardware and lists practical tips for reading and running benchmarks yourself.

• Tokens are the unit of both billing and speed for LLMs; tokenization affects cost and measured tok/s.
• Under a Red Hat benchmark on an A100 40GB with Llama 3.1 8B, vLLM peaked around 793 tok/s combined throughput versus about 41 tok/s for Ollama at high concurrency (~19x gap).
• vLLM's key innovation is continuous batching (plus PagedAttention), which increases total throughput under concurrency compared with single-request processing tools.