Azure Machine Learning

API vs Self-Hosted LLM: 2026 Cost Comparison

This 2026 analysis compares the real costs of self-hosting open LLM weights versus calling third-party APIs. It argues APIs are typically cheaper for most teams until sustained volumes reach roughly 5–10 million tokens per month on premium models; beyond that, raw compute costs can favor self-hosting but hidden costs (networking, storage, redundancy, monitoring, and engineering headcount) often erase savings. Example 2026 list prices cited: Claude Sonnet 5 at ~$2 input / $10 output per million tokens, OpenAI GPT-5.6 starting near $1 per million input, and Meta Muse Spark around $1.25 input / $4.25 output. NVIDIA H100 rental ranges from $2–3/hr on specialized clouds to ~$7/hr on AWS and ~$12/hr on Azure. The piece recommends starting on APIs and migrating high-volume, stable workloads to self-hosting when full-cost math justifies it.

Karpenter consolidation: 6 settings to tune in 2026

A Dev.to post (published 2026-06-23) explains how Karpenter's consolidation defaults prioritize compute cost over workload disruption tolerance and recommends tuning six settings to balance cost savings and stability. The author notes Karpenter 1.0 reached GA in late 2024 and defaults changed in 1.2 (mid-2025). Rising spot interruption rates and multi-architecture fleets in 2026 increase consolidation churn. Key recommendations include using consolidationPolicy=WhenEmptyOrUnderutilized for cost-sensitive fleets (but WhenEmpty for stateful workloads), increasing consolidateAfter from the 1m default based on workload type, setting explicit disruption.budgets (not percentages) with cron schedules for deploy windows, using disruption.expireAfter to limit node age, increasing pod terminationGracePeriodSeconds for long-lived connections, and pinning NodePool instance families and weights to avoid poor instance selections.

Microsoft Open-Sources OmniVec Embedding Platform

Microsoft (Azure Cosmos DB team) open-sourced OmniVec, a platform to build and operate embedding pipelines that keep vector representations of operational data in sync. OmniVec lets users register sources, embedding models, destinations (vector stores) and pipelines; it handles backfill, change tracking, model invocation, retries and writes to vector stores. The release includes support for Azure Cosmos DB, PostgreSQL, SQL Server (source and destination) and Azure Blob Storage (destination). OmniVec is deployed into an Azure subscription and provisions AKS, an Azure Cosmos DB metadata store and an ACR; it can call hosted Azure OpenAI models or self-hosted GPU models. The project is available on GitHub (AzureCosmosDB/OmniVec) and is published as a public preview.