Observed Signal · Jun 14, 2024 · Analysis · Source: CMSWire · Impact: 3/5 · Sentiment: Positive
AI Inference: The New S-Curve After Training
The article examines the shift from AI training to inference as the next major market. It argues that while training has dominated compute demand, inference will soon become 100 times larger as AI moves into production. Nvidia currently dominates training with over 95% of specialized AI chips, but new entrants like Groq and open-source tools like GGML are enabling efficient inference on commodity devices. The piece also discusses the need for new benchmarks to measure AGI progress, as current LLMs excel at pattern matching but lack reasoning and creativity. Enterprises should prepare for inference-centric workloads and the emergence of edge computing.
Highlights the significant market shift from AI training to inference, impacting AI infrastructure and potentially enabling new AdTech use cases.
Track Amazon Signals & Market Shifts in Real-Time
Polaris7 autonomous intelligence agents track regulatory filings, primary sources, executive changes, and deal flow 24/7. Create your free Explorer workspace to monitor these entities.
Key Takeaways & Evidence Grounding
- Nvidia controls more than 95% of the specialized AI chip market.
- Groq's LPU architecture is designed to reduce latency and outperform GPUs for inference.
- GGML is a tensor library for running LLMs efficiently on commodity CPUs.
- Inference is expected to be 100 times larger than the training market.
- Nvidia's Blackwell GPU increases computational speed by over a thousand times in eight years.
Connected Companies & Entities
5 Entities mapped“They sell to major tech companies like Amazon, Meta, and Microsoft, which are believed to make up roughly 40% of its revenue....”
“They sell to major tech companies like Amazon, Meta, and Microsoft, which are believed to make up roughly 40% of its revenue....”
“They sell to major tech companies like Amazon, Meta, and Microsoft, which are believed to make up roughly 40% of its revenue....”
“Nvidia, controls more than 95% of the specialized AI chip market....”
“New companies like Groq are emerging that have the potential to be serious competitors in the AI chip market....”
Ontology Mapping & Concepts
Related Market Signals & Shifts
Recent verified developments and strategic activity across this market segment.
Microsoft Xbox creates new division for films, series, parks
Microsoft's Xbox gaming division announced the creation of a new business unit dedicated to films, television series, and theme park attractions. The move signals an expansion into entertainment and immersive experiences beyond video games, leveraging Xbox's intellectual property. This strategic diversification is part of Microsoft's broader ambition to grow its media and entertainment footprint, following trends seen across the industry. The new division will focus on developing and producing content based on Xbox franchises, potentially opening new revenue streams for the company. Financial details or a timeline for the division's operations have not been disclosed.
US Government Excludes Microsoft from Visa Program
The US government has barred Microsoft from participating in the permanent residency process for foreign workers with H-1B visas, accusing the company of abusing the program. Vice President JD Vance stated that Microsoft laid off 6,000 American employees last year while benefiting from 6,300 H-1B visa holders. The Department of Labor, led by Keith Sonderling, will not accept new permanent residency applications from Microsoft, as well as several consulting firms and Adobe. This action comes weeks before the midterm elections and reflects the Trump administration's broader criticism of the H-1B program, which it claims disadvantages American workers. Microsoft has not yet responded. The move could impact the tech industry's ability to retain skilled foreign talent.
Outlook to Block MSIX Files as Email Attachments
Microsoft has announced that Outlook will begin blocking MSIX file attachments, a move aimed at preventing malicious distribution through email. The change affects both the classic Outlook for Windows and the new Outlook for Windows, as well as other clients like Outlook on the web. Microsoft's decision comes in response to the increasing use of MSIX files in phishing and malware attacks. The block will be implemented in phases, starting with a controlled rollout and then expanding to all users. The company advises users to use alternative secured methods for sharing such files. This technical update is part of Microsoft's broader security enhancements for its email clients.
Track Real-Time Market Signals & Shifts
Set up custom watchlists to receive automated, evidence-grounded executive digests whenever material signals or shifts occur across your tracked landscape.