Preference Model Raises Funding for AI RL Environments
Preference Model, a startup founded by former Anthropic and DatologyAI engineers Jennifer Zhou and Ning Cao, announced it has built reinforcement learning (RL) environments for leading AI labs and is open-sourcing its production framework, Karotte. The company focuses on creating robust environments for AI research and ML engineering tasks, addressing challenges like reward hacking. The announcement, made on a16z's newsletter, indicates that a16z is partnering with and investing in the company. Karotte has been hardened through over a million evaluation runs and controlled red-teaming. The demand for RL environments has surged over the last 18 months as labs apply them to real-world tasks in coding, research, and computer use. Preference Model aims to accelerate AI progress by enabling models to assist in building better AI.
- •Preference Model announced a partnership and investment from a16z.
- •The company is open-sourcing its production framework, Karotte.
- •Karotte has been used in over a million evaluation runs and red-teaming.