Snorkel AI Logo
Snorkel AI· HQ in Redwood City, California

Research Scientist, Open Benchmarks Red Team

Location: New York City, NY (Hybrid); San Francisco, CA (Hybrid); United States (Remote) · Category: Data, AI & Analytics · Seniority: Senior · USD200,000 – USD340,000 / yearly · Posted: November 13, 2026 · Verified: Nov 16, 2026

Role Overview & Description

This role focuses on red-teaming frontier AI benchmarks to identify weaknesses such as reward hacking, grader failures, and task ambiguity. The researcher will develop and validate fixes, create reusable methods for benchmark quality improvement, and collaborate with benchmark authors to publish improved versions. Requires strong Python skills, experience with LLM/agent evaluation, and a research or engineering background in machine learning.
Hiring Telemetry · Expansion Signal

Track team expansion and market signals for Snorkel AI

Polaris7 tracks open roles, strategic market movements, and M&A developments from verified primary sources. Receive structured Watchlist briefings on Snorkel AI and benchmark against direct competitors.

Source & Transparency: This position was structured directly from the publicly accessible career portal of Snorkel AI. Details on the employer’s original job posting are authoritative. Polaris7 charges no placement fees and conducts no candidate screening.
Company IntelligenceVerified Profile
Snorkel AI Logo

Snorkel AI

snorkel.ai

Enterprise platform for AI data development and deployment.

Headquarters: Redwood City, California

Similar Positions in DACH

Related roles across advertising, media, and data

Verified 3w agoSan Francisco, CAData, AI & AnalyticsSenior

This research-first role designs rubrics, benchmarks, and evaluation methods to measure whether AI works for real people over long horizons. The AI Research Engineer will build Dscout's first real-user benchmark, study human judgment as a training signal, and publish findings to the research community. Requires a PhD or MS with strong research experience in evals, preference learning, or human-AI interaction, plus strong Python and applied statistics skills.

Re-check scheduledSan Francisco HQ; Toronto HubData, AI & AnalyticsSenior

This role builds and runs rigorous, automated, public benchmarks of data providers, publishing weekly results that rank providers on accuracy, coverage, freshness, speed, and cost. The engineer owns the test datasets, ground truth, scoring methodology, and automated harness, and feeds results back into Alexandria's provider selection. Requires 4+ years in ML or data engineering with shipped evaluation or benchmark work, strong Python skills, and experience with messy vendor APIs.

Re-check scheduledSan FranciscoData, AI & AnalyticsSenior

This researcher role focuses on ensuring that increasingly capable AI agents act safely and remain aligned with user intent, spanning training, measurements, and oversight. The researcher will train and evaluate frontier models, mine incidents, and build scalable measurement and evaluation systems. The role requires strong research engineering, ML engineering, or applied model research experience, with the ability to own ambiguous projects end to end.

Re-check scheduledSan FranciscoData, AI & AnalyticsSenior

This applied research role focuses on post-training open-source models for customers, building reward functions, training environments, and eval harnesses to outperform frontier closed models on specialized tasks. The researcher will design and run pipelines including SFT, GRPO, DPO, and RLVR, and work directly with customers to translate production data into training signal. Requires hands-on LLM fine-tuning and reinforcement learning experience, with strong intuition for reward engineering and multi-turn agent environments.

More Positions at Snorkel AI

3 more open roles

Verified 2d agoNew York City, NY (Hybrid); San Francisco, CA (Hybrid); United States (Remote)RemoteData, AI & AnalyticsLead

This research lead role heads the Open Benchmarks Grants program, providing technical direction and oversight for the development of open benchmarks. The lead will define research priorities, guide grant recipients, and ensure the benchmarks are rigorous, relevant, and impactful for the AI field.

Verified 2d agoSan Francisco, CA (Hybrid)Engineering & DevelopmentSeniorUSD208,000 – USD315,000

This role focuses on building backend product capabilities for Snorkel's AI data platform, including data cataloging, agent benchmarking, and workflow automation. The engineer will own features from concept to production, working closely with Product and Design teams. Strong backend engineering fundamentals are required, with experience in Python or other languages like Java, Go, or Rust.

Verified 2d agoSan Francisco, CA (Hybrid); United States (Remote)RemoteEngineering & DevelopmentSenior

This engineering management role leads the team responsible for building and maintaining infrastructure for federal and regulated industry clients. The manager will oversee the development of secure, compliant, and scalable systems, ensuring they meet strict government and industry standards.

Start with a company or market. Track what changes.

Create your Polaris7 workspace to explore extended company profiles, organize relevant players into Watchlists, and receive focused briefings when material changes occur.