Observed Signal · Aug 1, 2026 · Technical Release · Source: DEV Community · Impact: 3/5 · Sentiment: Neutral
Enterprise Resource Planning & HR Market: Aggregators block scrapers; ATS job APIs are public
An analysis published on 2026-08-01 found that major job aggregators (Indeed, Glassdoor, ZipRecruiter, Upwork) often return 403 responses with empty bodies (blocked by CDNs/WAFs), while the original sources—company career boards served by Applicant Tracking Systems (ATS)—expose public JSON job-board APIs that answer anonymous requests with no API key. The author measured responses from Greenhouse, Ashby, Lever, Rippling, Workable, Recruitee and SmartRecruiters and found these endpoints return clear semantics (200 with jobs, 200 empty list for no openings, 404 for unknown board) in most cases, though SmartRecruiters behaves differently. Practical notes include the need to discover board tokens, differences in payload size when requesting full content (Greenhouse descriptions increase payload ~18x), and that some ATS endpoints publish compensation bands behind flags.
Practical measurements reveal that direct ATS job-board APIs are publicly accessible and often more current than aggregators, which matters for data collection, scraping design, and vendor integration but does not by itself change major industry infrastructure or policy.
Key Takeaways & Evidence Grounding
- Measurements taken on 2026-08-01 show indeed.com/jobs, glassdoor.com, ziprecruiter.com, and upwork.com/nx/search returning 403 responses with 0 bytes.
- Public ATS job-board endpoints responded to anonymous GETs: Greenhouse, Ashby, Lever, Rippling, Workable, Recruitee and SmartRecruiters; example: boards-api.greenhouse.io/v1/boards/stripe/jobs returned 200 with 548 jobs.
- Deriving company 'board tokens' from domains and probing ATS endpoints resolved ~90% of 30 companies versus ~50% when scraping careers pages for direct links.
- Greenhouse documents its job-board API as public (no authentication required); Lever explicitly permits third-party scraping of published postings.
- Requesting content/description fields can drastically increase payload size (Greenhouse: jobs endpoint 111 KB vs ?content=true 1.97 MB), and Ashby exposes compensation bands via an includeCompensation parameter.
Connected Companies & Entities
11 Entities mappedIndeed
Employment marketplace monetised through employer-paid sponsored job advertising.
“indeed.com/jobs 403 0 bytes...”
Social platform monetised through community advertising and data licensing.
“reddit.com/r/programming/hot.json 403 189,908 bytes (and .rss responses changing between 200 and 429)...”
Stripe
Financial infrastructure platform for payments, billing, fraud, and commerce operations.
“Example: boards-api.greenhouse.io/v1/boards/stripe/jobs returned 548 jobs; 'Stripe's 548 roles' cited as a sample...”
Visa
Global payments network and enterprise transaction infrastructure provider.
“SmartRecruiters example: 'real company Visa 200 {"totalFound":2, "content":[...]}'...”
Upwork
Freelance work marketplace with payments, subscriptions, and internal ads.
“upwork.com/nx/search 403 0 bytes...”
Notion
Unified workspace software for docs, knowledge and projects.
“Notion mentioned as returning compensation fields full of nulls (company choosing not to publish)...”
OpenAI
Foundation model company selling AI software, APIs and subscriptions.
“GET https://api.ashbyhq.com/posting-api/job-board/openai 200 753 jobs...”
Ramp
Finance automation platform for cards, spend, payments and procurement.
“Ashby example: GET /posting-api/job-board/ramp ?includeCompensation=true returns "$211.4K - $290.6K"...”
Palantir
Enterprise software for operational data integration, analytics and AI deployment.
“GET https://api.lever.co/v0/postings/palantir?mode=json 200 302 jobs...”
Databricks
Enterprise lakehouse platform for data, analytics and AI.
“Databricks cited with example job counts (Databricks' 803 roles) as results from direct ATS endpoints...”
Linear
Issue tracking and product development SaaS for software teams.
“Linear mentioned as returning compensation fields full of nulls (company choosing not to publish)...”
Ontology Mapping & Concepts
Track Real-Time Market Signals & Shifts
Set up custom watchlists to receive automated, evidence-grounded executive digests whenever material signals or shifts occur across your tracked landscape.