Observed Signal · Sep 8, 2026 · Research · Source: The Drum · Impact: 3/5 · Sentiment: Neutral
AI in Advertising Market: AI Can't Spot AI Ads but Can Judge Human Creative Quality
Andrew Tindall, a senior leader at System1, explores whether AI can effectively evaluate human-made advertising, given that humans cannot reliably identify AI-generated ads. He cites four major studies over ten months, including NYU Stern's 'AI Advertising Paradox' (AI ads outperformed human ads by up to 19% without disclosure, but performance dropped 31.5% when disclosed) and a Taboola study with 500M+ impressions finding similar results for short-term metrics like CTR. In his own experiment, he ran ten well-known human-made ads through System1's AI testing tool, comparing its predictions to human-based ratings. Results show the AI predicted the Fluency Rating correctly 10 out of 10 times and the Star Rating 9 out of 10 times, with one refusal to score. AI performed best at extreme high and low performers, struggled with mid-table ads, and highlighted the importance of acknowledging uncertainty. Tindall concludes that AI is a probabilistic prediction machine, better used to supplement rather than replace human judgment in creative testing.
Highlights the growing role of AI in creative testing and its limitations, relevant to AdTech and marketing effectiveness.
Key Takeaways & Evidence Grounding
- NYU Stern and Emory study found AI ads outperformed human ads by up to 19% without disclosure, but performance dropped 31.5% when AI involvement was disclosed.
- Taboola, Columbia, Harvard, TUM, and Carnegie Mellon study used 500M+ impressions and found 45% of AI ads were rated as likely human-made.
- System1's AI testing tool correctly predicted Fluency Rating 10/10 times and Star Rating 9/10 times on ten human-made ads.
- The AI declined to score Claude's Super Bowl ad, which received the lowest human testing score.
- In both human and AI testing, Coca-Cola's AI Christmas ad scored 5-Stars, indicating high performance.
Connected Companies & Entities
10 Entities mappedMeta
Consumer internet platforms monetised through advertising, apps, subscriptions and VR.
“Meta's Ray-Ban campaign was correctly pegged as low-performing by AI....”
Anthropic
Foundation model company selling AI assistants and model APIs.
“Claude, made by Anthropic, had a Super Bowl ad that the AI refused to score....”
Samsung Electronics
Global electronics maker monetising devices, streaming, advertising and digital services.
“Samsung's Galaxy S26 launch ad was rated 4-Star by humans, but AI could not narrow it beyond 'not a 1-Star'....”
Apple
Consumer electronics giant with integrated software, services and advertising platforms.
“Apple's email-rewriting ad scored 1-Star in both human and AI testing....”
System1
Performance advertising platform with owned media and intent-driven monetisation.
“Andrew Tindall leads growth at System1, and the AI testing tool, Test Your Ad, is a System1 product....”
Taboola
Open-web performance advertising and publisher monetisation platform.
“Taboola's study found 45% of AI-generated ads were rated as likely human-made....”
Coca‑Cola
Global beverage brand owner selling soft drinks and related beverages.
“Coca-Cola's AI Christmas ad scored 5-Stars in both human and AI testing....”
OpenAI
Foundation model company selling AI software, APIs and subscriptions.
“ChatGPT, made by OpenAI, had a pre-pivot Super Bowl ad that scored 1-Star in both human and AI testing....”
Harvard Business Review
Management media, subscriptions, advertising and executive research for business leaders.
“HBR study of 20 ads found AI underperforms compared to human-made ads....”
Search, video, adtech and cloud giant within Alphabet.
“Google's Gemini 'Dream Job' ad scored 3.1-Star in human testing and was correctly pegged as high performing by AI....”
Ontology Mapping & Concepts
Track Real-Time Market Signals & Shifts
Set up custom watchlists to receive automated, evidence-grounded executive digests whenever material signals or shifts occur across your tracked landscape.