Observed Signal · Sep 8, 2026 · Research · Source: The Drum · Impact: 3/5 · Sentiment: Neutral

AI Can't Spot AI Ads but Can Judge Human Creative Quality

Executive Signal Summary

Andrew Tindall, a senior leader at System1, explores whether AI can effectively evaluate human-made advertising, given that humans cannot reliably identify AI-generated ads. He cites four major studies over ten months, including NYU Stern's 'AI Advertising Paradox' (AI ads outperformed human ads by up to 19% without disclosure, but performance dropped 31.5% when disclosed) and a Taboola study with 500M+ impressions finding similar results for short-term metrics like CTR. In his own experiment, he ran ten well-known human-made ads through System1's AI testing tool, comparing its predictions to human-based ratings. Results show the AI predicted the Fluency Rating correctly 10 out of 10 times and the Star Rating 9 out of 10 times, with one refusal to score. AI performed best at extreme high and low performers, struggled with mid-table ads, and highlighted the importance of acknowledging uncertainty. Tindall concludes that AI is a probabilistic prediction machine, better used to supplement rather than replace human judgment in creative testing.

Polaris7 AgentPolaris7 Strategic Assessment
High Confidence

Highlights the growing role of AI in creative testing and its limitations, relevant to AdTech and marketing effectiveness.

MARKET SIGNALS

Track System1 Signals & Strategic Shifts

Polaris7 connects regulatory filings, primary sources, leadership changes, and M&A activity into structured intelligence. Set up your Polaris7 workspace to monitor relevant companies in your executive briefing.

  • Verified primary sources
  • Automated briefings
  • Instant watchlist setup

Key Takeaways & Evidence Grounding

  • NYU Stern and Emory study found AI ads outperformed human ads by up to 19% without disclosure, but performance dropped 31.5% when AI involvement was disclosed.
  • Taboola, Columbia, Harvard, TUM, and Carnegie Mellon study used 500M+ impressions and found 45% of AI ads were rated as likely human-made.
  • System1's AI testing tool correctly predicted Fluency Rating 10/10 times and Star Rating 9/10 times on ten human-made ads.
  • The AI declined to score Claude's Super Bowl ad, which received the lowest human testing score.
  • In both human and AI testing, Coca-Cola's AI Christmas ad scored 5-Stars, indicating high performance.

Connected Companies & Entities

“Andrew Tindall leads growth at System1, and the AI testing tool, Test Your Ad, is a System1 product.”

“Taboola's study found 45% of AI-generated ads were rated as likely human-made.”

“Claude, made by Anthropic, had a Super Bowl ad that the AI refused to score.”

“ChatGPT, made by OpenAI, had a pre-pivot Super Bowl ad that scored 1-Star in both human and AI testing.”

“Google's Gemini 'Dream Job' ad scored 3.1-Star in human testing and was correctly pegged as high performing by AI.”

“Samsung's Galaxy S26 launch ad was rated 4-Star by humans, but AI could not narrow it beyond 'not a 1-Star'.”

“Samsung's Galaxy S26 launch ad was rated 4-Star by humans, but AI could not narrow it beyond 'not a 1-Star'.”

Ontology Mapping & Concepts

Primary Source Grounding & Direct Attribution
Direct Origin Attribution
Primary Reporting: The Drum•Published: Sep 8, 2026
Original Coverage Title: “Andrew Tindall: Humans can’t spot AI ads. Can AI rate human ones?”

Related Market Signals & Shifts

Recent verified developments and strategic activity across this market segment.

Creation & Asset ManagementFeb 10, 2026

AI Ads Match Human Performance in Groundbreaking Study

A large-scale study titled "AI Ads That Work: How AI Creative Stacks Up Against Humans" analyzed several hundred thousand live ads on Taboola’s Realize platform to measure the impact of generative AI on user behavior. The dataset included over 500 million impressions and about three million clicks. Using a sibling-ads methodology—pairing AI-generated and human-created ads from the same advertiser, campaign and day—the study controlled for targeting and timing to isolate creative effects. Raw click-through-rates were 0.76% for AI ads versus 0.65% for human ads, but after statistical controls overall performance was comparable. Perception mattered: ads visually identified as AI-generated performed worse, while AI ads that lacked typical AI cues achieved the highest engagement. The presence of clear human faces increased perceived authenticity; AI-generated creatives more often included such faces. The authors conclude AI is a tool that can match or improve creative outcomes when human perception and trust signals are considered.

Read assessment
Generative AIOct 19, 2026

Study: AI Ads Underperform Human-Made Spots

A US study by Syracuse University and Ipsos reveals that AI-generated advertisements, while visually convincing, significantly underperform human-created spots in driving consumer purchase intent and brand equity. The study involved 3,000 US participants who viewed 20 ad videos for major brands, half of which were original human-made spots and half were AI recreations using Gemini and Sora 2. Although participants often struggled to distinguish between the two, AI-generated ads showed a 14% lower short-term sales potential and 17% lower long-term brand value. The researchers warn that the strategic risk lies in 'good enough' AI ads that cut costs but fail to build emotional connection, urging executives to measure actual outcomes rather than just production efficiency.

Read assessment
Creative & Strategy in Social AdvertisingJul 22, 2026

AI in Social Advertising: Creatives and Strategy Matter

The article explains that AI now automates large parts of social-media campaign operations—especially audience analysis, ad delivery, and ongoing optimization—shifting the advertiser's focus away from manual targeting. As platforms increasingly identify audiences and serve ads autonomously, the creative quality (images, video, copy, format) and a clear campaign strategy become the decisive factors for reach, clicks and conversions. AI can accelerate creation and testing of many creative variants, but strategic decisions (campaign goals, messaging, brand fit, hypothesis design) and final quality control must remain human responsibilities. The piece recommends systematic testing using explicit creative hypotheses and warns to critically review AI-generated content for errors before publishing.

Read assessment

WATCHLIST & BRIEFINGS

Receive future signals for System1 & competitors in your briefing

Stay ahead of material M&A deals, leadership changes, account wins, and product rollouts. Polaris7 connects primary filings, company newsrooms, and trade reporting directly into your executive briefing.

  • Regular primary-source checks
  • Structured Watchlist briefings
  • Preserves current context