The Label is Broken
A follow-up investigation into DriveThruRPG's AI content filter
In The AI Hypocrisy, I reported that a scan of DriveThruRPG's public API returned roughly 17,400 AI-tagged products. That number came from a straightforward query: count the number of products where the AI disclosure flag is set to true.
After publishing, I got messages from people who had been wrongly accused of using AI. I went back to look at the data more carefully. What I found suggests the labeling system itself has problems nobody has reported.
The filter
DriveThruRPG introduced its customer-facing AI content filter around October 2024. Every product now carries one of three labels: "Handcrafted" (no AI involved), "Contains AI-Generated Content," or "Creation Method Not Chosen By Publisher." The last one is the default for products uploaded before the system existed.
The filter is what makes DriveThruRPG feel safe to shop. You set it in your account preferences, and AI content disappears from your browse results.
The system relies entirely on publisher self-reporting. DriveThruRPG's own policy page states that they "acknowledge enforcement challenges and trust in the goodwill of our partners." Enforcement is "largely manual."
What the data actually shows
I queried the API for every product with the AI flag set to true, then checked their creation dates.
The first page of results contains fifty products. All fifty were created before 2020. The oldest is from 2009.
Consumer-accessible generative AI tools did not exist until mid-2022, when Midjourney and Stable Diffusion launched.
Across the first several pages, roughly 350 to 400 products tagged as "Contains AI" predate the technology they are supposedly disclosing. These are not edge cases at the margins of the dataset. They are the first results the API returns, sorted by product ID.
The contradictory state
The problem goes deeper than wrong dates. DriveThruRPG's database stores the label as two separate boolean fields: one for "Contains AI" and one for "Handcrafted." In a three-option system, these should be mutually exclusive. A product cannot be simultaneously handcrafted and AI-generated.
But many products are exactly that. On the first page of AI-tagged results, 26 of 50 products carry both flags at once. One publisher has 22 products in this contradictory state. Their catalog is a children's RPG line published between 2009 and 2019, with no public connection to generative AI. These games were on the market years before Midjourney or ChatGPT existed.
This is not a publisher making a mistake on a dropdown menu. This is a data integrity issue. The database allows a state that the labeling system's own logic says should not exist.
The other direction
The labeling problem runs both ways.
Geek Native's December 2025 analysis of DMsGuild, DriveThruRPG's sibling platform, found the opposite problem. They called it "The Metadata Crisis." Every single Gold best-seller released on DMsGuild in 2025 was listed as "Creation Method Not Chosen By Publisher." The top-selling creators on the platform had simply opted out of the labeling system entirely.
At the bottom of the sales tiers, products are tagged AI when they are not. At the top, products are not tagged at all.
What this means
The number I originally reported is based on real API data. But the label behind that number has problems no one appears to have audited.
Products from before generative AI existed carry the AI flag. Products can be simultaneously marked as handcrafted and AI-generated. And on the sibling platform, the biggest sellers don't engage with the system at all.
The customer-facing filter that shoppers rely on to separate human-made content from AI content is built on data that contains contradictions, pre-AI ghost entries, and systematic gaps.
DriveThruRPG's AI filter is not broken in the sense that it doesn't function. It is broken in the sense that nobody has checked whether the data behind it is accurate.
The total number of AI-tagged products on the platform at time of this follow-up query is approximately 15,900. Whether that number reflects reality is a question the platform has not answered, because as far as I can tell, nobody has asked.
All data queried from DriveThruRPG's public API on August 3, 2026. Product creation dates and AI/Handcrafted flag data are publicly accessible without authentication. The contradictory-flag findings and pre-2023 date anomalies are reproducible by anyone with API access.