TP The business of what comes next Saturday 21 November 2026 · updated 02:02 Get the weekly briefing
Tomorrow’sPublisher.
Policy & Rights Policy Rights

Bay City News blocks two AI bots as licensing talks continue

Noah Wire · 3 min read

Editorial cartoon accompanying: Bay City News blocks two AI bots as licensing talks continue

Cartoon · AI-generated

After requests from ChatGPT-User and Claude-User fell to zero, the newsroom says blocking bots can give publishers leverage without cutting off ordinary Google Search traffic.

Bay City News says it sharply reduced requests from two AI bots after changing how its websites handle automated traffic, offering a practical example of how publishers can restrict access while debates over payment and licensing continue.

The newsroom’s LocalNewsMatters.org site began using Cloudflare in early November and, several days later, blocked ChatGPT-User and Claude-User, which can fetch pages in response to user requests. The publication reported that requests from those bots fell to zero after the change, while clicks arriving from AI services and Google Search remained steady. Cloudflare’s documentation confirms that its AI Crawl Control tools let participating site operators inspect crawler activity and block selected bots, although the company’s general guidance does not independently verify the newsroom’s reported results.

The experiment followed an unsuccessful attempt to use robots.txt, a file through which site owners can ask crawlers not to visit specified pages. OpenAI’s documentation says ChatGPT-User visits may be initiated by a user or custom GPT and that robots.txt rules may not apply to those requests. It also distinguishes that bot from OAI-SearchBot, which helps surface websites in ChatGPT search, and GPTBot, which may crawl material for model training. The distinctions mean that blocking one kind of access need not require publishers to shut off every OpenAI-related use.

Advertisement · Noah

The newsroom also used Google Search Console to manage whether its material appears in Google’s generative AI features. Google’s guidance treats that setting separately from controls governing ordinary search indexing, snippets and certain media files. As a result, changes to participation in generative AI features should not be confused with a blanket restriction on Google Search visibility.

The intervention has limits. Bay City News distributes stories to partner websites, where the material can still be accessed and scraped; controlling requests to its own site does not prevent other publishers from making their copies available. Cloudflare’s tools likewise apply to domains routed through its network, and the company’s documentation sets out different monitoring capabilities according to account plan. The practical reach of blocking therefore depends partly on whether other sites carrying the same reporting take similar steps.

For publishers, the dispute is about more than whether a bot visits a page. AI-generated answers may draw on reporting without sending readers to the organisations that produced it, potentially weakening referral traffic and other routes to revenue. A Brookings analysis argues that journalism also supplies AI systems with current information and factual grounding, while warning that local and specialist outlets may have less influence in emerging licensing arrangements than larger publishers.

Licensing is one possible alternative to restricting access, but it is not a settled solution. A Nieman Journalism Lab account of an Open Markets Institute report describes proposals including payment per crawl or use, alongside concerns about who controls the marketplaces and how revenue is divided. Research by Columbia Journalism School’s Tow Center also records mixed views among publishers: some see licensing payments as recognition of journalism’s value, while others worry that agreements could weaken their future bargaining power or prove temporary.

The financial and legal arguments are already playing out in court. The Associated Press reported that The Intercept, Raw Story and AlterNet sued OpenAI in 2024, alleging that their stories had been used to train chatbots without permission, payment or credit. Those claims remain allegations in the supplied account; the case illustrates how concerns about the use and attribution of journalism have moved beyond negotiations over commercial deals.

Bay City News presents blocking as leverage rather than a preferred end state, arguing that publishers would rather have their work used in AI services on fair terms. Its proposed next steps include examining how chatbot responses differ when local reporting is unavailable and how a licence might follow content as it moves around the web. The broader challenge is to find arrangements that compensate reporting without leaving smaller newsrooms with little say over the systems that set the terms.

Inspired by headline at: [1]

Sources by paragraph: - Paragraph 1: [2] - Paragraph 2: [2] - Paragraph 3: [3] - Paragraph 4: [4] - Paragraph 5: [2] - Paragraph 6: [6] - Paragraph 7: [5], [8] - Paragraph 8: [7] - Paragraph 9: [5], [6]

Source: Noah Wire Services · Original

Make sense of what's next.

A weekly briefing for people shaping the future of publishing.

Free. Email requests are handled by our team.