Editor’s Pick Platforms & Discovery Editors Picks

AI’s shared opt-out still leaves publishers with separate bargains

Ivan Massow 10 Nov 2026 · 6 min read

A helper in a red cardigan pulls one blue thread from a publisher’s jumper, unravelling its sleeve and body. The caption reads: Just pull this one thread.

Cartoon · AI-generated

Publishers are being offered one instruction for telling AI companies not to train on their journalism. The first question is what else that instruction turns off. Google’s planned implementation links a general training refusal to an existing control that also governs whether material can inform answers in Gemini Apps. Microsoft is taking a different route. A common instruction is useful progress, but it is not a common bargain. Publishers gain a simpler way to state a preference; they still carry the risk of losing useful visibility or permitting uses they meant to exclude. The decision is which uses to allow before simplifying the paperwork.

The change sits behind announcements published on 30 October by Google, Microsoft and OpenAI. A proposed shared signal would let a website state a training preference for participating providers in robots.txt, the public file through which sites communicate instructions to automated visitors. That reduces duplicated administration. It does not mean every company is already applying the instruction.

Two activities must be kept separate. Training changes what a model has learned: it helps build the model. Grounding supplies material when an answer is being produced: it helps inform a particular response. A publisher might want to refuse the first while allowing the second, especially where an answer offers a useful route back to its reporting.

“Neither choice guarantees inclusion, traffic, or payment,” Krishna Madhavan, principal product manager at Microsoft Bing, wrote in the company’s 30 October announcement. Answer eligibility is an opportunity to evaluate, not income to book.

Advertisement · Noah

The UK publisher’s uncertainty is visible in a related, separate decision. Discussing the Competition and Markets Authority’s controls for Google Search, Josh Muncke, vice-president of generative AI at The Economist Group, said: “I don’t believe we’ve tested it yet,” according to Digiday’s 9 November report from its Media Strategies event. That comment concerns Search controls, rather than adoption of the new shared training signal.

The distinction is becoming a management problem. A publication needs an intelligible policy for how its work may be used, but the services reading that policy do not offer identical products or identical choices. Asking the technology team to put a single refusal on the website cannot settle the business question.

A common language, different consequences

The draft mechanism is called __usage_ai-training. It is a policy token, a label that participating systems can recognise. Kevin Kelley of Anthropic published the proposal on 30 October. The Internet Engineering Task Force’s tracker describes it as an individual Internet-Draft, not an endorsed standard. The proposal is work in progress.

Its scope is narrower than a blanket defence against scraping. The draft explicitly separates the usage preference from crawler rules. A provider may still fetch a page for another permitted purpose; a preference about training is not itself an instruction to stop that visit. Counting bot requests would therefore be a poor test of whether a training refusal was being respected.

That is a useful distinction for a publisher that wants discovery without contributing to future model training. It also means the shared signal cannot be sold internally as a technical barrier around the archive. Communicating a preference and preventing access are different jobs.

Google’s October announcement makes the commercial wrinkle explicit. If a site expresses the shared training refusal without explicitly specifying Google-Extended, Google plans to treat it as though Google-Extended had been disallowed. Google-Extended covers both future Gemini model training and grounding in Gemini Apps and Google Search grounding on its Vertex AI platform for businesses. Google says it does not affect inclusion or ranking in ordinary Google Search.

The result is more specific than the claim that refusing AI training makes a publisher disappear from Google. That would be wrong. The consequence concerns the uses covered by Google-Extended, while Search has a separate generative AI control. But the training-only label on the shared instruction does not, by itself, describe the full consequence of Google’s planned fallback.

Google says provider-specific controls will continue alongside the shared preference. That leaves room for exceptions, but it also leaves the publisher responsible for understanding them. A general preference and a company-specific permission can express different choices. A shorter configuration is not automatically a more accurate one.

Microsoft separates training and grounding. Its planned DisallowExtended grounding choice would permit Microsoft-operated experiences but exclude grounding services supplied to third parties. That distinguishes routes to market, not just welcome and unwelcome bots.

Consider a hypothetical publisher of local restaurant reviews. It wants readers to find its recommendations through an AI answer, but does not want the archive used to train a future model. A blanket refusal that also removes answer eligibility could sacrifice the first objective while achieving the second. An instruction that permits grounding may preserve that opportunity, but cannot guarantee a citation, a visitor or a paying subscriber.

The useful commercial unit is the permitted use: training a model, informing an answer, or supplying material through someone else’s service. Each can have a different value. Collapsing them into one decision makes administration easier by discarding information the business needs.

A rollout is not an implementation

OpenAI says it aims to implement the signal in the coming months and will publish details when it does. Existing crawler controls remain in place. Google’s crawler documentation points to early 2027, while its announcement gives the broader date of 2027. These are stated plans, not evidence that the shared preference is already honoured across those services.

Microsoft plans to honour the new preferences as it crawls from 1 February 2027. Sites relying on its older NoArchive or NoCache restrictions face a transition: from 1 March, those instructions will cease to express training or grounding preferences.

Sites using those restrictions as of 15 December 2026 qualify for a temporary Bing Webmaster Tools grounding preference. They need replacement page preferences and that temporary setting before 1 March to preserve restrictions pending recrawling. The safeguard ends when the replacement is discovered or on 30 September 2027, whichever comes first.

Publishing a setting is therefore not the same as having it take effect. The provider must discover it. A newsroom can approve a policy, its web team can implement it, and a provider can still be working from an earlier version. Verification needs a date and a defined service, rather than a screenshot of a file and an assurance that the job is finished.

Evidence belongs beside the choice

The publishers’ commercial question remains whether participation produces something worth having. An appearance in an AI answer, a visit to the website and a subscription are different events. A permission dashboard can help administer the first decision without demonstrating the value of the subsequent chain.

Digiday’s latest reporting places that problem inside a UK business. Chris Dicker, chief executive of Candr Media, which owns Trusted Reviews, identifies click data as essential to judging whether to use Google’s Search opt-out. That is a question about a particular distribution service, not evidence that one shared training setting determines every publisher’s traffic.

The same discipline should apply to a refusal. A publisher ought to know what it is withdrawing, which service should change and what it expects to learn. A change in referrals after an opt-out would still need to be assessed against search demand, editorial output and other product changes. A coinciding fall is not automatically a measured cost of restricting a particular use.

Nor does a machine-readable instruction establish a payment formula. A licensing agreement and a public preference file serve different purposes. The draft even allows private agreements to override preferences expressed through the mechanism. Commercial and technical records need to describe the same intended uses if either is to be useful.

The shared signal deserves support. Maintaining a separate instruction for every provider is needless friction, particularly for publishers with small technology teams. Making preferences easier to express can improve the conditions for useful AI services as well as strengthen publishers’ control.

But the test is whether a publisher can make a deliberate, observable choice without accidentally bundling away something it still values. A common vocabulary is a beginning. The commercial bargain remains in the details.

Sources

Source: Tomorrow’s Publisher