ChatGPT Search struggles with citation accuracy, raising concerns for publishers

OpenAI’s ChatGPT Search is reportedly facing significant challenges in accurately citing news publishers, as highlighted in a recent study conducted by Columbia University’s Tow Center for Digital Journalism. This scrutiny arrives shortly after OpenAI’s launch of ChatGPT Search, which had previously claimed extensive collaboration with the news industry to ensure proper attribution and visibility for publishers.

The study’s findings raise concerns regarding the potential misquoting and incorrect attributions that could undermine the credibility of news organisations. With 200 queries evaluated across 20 publications, the results reveal that 153 responses were incorrect. This misrepresentation can jeopardise brand visibility and erode the control publishers have over their own content, a critical factor in an increasingly competitive media landscape.

OpenAI’s push into news publishing follows its rollout of ChatGPT in 2022, where many publishers expressed concern over their content being used to train AI models without consent. In a bid to rectify these issues, OpenAI now allows publishers to dictate their participation in ChatGPT Search results through the robots.txt file. However, the Tow Center’s research indicates that whether a publisher opts in or out, the risks of misattribution and misrepresentation still loom large.

The accuracy problems identified during the evaluation are particularly concerning. The AI’s frequent inability to acknowledge its mistakes raises questions about user trust and the quality of information being disseminated. Instances where phrases like “possibly” were used occurred in merely seven responses, suggesting a tendency for the model to prioritise user satisfaction over factual correctness. Additionally, ChatGPT Search exhibited considerable inconsistency: identical questions yielded different answers, highlighting a potential flaw in its underlying language model.

The scope of the problem extends beyond flaws in attribution. The report concluded that the chatbot sometimes cites copied or syndicated articles instead of the original source, which can induce confusion and diminish the authority of the original publishers. For instance, queries pertaining to content from the New York Times revealed instances where ChatGPT linked to unauthorised versions of articles, even while facing an ongoing lawsuit against OpenAI and imposing restrictions on its crawlers. Likewise, a request for quotes from MIT Technology Review yielded citations from syndicated content rather than direct links to the original pieces.

These challenges illuminate a broader issue regarding OpenAI’s content filtration process. As the study indicated, enabling crawlers to access content does not guarantee visibility, nor does disabling them completely prevent material from appearing in search results. This reality presents significant implications for publishers, who may find their brand misrepresented or their content inadequately credited in an environment increasingly dominated by AI-driven tools.

In response to the findings, an OpenAI spokesperson reiterated the organisation’s commitment to supporting publishing partners by facilitating the access of ChatGPT’s 250 million weekly users to quality content through improved summaries and attribution methods. While OpenAI has acknowledged the difficulties in resolving specific attribution errors, they say they have an ongoing commitment to refining their search product.

As the news industry watches these developments keenly, the outcomes of ongoing legal challenges could lead to more robust control for publishers over their content.

Source: Noah Wire Services