When AI Invents the Evidence: A Warning for Every Author

3 hours ago

5 min read

When AI Invents the Evidence: A Warning for Every Author

AI & The Author | Wednesday 4 November 2026 | By Rob Frankson

A British prosecutor's use of a non-existent legal authority has brought an uncomfortable question into focus: when a convincing answer turns out to be invented, who takes responsibility for repeating it? For authors, researchers and publishers, this is not abstract. It goes to the heart of the trust readers place in our work.

The story behind the warning

On 3 November, the Guardian reported that a Crown Prosecution Service prosecutor cited a fictional case in High Court proceedings. The court criticised the explanation for the mistake, although the invented authority did not change the outcome of the particular challenge. The report says an online search produced an AI-generated summary containing the false reference. I have not independently examined the court transcript, so this account is attributed to that reporting.

The Guardian's report, 3 November

The presence of a fabricated citation does not establish that an entire document was machine-written. Nor does it prove that every AI tool is unreliable. It demonstrates something narrower and more important: a plausible-looking source reference is not proof that a source exists. That lesson travels far beyond the courtroom.

Why writers should pay attention

Most novelists will never prepare a legal argument. Yet the underlying problem is familiar. A writing assistant may supply a historical quotation, a scientific statistic, a publisher's submission policy or a supposed interview with an author. Each can arrive in polished language with the appearance of confidence. If we copy it into a manuscript, article or promotional claim without checking, the reader inherits our mistake.

The risk is especially acute in non-fiction and the supporting material around fiction. A historical novel's author's note, a science-fiction book's explanation of real astrophysics, or a blog post discussing copyright can blur the boundary between researched fact and persuasive invention. Elegant prose is no substitute for a source.

I use technology as an assistant, not an authority. A tool can help me discover a possible line of enquiry, organise competing arguments or suggest where to look next. It cannot relieve me of the responsibility to establish whether a claim is true before I put my name to it.

What official guidance says

The Crown Prosecution Service's guidance on digital material emphasises careful examination, proportionate lines of enquiry and transparency about the methods used. Its current public consultation on a revised Code for Crown Prosecutors is also concerned with clear principles and public confidence. These documents are not findings about the individual case; they are primary sources showing why verification matters institutionally.

CPS Disclosure Manual: digital material

CPS consultation on the Code for Crown Prosecutors

The same principle scales down to a home office desk. When an AI assistant offers a source, I want the original document, not merely another summary repeating it. When it offers a quotation, I want the words in context. When it offers a number, I want the dataset, the date and the method. Those checks are part of writing, not an optional administrative chore.

The false certainty problem

A fabricated citation is dangerous because it resembles the real thing. Names, dates and formal titles create an impression of provenance. The reader may reasonably assume the author has already checked them. Generative systems can reproduce that surface structure without establishing the underlying record. That is why the instruction 'include references' is insufficient on its own.

There is another trap. Once a dubious reference enters a draft, later editing can make it harder to spot. The surrounding paragraphs become more fluent, the argument more persuasive, and the writer more invested in the conclusion. A citation that should have been challenged at the beginning becomes the apparent foundation of a finished article.

For publishers, this is an editorial process issue rather than simply a technology issue. Human writers can misremember or misquote too. The difference is that automated systems can generate large volumes of polished material quickly, potentially multiplying the claims that need checking. Speed at the drafting stage can create a hidden bill at verification.

A practical verification rule

My rule is straightforward: if a factual claim matters to the argument, follow it back to the original source. For legal matters, that may be a judgment or official regulator's document. For publishing rules, it may be an agency's own submission page. For scientific claims, it may be the published study rather than a press release or a chatbot's retelling.

I also separate three kinds of material in working notes: facts I have independently checked; interpretations I am prepared to defend; and ideas that remain speculative. That simple division protects the freedom to explore possibilities without accidentally presenting them as established truth.

For science fiction, speculation is the point. I can imagine faster-than-light travel, self-aware starships and civilisations that have never existed. I do not need to pretend those inventions are verified science. The contract with the reader is different when I say 'imagine if' from when I say 'research proves'. A trustworthy author knows which contract is in force.

Copyright and attribution still matter

This week's broader debate about AI and creative work adds another dimension. The European Commission's consultation on technology and copyright closed on 3 November. It explicitly invited views on the use of protected content in AI. That is separate from the UK court report, but the issues meet at the same point: provenance, permission and accountability.

European Commission copyright consultation

When an AI answer reproduces a passage, a responsible writer should ask not only whether the information is accurate but whether the wording belongs to somebody else. Attribution and permission are not solved by adding a vague reference to an AI tool. Authors must continue to make deliberate choices about what they quote, paraphrase and publish.

What I would change in my daily workflow

First, use AI to identify questions rather than certify answers. Second, keep original links alongside the notes they support. Third, check names, dates, quotations and statistics before polishing prose. Fourth, revisit every source link at final proof. Finally, disclose meaningful AI assistance honestly, while keeping editorial responsibility with the named author.

None of this requires rejecting useful tools. It requires refusing to confuse assistance with authorship. The strongest use of AI may be to widen the range of possibilities we consider while leaving final decisions, factual checks and creative voice firmly human.

The author's line

The lesson from yesterday's report is not that one profession has failed and writers are somehow immune. It is that credibility is earned in unglamorous moments: opening the source, questioning a neat answer, removing an attractive but unsupported sentence. Readers rarely see that work. They notice when it has not been done.

For my own books and articles, I want AI to make the creative process more capable, not less accountable. Technology can be fast, articulate and useful. The responsibility for what appears under my name remains mine.

Sources and further reading

Guardian: fictional legal precedent cited

CPS: Digital Material guidance

CPS: Code consultation

European Commission: copyright consultation

About Rob Frankson

Rob Frankson writes original science fiction and publishes AI & The Author at 121minutes, exploring where emerging technology meets the craft, business and responsibilities of writing.

About Rob

Editorial transparency

AI & The Author is edited and published by Rob Frankson. Artificial intelligence is used to assist with news research, initial drafting, content organisation and supporting imagery. All articles are reviewed and, where necessary, edited by Rob Frankson before publication. The opinions, editorial position and final decision to publish remain the author's.