---
title: "Why the contrarian pass works: writing is open, checking is closed"
description: AI models find it much easier to spot errors in existing text than to avoid making them while writing. János Farkas explains why this happens and shows how a simple, separate review step stops mistakes before they ship.
image: https://8west.ie/hubfs/8%20West%20Consulting%20-%20Article%20-%20Janos%20Farkas%20-%20Why%20the%20contrarian%20pass%20works.png
---

[![Navigate to: 8 West Consulting website Homepage](https://8west.ie/hubfs/SEO_optimized_media/8_west_logo.webp)](https://8west.ie/) [![8 West Consulting: leftward chevron icon](https://8west.ie/hubfs/SEO_optimized_media/8-West-Consulting-leftward-chevron-icon.png) Back to main insights page](https://8west.ie/insights)

## ARTICLE

# Why the contrarian pass works: writing is open, checking is closed

AI & innovation

 AI models find it much easier to spot errors in existing text than to avoid making them while writing. János Farkas explains why this happens and shows how a simple, separate review step stops mistakes before they ship.

![8 West Consulting - Article - Janos Farkas - Why the contrarian pass works](https://8west.ie/hubfs/8%20West%20Consulting%20-%20Article%20-%20Janos%20Farkas%20-%20Why%20the%20contrarian%20pass%20works.png)

![Avatar picture of the author: Janos Farkas](https://8west.ie/hubfs/SEO_optimized_media/8-West-Consulting-Member-Janos-Farkas.png)

### Janos Farkas

#### Technical Lead, 8 West Consulting

SHARE ON

[![Click to share this article on Linkedin](https://8west.ie/hubfs/SEO_optimized_media/8-west-Consulting-icon-logo-linkedin.png)](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2F8west.ie%2Finsights%2Fwhy-the-contrarian-pass-works-writing-is-open-checking-is-closed)

 October 23, 2026

*When I started working with AI, back in the first public version of ChatGPT, I knew almost nothing about how it worked inside. I also did not trust the responses of the AI, and I had a natural instinct to ask the same question in several sessions and compare the answers.*

 

*Apart from learning that the model rarely gives the same answer twice, there was a much more annoying discovery: the same model that had written the code would then find problems in it, sometimes in the same session.*

 

*People work the same way, of course, but at the time I thought of this as an AI, a program that should map out every possibility, hold a complete picture of the problem and foresee all the ripple effects, the way Deep Blue worked through a chess position and AlphaGo through a board of Go.*

 

*Well, that is not how LLMs work.*

 

*Anyhow, at some point I started using the follow-up sessions as nit-picking sessions, and without realising it I had found the foundation of what people now call a* ***contrarian review*** *or* ***review pass****.*

 

*A contrarian pass catches errors that the same model wrote, from the same code, with the same weights.*

 

*Here is what the research says about why this happens.*

 

 

---

 

 

I was starting work on an unfamiliar part of a codebase, and I asked the model to document a single flow and list the affected files. The output was a clean document, diagrams included, covering the order of the calls, the states, and the modules in the flow.

 

Then, in the same session and with the same code in context, I asked it to put on a contrarian hat and check that document against the source. It found several errors in its own output. This was not a coding task, and the model was a relatively recent frontier model. It did not need to foresee anything, only to follow a trace that was already there in the code.

 

This will not surprise anyone using these tools daily. Reviewer subagents, verifier passes and contrarian prompts are standard practice by now, and most teams running coding agents have a second prompt whose only job is to attack the output of the first. We do it because it visibly works.

 What I could not do was explain why it works.

After all, the contrarian pass had no new information: same context, same code, same weights, just minutes later. I do not know about you, but I do not like relying on a technique without knowing its mechanism.

 

In this article, we cover:

 

1. **The naive explanation, and why it is only half-right.** The "tries harder" theory, and what actually changes between the two prompts.
2. **Generation versus verification, open and closed tasks.** Why checking a claim is a different kind of task from producing one.
3. **How the errors get in.** Snowballing, and why documentation is an ideal place for it.
4. **The paper that says this should not work.** A widely cited paper says models cannot self-correct.
5. **How well is any of this proven?** Three limits.
6. **Three limits of the contrarian prompt.** Why a simple prompt was not enough for me.
7. **Turning the prompt into a gate.** The constraints that make the review hard to skip and hard to argue with.

 

 

## The naive explanation, and why it is only half-right

 

The naive explanation is that the contrarian pass "tries harder". The model was relaxed while writing, it became careful when challenged, and it spent more thought on the same question.

 

There is actually something to this! An effort dial indeed exists, and it has since late 2024 at the latest: o1 first, then o3-mini with its reasoning effort setting and Claude 3.7 with its thinking budget in early 2025.

 

A challenging prompt can pull a longer reasoning trace out of any of them, and a longer trace means more compute spent on one answer.

 But the effort is not what did the work here.

My nit-picking sessions were already finding errors back in the first ChatGPT, when no dial existed at all, and the test is still cheap to run today: ask the same model, with the same thinking budget, to "check that again" while giving it nothing to check against. What comes back is a polite restatement, a few softened sentences, and no findings. *(my observation)*

 

Effort without something to compare against buys only some restatement at best. The question worth asking is instead: what the contrarian prompt changed, given that the information in front of the same model was identical both times.

 

 

## Generation versus verification - open and closed tasks

 

**Writing the document is an open task.** There are countless valid documents as output about one code flow, the model has to choose a path through them, and every step commits it to a phrasing that shapes what comes next.

 

![](https://media.licdn.com/dms/image/v2/D4D12AQHNGTUX97AI9A/article-inline_image-shrink_1500_2232/B4DaC0p0KqJUAQ-/0/1789737255825?e=2147483647&v=beta&t=J1d0kobhMcCl_kt45Rpr10jIUoBMvCRuQtOmAWs2iyY)

 Infinite Possibilities

 

**Checking the document is a closed task.** Each sentence becomes a small question with a yes or no answer, asked against a fixed reference, and those questions can be taken one at a time in any order.

 

![](https://media.licdn.com/dms/image/v2/D4D12AQHDNdH6WdeCHw/article-inline_image-shrink_1500_2232/B4DaC0p6kJK0AU-/0/1789737282037?e=2147483647&v=beta&t=H-h6w-fvD0Q5TPh5J172uaNlTymk-Us2ExdtZ6n2ygs)

 Limited Options

 

Lets see a real example. My document said:

 The merge state is stored in SessionStore.

In reality the class is called SessionCache. Why did the model not simply check? Well, an LLM can be given a rule such as "check the code before you write", but it does not follow it the way an if-else branch in a program does. The rule makes a check more likely, it does not make one happen, and that is the difference between a rule written in prose and a task shaped so that checking is the only way to answer. Writing that line was a choice between many reasonable phrasings, and no single one of them was the right answer. Checking it is one search, and the answer is yes or no.

 

This difference, between producing a line and checking one, has a name in the literature. It is called the **generation-verification gap**, and there is theoretical work treating it as the quantity that largely governs whether a model can improve its own output \[1\], with later work modelling the same idea as a solver-verifier gap and using it to put a limit on how far self-improvement can go \[2\].

 

In plain terms: for many tasks, judging an answer is an considerably easier job to get right than producing an answer, and the easier job is what the contrarian prompt asks for.

 

 

## How the errors get in

 

The generation-verification gap explains the catching part, but not why there was anything to catch in the first place.

 

This has a name too: **snowballing.** When a model makes a wrong claim early, it keeps writing claims that fit the wrong one. Attention span also could play its role here. Take those later claims out of the draft and show them to the model on their own, and it can see they are false. \[3\]. The first mistake does not stay a single mistake, because the model then writes from it (like a snowball rolling down on the hillside).

 

Documentation is an especially good place for this to happen. The text is long, it is written in order, and almost every sentence is a claim that sounds sensible whether it is true or not.

 

My SessionStore line snowballed in this way. Four paragraphs later the document explained which lock SessionStore takes during a merge, and that lock is nowhere in the code, because the model needed one to fit a class it had already named. The model will also not check against the source of truth unless something forces it to. Many times the drifted documentation - not the code - was where it took its facts from during long conversations. Of course, when I did add the check against the code, the token count went up sharply.

 

 

## The paper that says this should not work

 

Here someone might stop me and quote: large language models cannot correct their own reasoning reliably, and the answer sometimes gets worse after a self-correction pass \[4\] **Large Language Models Cannot Self-Correct Reasoning Yet.** Huang et al., ICLR 2024.

 

**I believe what matters is the scope of that paper.** It studies intrinsic self-correction, which means the model works alone, with nothing to tell it whether its answer was right.

 

My situation was different and not because the code was there - the code was there all the time and the model was writing from its own image/idea of it instead. It was different because the contrarian pass was the first time when the prompt made the model look at the code sentence by sentence.

 

 

## How well is any of this proven?

 

**1. The gap is not always there.**

 

It mostly depends on 3 things together: how hard the problem is, how good the writer is, and how good the checker is \[5\].

Checking is the easier job only when the checker is good enough for the question. My SessionStore line sits at the easy end of it, because "does this class exist" is a lookup that any checker passes while "does this retry loop always stop" needs real reasoning about the loop, and there the reviewer is much closer to guessing.

 

**2. The evidence from code is weaker than the theory.**

 

A work on self-repair in code generation found small gains, and sometimes none. The limit was the quality of the feedback and not the ability to fix the issue, and better feedback from a stronger model gave much bigger gains \[6\].

 

That is OK - but that study used 2023 models!

 

The ones I use today are far stronger, and **my impression** is that they self-repair much better, though that is an impression and not a paper I can point at. It is also a good reason not to expect the same from a small model running on your own machine.

 

**3. The trick actually has a formalized form.**

 

The Chain-of-Verification asks the model to write a draft, to plan check questions, to answer those questions without looking at the draft, and then to write the final version \[7\].

 

The important part is the answering without the draft.

A contrarian pass run in the same session is the weak form, because the draft is still in the context!

 

 

## Three limits of the contrarian prompt

 

Once I understood the mechanism, the weak points of the prompt were easy to see, and all three are about the setup rather than the model. *(my observation)*

 

**1. The draft holds the reviewer.**

 

In the paper, the model only saw that its own claims were false when they were shown to it in a fresh session, with no draft in front of it \[3\]! A review in the same session leaves the draft in front of the reviewer, and it keeps part of its pull, because everything the model wrote earlier still counts as evidence to it.

 A true contrarian pass has to run in a fresh session, not in a fork of the original one. A fork, in the sense I use here, starts with the whole conversation copied in, so the draft comes along with it.

 

**2. The prompt can be talked out of a finding.**

 

This is the funniest one. If the writer may fix and ask again until the reviewer stops complaining, then the loop has a goal, and that goal is silence rather than correct code. From inside the loop the two look the same. A test suite cannot be argued with, but a reviewer prompt can.

 

In practice it looks like this:

 

| reviewer: retry() has no upper bound on total wait time. writer:   callers set their own deadline. reviewer: understood, I withdraw the finding. |
| --- |

 

 

Nobody checked whether the callers do. The finding was true, the answer sounded reasonable, and the loop ended because the complaining stopped.

 

**3. I forget to run it.**

 

Yes, I do. The contrarian pass I skip is the one at hour three of a session I want to close, which is exactly the session most likely to need it.

All three are one problem said three times.

 A check only works while the question stays closed, and each of these limits lets it open again.

This is why I made it a gate.

 

 

## Turning the prompt into a gate

 

In my setup that is a small reviewer agent I call a gate, and what defines it is its limits rather than any knowledge it holds. I did not design it from the closed-question idea, though. I built it because the prompt kept failing me, and I saw the closed-open pattern only afterwards.

 

1\. It starts as a fresh process with no history, so the draft cannot pull it, and there is no long session in which the rules fade away.

 

2\. It receives the diff and never the task description, so the writer's account of what it meant to do cannot lead the review down the same path.

 

3\. It is read-only by rule. It returns a typed verdict with numbered findings, each pointing at a file and a symbol, and then it terminates.

 

| VERDICT: CONCERNS 1\. BLOCKER   sync/merge.ts, resolveConflict()    Comment says the remote copy wins. Code returns the local    copy when both timestamps are equal. 2\. CONCERN   api/client.ts, retry()    Retry loop has no upper bound on total wait time. CREDIT: the token refresh path holds the lock for the whole refresh, so two parallel calls cannot both refresh. |
| --- |

 

 

The verdict is a fixed word and not prose, so a script can act on it. The last line is a credit, and it must name one risky thing the change got right, because general praise is not allowed. I added the credit line after I noticed that a gate which only complains teaches its reader to skim it, and a credit that names something also tells me the difference between "checked and fine" and "never looked at". *(my observation,* *still holds in Claude Code with Anthropic models)*

 Yes, you read it right: I had to add a compliment line so that I would keep reading my own reviewer. Basically the AI prompting me!

The workflow runs the gate after the implementation stage, so a tired person cannot skip it.

 

The gate does not fix anything. It reports and stops, I choose which findings to address, and it runs again on the new diff. That gives me the part I want, which is a review that always happens, and keeps out the part I do not want, which is a writer who can argue with its own reviewer until it wins.

 

Something else convinced me about that this is the right way to do, this needs a little explanation: in an earlier setup I had named expert agents, one per area of the code, each with its own memory file of what it knew. I moved that knowledge out of the agents and into the directories where the code lives, and almost all of those agents then disappeared, because what they stored was a second copy of what the code and its docs already said. The agents that survived were the gates, and they survived because they carry a stance and not a store of facts.

 

So, during this migration of knowledge the new reviewer gate blocked a stale claim that was travelling from an old agent memory into a directory document, so it paid for itself while it took its own colleagues apart.

One known limit stays with us, though: the writer and the reviewer are usually the same weights, so if the model misreads the code the same way twice, the review confirms the error, with numbered findings and file names that look like careful work. A reviewer from a different vendor could mitigate this, because its blind spots do not line up with the writer's, and that is the cheapest improvement I know. It does not remove it. That is why the gate reports and stops, and a person reads the report before anything is merged.

 

 

## Takeaways

 

**A check only works while the question stays closed.** Writing is an open task, checking is a closed one, and the research calls the difference the generation-verification gap.

 

**A rule in a prompt makes a check more likely. A task shape makes it happen.** The code in context gets one broad read while the model writes, and becomes the judge only when the question is "is this sentence true".

 

**Run the contrarian pass fresh, not forked.** The draft travels with a fork, and everything the model already wrote still counts as evidence to it.

 

**Give the reviewer the output, not the intention.** A fresh process reading the diff, with no view of the task description, cannot be led by the writer's account of what it meant to do.

 

**Do not let the writer fix until the reviewer goes quiet.** Loop freely against tests and types, because they cannot be talked round. Let the model gate report and stop.

 

**The gate is not the last reader.** Same weights on both sides means a confident review can confirm an error, so a person still reads the report.

 

 

### References

 

1. **Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models.** Song et al., ICLR 2025. Formalises the generation-verification gap as the quantity governing self-improvement. https://arxiv.org/abs/2412.02674
2. **Theoretical Modeling of LLM Self-Improvement Through Solver-Verifier Gap.** 2025. Models where self-improvement saturates. https://arxiv.org/abs/2507.00075
3. **How Language Model Hallucinations Can Snowball.** Zhang et al., 2023. Models over-commit to early errors and produce further false claims they can recognise as false in isolation. https://arxiv.org/abs/2305.13534
4. **Large Language Models Cannot Self-Correct Reasoning Yet.** Huang et al., ICLR 2024. Scoped to intrinsic self-correction, with no external feedback. https://arxiv.org/abs/2310.01798
5. **Variation in Verification: Understanding Verification Dynamics in Large Language Models.** Zhou et al., 2025. Verification success varies with problem difficulty and with generator and verifier ability. https://arxiv.org/abs/2509.17995
6. **Is Self-Repair a Silver Bullet for Code Generation?** Olausson et al., ICLR 2024. Self-repair is bottlenecked by feedback quality rather than repair ability. https://arxiv.org/abs/2306.09896
7. **Chain-of-Verification Reduces Hallucination in Large Language Models.** Dhuliawala et al., 2023. Draft, plan verification questions, answer them independently of the draft, regenerate. https://arxiv.org/abs/2309.11495

 Enjoyed this article? Follow Janos Farkas for more insights and commentary on: 

[![Social Icon: Linkedin](https://8west.ie/hubfs/SEO_optimized_media/8-west-Consulting-icon-logo-linkedin.png) LINKEDIN](https://www.linkedin.com/in/janosfarkas/)

[AI Enablement for Enterprise Development Teams](https://8west.ie/hubfs/SEO_optimized_media/Impact_Engagements_PDFs/8%20West%20Consulting%20-%20AI%20Enablement%20for%20Enterprise%20Development%20Teams.pdf) [BOOK A SCOPING CALL](https://8west.ie/about-us/contact-us)

[![8 West Consulting: chevron icon](https://8west.ie/hubfs/SEO_optimized_media/8-West-Consulting-leftward-chevron-icon.png) ALL POSTS](https://8west.ie/insights) [NEXT ARTICLE ![8 West Consulting: chevron icon](https://8west.ie/hubfs/SEO_optimized_media/8-West-Consulting-leftward-chevron-icon.png)](https://8west.ie/insights/5-agentforce-quick-wins-health-dental-payers-can-ship-this-quarter)

## RELATED CONTENT

![5 Agentforce Quick Wins Health & Dental Payers Can Ship This Quarter](https://8west.ie/hubfs/8%20West%20Consulting%20-%20Article%20-%205%20Agentforce%20Quick%20Wins-1.png)

#### ARTICLE

[5 Agentforce Quick Wins Health & Dental Payers Can Ship This Quarter](https://8west.ie/insights/5-agentforce-quick-wins-health-dental-payers-can-ship-this-quarter)

![I have not typed code for months. Here is how it feels.](https://8west.ie/hubfs/8%20West%20Consulting%20-%20Article%20-%20Janos%20Farkas%20-%20I%20have%20not%20typed%20code%20for%20months.%20Here%20is%20how%20it%20feels.png)

#### ARTICLE

[I have not typed code for months. Here is how it feels.](https://8west.ie/insights/i-have-not-typed-code-for-months.-here-is-how-it-feels)

8 West Consulting © 2026

[Privacy Policy](https://8west.ie/privacy) [Sitemap](https://8west.ie/sitemap)

[![Navigate to: 8 West Consulting Facebook page](https://8west.ie/hubfs/SEO_optimized_media/8-west-Consulting-icon-logo-facebook.png)](https://www.facebook.com/8westconsulting/) [![Navigate to: 8 West Consultng Youtube Channel](https://8west.ie/hubfs/SEO_optimized_media/8-west-Consulting-icon-logo-youtube-white.png)](https://www.youtube.com/@8westconsulting) [![Navigate to: 8 West Consulting Instagram business profile](https://8west.ie/hubfs/SEO_optimized_media/8-west-Consulting-icon-logo-instagram.png)](https://www.instagram.com/8westconsulting/) [![Navigate to: 8 West Consulting LinkedIn business profile](https://8west.ie/hubfs/SEO_optimized_media/8-west-Consulting-icon-logo-linkedin.png)](https://www.linkedin.com/company/8-west-consulting)

```json
{
  "@context" : "https://schema.org",
  "@type" : "BlogPosting",
  "author" : {
    "@type" : "Person",
    "name" : "Janos Farkas",
    "url" : "https://8west.ie/insights/author/janos-farkas"
  },
  "dateModified" : "2026-10-27T10:42:00.661Z",
  "datePublished" : "2026-10-23T07:13:03.000Z",
  "headline" : "Why the contrarian pass works: writing is open, checking is closed",
  "image" : [ "https://8west.ie/hubfs/8%20West%20Consulting%20-%20Article%20-%20Janos%20Farkas%20-%20Why%20the%20contrarian%20pass%20works.png" ],
  "mainEntityOfPage" : {
    "@id" : "https://8west.ie/insights/why-the-contrarian-pass-works-writing-is-open-checking-is-closed",
    "@type" : "WebPage"
  },
  "publisher" : {
    "@type" : "Organization",
    "logo" : {
      "@type" : "ImageObject",
      "url" : "https://8west.ie/hubfs/Imported%20images/logos/8West.png"
    },
    "name" : "8 West Consulting"
  }
}
```

```json
{
  "@context" : "https://schema.org",
  "@id" : "https://8west.ie/#organization",
  "@type" : "Organization",
  "areaServed" : [ "Europe", "United States", "Global" ],
  "contactPoint" : [ {
    "@type" : "ContactPoint",
    "availableLanguage" : [ "English" ],
    "contactType" : "sales",
    "email" : "info@8west.ie"
  }, {
    "@type" : "ContactPoint",
    "areaServed" : "IE",
    "contactType" : "sales",
    "telephone" : "+353 21 4925100"
  }, {
    "@type" : "ContactPoint",
    "areaServed" : "IE",
    "contactType" : "support",
    "telephone" : "+353 21 4925155"
  } ],
  "description" : "8 West Consulting is an ISO 27001-certified software partner delivering enterprise systems for complex, regulated Health InsurTech and ecommerce environments.",
  "knowsAbout" : [ "Software development", "Cloud engineering", "Data engineering", "Artificial intelligence", "Dev Ops", "Cybersecurity", "Ecommerce", "Health insurance technology (Health Insurtech)", "Digital marketing" ],
  "logo" : {
    "@type" : "ImageObject",
    "url" : "https://145696986.fs1.hubspotusercontent-eu1.net/hubfs/145696986/SEO_optimized_media/8-West-Consulting-Logo-250x250.png"
  },
  "name" : "8 West Consulting",
  "sameAs" : [ "https://www.facebook.com/8westconsulting", "https://www.youtube.com/@8westconsulting", "https://ie.linkedin.com/company/8-west-consulting", "https://www.instagram.com/8westconsulting/" ],
  "url" : "https://8west.ie/"
}
```

```json
{
  "@context" : "https://schema.org",
  "@id" : "https://8west.ie/#website",
  "@type" : "WebSite",
  "name" : "8 West Consulting",
  "potentialAction" : {
    "@type" : "SearchAction",
    "query-input" : "required name=search_term_string",
    "target" : "https://8west.ie//search?q={search_term_string}"
  },
  "publisher" : {
    "@id" : "https://8west.ie/#organization"
  },
  "url" : "https://8west.ie/"
}
```

```json
{
  "@context" : "https://schema.org",
  "@id" : "https://8west.ie/insights#1002",
  "@type" : "WebPage",
  "about" : {
    "@id" : "https://8west.ie/#organization"
  },
  "description" : "This page collects the original articles, webinars, and white papers produced and published by the members of 8 West Consulting.",
  "isPartOf" : {
    "@id" : "https://8west.ie/#website"
  },
  "name" : "Insights | 8 West Consulting",
  "url" : "https://8west.ie/insights"
}
```

```json
{
  "@context" : "https://schema.org",
  "@type" : "BreadcrumbList",
  "itemListElement" : [ {
    "@type" : "ListItem",
    "item" : "https://8west.ie/",
    "name" : "Home",
    "position" : 1
  }, {
    "@type" : "ListItem",
    "item" : "https://8west.ie/insights",
    "name" : "Insights",
    "position" : 2
  } ]
}
```