From The Trail, a site by ACG · www.christiantrail.site/faq/text/cherry-picking
← About this siteHow do I know you are not cherry-picking?
Read it as an illustrated story · 6 minDon't take anyone's word for it, The Trail's included. It's built to be checked: quotes link to their sources, every claim keeps the best case against it, and scripts test each quote against a saved copy of its page. Which questions get asked is still a human choice.
The worry is fair. Anyone can build a case from real quotes by leaving out the awkward ones. And a Catholic will tend to find the friendly ones.
Cherry-picking comes in two kinds. One is cutting a quote so it says what the writer didn't mean. The other is choosing which questions get asked at all.
The first is much easier to guard against than the second. This is how The Trail tries to do both, and where it still falls short.
The whole record, kept together
Every claim sits in its own small file in the project. It holds one sentence, the source's exact words, a link to the original, what the claim can't prove, and how others read the same evidence.1
The best case against a claim is kept right beside it.
You can see those files on the site. At the foot of every chapter, a folded section called Check it yourself lists each claim the chapter leans on, with its source, its other readings, its limits and how far it's been checked.1
To see one in full, the method page's chapter IV, One claim, start to finish, follows a single claim all the way from an old letter to the page you read.1
Think of it like this
Think of buying a second-hand car.
Any seller can park it with the shiny side facing you and the dented door against the wall. They're not lying, exactly. Everything you can see is real. You just can't see everything, and they know it. You find the rust after you've paid.
A seller you can trust does it differently. They walk you round the whole car, dent included. They hand you the full service log, with the accident report still in it. And they give you the number of the garage that did the repairs, so you can call and check for yourself.
That's the idea here: show the whole record, and tell you how to check it. You shouldn't have to trust the seller's smile.
Good intentions aren't enough, so scripts check the files. Every source page is downloaded and saved as plain text, and every quote must be found in one, word for word.1
The saved copies stay in the project's own files, not on the site, because they belong to the sites they came from. Every source still links to the original, so you can check it there.1
And the checks work. In September 2026 they caught eight quotes the AI had written from memory in one day's writing: quotes that couldn't be traced to any saved page.1
Under the site's rule, a quote like that is shown to you as unchecked, and its claim stays a draft.1
So what happened to those eight? Each one was looked up in its document. Seven turned up there word for word: the wording had been right, it just hadn't been checked.2
The eighth was in a translation whose web page couldn't be reached to check it. So it was replaced with another English translation of the same passage, one that could be checked.2
All eight were settled by 8 October 2026. Each claim's Check it yourself card tells you how, in its audit note.2 And as of October 2026, the method page counts no claim left at draft.1
What a script can't see
Now the honest part. A script can't tell if the sentence around a quote says more than the quote does. Spot checks in September 2026 found about one sentence in five doing that.1
When 120 diocese histories picked at random were read that way, about one sentence in sixteen was plainly wrong, too. Each one was corrected.3
That's a lot. So a second AI agent, not the one that wrote the page, now reads each sentence beside its quote and cuts whatever says more.1 To be fair, an AI reading an AI isn't independent review. The read-through by someone from another tradition hasn't happened yet.
Every saint's life, every history of a diocese (the area a looks after) and the Catholic history of every country has now had that reading. Each one shows a mark saying so: Checked line by line.3,4
The and stories on the country pages haven't had it yet, and say so: Sourced, not yet line-checked.5 The chapters and these FAQ answers don't carry a mark like that yet. So for them, you can't yet see which have had the second reading.
And no script notices what's missing.
A page can quote every source correctly and still mislead by never raising the awkward question.
Who picks the questions? ACG does. The questions, the direction and the argument are mine, he says.6
There's no public list yet of hard questions the site hasn't answered. If you think one is missing, that's the most useful thing you could tell the project: thetrailofchristianityproject@gmail.com.1
An old rule for historians
The Trail holds itself to a rule much older than any website. In August 1883 Leo XIII wrote about how Church history should be written. He told writers to answer false stories by going back to the original sources.7
And he gave them one rule to keep in mind. In English it runs roughly like this: the first law of history is not to dare to say anything false, and then not to dare to leave out anything true.7
The point was that no one could suspect the writer of favour or hostility.7 Leo wrote mainly to defend the popes against histories he thought unjust. But his rule binds his own side too: you don't get to drop the true things that hurt your case.
He backed it up. He had already opened the Vatican's archives, as far as possible, to serious study. In the same letter he opened the Vatican Library's collections to the writers too.7
Still, a rule on paper proves nothing. The real test is one you can run yourself: open the sources, and look for what's missing.
Sources7
- 1The Trail, How we know (the method page), chapters I, III and IVClaim files and the 'Check it yourself' lists; saved pages and the quote-matching scripts; the eight quotes from memory; one sentence in five, and the second reading; what scripts can't see; no claim at draft; chapter IV follows one claim from source to page.
- 2The Trail, chapter: The Catholic Reformation and Trent, 'Check it yourself' audit notes (the same notes appear in 'What was wrong in the medieval Church' and 'Mary's Assumption')How each of the eight quotations flagged on 23 September 2026 was settled: re-fetched and found word for word (23 September and 8 October), or, for the Council of Florence passage, replaced with another translation that was checked.
- 3The Trail, front page: "How this map was made", note 6, "The histories"Two labels, checked line by line and sourced, not yet line-checked; in 120 diocese histories read at random, about one sentence in five said more than its source and about one in sixteen was plainly wrong, each corrected; 3,163 histories read line by line in full (October 2026). Each history shows its mark.
- 4The Trail, a saint's life: Thomas AquinasEach saint's page shows its mark under "The evidence": here "checked line by line".
- 5The Trail, a country page: SingaporeThe Catholic history is marked "Checked line by line"; the Orthodox and the Protestant stories are marked "Sourced, not yet line-checked".
- 6ACG, About The Trail ("What people ask me": "Do you use AI to build this?")AI helps find sources, draft pages and write code; the questions, the direction and the argument are his.
- 7Pope Leo XIII, Saepenumero considerantes, letter on historical studies (18 August 1883), Latin original; English renderings here are a paraphrase"Primam esse historiae legem ne quid falsi dicere audeat: deinde ne quid veri non audeat"; the archive and library put at research's service.
A question this page doesn’t cover?
Ask it in the question box at the bottom right of the page. It searches the whole site and answers from it. Or start with how this site checks what it says, and the full list of sources, each linked to the original.