paper-analyst: a six-round deep read per paper, with hard gates against templated hallucination

Why this line exists

paper-detail-generator v5.2.0 writes one deep-read detail page per paper in the library (bilingual HTML). In the digital twin the worker is paper-analyst: one paper, one task, status running for the whole six-round workflow. The reason it exists is an incident. v1 once used a script called gen_paper_details.py to batch-generate 207 detail pages from "title + abstract", and 205 of 214 failed the quality bar: templated hallucinated prose, invented formulas, references to figures that did not exist, identical structure across papers. That script was deleted and a repo iron law was written over it: never restore it, never create anything of the same kind.

So the first property of this skill is not "can write analysis". It is "cannot take the shortcut". Six mandatory rounds turn "actually read this paper" into artefacts that can be checked.

Six rounds, each with its own self-check

Round	What happens	Self-check
1 Full text	Confirm the version number on the abs page, then fetch https://arxiv.org/html/<id>v<ver>; for non-arXiv sources use defuddle parse	Did you get the real formulas and experiment tables from the method section? If not, switch version or switch method and keep going
2 Code	Extract the repo link from the full text (usually end of Introduction or the footer); if absent, search GitHub by title + authors, then git clone --depth 1 and read the README and core sources	Code released → cite ≥2 concrete "paper method ↔ source file" correspondences with paths. No code → state plainly that the paper ships none
3 Figures	scripts/fetch_paper_figures.py downloads the real figures from the arXiv HTML into data/static/img/papers/<id>/figN.png	How many downloaded, how many cited? Citations ≥3, and remember fig1/fig2 as cover candidates
4 Analysis	Internal, never surfaced as output: 1-3 core contributions, each mapped input → processing → output → why it is better; ≥5 real formulas with symbol meanings; the most convincing experiment; author-admitted weaknesses plus your own judgement; which file/function each component lives in	All six items have conclusions
5 Writing	Academic register, one fully independent HTML per language, no mixed-language typesetting	See hard bar below
6 Review + publish	Walk the self-review checklist, write via set_paper_detail.py, score with the machine inspector, verify live with curl	quality_score ≥ 8.0, and the output must contain cover_image: /static/img/papers/...

Round 1 has a practical wrinkle: curl arxiv.org from a local box is often very slow, so the standard move is to download on the production server and scp back. When the full text is huge, read it in slices with sed -n 'X,Yp' until abstract, intro, method, formulas, experiments and conclusion are all in hand. A minority of papers have no arXiv HTML, or only a stub file; then fall back to defuddle parse or convert the PDF to text.

The hard bar: the numbers are gates, not suggestions

• ≥ 4000 words per language (HTML tags excluded), ≥ 20 paragraphs
• ≥ 5 real formulas from the paper, extracted from the full text, rendered by KaTeX
• ≥ 3 real figures, and they must come from the Round 3 download
• ≥ 2 experiment tables carrying real numbers
• ≥ 1 Mermaid diagram, built from that paper's own pipeline rather than a generic template
• ≥ 2 stated limitations, at least one of them the authors' own admission
• A closing section, with a memorable line

The section order is fixed too: metadata (title, authors, link, code status) → one-sentence summary (under 100 words) → background and motivation (4-5 paragraphs) → preliminaries (2-3, if needed) → method walkthrough (≥8 paragraphs, the centre of gravity) → experiments (≥3 paragraphs, tables and figures interpreted rather than pasted) → limitations → conclusion and outlook → the memorable line. The writing brief explicitly bans AI filler ("delve into", "of paramount importance", "it is worth noting that") and requires each Mermaid to reflect that paper's specific pipeline. Both rules exist to kill the smell batch generation leaves behind.

Three gates: machine score, metadata integrity, human sign-off

Writing to the database is not the finish line. set_paper_detail.py calls quality_inspector.score_detail() on insert, scoring 0-10 into papers.quality_score. The scorecard is structural completeness 40% (word count, formulas, figures, tables) + technical accuracy 30% (formula delimiter safety) + factual traceability 15% + language quality 15%. Under 8.0 means not acceptable: follow the inspector's pointers to the weak dimension, rewrite the HTML, re-score until it passes. Re-score an existing row with quality_inspector.py <paper_id> --write.

Next is the metadata gate (since 2026-09-01): the confirm-publish endpoint validates completeness of bilingual titles, summaries and tags (paper_metadata_issues) and rejects with HTTP 409 if anything is missing, so the Chinese site can never display an English title. That gate came from a real incident: after the ZEST paper was ingested, the SQL that backfilled the Chinese title and summary failed silently on shell quoting, and the agent kept going without reading the row back. The Chinese page ended up with an English title, an English abstract, and a tag list containing the literal placeholder {Twitter}. The hard rule now: after any SQL or script touches metadata, read it back immediately; a zero exit code is not evidence of a write. UPDATE statements with nested quoting always go into /tmp/fix.sql, executed with docker cp + psql -f.

Last is human sign-off. Once the detail page lands, the paper enters the /manage/confirm queue (confirmed_at IS NULL); the owner clicks confirm to set published=true and record who and when, or reject to pull it back for a rewrite. The queue only shows agent output, and the pre-existing published corpus was backfilled, so it never drowns. Anything whose title/summary/tags were edited with raw SQL must be followed by reindex_search.py --fix, otherwise search and display disagree.

After publish: two webp tiers, then the social kit

arXiv source figures are heavy (median long edge 997px, mostly PNG); measured detail-page image load has a median of 2.64MB and a maximum of 19.89MB. Once published=true, Round 6.4 runs build_card_thumbs.py --only papers and build_body_thumbs.py --only papers inside the api container, pre-generating 800px/q78 and 1400px/q80 webp siblings for body figures and covers. Idempotent and incremental, no container restart, no code deploy. The webp files are derived siblings only, the original bytes are untouched, click-to-zoom swaps back to the original, and og:image uses the original.

Round 6.5 adds the social kit, and the order matters: OG card first (POST /api/social/card?kind=paper&id=<id>&both_langs=true), then a hand-written social rewrite which also renders the 1080x1350 knowledge card, then pull the six-platform English post bundle, then eyeball the result. The rewrite is authorship: no script may fill that table from title+summary with a template, which is the same failure class as the deleted gen_paper_details.py. Before eyeballing, compress cards with scripts/rw_tools.py small to ≤1280px long edge and ≤200KB; feeding view_image an original PNG burns the context window.

Live verification and legacy repair

curl -s "https://robot.agientry.com/api/papers/<paper_id>?lang=zh" | python3 -c \
"import sys,json; d=json.load(sys.stdin); h=d.get('detail_html',''); print('len:',len(h),
'figs:',h.count('/static/img/papers/'),'mermaid:',h.count('mermaid'),
'table:',h.count('<table'),'cover:',d.get('cover_image'))"

All four numbers get read: len in the 4000-word ballpark, figs ≥3, table ≥2, and cover shaped like /static/img/papers/<id>/figN.png. When an existing paper misses the bar, the repair path is to re-read that paper properly, not to pad it with a template; then re-run the inspector and the verification. If SSH drops mid-run, reconnect with -o ConnectTimeout=60 -F /dev/null, and put long batches under setsid so a timeout cannot interrupt a write.