# An inquiry framework for unsolved artifacts Research report, 22 October 2026. First case: the Voynich Manuscript (Beinecke MS 408). Later cases: the Rohonc Codex, Codex Seraphinianus, Linear A, Rongorongo, the Phaistos Disc, the Antikythera mechanism, the Shugborough inscription and the Tamam Shud case. Citation tags: **VERIFIED-FETCHED** means I read the source in this session. **VERIFIED-URL** means a search result confirmed the bibliographic record (title, authors, venue) but I did not read the text. **UNVERIFIED** means it is cited from memory and should be checked before print. Full references are in section 13. --- ## 0. Summary The project already does the hard part. It has a hypothesis registry, pre-registered decision rules, null models and positive controls, and a frozen hash. What it lacks is the layer above the tests: a record of *why* each hypothesis was asked, *what it would mean* if true, *how the results add up* across hypotheses, and *what to do next*. Because that layer is missing, the narrative reads as a sequence of tests instead of an argument. The framework proposed here adds that layer with six changes: 1. **One data model for the whole inquiry.** Questions, observations, hypotheses, readings, predictions, tests, runs, results, evidence links, theories and next steps are stored as plain YAML/JSONL in git with stable IDs. Every finding traces back to a script run, and every hypothesis records where it came from. 2. **Readings are first-class objects.** Every hypothesis must carry at least one concrete reading: what each section of the artifact would be, what a passage would say, and which measurable properties follow. A reading that cannot be written down is a sign the hypothesis is too vague to test. 3. **Systematic generation.** Hypotheses come from four named generators: morphological boxes, analogy to solved artifacts, differential lists, and LLM proposals passed through filters. Each generator logs its candidates, including the rejected ones. 4. **Automatic synthesis.** Evidence links carry a direction and a diagnosticity. From these, an ACH matrix and a capped log-likelihood-ratio score are computed for each set of competing hypotheses. Confidence is stated in IPCC/GRADE language and mapped to the project's tiers (Established / Supported / Working hypothesis). 5. **A lifecycle with exit criteria.** Every mystery runs the same cycle: induce, abduce, deduce, synthesise, publish. Each stage has a rule for when it is finished. 6. **Generated outputs.** The book appendix, the website, the hypothesis–evidence matrix, the evidence-and-gap map and a `resume` brief for the next worker are all built from the registry. None are maintained by hand. The research below supports these choices. The most useful lessons do not come from philosophy of science, which mostly supplies vocabulary. They come from how actual hard problems were cracked: - **Zodiac 340.** The team enumerated the hypothesis space exhaustively, sieved it with a cheap statistic, kept an "encyclopedia of observations" that separated verified facts from speculation, and verified the answer against external facts. - **Linear B.** Kober built grids of the data before attempting any phonetic reading, and Ventris circulated numbered work notes so others could follow and challenge each step. - **Kryptos K4 and Tamam Shud.** Both answers came from archives and DNA, not from the text. The framework must keep a non-textual track open. - **Polymath.** Small public contributions worked, distilled into a wiki, but linear threads stopped scaling. Issue tracking was needed. --- ## 1. Where the project stands | Present | Gap the framework fills | |---|---| | `hypotheses.yaml` (25 entries: id, family, statement, predictions, tests, result, status, files) | No origin or generator for each hypothesis. No reading attached. Predictions are free text, not linked to the test rows that check them. Result and status are prose, so they cannot be aggregated. | | `results.csv` (~80 rows: hypothesis, test, statistic, value, baseline, decision, note, date) | No run ID, script hash or input hash in the row. Some decisions are "see note". No evidence direction or diagnosticity per competing hypothesis. A result counts only for the hypothesis that ran it, even when it bears on others. | | `PROTOCOL.md` (statement → predictions → nulls/controls → decision rules → held-out sheets → reporting) | This is sound deductive machinery. There is no matching protocol for exploratory work, which produces the observations that new hypotheses come from. | | SHA-256 freeze of the registry (Part VII) | The freeze covers the whole file. It needs to be per hypothesis, with a record of which frozen tests have run and which are still pending. | | FINDINGS.md tiers (Established / Supported / Working hypothesis) | Tiers are assigned by hand. No rule maps evidence to tier. | | `ASSESSMENT.md`: the v0.1 zodiac/planet/Picatrix reading was withdrawn because its strings do not occur in ZL3b | The withdrawn reading exists only as a memo. It should be a registered hypothesis with specific claims and a recorded refutation, so the reasoning is preserved and the same mistake cannot be re-made silently. | | Book appendices E (registry) and F (methods inventory), built by `make_registry_tex.py` | These are generated, which is good. Nothing similar exists for the question map, the evidence matrix or the "what next" list. | --- ## 2. Philosophy and methodology of science Each tradition below solves one part of the problem. The table states what it contributes and where it lands in the framework. | Source | Core idea | Contributes | Where it plugs in | |---|---|---|---| | Hypothetico-deductive method | Derive observable consequences from a hypothesis and check them | The existing PROTOCOL.md | `Prediction` and `Test` entities; freeze before run | | Peirce's abduction (SEP entry, Douven) | A surprising fact C is observed. If A were true, C would be a matter of course. So there is reason to suspect A | The only inference that *creates* hypotheses. It starts from a surprise | Every `Observation` flagged `surprising: true` must spawn an abduction task (section 8) | | Lipton, *Inference to the Best Explanation* (2004) | Prefer the hypothesis that would, if true, provide the "loveliest" explanation (scope, precision, mechanism, unification), but only among those that are also "likeliest" | Separates two criteria. Explanatory virtue guides which hypotheses to pursue; likelihood decides which to believe | Two separate scores on `Hypothesis`: `pursuit_value` (for prioritising) and `standing` (from evidence). They are not mixed | | Grounded theory (Glaser & Strauss 1967) | Code the data, compare constantly, stop when new data yield no new categories (saturation) | A stopping rule for exploration | The *induce* stage ends at saturation: N consecutive exploratory runs yield no new `Observation` class | | Abductive analysis (Timmermans & Tavory 2012) | Theory construction starts from anomalies against existing theories. The researcher is an "informed theoretical agnostic". Heuristics: *revisiting* the same data, *defamiliarization* through inscription, *alternative casing* (reading one datum through several theories). Most abductions are dead ends | The most directly usable method. It ties induction and deduction together through surprise, and admits that most ideas die | Alternative casing becomes a required step: every new `Observation` is read against every live `Theory`, and the fit is recorded as `EvidenceLink`s | | Box's loop (Box 1976; Blei 2014, "Build, compute, critique, repeat") | Build a model, fit it, critique it with posterior predictive checks, revise | The model-criticism step that the project already does with controls and held-out sheets | `Test.kind = critique` for checks of a fitted generator against held-out statistics (T15 already does this) | | Bayesian model comparison (Kass & Raftery 1995) | Bayes factors measure evidence for one model over another; Jeffreys-style scales translate them into words | Rules for adding up evidence across tests | The scoring engine in 11(d) uses capped log-likelihood ratios per independent evidence cluster | | Lakatos (1970), research programmes | A programme has a hard core and a protective belt. It is *progressive* if its changes predict novel facts that are then confirmed, *degenerating* if changes only absorb anomalies after the fact | A way to judge whole families such as "cipher" or "generated text", not just single hypotheses | `Theory.trajectory`: count the novel predictions confirmed against the ad-hoc patches over time. H-CIPHER-MIXED's chain of untested "beyond rung L4" clauses should be watched for this | | Platt (1964), strong inference | Devise alternative hypotheses, then a crucial experiment that excludes some, run it cleanly, and repeat | Tests are chosen for their power to discriminate *between* hypotheses, not just to support one | The `NextStep` ranker in 11(c) prefers tests with high expected diagnosticity across the live contrast set | | Chamberlin (1890/1965), multiple working hypotheses | Keep several hypotheses alive at once to avoid "parental affection" for one | A guard against the v0.1 failure, where one appealing reading was developed alone | Every `Question` must hold at least three live hypotheses, including a "none of the above / meaningless" option, before any test is frozen | **Synthesis.** Induction produces observations. Abduction turns surprising observations into candidate explanations. Deduction turns those into frozen predictions with controls. Bayesian/ACH synthesis weighs the results. Lakatos judges the families over time. Timmermans and Tavory supply the glue: a recorded surprise is what triggers abduction. The project already runs the deductive step well. The framework makes the other steps equally explicit. --- ## 3. Intelligence, forensic and legal analysis ### 3.1 Analysis of Competing Hypotheses (Heuer 1999, ch. 8) ACH has eight steps: 1. List all hypotheses. 2. List significant evidence and arguments, including assumptions and absences. 3. Build a matrix with hypotheses across the top and evidence down the side, and mark each cell consistent, inconsistent or not applicable. 4. Refine the matrix by deleting evidence that has no diagnostic value. 5. Draw tentative conclusions by trying to *disprove* hypotheses. The winner is the one with the least evidence against it, not the most in favour. 6. Analyse how sensitive the conclusion is to a few critical items of evidence. 7. Report the relative likelihood of all hypotheses, not only the winner. 8. Identify milestones that would change the judgement. The key concept is **diagnosticity**. Evidence consistent with every hypothesis is useless however strong it feels. "The text has Zipfian word frequencies" is the standard Voynich example: it fits a natural language, a cipher of one, and many generators. **In the framework:** each `EvidenceLink` has a `direction` per hypothesis, and ACH is computed per `Question` over its contrast set. Step 4 becomes automatic: rows whose ratings are the same across all hypotheses are greyed out and listed as "non-diagnostic". Steps 6 and 8 become fields: `Theory.critical_evidence` and `Theory.would_change_if`. ### 3.2 Structured analytic techniques (US Government Tradecraft Primer 2009; Heuer & Pherson 2010) | Technique | What it does | Use here | |---|---|---| | Key assumptions check | List every assumption a judgement rests on and rate each solid, caveated or unsupported | Mandatory before any `Theory` is moved up a tier. In Voynich work the classic hidden assumptions are: EVA word breaks are real word breaks; the transliteration's glyph inventory is the scribe's; the illustrations and text belong together; the binding order is original (the project already tested and rejected this last one). Each becomes an `Assumption` record with a test, or a flag saying it is untestable | | Devil's advocacy | One person argues against the leading view | A scheduled "adversarial review" step in *synthesise*. An LLM can serve as a cheap first advocate, but its output is input to a human reviewer, not a verdict | | Team A / Team B | Two teams build the strongest case for rival views on the same evidence | Run on the top two families, e.g. "enciphered language" against "generated/meaningless". Each team must produce a reading and a test it expects to win | | Red teaming | Attack your own method | Apply to the instruments: can a generator beat the test battery while lacking the property claimed? The project's positive controls are already a form of this | | Premortem (Klein 2007) | "It is a year from now and this theory turned out wrong. Why?" | Required before freezing a high-stakes pre-registration. The answers become extra predictions or kill criteria | | What-if / high-impact low-probability | Assume an unlikely event and trace its consequences | Use for outside-the-text hypotheses: a later forgery, a key surviving in an archive (as with K4), a second manuscript in the same script | | Words of estimative probability (Kent 1964) | Fix the meaning of words such as "probable" to numeric ranges | Merged with the IPCC scale in 11(d) | ### 3.3 Argument maps, Wigmore charts and Bayesian networks - **Wigmore charts** (Wigmore 1913; modernised by Anderson, Schum & Twining 2005) break a case into ultimate probanda, penultimate probanda and evidential propositions, with numbered nodes and symbols for testimony, circumstantial evidence, corroboration and explanation. Schum's main contribution is the split between the *credibility* of a piece of evidence and its *relevance*. - **In the framework:** `EvidenceLink` gets separate `credibility` and `diagnosticity` fields. In Voynich work, credibility covers questions like "is this a property of the manuscript or of the transliteration?". The project already measures this with H-TRANSLIT-ROBUST. - **Argument maps** (Twardy 2004 reports gains in critical thinking from teaching with argument maps) lay out claim, reasons and objections as a tree. - **In the framework:** every `Theory` page on the site shows its map: the claim at the root, supporting and opposing evidence links as children, and assumptions as leaves. - **Bayesian networks for legal evidence** (Fenton, Neil & Lagnado 2013) give reusable idioms: evidence–accuracy, explaining-away, alibi, dependency between items. The **dependency idiom** matters most here. Two tests that rest on the same transliteration and the same word segmentation are not independent, and counting both double-counts. - **In the framework:** the scorer in 11(d) groups results into *evidence clusters* that share an input representation and statistic family, and caps each cluster's contribution. A full BN is not proposed. It would be over-engineering at this scale. --- ## 4. Medicine ### 4.1 Differential diagnosis The clinical routine is a good template for generating explanations and then ruling them out: 1. Write down the chief complaint, which here means the anomaly. 2. List *all* plausible causes, ordered by prevalence and by danger. "Can't miss" diagnoses go on even if rare. 3. Choose tests that separate the top candidates. 4. Rule causes out, not in. 5. Revisit the list when a test result surprises. Two habits transfer directly: - **The "can't miss" list.** In mystery work, these are hypotheses that would be costly to ignore: a modern forgery, a transliteration artefact, a misordered binding. Each is ruled out early and explicitly. The project has done this for binding order (H-ORDER-BINDING) and transliteration (H-TRANSLIT-ROBUST). Forgery is handled by the radiocarbon date and the ink work, but it is not registered as a hypothesis with a status. - **Prevalence priors.** A clinician starts from base rates. The analogue is the base rate of each explanation among comparable artifacts. Among historical enciphered manuscripts catalogued by the DECRYPT/Copiale community, nomenclators and homophonic ciphers dominate. Among undeciphered *scripts* with substantial corpora, unknown language in a known script type dominates. These base rates set priors in 11(d). Uniform priors are the fallback. ### 4.2 GRADE certainty of evidence GRADE rates certainty as high, moderate, low or very low (Guyatt et al. 2008; Balshem et al. 2011). - **Downgrade for:** risk of bias, inconsistency, indirectness, imprecision, publication bias. - **Upgrade for:** large effect, dose–response, and confounders that would reduce the effect. The mapping to this project is close: | GRADE domain | Analogue in artifact inquiry | Measured by | |---|---|---| | Risk of bias | No positive control; decision rule set after seeing data; exploratory result reported as confirmatory | `Test.prereg` flag; control pass/fail | | Inconsistency | Result changes across transliterations, segmentations or reasonable analysis choices | Multiverse spread (section 5) | | Indirectness | The statistic is a proxy for the property claimed (e.g. h2 as a proxy for "alphabetic") | `Prediction.proxy_for` field, rated by hand | | Imprecision | Small samples (e.g. the 500-word scribe samples in H-HANDS-PROCEDURE) | Confidence interval or permutation-null width | | Publication bias | Only "interesting" runs logged | Every `Run` is logged automatically by the CLI, including failed and null runs | | Upgrade: large effect | z ≈ 9.6 for sheet vocabulary cohesion (H-ORDER-SHEET) | Effect size against null | ### 4.3 Living systematic reviews and evidence and gap maps - A **living systematic review** (Elliott et al. 2014) is updated continually as new evidence appears, with a declared search frequency and a declared rule for when a conclusion changes. - **In the framework:** the literature side of the inquiry, meaning published Voynich claims, is a living review. Each external claim becomes a `Hypothesis` with `origin: literature` and is re-scored when new tests land. - An **evidence and gap map** (Snilstveit et al. 2013, World Bank WP 6725) is a matrix of interventions by outcomes. Bubbles show how much evidence exists and of what quality, and empty cells are gaps. - **In the framework:** rows are hypothesis families and columns are artifact sections or evidence types (text statistics, codicology, imagery, materials, provenance). Each cell shows the number of independent result clusters and the best GRADE level. Empty cells are candidate next steps. This is the single most useful "what has been asked and what has not" view. --- ## 5. Exploratory versus confirmatory practice | Practice | Idea | Use here | |---|---|---| | Tukey (1977; 1980, "We need both exploratory and confirmatory") | Exploration finds patterns and confirmation checks them. The danger is using the same data for both | Two run modes in the CLI: `explore` and `confirm`. Exploratory runs may produce `Observation`s but never move a `Hypothesis` status | | Preregistration (Nosek et al. 2018) | Fix hypotheses, analyses and decision rules before seeing outcome data. Deviations are allowed if they are reported | Per-hypothesis freeze in 11(c), with a `deviations` list that must be filled in if the run differs from the frozen plan | | Multiverse analysis (Steegen et al. 2016) | Run the analysis under every reasonable combination of processing choices and report the whole distribution | Voynich has a natural multiverse: transliteration (ZL, Takahashi, Currier–D'Imperio), glyph segmentation (EVA characters, merged units such as `ch`/`sh`/`ain`, Currier units), treatment of uncertain spaces, inclusion of labels, Currier A/B split, scribe split. Every confirmatory test declares which axes it is robust over. The project's H-TRANSLIT-ROBUST is one axis of this, done by hand | | Specification curve (Simonsohn, Simmons & Nelson 2020) | Plot the estimate across all specifications, sorted, with the choices shown underneath, and test the whole curve against a null | The standard figure for any statistic whose sign or size depends on processing choices | | Many analysts (Silberzahn et al. 2018) | 29 teams analysing one dataset reached a wide range of conclusions | Cheap version: two independent implementations of each key statistic (one human, one LLM-written from the spec), compared before a result is accepted. For ratings in the ACH matrix, independent raters score before discussion. The IPCC guidance recommends the same step | | Holdout and replication splits | Keep data the exploratory work never touches | Already in PROTOCOL.md §5 (sheet-level split). Add a second, **sealed holdout**: a set of sheets no exploratory run may read. The CLI enforces this through the dataset loader. Every surviving confirmatory result is re-run once on the sealed set before reaching "Established" | The sealed holdout is the most important addition. Thirty sessions of exploration have touched the full corpus. That is fine for generating hypotheses. It is not fine for confirming them unless a part of the data has stayed unseen. --- ## 6. Automated and closed-loop discovery | System | What it does | Lesson for this framework | |---|---|---| | Swanson's literature-based discovery (1986) | Linked "fish oil" and "Raynaud's syndrome" through shared intermediate terms (blood viscosity, platelet aggregation) in two literatures that never cited each other: A–B–C linking | Analogous move: link Voynich features (B terms) to properties of other corpora (C) that the Voynich literature never cites, for example monastic receipt books, Hebrew or Arabic abbreviation systems, or tally and accounting notations. A "bridge search" generator in section 8 | | Robot scientist Adam (King et al. 2004) | Generated functional-genomics hypotheses, chose experiments by expected cost and information, ran them, and updated, closing the loop | The `NextStep` ranker. Choose the next test by expected information gain per unit cost over the live contrast set | | Eve (Williams et al. 2015) | Drug screening with active learning; found repositioning candidates | Active learning: after each result, re-rank pending tests | | Sakana's AI Scientist (Lu et al. 2024) | End-to-end ideation, code, experiment, paper and automated review in ML | Its weak point, as later critiques note, is self-review. Automated review cannot replace an external check. Use LLM review only as a pre-filter | | Google's AI co-scientist (Gottweis et al. 2025) | Gemini-based agents: Generation, Reflection (initial review, full review with search, deep verification by breaking a hypothesis into assumptions, observation review, simulation review), Ranking (Elo tournament from 1200 with pairwise debates), Proximity (similarity graph for de-duplication), Evolution (combine, simplify, ground) and Meta-review (feeds recurring critiques back into prompts) | The best published template for filtering and ranking generated hypotheses. Borrow deep verification, the proximity de-duplication, the pairwise tournament and the meta-review loop. Do not borrow the claim that tournament Elo measures truth. It measures persuasiveness to a model | | FutureHouse Robin (Ghareeb et al. 2025) | Multi-agent loop of literature search, hypothesis, experiment design and data analysis. Proposed ripasudil for dry AMD | Shows the loop working when a wet lab provides real tests. Here the "lab" is the test battery, which is cheap to run, so the loop can be tighter | | Si, Yang & Hashimoto (2024) | In a study with 100+ NLP researchers, LLM ideas were rated *more novel* than expert ideas, slightly less feasible | Novelty is cheap | | Si, Hashimoto & Yang (2025), "The Ideation–Execution Gap" | After the ideas were actually executed, LLM-idea scores dropped significantly more than human-idea scores on every metric. The ideation-stage advantage disappeared and several rankings flipped | **Judge generated hypotheses after a cheap test, not on how they read.** This is the main design rule for the LLM generator | | HypoGeniC (Zhou et al. 2024) | LLM proposes hypotheses from labelled examples and updates them with a bandit-style reward as they are tested against data | Good pattern for the inductive generator: propose rules from observations, keep those that predict held-out observations | ### 6.1 Filtering and ranking generated hypotheses Every generated candidate passes through six gates in order. Each gate is logged, so the rejection record is part of the inquiry. 1. **Well-formedness.** The candidate states who, when, where, language, encoding and purpose, or marks each one "unspecified". It names at least one prediction and one kill criterion. *Automatic check.* 2. **Precondition check.** Does it rely on any string, glyph, count or fact that does not exist in the data? This is the gate the v0.1 reading would have failed. It costs one search over the transliteration. *Automatic.* 3. **Novelty and de-duplication.** Embedding similarity to existing hypotheses and to the literature claims registry. Near-duplicates are merged, keeping the more specific one (the co-scientist's proximity agent). *Automatic, with human override.* 4. **Deep verification.** Break the candidate into assumptions and check each against the `Assumption` records and established findings. For example, "Assumes word order carries information": conflicts with H-LANG-WORDLEVEL (word sequence carries less information than any attested language), so it must say why. *LLM plus rule checks.* 5. **Cheap discriminating test.** Run the fastest test in the battery that the candidate's predictions bear on. Candidates that fail a gate-5 test with a working control are recorded as `refuted-at-screen`. *Automatic.* 6. **Ranking of survivors.** Score by `pursuit_value` = (expected diagnosticity of the next test against the current leader) × (specificity of the reading) ÷ (cost). Pairwise tournament comparisons by model and human are allowed as a tie-breaker, never as the main score. --- ## 7. Unconventional and non-academic practice that solved hard problems These are the cases the user asked for. For each one: what happened, what it contributes, and where it plugs in. ### 7.1 Zodiac 340 (Oranchak, Blake & Van Eycke 2024) **What happened** (from the arXiv paper): - Decades of attempts assumed simple homophonic substitution. - The key statistic was the count of repeating bigrams by period. Z340 had 25 repeats at period 1 (shuffle average about 20) and 37 at period 19. Van Eycke and a forum user independently noticed the period-19 spike in 2015. - Blake then enumerated *billions* of candidate transposition schemes and sieved them by repeating-bigram count down to 655,088 variations. - Oranchak ran these through AZdecrypt, Van Eycke's simulated-annealing solver, with 6-gram scoring. One variation, which split the grid into four sections with the same decimation in each, produced "HOPE YOU ARE" and "TRYING TO CATCH ME". - Section 1 needed cribs before the solver converged. - Verification: - the plaintext matched a contemporary 1969 TV call-in reference; - the text length was far beyond the unicity distance; - the FBI independently confirmed the solution. - Oranchak kept an **"Encyclopedia of Observations"** that separated verified facts from speculation scattered across forums. - The paper also warns about the look-elsewhere effect. Searching more variations produces more spurious "significant" features. **Contributes:** - Exhaustive enumeration followed by a cheap sieve. - Curated observations kept separate from speculation. - External verification. - An explicit look-elsewhere correction. **Plugs in as:** - The *morphological box + gate 5* pattern in section 8. - The `Observation` entity, with `status: verified | reported | speculative`. - `Test.search_size`, so the decision rule can correct for how many variants were tried. The project's Benjamini–Hochberg rule already does this for languages; extend it to solver configurations. ### 7.2 Kryptos, Beale and Cicada 3301 - **Kryptos K4.** Journalists Jarett Kobek and Richard Byrne found K4's plaintext in 2025 in Jim Sanborn's papers at the Smithsonian, not by cryptanalysis (Scientific American). The method of encipherment is still unknown. - **Lesson:** a provenance and archive track can beat the text track outright. **Plugs in as** a standing `Question` for every artifact: "What external records could contain the answer?" (archives, correspondence, collectors' papers, conservation files). It has its own next steps. - **Beale papers.** One of the three ciphers was "solved" with the Declaration of Independence as a book key. The other two have absorbed a century of effort. - **Lesson:** a partial success makes the unsolved remainder look solvable, which strongly encourages degenerating programmes. **Plugs in as** a Lakatos `trajectory` warning when a family keeps adding auxiliary assumptions without new confirmed predictions. - **Cicada 3301.** Solvers organised in public chat channels and community wikis. A DEF CON 31 talk (2023) by community members discussed collaborative solving. The Liber Primus remains largely unsolved. - **Lesson:** public wikis preserve the trail of attempts. Without a structured status for each attempt, newcomers re-try dead ends. **Plugs in as** the "tried and failed" register on the public site (section 10). ### 7.3 Puzzle-hunt teams (MIT Mystery Hunt) Advice from experienced hunters (Irpan 2024) is very practical: - count the parts of a puzzle and look for matching totals; - copy the data faithfully into a shared sheet before theorising; - ask whether a constructor could realistically have built the puzzle the way you imagine; - use the extraction step to decide which clues matter; - when switching away, leave a clean sheet with speculative guesses removed; - share half-formed ideas; - use known constraints (such as alphabetical answer order) as early error checks. **Contributes:** - The **"constructor test"**: is the hypothesised mechanism something a 15th-century person could have built and used? This is the most useful single filter for Voynich cipher hypotheses. It is the logic behind the Naibbe proposal (a cipher executable with cards and dice). - **Clean handoff.** Nothing speculative is left mixed into the shared state. **Plugs in as:** - `Hypothesis.constructor_plausibility`, which needs a sketch of how the author would have worked: materials, time per page, error modes. Error modes give predictions, for example the line-dependence of variant choice that the project found. - The `resume` command (11(c)), which only shows verified state plus explicitly labelled open threads. ### 7.4 Bellingcat and open-source investigation Bellingcat's workflow, and the Berkeley Protocol on Digital Open Source Investigations (OHCHR & UC Berkeley 2022) that codifies the field, rest on a few principles: - preserve the original with a hash; - record the chain of custody; - verify through *independent* lines (geolocation, chronolocation, metadata, corroborating sources); - publish the method so others can repeat it; - be explicit about what is not known. **Contributes:** - Source preservation with hashes. - A rule that verification needs independent lines, which is the same as the dependency idiom in 3.3. - Publishing the method alongside the conclusion. **Plugs in as:** - `Dataset` records with source URL, retrieval date and SHA-256. For example, ZL3b dated 13 May 2025 and the Beinecke images. - A rule that an `Observation` reaches `verified` only with two independent representations or methods. ### 7.5 Cold-case and detective method Rossmo (2009) documents how investigations fail: - **tunnel vision**, where the first plausible suspect is fixed on; - **confirmation bias**; - **groupthink**; - **"information overload with no case management"**. Cold-case review units counter these by giving the case to fresh reviewers who re-read the original evidence rather than the earlier conclusions. They also re-examine physical evidence with new technology. The Tamam Shud case shows the second point. The Somerton Man was identified in 2022 as Carl "Charles" Webb through DNA from hair on the death mask and genetic genealogy, a project led by Derek Abbott with genealogist Colleen Fitzpatrick (news coverage; South Australian authorities have not formally confirmed). The code in the *Rubaiyat* is still unread. The identification came from a new instrument applied to old physical evidence. **Contributes:** - Fresh-eyes review against primary evidence. - A standing re-examination when new instruments appear. **Plugs in as:** - A scheduled **cold review** in the lifecycle: every N sessions, a worker with no memory of earlier sessions (a fresh LLM context is a useful stand-in) reads only `Dataset` and `Result` records, not `Theory` prose, and writes an independent synthesis to compare with the current one. - A `Method` watch-list of instruments that could reopen physical questions: multispectral imaging, ink XRF and isotopes, and parchment proteomics/DNA as used in other manuscript studies. ### 7.6 Superforecasting and calibration In the IARPA tournament, Mellers et al. (2014) found that training in probabilistic reasoning, teaming and tracking the best forecasters improved accuracy. Tetlock & Gardner (2015) describe the habits: Fermi decomposition, starting from base rates, small frequent updates and scoring by Brier score. **Contributes:** numeric probabilities that are later scored. **Plugs in as** a `Forecast` field on each frozen prediction. Before a run, each contributor records the probability that the prediction holds. After the run, the Brier score goes into a per-contributor calibration log. Over 25+ hypotheses this shows whether the team's "likely" means likely. Cost: one number per person per test. ### 7.7 Crowdsourced science | Project | How it worked | Contributes | Plugs in as | |---|---|---|---| | Foldit (Cooper et al. 2010) | Players folded proteins in a game. Their strategies beat algorithms on some targets and were later encoded as recipes | Humans are good at spatial and pattern search; their strategies can be harvested and automated | Visual exploration tools (glyph-position heatmaps, page-similarity views) where a contributor's pattern claim becomes an `Observation` with a reproducible query | | Galaxy Zoo (Lintott et al. 2008) | Many independent classifications per image, aggregated with weighting by user reliability | Redundant independent judgements, weighted | Image-side tasks (plant parts, nymph poses, ring counts) for the IMAGE family, each item judged several times | | Zooniverse transcription (Ancient Lives, Oxyrhynchus papyri; Williams et al. 2014 pipeline) | Volunteers marked individual Greek letters on papyrus fragments; consensus algorithms merged them | Character-level consensus transcription | A route to an independent Voynich transliteration, whose disagreements with ZL quantify transliteration uncertainty | | Polymath (Gowers & Nielsen 2009) | Blog comments, "one idea per comment", a wiki distilling progress. 27 contributors and about 800 comments in 37 days. Papers signed by "D.H.J. Polymath". The authors note the linear format stopped scaling and suggest issue trackers | Small contributions, a distilled shared memory, collective credit | The public site's "open questions" use the issue-tracker model: one issue per `Question`/`NextStep`, distilled into wiki pages generated from the registry | ### 7.8 Hackathons, diverge–converge, TRIZ and morphological analysis - **Hackathon-style parallel exploration.** Many small teams each take one hypothesis for a fixed time and report back in a fixed format. In this project the "teams" can be parallel worker sessions, each given one `NextStep` and the `resume` brief. - **Diverge–converge (the Design Council's Double Diamond).** Discover → define (problem) → develop → deliver (solution), alternating divergent and convergent phases. It maps directly onto the lifecycle in 11(f). - **TRIZ (Altshuller).** Inventive principles for resolving contradictions. The idea that transfers is to *state the contradiction*. The central Voynich contradiction is: low character entropy and rigid word structure (cipher- or generator-like) *versus* topical locality by sheet and section (language-like). Hypotheses that resolve the contradiction rather than ignore one side deserve higher pursuit value. H-PROCESS-GENERATORS already states it ("a verbose cipher's word statistics with a language's topical locality"). - **Zwicky's morphological analysis** (Zwicky 1969; Ritchey 2011, general morphological analysis with *cross-consistency assessment*). List the dimensions of a problem and the values each can take, generate all combinations, and remove pairs of values that cannot coexist. This is the core of the generator in section 8. ### 7.9 How decipherments organised and communicated their work - **Linear B.** - Alice Kober built, by hand on index cards, a catalogue of every sign and every word. From it she drew her "triplets" (words sharing a stem with different endings) and grids that grouped signs sharing a consonant or vowel without assigning sound values (Kober 1946; Fox 2013). - Ventris extended the grids. From 1951–52 he circulated numbered **Work Notes** to a small group of scholars, setting out each step, including wrong turns. When place names on Knossos tablets fitted his grid as Greek, he published with Chadwick (Ventris & Chadwick 1953). The correspondence is preserved by Cambridge's Mycenaean Epigraphy Group. - **Contributes:** structure before sound (grids that constrain any reading, built before readings are attempted); numbered, circulated work notes as the communication unit; and a verification test that was *external*, through place names known from Greek. - **Plugs in as:** session logs become numbered Work Notes generated from the registry; and a `Prediction.type = external_anchor` for predictions checked against independent knowledge (place names, plant names, star names). - **Copiale cipher** (Knight, Megyesi & Schaefer 2011). - The team first tested the hypothesis that the plain Roman letters were the plaintext (they were not; they were nulls or spaces). They clustered symbols, tried languages, found German, and then recovered a homophonic cipher with logograms for key terms. - **Contributes:** test the "obvious" reading first and rule it out formally; let a language-identification screen choose the target language. The project already has the language screen (H-LANG-SUBST). - **Antikythera mechanism.** - The Antikythera Mechanism Research Project combined microfocus X-ray CT and reflectance imaging (Freeth et al. 2006). It produced a model of gear trains constrained by tooth counts and inscriptions. In 2021 the UCL team published a full reconstruction of the cosmos display, constrained by inscriptions and the number-theory of planetary cycles (Freeth et al. 2021). - **Contributes:** new instruments on physical evidence, and models that must satisfy *every* constraint at once, not most of them. - **Plugs in as:** `Reading.constraints_satisfied` and `constraints_violated`, counted over a fixed list. - **Indus script.** - Rao et al. (2009) argued from conditional entropy that the signs behave linguistically. Farmer, Sproat & Witzel (2004) had argued that the signs are not writing at all, and Sproat disputed the entropy argument. The exchange shows a single statistic being made to carry a whole conclusion, and the argument turning on which non-linguistic controls are fair. - **Lesson:** the "meaningless / non-linguistic" hypothesis needs its own positive controls drawn from *real* non-linguistic sign systems (heraldry, Pictish symbols, kudurru symbols, notation systems), not only from shuffles. **Plugs in as** a requirement on the MEANINGLESS and STRUCT families. - **Rongorongo.** - Radiocarbon dating of one wooden tablet in Rome (Ferrara et al. 2024) gave a date before European contact in 1722, which supports independent invention. Corpus work uses Barthel's sign catalogue. - **Lesson:** materials dating can reorder the hypothesis space more than any text statistic. - **Linear A.** - The SigLA palaeographic database (Salgarella & Castellan) puts every sign occurrence online with its context. - **Lesson:** a sign-level database with images is the basic infrastructure. Voynich has transliterations but no public, per-glyph image database tied to them. ### 7.10 The Voynich community resources - **voynich.nu** (René Zandbergen, 2004–2026). - *Does well:* the most complete and careful reference on the object, its history, provenance, transliterations (the ZL transliteration used here), illustrations and earlier solvers. - *Lacks:* it does not, as far as the page structure shows, keep a structured register of theories with current standing, the tests each has passed or failed, or a tracker of open questions. Views are kept in the author's own section, which is honest but not aggregable. - **Voynich Ninja** (forum). - *Does well:* active discussion with a "Theories & Solutions" sub-forum. It works as a fast peer-review channel: threads on the Naibbe cipher and on statistical claims gather informed critique quickly. - *Lacks:* threads are linear (the Polymath scaling problem). There is no canonical status per claim, so newcomers re-propose refuted readings, and there is no link from a theory to a reproducible test. - **What the framework adds that neither has:** a public, machine-readable registry of theories with standing, linked to reproducible tests and runs, with an evidence-and-gap map and an open-questions tracker. It should *link to* voynich.nu for object facts rather than duplicate them. --- ## 8. Hypothesis generation at scale, and readings ### 8.1 Four generators Each generator is a script that writes candidate `Hypothesis` records with `origin` and `generator` filled in. All candidates then pass through the gates in 6.1. **G1. Morphological box (Zwicky/Ritchey).** The dimensions and values for the Voynich case: | Dimension | Values | |---|---| | Who | professional scribe-copyists of an author's draft · physician/apothecary · cleric in a religious house · university-trained natural philosopher · alchemist/distiller · merchant/engineer · a single author-scribe · a forger | | When (content composed) | before 1404 (copy of earlier work) · 1404–1438 (radiocarbon window of the parchment) · later than the parchment (old stock reused) | | Where | Northern Italy · Alpine/South German lands · Bohemia · Iberia · France/Savoy · Eastern Mediterranean | | Underlying language | Latin · Italian dialect · German dialect · Czech · Hebrew/Judaeo-language · Romance mixed with Latin · none | | Encoding | plain unknown alphabet · syllabary/abugida · verbose homophonic cipher · nomenclator/codebook · abbreviation/stenography system · constructed philosophical language · generated (procedural, no plaintext) · mixed (cipher + abbreviation + nulls) | | Purpose | secrecy (trade secrets, heterodox content) · prestige/curiosity object · mnemonic/teaching device · ritual/talismanic use · commercial fraud · private notebook | | Content genre | herbal of simples · regimen of health / balneology · iatromathematical (astrological medicine) compendium · alchemical/distillation text · women's medicine · encyclopaedic miscellany · none | That gives 8 × 3 × 6 × 7 × 8 × 6 × 7 ≈ 1.35 million configurations. **Cross-consistency assessment** removes incompatible pairs before any test. Examples: - "generated (no plaintext)" × any underlying language other than "none"; - "forger" × "content composed before 1404"; - "plain unknown alphabet" × established findings, since H-LANG-ANY places h2 below every sampled language. It is not marked impossible, but it is marked "requires an explanation of the low h2". Established findings act as constraints that remove whole slices. After cross-consistency, the box typically shrinks by one to two orders of magnitude. Survivors are then **grouped into contrast sets** by the dimension that the next available test discriminates. That dimension is usually *encoding*, because most of the battery measures encoding properties. Content genre is mostly tested through readings. **G2. Analogy to known artifacts.** Keep a reference table of solved or explained artifacts with their properties: the Copiale cipher, the Zodiac ciphers, papal nomenclators, the Codex Seraphinianus (stated by its author to be asemic; UNVERIFIED in this session), Hildegard of Bingen's *Lingua Ignota* (a constructed vocabulary), medieval shorthand and Tironian notes, Hebrew and Latin abbreviation systems, Linear B, Ugaritic, and the Naibbe simulation. For each, compute the same battery statistics where text is available. The generator proposes "the Voynich text is like X" whenever the Voynich profile falls within X's profile on a declared subset of statistics. It also states, as a prediction, the statistics where they differ. This is analogy made measurable, and it reuses the project's 100-language infrastructure. **G3. Differential list from each surprise.** For each `Observation` marked `surprising`, list every mechanism that would produce it, clinically. For example, for the finding that line position governs variant choice (H-CIPHER-MIXED), the differential is: - a scribal layout convention (justification by variant spellings); - a cipher with position-dependent tables; - a generator that resets at line start; - line-initial and line-final abbreviation marks; - copying from an exemplar with a different line length. Each becomes a candidate with a discriminating prediction. **G4. LLM-assisted generation with filters.** Prompt one model, with the registry, the established findings and the assumption list, to propose hypotheses in the full schema, including a reading and kill criteria. Prompt a second model to attack each one, running deep verification as in the co-scientist. Then apply gates 1–5. The ideation–execution gap (Si et al. 2025) is the reason gate 5 exists: candidates are ranked only after a cheap test. The prompts and model versions are logged in `Run` records, so generation is reproducible and audit-able. A fifth, lighter source is the **literature registry**. Every published or forum-prominent solution claim becomes a `Hypothesis` with `origin: literature`. Its reading is taken from the claimant's own words, and it passes through gate 2 first. Most fail there, which is itself a useful public record. ### 8.2 Readings: the concrete layer A **Reading** makes a hypothesis concrete enough to say what is on the page. Schema: ```yaml id: R-0007 hypothesis: H-0102 # "monastic or cloister infirmary compendium" status: candidate # candidate | screened | tested | refuted | standing sections: # one entry per codicological section herbal_A: "simples, one plant per page: name, qualities (hot/cold, degree), virtues, preparation" astronomical: "iatromathematical calendar: which sign governs which body part; bloodletting days" zodiac: "lunar days per sign for bleeding/bathing; each nymph label a day number or saint" balneological: "regimen of baths: which waters, which ailments, seasons" rosettes: "map or schema: the cosmos as the physician needs it; or the house's water system" pharmaceutical: "materia medica of roots/parts kept in jars; labels name the drug" recipes: "antidotary: one receipt per starred paragraph: Recipe, ingredients, quantities, use" sample_gloss: # what a passage would say, as a template, not a translation f103r_para1: "Recipe [herb] et [herb] ana [quantity]; coque in [liquid]; da [patient] contra [illness]" properties: # measurable consequences of this reading - id: P-0311 text: "recipe paragraphs share a small set of repeated incipits and connectives" statistic: repeated_3_4gram_rate_recipes expect: ">= Latin receipt-book baseline" - id: P-0312 text: "herbal paragraph-first words are mostly unique to their page (plant names)" statistic: first_word_page_uniqueness expect: "> paragraph-internal words, > shuffled baseline" - id: P-0313 text: "zodiac labels behave as an ordered sequence (day numbers), not free words" statistic: label_sequence_order_gain expect: "> 0 vs shuffled labels within ring" - id: P-0314 text: "number-like tokens (quantities) cluster in the recipe section" statistic: section_skew_of_short_repeated_tokens expect: "recipes > herbal" external_anchors: # checked against independent knowledge - "illustrated plants in herbal_A include common infirmary simples (sage, rue, plantain...)" - "a homo signorum (zodiac man) figure would be expected; its absence counts against" constraints_checked: [C-LOWH2, C-NO4GRAMS, C-SHEETCOHESION, C-WORDORDER] kill_criteria: - "P-0311 fails with the cipher-adjusted baseline (see note) AND P-0312 fails" notes: "If the text is a verbose cipher, repeated incipits are hidden by homophone choice (H-CIPHER-HOMOPHONE). The reading must state which encoding it assumes; each encoding gives a different expected value for P-0311." ``` Three points about this design: 1. **A reading is conditional on an encoding.** The infirmary-compendium reading under "plain language" predicts repeated phrases. The project measured zero repeated 4-grams in 34k words against Pliny's 16 and Bible texts' 170–1,700, so that combination is already refuted. Under "verbose homophonic cipher", repeated phrases are expected to be broken, so P-0311 is not diagnostic and P-0312–P-0314 carry the weight. The framework should therefore store **reading × encoding** pairs and score each one separately. This is the same split that ACH needs. 2. **Sample glosses are templates, not translations.** They show what kind of sentence would sit there. That is enough to derive structural predictions (incipits, quantities, names in first position) without claiming to read anything. 3. **External anchors** carry the Linear B lesson. A reading that matches plant identifications or star groups that independent experts would also accept gains more weight than one that matches only text statistics. ### 8.3 Scoring readings Readings are scored on the same evidence links as hypotheses, plus three reading-specific counts: | Score | Definition | |---|---| | Constraint satisfaction | Fraction of the fixed constraint list (established findings) the reading is consistent with, under its stated encoding | | Prediction record | Confirmed / refuted / pending predictions. Only frozen predictions count | | Anchor agreement | Fraction of external anchors confirmed by an independent source (botanical identification literature, astronomical tables) | | Specificity | Number of section-level claims that could fail. Vague readings score low and are not promoted however well they "fit" | A reading is **refuted** if any kill criterion is met. It is **standing** if it has at least one confirmed frozen prediction that its rivals in the same contrast set did not predict. ### 8.4 Re-registering the withdrawn v0.1 reading The v0.1 draft is recorded as a hypothesis with its own claims, so its refutation is preserved and visible: ```yaml id: H-V01-ASTROMED origin: earlier_draft generator: "v0.1 draft (LLM description of low-resolution folio images, March 2026)" statement: "The labels on ff. 70-73 and 87-94 are Latin-derived astrological terms (zodiac names, planet roots) and the pharmaceutical section follows a Picatrix-style seven-planet classification." readings: [R-V01-ZODIAC, R-V01-PLANETS, R-V01-PICATRIX] predictions: - P-V01-1: "strings qotain, otchor, shcamed, qodar, nottbas, occtird, obecand, olpidag, spato occur on the zodiac pages of the matching signs" - P-V01-2: "planet-root strings (Merc-, Satur-, Iupit-, Ven-, Sol-, Lun-) occur on ff. 87-94" - P-V01-3: "the seven-planet assignment of jars matches Picatrix correspondences" tests: [T-PRECOND-STRINGS] # gate-2 precondition: search ZL3b, IT2a, CD2a result: "6 of 9 zodiac strings occur nowhere; 3 occur but never on the claimed page; no planet-root strings exist; P-V01-3 depends on P-V01-2" status: refuted-at-precondition lessons: ["readings must be checked against a transliteration before any interpretation", "image-only LLM glyph descriptions are not a Representation of the text"] successors: [H-IMAGE-ZODIAC-LABELS] # the weaker, testable form already registered ``` The "successor" link records that the idea behind it survives in a testable form: zodiac labels encoding month or sign names in *some* encoding. That form is H-IMAGE-ZODIAC-LABELS, currently "not supported". Any other earlier-draft claim (the rosette as al-Idrisi map, bathers as decans, the `follan` lexeme) goes in the same way, with one `Reading` per claim. The ASSESSMENT.md tables already contain everything needed. --- ## 9. Provenance and reproducibility | Tool / standard | What it gives | Recommendation | |---|---|---| | W3C PROV-O (2013) | Entity, Activity and Agent, linked by `used`, `wasGeneratedBy`, `wasDerivedFrom`, `wasAttributedTo` | Use the *vocabulary* in the schema. `Run` is an Activity, `Result` and output files are Entities, and workers and models are Agents. No triple store; export to PROV-JSON on demand | | RO-Crate (Soiland-Reyes et al. 2022) | A JSON-LD manifest (`ro-crate-metadata.json`) that packages data, code and provenance | Generate one RO-Crate per release of the book or site, so the release is a citable, self-describing package | | DVC | Git-tracked pointers to large data files, with pipelines | Only needed if images or derived arrays grow beyond what git handles comfortably. For now, store the SHA-256 of each `Dataset` and keep large files outside git with a fetch script | | MLflow | Experiment tracking: parameters, metrics, artefacts per run | Over-sized for this. The JSONL run log in 11(a) gives the same audit trail with no server | | make / Snakemake (Mölder et al. 2021) / Nextflow | Declarative pipelines that rebuild only what changed | **make** at the top level (the project already uses shell scripts). Snakemake is worth it only if the dependency graph from tests to outputs to figures gets deep. Nextflow is aimed at clusters, so not needed | | Jupyter Book / Quarto | Books and sites from Markdown or notebooks | Keep the LaTeX book. Use **Quarto** for the website, fed by Markdown generated from the registry. One tool, static output, cross-references and citations included | | ORKG (Jaradeh et al. 2019) | Research contributions described as structured comparisons (problem, method, result) | Borrow the idea of a **comparison table across claims**: the literature registry rendered as a table of method, data, claim and status | | Nanopublications (Groth, Gibson & Velterop 2010) | The smallest publishable assertion, with its provenance and publication info | Each `Finding` is effectively a nanopublication: assertion + provenance (runs) + publication info (date, version). It can be exported as one later if needed | | Micropublications (Clark, Ciccarese & Goble 2014) and SWAN (Ciccarese et al. 2008) | Claims with supporting and challenging evidence and arguments; hypotheses linked to claims in discourse | The model for `EvidenceLink` (supports / challenges) and for linking literature claims to the project's own findings | | Toulmin (1958) | Claim, data, warrant, backing, qualifier, rebuttal | Every `Theory` record has these six fields. *Qualifier* is the IPCC/GRADE confidence term. *Rebuttal* is the list of conditions under which the claim would fail, the same as `would_change_if` | **Minimum provenance rule.** Every `Result` row must carry: - `run_id`; - script path and git commit; - input `Dataset` IDs with their SHA-256; - `Representation` ID (which transliteration and segmentation); - parameters and seed; - wall-clock date. The CLI adds these when it runs a test. A row without them is shown as "unprovenanced" and cannot contribute to a tier above Working hypothesis. --- ## 10. Visualising the inquiry, and a public home ### 10.1 Visual forms | View | Shows | Built from | Where | |---|---|---|---| | **Question map** | Questions as a tree. Under each one: the contrast set of hypotheses, coloured by standing, with counts of tests run and pending | `Question`, `Hypothesis` | Book front matter; site home | | **Hypothesis–evidence matrix (ACH)** | Hypotheses as columns, evidence clusters as rows, cells ++/+/0/−/−− with diagnostic rows emphasised and non-diagnostic rows greyed | `EvidenceLink` | One per Question; book chapter on synthesis; site | | **Evidence and gap map** | Hypothesis families × evidence types (text statistics, codicology, imagery, materials, provenance, external anchors). Bubble size is the number of independent clusters, colour is the best certainty | `EvidenceLink`, `Test` | Site "what we know" page; appendix | | **Argument map** | One Theory: claim at the root, evidence for and against as branches, assumptions as leaves | `Theory`, `EvidenceLink`, `Assumption` | Theory pages | | **Provenance graph** | Dataset → Representation → Run → Result → Finding | PROV fields | Expandable from any finding on the site | | **Inquiry flow** (Sankey-style) | Questions → hypotheses generated (by generator) → screened out (by gate) → tested → refuted / standing | Generator and gate logs | Shows that the inquiry is systematic. A good opening figure for the book's method chapter | | **Programme timeline** | Per family: confirmed novel predictions (up ticks) against ad-hoc patches (down ticks) over sessions | `Theory.trajectory` | Lakatos view; site "journey" page | | **Specification curve** | One statistic across all processing choices | Multiverse runs | Per key finding | **Confidence language.** Use the IPCC scales (Mastrandrea et al. 2010) without change. The likelihood terms are: | Term | Probability | |---|---| | virtually certain | 99–100 % | | very likely | 90–100 % | | likely | 66–100 % | | about as likely as not | 33–66 % | | unlikely | 0–33 % | | very unlikely | 0–10 % | | exceptionally unlikely | 0–1 % | Confidence is very low / low / medium / high / very high, set from evidence (limited, medium, robust) × agreement (low, medium, high). The IPCC guidance also asks for a **traceable account** for each key finding (how evidence was evaluated and combined). In this framework that account is generated: it is the finding's evidence links and runs. The house style for visuals already exists in the project's figure kit and should be reused. These views are mostly tables and node-link diagrams, so they render cleanly in both LaTeX (TikZ or pre-rendered SVG→PDF) and HTML. ### 10.2 Models to borrow from | Model | What to borrow | |---|---| | Our World in Data | Every chart has a "sources and processing" tab and downloadable data. Short text sits under each chart saying what it shows | | Distill (Olah & Carter 2017, "Research Debt") | The argument that explanation and distillation are research work. Interactive figures where the reader changes an assumption and sees the result change, e.g. toggling the transliteration in a specification curve | | Wikipedia featured articles | Neutral point of view, every claim cited, a "theories" section that states each theory's standing in the literature. A consistent section structure from article to article | | Polymath wiki | Distilled progress pages separate from discussion; clear "current state" summaries | | Erdős Problems (erdosproblems.com) | One page per problem with a stable number, a status (open / solved / partially solved), references, and a changelog of status changes. This is the right model for `Question` pages | | Zooniverse project pages | "About", "Research", "Results", "Talk" and the task itself, with a clear statement of what volunteer work produces and how it is used | ### 10.3 What the public site should contain The site should be one per inquiry, generated from the registry with Quarto: 1. **The artifact.** A page viewer, using IIIF images if the holding library provides them (Yale's digital collections should be checked for MS 408). Overlays: transliteration by line, scribe, Currier language, section, and any per-glyph data. Clicking a word shows its occurrences and the findings that cite it. This is the infrastructure SigLA provides for Linear A. 2. **What we know.** Findings by tier, each with the IPCC/GRADE term, a traceable account and a provenance link. The evidence-and-gap map sits on this page. 3. **Competing theories.** One page per Theory/family: Toulmin fields, argument map, ACH row, readings, current standing, trajectory, and "what would change this". 4. **Ruled out.** Refuted hypotheses, including literature claims and the project's own v0.1 reading, each with the test that refuted it and a link to re-run it. This is the most valuable page for the community, because it stops dead ends being re-proposed. 5. **Open questions.** Erdős-style numbered questions with status, the next steps queued, their expected diagnosticity, and how to take one on. 6. **The journey.** Numbered Work Notes (the Ventris model) generated from session logs, plus the inquiry-flow and timeline figures. 7. **The framework.** The protocol, the lifecycle, the decision rules, the confidence scales, and how to propose a hypothesis in the schema. 8. **Contribute.** Issue templates: "propose a hypothesis" (with schema and gates), "report an observation" (with a reproducible query), "challenge a finding". Image-classification and transliteration tasks if crowdsourcing is added later. 9. **Data and code.** Downloads, the RO-Crate for each release, and how to reproduce any figure with one command. --- ## 11. Proposed design for this repo ### (a) Data model The core does not depend on any one artifact. Each artifact gets an adapter that provides its datasets, representations, loaders, section scheme and battery. ``` inquiry/ # core package (artifact-agnostic) schema/ # JSON Schema per entity; validated on every write cli.py # `inq` command store.py # read/write YAML + JSONL, ID allocation, validation generate/ # G1 morph box, G2 analogy, G3 differential, G4 llm; gates.py synth/ # ach.py, score.py, confidence.py render/ # tex.py (book appendices), site.py (Quarto md), brief.py (resume) cases/ voynich/ adapter.py # datasets, representations, section map, loaders, battery hooks registry/ # human-authored records, one YAML file per record questions/ Q-*.yaml hypotheses/ H-*.yaml readings/ R-*.yaml theories/ TH-*.yaml assumptions/ A-*.yaml methods/ M-*.yaml datasets/ D-*.yaml representations/ REP-*.yaml log/ # machine-written, append-only runs.jsonl results.jsonl observations.jsonl evidence_links.jsonl generator_log.jsonl # every generated candidate and the gate it stopped at forecasts.jsonl prereg/ # frozen plans: ..yaml + .sha256 tests/ # existing t_*.py move here unchanged; each declares its Test id out/ # existing outputs rohonc/ ... # later cases: same layout ``` **Entities.** Human-authored records are stored one YAML file per record, which gives clean diffs and few merge conflicts. Machine-written records are stored as JSONL lines. | Entity | ID | Key fields | |---|---|---| | Artifact | `ART-voynich` | holding, shelfmark, dates, physical description, sections scheme | | Dataset | `D-*` | source URL, retrieved, sha256, licence, kind (transliteration, image set, reference corpus) | | Representation | `REP-*` | dataset, segmentation (EVA chars / merged units / Currier), space handling, label inclusion. These are the multiverse axes | | Method | `M-*` | name, description, implementation path, validated_on (control IDs), known failure modes. The methods inventory appendix is generated from these | | Question | `Q-*` | text, parent, contrast_set (hypothesis IDs, mutually exclusive), status (open / answered / dissolved), priority | | Observation | `OBS-*` | statement, run_ids, status (verified / reported / speculative), surprising (bool), against_expectation_of (theory IDs) | | Hypothesis | `H-*` | statement, family, question, origin (inductive / deductive / abductive / literature / earlier_draft / generated), generator, parent_observations, morph_coordinates (who/when/where/language/encoding/purpose/genre), constructor_sketch, assumptions, readings, predictions, status, pursuit_value | | Reading | `R-*` | hypothesis, encoding, sections, sample_gloss, properties, external_anchors, constraints_checked, kill_criteria, status | | Prediction | `P-*` (inline in H or R) | text, statistic, expect, proxy_for, type (internal / external_anchor), forecast (per contributor) | | Test | `T-*` | predictions tested, method, representations (multiverse axes), controls (positive and null), decision_rule, search_size, prereg (lock ID), script | | Run | `RUN-*` (jsonl) | test, mode (explore / confirm / sealed), git commit, script sha, inputs (dataset sha256), representation, params, seed, started, ended, outputs, agent (human / model + version) | | Result | `RES-*` (jsonl) | run, statistic, value, baseline, interval, decision (pass / fail / inconclusive, never "see note"), control_passed, deviations | | EvidenceLink | `EL-*` (jsonl) | from (RES or OBS), to (H or R), direction (++ / + / 0 / − / −−), diagnosticity (log10 LR, capped), credibility (high / med / low), cluster (dependency group), rater, rationale | | Theory | `TH-*` | question, hypotheses (a family or a compound), claim, data, warrant, backing, qualifier (computed), rebuttal / would_change_if, critical_evidence, trajectory, tier (computed) | | NextStep | `NS-*` | kind (test / generate / review / archive / instrument), target IDs, expected diagnosticity, cost, owner, status | | Assumption | `A-*` | text, rating (solid / caveated / unsupported), test (optional) | **ID rules.** IDs are never reused or renamed. Existing `H-LANG-ANY`-style IDs are kept as they are. New records get `H-0101`-style numbers allocated by `store.py`. Each record carries `created`, `updated` and `supersedes` / `superseded_by`. ### (b) Migration `inq migrate --from framework/hypotheses.yaml --results framework/results.csv` performs these steps once, writing a migration report: 1. **Hypotheses.** - Each of the 25 entries becomes `hypotheses/.yaml` with the same ID. - `family` is kept as is. - `origin` is set to `deductive` for registry entries written before testing, and `literature` for entries named after published work (H-MEANINGLESS-AUTOCOPY → Timm & Schinner; H-CIPHER-WORDLETTER → Naibbe). - `predictions` (free text) become `P--n` records with `statistic: unparsed`. A checklist asks a human to fill in `statistic` and `expect`. - `tests` are matched to `T-*` records by test ID, and test IDs are matched to `t_*.py` scripts by grep. - `result` prose is kept verbatim as `legacy_result`. - `status` is mapped to a controlled vocabulary (`supported`, `not supported`, `inconclusive`, `untested`, `unfalsifiable`, `refuted-at-precondition`). The prose qualifier is kept as `status_note`. Mixed statuses, such as H-CIPHER-HOMOPHONE's "two structural predictions supported; the solver finds no key", are split per prediction. - `files` become Run outputs. 2. **Questions.** Seeded by hand from the families. A first cut: - `Q-1` What produced the text? (contrast set: LANG, CIPHER, CONSTRUCTED, MEANINGLESS, PROCESS); - `Q-2` What is the structure of a line, paragraph and page? (STRUCT); - `Q-3` In what order and by whom was it made? (ORDER, HANDS); - `Q-4` What do the pictures depict and how do they relate to the text? (IMAGE); - `Q-5` What is the content genre?; - `Q-6` What external records could bear on it? (new, the K4/Tamam Shud lesson). 3. **Results.** - Each `results.csv` row becomes a `RES-*` line with `run: legacy--`. A `Run` stub records the date and the git hash if `out/` files can be matched. - Rows with `decision: see note` are listed for a human to resolve into pass / fail / inconclusive. - Rows that cannot be tied to a script are marked `unprovenanced`. 4. **Evidence links.** - For each result, a link to its own hypothesis is created from the decision. - The migrator then proposes **cross-links**: the same result applied to every other hypothesis in the contrast set, with direction left `unrated`. - A rating session, where two raters work independently and then compare, fills them in. This is where the deductive results start to count for the hypotheses that did not commission them. 5. **Pre-registrations.** The existing Part VII freeze file and its SHA-256 become `prereg/H-CIPHER-MIXED..yaml` with the hash preserved. Earlier hypotheses get `prereg: none (retrospective)`, which caps them at "Supported" until re-run under a frozen plan on the sealed holdout. 6. **Earlier draft.** ASSESSMENT.md's tables become `H-V01-*` records and readings, as in 8.4. 7. **Methods.** The methods inventory table in appendix F is parsed into `M-*` records. After that the appendix is generated from them. 8. **Compatibility.** `framework/hypotheses.yaml` is regenerated from the new store by `inq export legacy` until the old scripts that read it are updated. Nothing breaks during the transition. ### (c) Workflow A single CLI, `inq`, wrapped by make targets. Every command validates against the schema and refuses to write invalid records. | Command | Does | |---|---| | `inq resume` | Prints a one-page brief for a new worker: what the artifact is; the question map with standings; findings by tier; what changed in the last session; frozen pre-registrations with tests not yet run; open `NextStep`s ranked; unresolved migration items; stale results (script changed since last run); hypotheses with no reading; readings with no frozen prediction; the latest cold-review divergences. Ends with "Start here: NS-xxx". This is what every session reads first | | `inq new question\|hypothesis\|reading\|test` | Scaffolds a record from a template with required fields. `new hypothesis` refuses to save without at least one reading and one kill criterion (it may be marked `draft`) | | `inq generate --from OBS-… --gen morph\|analogy\|diff\|llm` | Runs a generator and writes candidates to `generator_log.jsonl`. Every candidate passes through the gates, and survivors become `H-*` drafts | | `inq screen H-…` | Runs gates 2–5 on a candidate: precondition string checks, dedupe, deep verification, cheapest discriminating test | | `inq freeze H-…` | Writes `prereg/..yaml` containing the hypothesis, readings, predictions, tests, decision rules, representations, controls, the sealed-holdout declaration, contributor forecasts, and the script SHA of each test. Stores its SHA-256 and makes a git commit and tag `prereg//`. Further edits to those fields go into `deviations` | | `inq run T-… [--mode explore\|confirm\|sealed]` | Runs the test script with provenance capture, writes `Run` and `Result` lines, and applies the decision rule. Confirm mode requires a matching freeze. Sealed mode is allowed once per frozen test | | `inq observe` | Records an `Observation` from an exploratory run, asking whether it was surprising and against which theory's expectation. A surprising observation automatically queues a `generate --gen diff` NextStep | | `inq link` | Proposes evidence links for new results across the contrast set and opens a rating sheet | | `inq synth [Q-…]` | Recomputes ACH, scores, confidence and tiers (d). Writes `build/synthesis.json` and a diff against the previous synthesis ("H-… moved from likely to about as likely as not because RES-…") | | `inq next` | Ranks candidate tests by expected diagnosticity over the live contrast set ÷ cost, plus generate, review and archive tasks. Writes `NextStep` records | | `inq review --cold` | Builds the context for a cold reviewer (data and results only, no theory prose) and records the reviewer's independent synthesis for comparison | | `inq render book\|site\|appendix\|notes` | Generates LaTeX appendices (registry, methods inventory, ACH matrices, gap map, ruled-out list), Quarto Markdown for the site, and numbered Work Notes from session logs | | `inq check` | Lint: every hypothesis has a question, reading and prediction; every prediction has a test or `untestable`; every confirm-mode result has a freeze; no "see note" decisions; no orphan results; every result has provenance; every surprising observation has a generation task. CI runs this on every commit | Make targets (the existing `run_*.sh` scripts become make rules): ```make resume: ; inq resume check: ; inq check synth: ; inq synth book: synth ; inq render book && latexmk -pdf -cd ../book/main.tex site: synth ; inq render site && quarto render site/ release: check book site ; inq export rocrate --tag $(TAG) ``` ### (d) Theory synthesis Per question: 1. **Contrast set.** The live hypotheses (or reading × encoding pairs) under the question, plus a catch-all "other / none of the above" whose likelihood under each evidence cluster is set at the median of the others. This keeps an unconsidered explanation from being squeezed to zero. 2. **ACH matrix.** Rows are evidence clusters and cells are directions. The ordinal score per hypothesis is the weighted inconsistency count, Σ over clusters of (−− = 2, − = 1) × credibility weight. As Heuer recommends, the lowest inconsistency ranks first. Non-diagnostic rows are reported and excluded. 3. **Bayesian score.** Each cluster contributes log10 LR(h vs the contrast-set mean), taken from the link's diagnosticity: | Strength | Cap | |---|---| | Weak | ±0.3 (2:1) | | Moderate | ±1 (10:1) | | Strong | ±2 (100:1), only when backed by a positive control that the test passed | Clusters are summed (independent by construction). Priors come from base rates (section 4.1) or are uniform; both are reported. Posterior shares within the contrast set are computed. 4. **Sensitivity.** Recompute with each cluster dropped (Heuer step 6), and with each rater's ratings separately (many-analysts). Report the range. A hypothesis whose rank changes when one cluster is dropped is flagged "rests on RES-…". 5. **Confidence and tier.** - *Evidence* is set from the number of independent clusters and their GRADE level: limited (1 cluster, or only low-certainty ones), medium (2–3 clusters, at least one moderate-certainty), robust (≥4 clusters, at least two high-certainty, sealed-holdout replication). - *Agreement* is set from consistency across raters, representations and clusters. - These map to IPCC confidence. The likelihood term comes from the posterior share (sensitivity range in brackets). - Tiers: | Tier | Rule | |---|---| | Established | high or very high confidence; at least one confirm-mode result on the sealed holdout; robust across the declared multiverse | | Supported | medium confidence, or high confidence without sealed replication | | Working hypothesis | low confidence, or only exploratory support | | Ruled out | "very unlikely" or below with at least medium confidence, or refuted at precondition | | Open | everything else | 6. **Output.** One generated sentence per theory in a fixed form, for example: "*It is very likely (90–100 %; range 85–97 % across raters) that the text is not a plain alphabetic rendering of a natural language (high confidence; robust evidence, high agreement; 5 clusters; sealed replication: yes).*" These sentences go verbatim into FINDINGS.md, the book and the site, so all three always agree. The scoring is deliberately simple: capped LRs, explicit clusters, no learned weights. With roughly a hundred results it needs to be auditable, not clever. The matrices and ratings are data; only the arithmetic is automatic. ### (e) Narrative and visual outputs **Book.** The narrative problem is that the chapters follow the order in which tests were run. The registry makes another order possible: **by question**. Each part opens with the question map for that question and its ACH matrix. Chapters then follow each hypothesis from origin to reading, prediction, test, result and the link back to the matrix. Generated pieces: - each part's opening matrix and question map; - appendix E (registry, now with readings and evidence links); - appendix F (methods inventory from `M-*`); - a new appendix listing ruled-out hypotheses with the refuting result; - the evidence-and-gap map; - the inquiry-flow figure; - the confidence statements. The prose chapters cite records by ID (`\hyp{H-ORDER-SHEET}`, `\res{RES-0412}`), and a LaTeX macro renders the current status beside the ID, so the text cannot drift from the registry. **Website.** The section-10.3 structure, generated as Quarto Markdown from the same `build/synthesis.json`. Work Notes are numbered and dated, one per session, generated from the session log plus the synthesis diff. **Visual style.** Reuse the project's figure kit so book and site figures match. The ACH matrix and the gap map are the two signature figures. They should be designed once, carefully, and reused for every question and every later artifact. ### (f) Lifecycle Every mystery, and every question within it, runs through the same cycle. Stages can overlap, but a stage's outputs only count once its exit criteria are met. | Stage | Mode | Activities | Exit criteria | |---|---|---|---| | 0. **Intake** | — | Register the Artifact, Datasets (with hashes), Representations and the multiverse axes. Build the sign- or word-level database and viewer. Collect literature claims into the registry. Run gate 2 on all of them | Every dataset hashed. At least one representation validated by a second, independent one (H-TRANSLIT-ROBUST style). Literature claims registered | | 1. **Induce** (diverge) | explore | Exploratory statistics on the training sheets only. Record Observations with verified / speculative status and a surprise flag. Fresh-eyes "constructor" and puzzle-hunt passes | Saturation: the last *k* = 5 exploratory runs produced no new Observation class. Every surprising Observation has a G3 differential task queued | | 2. **Abduce** (diverge → converge) | generate | Run G1–G4 and the literature registry. Apply gates 1–5. Write readings. Hold the key assumptions check, the premortem and devil's advocacy on the leading candidates | Each open Question has ≥3 live hypotheses including a catch-all. Each live hypothesis has ≥1 reading with kill criteria and a constructor sketch. Generator logs complete | | 3. **Deduce** (converge) | confirm | Freeze, forecast, run with controls; sealed-holdout run for anything heading to Established | Every frozen prediction has a result with provenance. Control pass/fail recorded. Deviations documented. Forecasts scored | | 4. **Synthesise** | — | Link evidence across the contrast set, independent ratings, ACH plus Bayesian scoring, sensitivity, confidence and tiers. Cold review every *n* = 3 cycles. Team A / Team B on the top two | Synthesis diff written. Every tier change justified by named results. Cold-review divergences resolved or recorded as open questions. `inq next` has produced ranked NextSteps | | 5. **Publish** | — | Render book sections, site, Work Notes and the RO-Crate release | `inq check` clean. The site's "ruled out" and "open questions" pages are current. Release tagged | **Exit from the whole cycle for a question.** - *Answered:* one hypothesis is Established and its rivals are Ruled out. - *Dissolved:* the question rested on an assumption that was refuted, for example a question about binding order once binding order was shown to be secondary. - *Parked:* expected diagnosticity of every available test is below threshold. Record what instrument or data would reopen it. This is where the archive and new-instrument tracks (K4, Tamam Shud) live. **Why this order matters.** It joins the two threads the project currently runs separately: - induction feeds abduction through recorded surprise; - abduction feeds deduction through readings with kill criteria; - deduction feeds synthesis through evidence links that apply to *all* rival hypotheses, not just the one that commissioned the test; - synthesis feeds the next induction through the gap map and NextSteps. --- ## 12. Notes for the later cases The adapter for each artifact supplies datasets, representations, a section scheme and battery hooks. What differs per case: | Artifact | Representation | Distinctive feature for the framework | |---|---|---| | Rohonc Codex | Sign transcription of ~450 pages; sign inventory disputed (Láng 2010, Cryptologia) | Illustrations with Christian, Pagan and Islamic iconography give external anchors. The forgery hypothesis (19th-century) is a "can't miss" | | Codex Seraphinianus | Images plus pseudo-script | **Calibration case.** The author has said the script is asemic (UNVERIFIED here). Run the full framework on it blind: does it rank "meaningless" first? It is the best available positive control for the MEANINGLESS family | | Linear A | SigLA database; Linear B sound values as a partial key | Hypothesis space dominated by *language* (Luwian, Semitic, Hurrian, isolate) with a known script type. Readings are word-level (e.g. libation formula) and anchors are commodity ideograms | | Rongorongo | Barthel codes; ~25 objects | Small corpus, so the imprecision downgrade dominates. Dating (Ferrara et al. 2024) is a materials result that reorders hypotheses | | Phaistos Disc | 45 sign types, 241 tokens, one object | Very small corpus: most statistical tests are underpowered, and the framework should say so through the GRADE imprecision rule rather than run them. Forgery is a live "can't miss" | | Antikythera mechanism | CT volumes, surface imaging, inscriptions | Engineering hypothesis space. Readings are gear-train models scored by constraint satisfaction (Freeth et al. 2021) | | Shugborough inscription | 10 letters (O U O S V A V V, flanked by D and M) | Far below any unicity distance. The framework's main contribution is to rule out readings as underdetermined and to point to archives (Anson family papers) | | Tamam Shud | Five lines of letters; physical evidence; archives | Mixed case: the identity was resolved by DNA and genealogy, the code was not. Shows the non-textual track and the "parked" exit | --- ## 13. References **Fetched and read in this session (VERIFIED-FETCHED)** - Oranchak, D., Blake, S., Van Eycke, J. (2024). The Solution of the Zodiac Killer's 340-Character Cipher. arXiv:2403.17350. https://arxiv.org/abs/2403.17350 — VERIFIED-FETCHED - Timmermans, S., Tavory, I. (2012). Theory construction in qualitative research: from grounded theory to abductive analysis. *Sociological Theory* 30(3):167–186. doi:10.1177/0735275112457914 — VERIFIED-FETCHED - Gottweis, J., Weng, W.-H., Daryin, A., et al. (2025). Towards an AI co-scientist (arXiv v2 title: Accelerating scientific discovery with Co-Scientist). arXiv:2502.18864. https://arxiv.org/abs/2502.18864 — VERIFIED-FETCHED - Mastrandrea, M.D., Field, C.B., Stocker, T.F., et al. (2010). Guidance Note for Lead Authors of the IPCC Fifth Assessment Report on Consistent Treatment of Uncertainties. IPCC. https://www.ipcc.ch/site/assets/uploads/2017/08/AR5_Uncertainty_Guidance_Note.pdf — VERIFIED-FETCHED - Si, C., Hashimoto, T., Yang, D. (2025). The Ideation–Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas. arXiv:2506.20803. https://arxiv.org/abs/2506.20803 — VERIFIED-FETCHED - Gowers, T., Nielsen, M. (2009). Massively collaborative mathematics. *Nature* 461:879–881. doi:10.1038/461879a. https://michaelnielsen.org/papers/mcm.pdf — VERIFIED-FETCHED - Basilio, H. (2025). A Solution to the CIA's Kryptos Code Is Found after 35 Years. *Scientific American*. https://www.scientificamerican.com/article/a-solution-to-the-cias-kryptos-code-is-found-after-35-years — VERIFIED-FETCHED (year inferred from the article) - Irpan, A. (2024). Puzzlehunting 201. Blog. https://www.alexirpan.com/2024/04/30/puzzlehunting-201.html — VERIFIED-FETCHED - Zandbergen, R. (2004–2026). The Voynich MS (voynich.nu). https://www.voynich.nu/ — VERIFIED-FETCHED (home page structure only) **Records confirmed by search (VERIFIED-URL)** *Philosophy and statistics* - Douven, I. Abduction. *Stanford Encyclopedia of Philosophy*. https://plato.stanford.edu/entries/abduction/ — VERIFIED-URL - Platt, J.R. (1964). Strong inference. *Science* 146(3642):347–353. doi:10.1126/science.146.3642.347. https://www.whoi.edu/cms/files/platt64sci_72743.pdf — VERIFIED-URL - Chamberlin, T.C. (1890; reprinted 1965). The method of multiple working hypotheses. *Science* 148:754–759. https://www.whoi.edu/cms/files/chamberlin65sci_72744.pdf — VERIFIED-URL - Blei, D.M. (2014). Build, compute, critique, repeat: data analysis with latent variable models. *Annual Review of Statistics and Its Application* 1:203–232. https://www.cs.columbia.edu/~blei/papers/Blei2014b.pdf — VERIFIED-URL - Kass, R.E., Raftery, A.E. (1995). Bayes factors. *Journal of the American Statistical Association* 90(430):773–795. https://sites.stat.washington.edu/courses/stat527/s13/readings/KassRaftery_jasa1995.pdf — VERIFIED-URL - Lakatos, I. (1970). Falsification and the methodology of scientific research programmes. In Lakatos & Musgrave (eds), *Criticism and the Growth of Knowledge*, Cambridge UP. doi:10.1017/CBO9781139171434.009 — VERIFIED-URL - Tukey, J.W. (1980). We need both exploratory and confirmatory. *The American Statistician* 34(1):23–25. https://www.causeweb.org/cause/node/5603 — VERIFIED-URL - Nosek, B.A., Ebersole, C.R., DeHaven, A.C., Mellor, D.T. (2018). The preregistration revolution. *PNAS* 115(11):2600–2606. doi:10.1073/pnas.1708274114 — VERIFIED-URL - Steegen, S., Tuerlinckx, F., Gelman, A., Vanpaemel, W. (2016). Increasing transparency through a multiverse analysis. *Perspectives on Psychological Science* 11(5):702–712. doi:10.1177/1745691616658637 — VERIFIED-URL - Simonsohn, U., Simmons, J.P., Nelson, L.D. (2020). Specification curve analysis. *Nature Human Behaviour* 4:1208–1214. doi:10.1038/s41562-020-0912-z — VERIFIED-URL - Silberzahn, R., et al. (2018). Many analysts, one data set. *Advances in Methods and Practices in Psychological Science* 1(3):337–356. https://dare.uva.nl/id/0785a9cc-2470-4a3a-bea8-0f34828953c2 — VERIFIED-URL *Intelligence, forensic and legal analysis* - Heuer, R.J. (1999). *Psychology of Intelligence Analysis*, ch. 8 "Analysis of Competing Hypotheses". CIA Center for the Study of Intelligence. https://www.cia.gov/resources/csi/static/Pyschology-of-Intelligence-Analysis.pdf — VERIFIED-URL - US Government (2009). *A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis*. https://www.cia.gov/resources/csi/static/955180a45afe3f5013772c313b16face/Tradecraft-Primer-apr09.pdf — VERIFIED-URL - Kent, S. (1964). Words of Estimative Probability. *Studies in Intelligence*. https://www.cia.gov/resources/csi/static/Words-of-Estimative-Probability.pdf — VERIFIED-URL - Klein, G. (2007). Performing a project premortem. *Harvard Business Review*, September 2007. Record via https://en.wikipedia.org/wiki/Pre-mortem — VERIFIED-URL (secondary record) - Anderson, T., Schum, D., Twining, W. (2005). *Analysis of Evidence*, 2nd ed. Cambridge UP (chapter "Chart method"). https://www.cambridge.org/core/books/analysis-of-evidence/chart-method/65FC65C7FB00E21100B321A15940AB18 — VERIFIED-URL - Twardy, C. (2004). Argument maps improve critical thinking. *Teaching Philosophy* 27(2):95–116. https://philarchive.org/rec/TWAAMI — VERIFIED-URL - Fenton, N., Neil, M., Lagnado, D.A. (2013). A general structure for legal arguments about evidence using Bayesian networks. *Cognitive Science* 37(1):61–102. Record: https://eld.bl.uk/catalog/100024502976_0x000001 — VERIFIED-URL - Rossmo, D.K. (2009). *Criminal Investigative Failures*. CRC Press. https://www.routledge.com/products/9780429248306 — VERIFIED-URL - OHCHR & Human Rights Center, UC Berkeley (2022). *Berkeley Protocol on Digital Open Source Investigations*. https://humanrights.berkeley.edu/programs/investigations/berkeley-protocol — VERIFIED-URL *Medicine* - Guyatt, G.H., et al. (2008). GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. *BMJ* 336:924–926. https://www.bmj.com/content/336/7650/924 — VERIFIED-URL - Balshem, H., et al. (2011). GRADE guidelines: 3. Rating the quality of evidence. *Journal of Clinical Epidemiology* 64(4):401–406. https://experts.mcmaster.ca/scholarly-works/973129 — VERIFIED-URL - Elliott, J.H., et al. (2014). Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap. *PLoS Medicine* 11(2):e1001603. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3928029/ — VERIFIED-URL - Snilstveit, B., Vojtkova, M., Bhavsar, A., Gaarder, M. (2013). Evidence Gap Maps: A Tool for Promoting Evidence-Informed Policy and Prioritizing Future Research. World Bank Policy Research Working Paper 6725. https://documents1.worldbank.org/curated/en/212651468163487838/pdf/WPS6725.pdf — VERIFIED-URL *Automated discovery and LLM hypothesis generation* - Swanson, D.R. (1986). Fish oil, Raynaud's syndrome, and undiscovered public knowledge. *Perspectives in Biology and Medicine* 30(1):7–18. https://muse.jhu.edu/article/403510 — VERIFIED-URL - King, R.D., et al. (2004). Functional genomic hypothesis generation and experimentation by a robot scientist. *Nature* 427:247–252. doi:10.1038/nature02236 — VERIFIED-URL - Williams, K., et al. (2015). Cheaper faster drug development validated by the repositioning of drugs against neglected tropical diseases. *Journal of the Royal Society Interface* 12:20141289. https://pmc.ncbi.nlm.nih.gov/articles/PMC4345494 — VERIFIED-URL - Lu, C., Lu, C., Lange, R.T., Foerster, J., Clune, J., Ha, D. (2024). The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery. arXiv:2408.06292. https://arxiv.org/abs/2408.06292 — VERIFIED-URL - Ghareeb, A.E., et al. (2025). Robin: A multi-agent system for automating scientific discovery. arXiv:2505.13400. https://arxiv.org/abs/2505.13400 — VERIFIED-URL - Si, C., Yang, D., Hashimoto, T. (2024). Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers. arXiv:2409.04109 (ICLR 2025). https://arxiv.org/abs/2409.04109 — VERIFIED-URL - Zhou, Y., Liu, H., Srivastava, T., Mei, H., Tan, C. (2024). Hypothesis Generation with Large Language Models. arXiv:2404.04326. https://arxiv.org/abs/2404.04326 — VERIFIED-URL *Crowdsourcing, forecasting and community practice* - Mellers, B., et al. (2014). Psychological strategies for winning a geopolitical forecasting tournament. *Psychological Science* 25(5):1106–1115. doi:10.1177/0956797614524255 — VERIFIED-URL - Cooper, S., et al. (2010). Predicting protein structures with a multiplayer online game. *Nature* 466:756–760. doi:10.1038/nature09304 — VERIFIED-URL - Lintott, C.J., et al. (2008). Galaxy Zoo: morphologies derived from visual inspection of galaxies from the Sloan Digital Sky Survey. *MNRAS* 389:1179–1189. arXiv:0804.4483 — VERIFIED-URL - Williams, A.C., et al. (2014). A computational pipeline for crowdsourced transcriptions of Ancient Greek papyrus fragments. IEEE Big Data 2014. https://experts.umn.edu/en/publications/a-computational-pipeline-for-crowdsourced-transcriptions-of-ancie/ — VERIFIED-URL - MIT Puzzle Club. History of the Mystery Hunt. https://web.mit.edu/activities/puzzle/history.html — VERIFIED-URL - Taiiwo, Artorias, Puck, TheClockworkBird (2023). Cracking Cicada 3301: The Future of Collaborative Puzzle-Solving. DEF CON 31. https://forum.defcon.org/node/246097 — VERIFIED-URL - Ciphermysteries (2010). Benedek Láng's Rohonc article in Cryptologia. https://ciphermysteries.com/2010/08/23/benedek-langs-rohonc-article-in-cryptologia — VERIFIED-URL (record of Láng 2010; exact title not confirmed) - Design Council. Framework for Innovation (Double Diamond). https://www.designcouncil.org.uk/resources/framework-for-innovation/ — VERIFIED-URL - Zwicky, F. (1969). *Discovery, Invention, Research through the Morphological Approach*. Macmillan. Catalogue: https://libcat.library.tamu.edu/Record/in00000127994 — VERIFIED-URL - Ritchey, T. (2011). *Wicked Problems – Social Messes: Decision Support Modelling with Morphological Analysis*. Springer. https://ideas.repec.org/h/spr/rischp/978-3-642-19653-9_2.html — VERIFIED-URL - Bloom, T. Erdős Problems. https://www.erdosproblems.com/faq — VERIFIED-URL - Olah, C., Carter, S. (2017). Research Debt. *Distill*. https://distill.pub/2017/research-debt/ — VERIFIED-URL *Decipherment and artifact research* - Kober, A.E. (1946). Inflection in Linear Class B: 1 – Declension. *American Journal of Archaeology* 50(2):268–276. Record: https://en.wikipedia.org/wiki/Alice_Kober — VERIFIED-URL (secondary record) - Ventris, M., Chadwick, J. (1953). Evidence for Greek dialect in the Mycenaean archives. *Journal of Hellenic Studies* 73:84–103. https://keibi.uni-tuebingen.de/Record/KEI00120003 — VERIFIED-URL - Fox, M. (2013). *The Riddle of the Labyrinth: The Quest to Crack an Ancient Code*. Ecco. https://en.wikipedia.org/wiki/The_Riddle_of_the_Labyrinth — VERIFIED-URL - Cambridge Mycenaean Epigraphy Group, Ventris–Chadwick correspondence archive. https://www.classics.cam.ac.uk/research/projects/mycep/archive/correspondence — VERIFIED-URL - Knight, K., Megyesi, B., Schaefer, C. (2011). The Copiale Cipher. Proc. 4th Workshop on Building and Using Comparable Corpora (ACL). https://aclanthology.org/W11-1202/ — VERIFIED-URL - Freeth, T., et al. (2006). Decoding the ancient Greek astronomical calculator known as the Antikythera Mechanism. *Nature* 444:587–591. https://orca.cardiff.ac.uk/id/eprint/1622/ — VERIFIED-URL - Freeth, T., et al. (2021). A model of the Cosmos in the ancient Greek Antikythera Mechanism. *Scientific Reports* 11:5821. https://pmc.ncbi.nlm.nih.gov/articles/PMC7955085 — VERIFIED-URL - Rao, R.P.N., et al. (2009). Entropic evidence for linguistic structure in the Indus script. *Science* 324:1165. https://homes.cs.washington.edu/~rao/ScienceIndus.pdf — VERIFIED-URL - Farmer, S., Sproat, R., Witzel, M. (2004). The collapse of the Indus-script thesis: the myth of a literate Harappan civilization. *Electronic Journal of Vedic Studies* 11(2). https://hasp.ub.uni-heidelberg.de/journals/ejvs/article/view/620 — VERIFIED-URL - Ferrara, S., et al. (2024). The invention of writing on Rapa Nui (Easter Island). New radiocarbon dates on the Rongorongo script. *Scientific Reports* 14. https://doaj.org/article/05029bdd384945ef8acb67bd5593e39f — VERIFIED-URL - Salgarella, E., Castellan, S. SigLA: The Signs of Linear A. A palaeographical database. https://www.repository.cam.ac.uk/items/25c5d712-6e4c-4435-86a5-30f769ea4072 — VERIFIED-URL - Luo, J., Cao, Y., Barzilay, R. (2019). Neural decipherment via minimum-cost flow: from Ugaritic to Linear B. ACL 2019. arXiv:1906.06718 — VERIFIED-URL - Reddy, S., Knight, K. (2011). What we know about the Voynich manuscript. Proc. 5th ACL-HLT Workshop LaTeCH. https://aclanthology.org/W11-1511/ — VERIFIED-URL - Bowern, C.L., Lindemann, L. (2021). The linguistics of the Voynich manuscript. *Annual Review of Linguistics* 7:285–308. https://alumniacademy.yale.edu/sites/default/files/2021-07/The%20Linguistics%20of%20the%20Voynich%20Manuscript.pdf — VERIFIED-URL - Davis, L.F. (2020). How many glyphs and how many scribes? Digital paleography and the Voynich manuscript. *Manuscript Studies* 5(1). https://muse.jhu.edu/article/754633 — VERIFIED-URL - Schneier, B. (2025). Substitution cipher based on the Voynich Manuscript [on Greshko's Naibbe cipher]. https://www.schneier.com/blog/archives/2025/12/substitution-cipher-based-on-the-voynich-manuscript.html — VERIFIED-URL - The Voynich Ninja forum, Theories & Solutions. https://www.voynich.ninja/archive/index.php/forum-58-2.html — VERIFIED-URL - Somerton Man identification, news coverage (2022). https://iflscience.com/somerton-man-mystery-may-have-finally-been-solved-using-dna-from-his-death-mask-64614 — VERIFIED-URL *Provenance, reproducibility and claim models* - W3C (2013). PROV-O: The PROV Ontology. W3C Recommendation, 30 April 2013 (Lebo, Sahoo, McGuinness, eds). https://www.w3.org/TR/prov-o/ — VERIFIED-URL - Soiland-Reyes, S., et al. (2022). Packaging research artefacts with RO-Crate. *Data Science* 5(2). arXiv:2108.06503 — VERIFIED-URL - Mölder, F., et al. (2021). Sustainable data analysis with Snakemake. *F1000Research* 10:33. https://f1000research.com/articles/10-33/v2 — VERIFIED-URL - Jaradeh, M.Y., et al. (2019). Open Research Knowledge Graph: Next Generation Infrastructure for Semantic Scholarly Knowledge. K-CAP 2019. arXiv:1901.10816 — VERIFIED-URL - Groth, P., Gibson, A., Velterop, J. (2010). The anatomy of a nanopublication. *Information Services & Use* 30(1–2):51–56. doi:10.3233/ISU-2010-0613 — VERIFIED-URL - Clark, T., Ciccarese, P.N., Goble, C.A. (2014). Micropublications: a semantic model for claims, evidence, arguments and annotations in biomedical communications. *Journal of Biomedical Semantics* 5:28. doi:10.1186/2041-1480-5-28 — VERIFIED-URL - Ciccarese, P., et al. (2008). The SWAN biomedical discourse ontology. *Journal of Biomedical Informatics* 41(5). https://force11.org/post/the-swan-biomedical-discourse-ontology/ ; W3C Note (2009) https://www.w3.org/TR/hcls-swan/ — VERIFIED-URL **Cited from memory (UNVERIFIED; check before print)** - Peirce, C.S. *Collected Papers*, vol. 5 (abduction schema, CP 5.189). Harvard UP, 1931–58 — UNVERIFIED (the SEP entry above covers the same content) - Lipton, P. (2004). *Inference to the Best Explanation*, 2nd ed. Routledge — UNVERIFIED - Glaser, B.G., Strauss, A.L. (1967). *The Discovery of Grounded Theory*. Aldine — UNVERIFIED - Box, G.E.P. (1976). Science and statistics. *JASA* 71(356):791–799 — UNVERIFIED - Heuer, R.J., Pherson, R.H. (2010). *Structured Analytic Techniques for Intelligence Analysis*. CQ Press — UNVERIFIED - Wigmore, J.H. (1913). *The Principles of Judicial Proof*. Little, Brown — UNVERIFIED - Toulmin, S. (1958). *The Uses of Argument*. Cambridge UP — UNVERIFIED - Tukey, J.W. (1977). *Exploratory Data Analysis*. Addison-Wesley — UNVERIFIED - Tetlock, P.E., Gardner, D. (2015). *Superforecasting: The Art and Science of Prediction*. Crown — UNVERIFIED - Altshuller, G. (1984). *Creativity as an Exact Science*. Gordon & Breach — UNVERIFIED - Di Tommaso, P., et al. (2017). Nextflow enables reproducible computational workflows. *Nature Biotechnology* 35:316–319 — UNVERIFIED - Zaharia, M., et al. (2018). Accelerating the machine learning lifecycle with MLflow. *IEEE Data Engineering Bulletin* 41(4) — UNVERIFIED - Timm, T., Schinner, A. (2020). A possible generating algorithm of the Voynich manuscript. *Cryptologia* 44(1):1–19 — UNVERIFIED (as in REPORT_bioinformatics_signal.md) - Ventris, M. (1988). *Work Notes on Minoan Language Research and Other Unedited Papers*, ed. A. Sacconi. Edizioni dell'Ateneo — UNVERIFIED - Serafini, L., statement that the Codex Seraphinianus script is asemic (reported 2009) — UNVERIFIED - Tool homepages, not checked in this session: DVC (dvc.org), MLflow (mlflow.org), Quarto (quarto.org), Jupyter Book (jupyterbook.org), Our World in Data (ourworldindata.org), Zooniverse (zooniverse.org) — UNVERIFIED **Project files read for this report.** `/home/claude/analysis/framework/hypotheses.yaml`, `/home/claude/analysis/framework/results.csv`, `/home/claude/analysis/framework/PROTOCOL.md`, `/home/claude/analysis/framework/make_registry_tex.py`, `/home/claude/analysis/ASSESSMENT.md`, and the chapter list of `/home/claude/book/chapters/`.