# datasetpapers — full reference > Assembled from the design docs by scripts/build.mjs. Short version: https://datasetpapers.com/llms.txt --- ## 20-concept-narrative.md # Deliverable 4a: concept narrative The written case for datasetpapers. This is the manifesto the landing page is built from. It is deliberately plain: no jargon theatre, each step explained. ## The short version A paper was always a lossy compression of the work behind it. When a person writes it, that compression is unavoidable. When a machine can generate the analysis and the prose, it is a choice, and a bad one. datasetpapers stops compressing. The unit of knowledge becomes a versioned, forkable, executable research object: the data, the code, the environment, the figures, the individual claims, the provenance, and a narrative that is just one rendered view of all of it. The object is built to be consumed, checked, and forked, by people and by AI agents alike. ## Why now Generating an analysis and its write-up is no longer the hard part. That is worth saying plainly, because it changes what is valuable. If anyone can produce a plausible analysis on a dataset in minutes, then plausibility is worthless and the scarce thing is trust: knowing which claim was actually re-executed from the data and which was merely asserted. datasetpapers is built around that scarce thing. The verification and the provenance are the product. The prose is a by-product. ## What it is not It is not a data descriptor. There is an established genre, the data paper, whose job is to describe a dataset so others can reuse it, deliberately without analysis or conclusions. datasetpapers is the opposite artifact: it is the analysis, with hypotheses, methods, results, and claims, made forkable and machine-readable. If you were expecting a catalogue record, this is not that. It is not an attempt to sneak machine-written papers past peer review. Every public attempt to do that so far has been withdrawn or disputed, and the major venues now reject undisclosed AI text. datasetpapers does not pretend a generated analysis is a peer-reviewed publication. It labels exactly what has and has not been verified, and it lets that verification accumulate over time. ## How it works, in plain steps 1. Start from an open dataset, pinned to a specific version so results are reproducible. 2. Ask a question the dataset can actually answer, and check it is not a restatement of something already known. 3. Run the analysis in a captured environment, so the exact computation can be re-run later. 4. Break the result into parts: the data reference, the code, the environment, each figure and table, each claim, and the provenance that links them. 5. Put it through a review gate that scores novelty, checks integrity markers, and runs an adversarial review before anything is published. 6. Give it persistent identifiers, package it as a standard research object, and publish it in formats that both people and machines can read. 7. Let others verify individual claims, and let anyone, including an AI agent, fork the whole thing to ask a new question. ## The object A datasetpaper has named parts, and each part is addressable on its own: - The dataset, referenced and pinned, never silently updated. - The code and the environment, so the analysis re-runs deterministically. - The figures and tables, each pointing back to the code that produced it. - The claims, written as atomic, individually citable, machine-checkable statements. - The provenance, a graph that ties every output back to the data and records whether a human or an AI produced each step. - The narrative, compiled from the object, available as a web page, a PDF, or structured XML. Because the parts are addressable, an agent can build on one claim without parsing an entire document, and a person can cite a single figure with a stable identifier. ## Built for AI to build on The reason to decompose the paper is not tidiness. It is that the future reader is often a machine. datasetpapers publishes the whole corpus through interfaces made for that reader: a research-object bundle, a machine-readable dataset description that ML tools already consume, a knowledge graph you can query for contradictions and dependencies, a persistent-identifier tree for granular citation, git for forking, and a Model Context Protocol server so any agent can search, read, verify, and fork without a human in the loop. Crucially, every machine-readable result carries its verification status and its distance from raw data. An agent cannot quietly treat an unverified assertion as a fact, because the label travels with the claim. That is what stops a corpus of machine-generated analyses from poisoning the next generation of machine-generated analyses. ## Credit where it is due Every datasetpaper starts from someone's dataset. datasetpapers records that debt explicitly, notifies the original depositor that their data was used, and gives them a first-class place in the credit graph. The people who share data have been under-credited for as long as data sharing has existed. A world where machines analyse open data at scale makes that worse unless credit is designed in. Here it is designed in. ## The honest risks - If generation is cheap, the corpus can drown in near-duplicates. The answer is a novelty and dedup gate that runs before any compute is spent, and a visible quality signal, not more generation. - A machine can be confidently wrong. The answer is mandatory re-execution, adversarial review, and never deleting a refuted claim. - Errors compound if unverified work becomes the input to new work. The answer is the trust-distance measure and a hard rule that unverified claims are never ground truth. - These platforms usually fail on adoption, not technology. Every decomposed-publishing predecessor was technically sound and adoption-starved. The missing piece was always credit and mandate. That is the problem to solve first. ## What we are asking people to do Read a datasetpaper the way you would read a pull request, not a journal article. Check the claims you care about. Fork it if you can do better. Bring your dataset if you want it analysed. Treat it as a starting point that is honest about its own uncertainty, not a finished verdict. That is a more useful thing than another PDF, and it is a thing only this shape can be. --- ## 30-object-model-spec.md # Deliverable 1: the datasetpaper object model spec The normative definition of a datasetpaper as a packaged research object: the RO-Crate profile, the fixed slot schema, the claim (nanopublication) schema, provenance, and the machine-readability contract. A conforming object is what the pipeline emits at stage 8 and what every consumption surface is projected from. The worked reference example that conforms to this spec is in [../examples/permeability/](../examples/permeability/). ## Conformance chain A datasetpaper version is one RO-Crate that conforms, in order, to: 1. RO-Crate 1.2 (`https://w3id.org/ro/crate/1.2`) — the base packaging. 2. Workflow RO-Crate 1.0 (`https://w3id.org/workflowhub/workflow-ro-crate/1.0`) — the analysis is an executable workflow. 3. Process Run Crate (`https://w3id.org/ro/wfrun/process/0.5`) — at least one retrospective run with PROV alignment. 4. The datasetpaper Profile Crate (`https://datasetpapers.com/profile/0.1`) — the additional MUSTs below. Verify current profile version pins before use; the Workflow Run Crate family was at 0.5 at time of writing. ## Required entities (datasetpaper profile MUSTs) The `@graph` MUST contain: - The metadata descriptor (`@id: "ro-crate-metadata.json"`) with `conformsTo` listing the full chain above. - The root data entity (`@id: "./"`, `@type: Dataset`) representing the version, carrying: `name`, `description`, `datePublished`, `license`, `identifier` (the version ARK), `version`, and, when a fork, `isBasedOn` the parent version ARK. - The source dataset as a data entity, referenced not embedded (the RO-Crate 1.2 detached pattern): a `Dataset` with `@id` the source DOI or URL, plus `license`, `identifier`, and `sha256`. It is pinned; `version` is required. - The analysis workflow as the `mainEntity`: a `File` typed `ComputationalWorkflow` (Bioschemas), with `programmingLanguage`, `version`, and declared `input`/`output` `FormalParameter` entities. - A pinned environment: a `SoftwareApplication` carrying the container image reference and digest, plus a lockfile reference. This is a MUST; a datasetpaper that cannot state its environment cannot be re-run. - At least one run: a `CreateAction` (Process Run Crate) with `instrument` (the workflow), `object` (inputs: dataset, params), `result` (outputs: figures, tables), `agent`, `startTime`, `endTime`, and `environment`. - Agents: the AI engine as a `SoftwareAgent` (with its model id), and any human as a `Person` (with ORCID) who `actedOnBehalfOf` an `Organization`. This distinction is a MUST so audits separate AI-driven from human-driven work. - The claims: each an entity of `@type: Claim` (profile term) referenced from the root, each `isSupportedBy` one or more figure or table entities, each with a `trustyURI` (its nanopub hash), `verificationStatus`, and `trustDistance`. - Trust declaration: the root carries `trustDistance` and every claim repeats it. ## The fixed slot schema Every datasetpaper fills the same seven slots, so objects are diffable and comparable. In the crate they are `CreativeWork` parts of the root with a stable `additionalType`: 1. summary — the background and question. 2. provenance-methods — how the analysis was produced. 3. data-records — the input datasets and derived tables. 4. technical-validation — machine-checkable QC and expected outputs. Required. 5. usage-notes — caveats and reuse guidance. 6. code-availability — the workflow and environment. 7. claims — the atomic assertions. The narrative is compiled from these slots as a MyST AST and exported to HTML and JATS; it is never the source of truth. ## Claim schema (nanopublication) A claim is emitted as a nanopublication with three named graphs: - assertion: the triple `subject predicate object` plus a human `assertionText`. - provenance: `wasDerivedFrom` the supporting figures and the dataset, `wasGeneratedBy` the run activity, `wasAttributedTo` the agents. - publicationInfo: `creator`, `created`, `hasTrustyURI` (the content hash), `verificationStatus`, `trustDistance`, `confidence`, `noveltyGrade`. Claims are generated as an output of authoring, never hand-written in RDF. Identity pairs the mutable claim ARK with the immutable trusty URI. ## Dataset description layers - Croissant (JSON-LD over schema.org) for the source dataset and every derived tabular output, so ML tooling and Google Dataset Search consume them directly. - Frictionless Data Package plus Table Schema as the inner descriptor of each derived table, giving column types, constraints, and keys, and machine validation the descriptive JSON-LD does not provide. - schema.org `Dataset` and `ScholarlyArticle` JSON-LD on the object page for web discovery. Croissant answers what is in the data and how to load it; the RO-Crate answers what produced it and how to re-run it; they compose, with the Data Package as the typed core. ## Identifiers Every entity in the crate carries an ARK from the object's tree (concept, version, component, claim) and, for the version, component, and claim, a content hash. See the ARK scheme. A promoted release additionally carries a DataCite DOI on the root. ## The machine-readability contract So that an agent can locate and re-run a datasetpaper without human parsing, a conforming object MUST expose: - stable `additionalType` values on the slot parts and typed components. - `encodingFormat` on every file entity. - a single machine entry point: the `CreateAction` names the workflow, the pinned environment, the exact inputs, and a re-run command, so an agent can reconstruct the run from the crate alone. - the seven integrity markers from the pipeline design, each as a typed entry, so a consumer can check provenance completeness programmatically. An object that satisfies these is not merely described; it is actionable. That is the difference between a datasetpaper and a PDF with attachments. --- ## 40-io-contract.md # Deliverable 5: the input, process, and output contract This is the contract to fix before any datasetpaper is generated. A datasetpaper must be a pure function of a pinned input and a versioned protocol: the same input bundle plus the same protocol should produce the same object, apart from declared, bounded nondeterminism. Get this contract right and everything downstream (the gate, forking, provenance, comparison) works. Get it wrong and the corpus is a pile of non-comparable one-offs. Three parts: what we pull (input), how it is processed (protocol), and what must come out (result). Companion files: the object model in [30-object-model-spec.md](30-object-model-spec.md), the pipeline in [10-pipeline-design.md](10-pipeline-design.md), the data model in [01-data-model.md](01-data-model.md). ## Part A. The input: what we pull from fairdata fairdata-ai already holds almost everything the framing and analysis need, so we pull structured facts, not raw guesses. Access is read-only against the fairdata Postgres (direct IPv6 for batch, pooler otherwise) or a thin fairdata API. We reference dataset bytes; we do not copy them into our store. What we take, by fairdata table: - datasets (keyed by doi): title, metadata, datacite_raw, landing_page_url, repository. This gives source identity, the citation, and the license (read from datacite_raw, not assumed). - dataset_files: content_url, media_type, size_bytes, md5, is_tabular. This is the pinning source. Each file is pinned by (content_url, md5, size_bytes); the md5 set defines the immutable input. - data_profiles: columns (per-column stats), row_count, column_count, correlations, anomalies, summary. This is the machine understanding of the data. We frame questions against real columns and distributions before a single byte is loaded, which is what makes framing cheap and grounded. - assessments: overall_score, scores, frameworks_used, framework_version. The FAIR and quality gate. A dataset below the quality threshold is not analysed; the score is carried into the datasetpaper. - paper_context: methodology, variables, temporal, spatial, population, limitations, ethics, subjects, granularity. This seeds the Summary, Methods, and Usage Notes slots and, importantly, the ethics and limitations guardrails the analysis must respect. - linked_papers and data_citations: the prior-art corpus for novelty scoring, so a question is checked against what has already been done with this data. - discoveries: hypothesis, surprise_score, novelty_status. fairdata already mines cross-dataset hypotheses. These are candidate questions the framing stage can adopt or sharpen rather than inventing from scratch. - dataset_subdomains: domain and subdomain, for routing and for domain rules (for example steering pharmacogenomics away from over-studied genes). The InputBundle. The pipeline assembles these into one pinned object, hashes it, and hands exactly that to the protocol. Nothing outside the bundle may enter the analysis. ```json { "bundle_id": "ark:/99999/dpINPUT.", "source": { "doi": "10.xxxx/....", "title": "...", "repository": "figshare", "landing_page_url": "https://...", "license": "CC-BY-4.0", "datacite_raw": { } }, "files": [ { "filename": "runs.csv", "content_url": "https://...", "md5": "…", "size_bytes": 12345, "media_type": "text/csv", "is_tabular": true } ], "profiles": [ { "filename": "runs.csv", "row_count": 812, "column_count": 6, "columns": [ ], "correlations": [ ], "anomalies": [ ] } ], "assessment": { "overall_score": 0.84, "frameworks_used": ["FAIR"], "framework_version": "…" }, "context": { "variables": [ ], "limitations": "…", "ethics": "…", "population": "…", "temporal": "…", "spatial": "…" }, "prior_art": { "linked_papers": [ ], "citation_count": 12 }, "seed_hypotheses": [ { "hypothesis": "…", "surprise_score": 0.7, "novelty_status": "candidate" } ], "subdomains": [ { "domain": "…", "subdomain": "…" } ], "bundle_hash": "sha256:…" } ``` Selection rules (which datasets get analysed): assessment.overall_score above threshold, at least one tabular file, a subdomain we support, not already covered by an existing datasetpaper (dedup on the file md5 set), and either a seed hypothesis or a framing that clears the novelty bar. ## Part B. The process: a standardised protocol, not a prompt Claude Science is the executor. The thing that makes the output a datasetpaper rather than a chat transcript is that it runs a versioned Analysis Protocol: a specification, not an ad-hoc prompt. The protocol is itself a citable, versioned object (protocol_id plus version), and every datasetpaper records which protocol produced it. Why standardise. Comparable objects require a comparable procedure. A free prompt yields objects that cannot be diffed, graded consistently, or checked against the integrity-marker contract. A protocol yields objects that can. A protocol has four fixed parts: 1. Specification: the role, the hard constraints (respect the license and ethics from context; declare uncertainty; never assert beyond the data), and the required outputs. 2. Toolset: the allowed libraries and operations (data load, validation, statistics, plotting) and nothing else, so runs are bounded and reproducible. 3. Input: exactly one InputBundle. 4. Output schema plus validators: the DatasetPaperResult below, with checks that must pass before the object may leave the run. The steps a protocol run must perform, in order: 1. Load and validate the files against the profiles. If the data does not match its profile (columns, types, row counts), stop; the input is not what fairdata described. 2. Pre-register the plan. Before running anything, declare the hypotheses, the exact tests, and the expected outputs. This is the control for the garden of forking paths: the executed analysis is checked against this plan, and deviations are recorded, so a machine cannot silently fish for significance. 3. Execute the pre-registered analysis in the pinned environment. 4. Self-check (Technical Validation): reproduce any reference values, run the declared robustness and sensitivity checks, and record expected-versus-actual. 5. Extract claims, each linked to the figure or table that supports it, each with a confidence and a novelty grade. 6. Emit the structured result and the compiled narrative. Determinism. Pin the protocol version, the model id, the container digest, the lockfile, and the random seeds. Numeric outputs must reproduce on re-run; only prose may vary within the bounds the pre-registered plan and validators allow. Declared nondeterminism is acceptable and labelled; undeclared nondeterminism is a bug. Where it runs. Claude Science is the primary authoring and exploration executor. The same protocol is packaged for headless, gated, batch execution (Agent SDK or a compute capsule) so production generation does not depend on an interactive session. The protocol is the shared artifact; the executor is swappable. ## Part C. The output: the DatasetPaperResult A protocol run must emit exactly this, and it must pass the validators before it advances to the gate. It maps one-to-one onto the object model. ```json { "protocol": { "id": "protocol:tabular-reduction", "version": "0.3", "model_id": "claude-opus-4-8" }, "input_bundle_hash": "sha256:…", "environment": { "container_digest": "sha256:…", "lockfile_hash": "sha256:…", "seeds": { "numpy": 12345 } }, "preregistered_plan": { "hypotheses": [ "…" ], "tests": [ { "name": "…", "method": "…", "expected": "…" } ] }, "slots": { "summary": "…", "provenance_methods": "…", "data_records": "…", "technical_validation": "…", "usage_notes": "…", "code_availability": "…" }, "components": [ { "type": "code", "path": "analysis.ipynb", "media_type": "application/x-ipynb+json", "content_hash": "…" }, { "type": "figure", "path": "figures/fig-1.png", "produced_by": "analysis.ipynb#cell-7", "content_hash": "…" }, { "type": "table", "path": "tables/tbl-1.csv", "produced_by": "analysis.ipynb#cell-9", "datapackage": "datapackage.json", "croissant": "croissant.json", "content_hash": "…" } ], "claims": [ { "subject": "…", "predicate": "…", "object": "…", "assertion_text": "…", "supported_by": ["figures/fig-1.png"], "confidence": 0.94, "novelty_grade": "B", "trust_distance": 0 } ], "provenance": [ { "subject": "figures/fig-1.png", "predicate": "wasGeneratedBy", "object": "activity:run" }, { "subject": "version", "predicate": "wasDerivedFrom", "object": "10.xxxx/..." } ], "integrity_markers": { "source-pinned": true, "env-captured": true, "figure-traced": true, "claim-supported": true, "provenance-complete": true, "license-cleared": true, "trust-distance-declared": true }, "narrative_myst_ast": { }, "self_report": { "deviations_from_plan": [ ], "limitations": "…", "confidence_overall": 0.9 } } ``` Validators that must pass before the result leaves the run (fail any and the run is rejected before it reaches the gate): - Every figure and table has a produced_by pointer to a real code cell. - Every claim has a non-empty supported_by list resolving to a figure or table. - The provenance graph reaches from the source dataset to every output with no dangling entity. - Every executed test appears in the pre-registered plan, or its deviation is recorded in self_report. - All seven integrity markers are present and true. - The license permits the derivative, read from the bundle, not assumed. This is the hand-off to pipeline stage 6. The gate then judges quality, novelty, and integrity on an object that is already internally consistent, rather than trying to salvage a free-form transcript. ## Why fix this first The user's instinct is right: define the workflow, inputs, and outputs before making a paper. Everything valuable about datasetpapers (forking, comparison, verification, machine consumption) depends on objects being uniform and self-describing. That uniformity comes from this contract, not from the model being clever. The model is clever inside a fixed frame; the frame is what we are building. --- ## 50-published-object.md # Deliverable 6: the published datasetpaper What a finished datasetpaper looks like at its ARK, what a reader can do with it, and how it persists. This answers the direct questions: can I fork it, is it interactive, what is the provenance like, what are the API and MCP, can I download a PDF, can I open it in Overleaf, and what keeps it alive. Companion files: the object model in [30-object-model-spec.md](30-object-model-spec.md), identifiers in [02-identifiers-ark.md](02-identifiers-ark.md), the agent API in [03-mcp-surface.md](03-mcp-surface.md). ## The reader view A published datasetpaper resolves at its version ARK. The default view is the compiled narrative with figures and tables inline, and it reads like a short paper. The difference is that every element is a live, addressable object rather than flat text: - Each figure and table shows a small provenance and verification badge and links to the exact code cell that produced it. - Each claim is its own element. Expanding it reveals the atomic assertion, the figures that support it, its verification status, its trust distance, and whether a human or an AI produced it. - The seven fixed slots (summary, methods, data records, technical validation, usage notes, code availability, claims) are always present and always in the same place, so any two datasetpapers can be read and compared the same way. The canonical published version is immutable. Anything a reader changes happens either in a throwaway sandbox or in a fork. ## Is it interactive Yes, in two bounded ways. - Re-run in place. The object ships with a pinned environment, so the page can offer a re-run backed by a compute session (a Binder or a compute capsule). A reader can re-execute the analysis, tweak a parameter, and see the figures update, without changing the published record. - Explore the object. Toggle raw versus derived data, open the notebook, expand any claim to its nanopublication, and walk the provenance graph. These are reads; they never mutate the published version. Heavy or open-ended changes are a fork, not an edit, which keeps the published version stable and every derivative attributable. ## Can I fork it Yes. Forking is the primary motion, for people and for agents. - A fork button on the page, and a fork tool on the MCP server, both create a new lineage from this exact version. - The fork opens the notebook and the pinned InputBundle in an execution environment, preloaded, so you start from a working analysis, not a blank page. - You change the question or the method and generate a child datasetpaper, which goes through the same gate before it is published. - A fork gets a new concept ARK and is a git fork. Its ancestry is recorded twice: as a git parent and as a wasDerivedFrom edge in the provenance graph. If it builds on the parent's derived outputs rather than the raw data, its trust distance increments. So building on someone's analysis is a first-class action with a clean lineage, which is the whole point of the format. ## What is the provenance like Provenance is a navigable graph, not a paragraph. From any claim you can walk to the figures that support it, to the code cell that produced them, to the pinned dataset they came from, and to the agents involved. It is expressed in W3C PROV and served as part of the object. Two things make it trustworthy rather than decorative: - The AI engine is recorded as a software agent, distinct from any human, so an audit can always separate machine-produced steps from human ones. - Every claim carries its verification status and its trust distance, and verifications (re-execution, sensitivity, adversarial review) attach to it over time. A refuted claim is never deleted; its refutation is shown in place. Each claim effectively answers, on demand, the question how do I know this. ## The API and the MCP Everything the page shows is available as data, through two surfaces. - REST and GraphQL for applications and people: fetch a work, a version, a component, or a claim; list and filter claims; walk related objects; query the knowledge graph. - MCP for agents: search, get, list claims, get provenance, get the RO-Crate or Croissant, find related objects, query the graph, and the write tools to fork, propose, and verify. Reads are open; writes are authenticated and gated. Content negotiation on the ARK returns the right representation from the same identifier: the narrative for a browser, schema.org JSON-LD for a crawler, the RO-Crate for a machine, a nanopublication for a claim ARK. Whatever a reader can see, an agent can fetch. ## Downloads and exports The narrative is a compiled build artifact, so it can be rendered to any target. From the object page and the API: - PDF: a typeset rendering compiled from the MyST AST (via Typst or LaTeX). A promoted release can carry a DataCite DOI so the PDF is formally citable. - LaTeX, open in Overleaf: the MyST AST exports to LaTeX, and the page offers an open-in-Overleaf action that hands Overleaf the LaTeX bundle by URL. Because the narrative is generated, LaTeX is just another output target, not a separate manuscript. - JATS XML: for archival and for indexing by scholarly infrastructure. - RO-Crate (zip): the entire object, data references, code, environment, figures, claims, and provenance, as one portable bundle. - Croissant and Frictionless Data Package: the dataset description and the typed derived tables, for ML and data tooling. - Jupyter notebook: the analysis code. - Nanopublications: the claims, individually. The rule is that no representation is privileged. The RO-Crate is the source of truth; the PDF, the LaTeX, the web page, and the JATS are all views compiled from it. ## The persistence strategy Persistence is designed so the objects outlive the platform. The lesson from Papers With Code, retired overnight, is taken literally: nothing important may depend on datasetpapers.com staying up. Three layers, decoupled: 1. Identity and resolution. Every object has an ARK (a concept ARK for the lineage, a version ARK per snapshot), resolvable at datasetpapers.com and mirrored at the global resolver N2T, each paired with a content hash. Identifiers are never deleted; a retracted object resolves to a tombstone that preserves its metadata and provenance. 2. Preservation. Every published version is deposited as an RO-Crate into an external preservation repository (Zenodo or figshare) that issues its own DOI and does the long-term keeping. Source datasets are referenced by pinned DOI and checksum, never re-hosted. The code, narrative, and crate metadata are also mirrored to git, so distributed, forkable copies exist outside our infrastructure. 3. Live service. datasetpapers.com, its object store, the knowledge graph, and the MCP server are the convenience layer: fast search, the reader view, re-run, forking. This layer can fail without losing a single object, because identity lives in the resolver and the bytes live in the preservation repositories and git. The whole corpus is exportable as a dump (objects, claims, provenance, and identifiers), so anyone, including you, can stand up a mirror. Durability comes from the standard formats and the external deposits, not from a promise that the site will always be online. ## In one line A published datasetpaper is an immutable, addressable, re-runnable research object that reads like a paper, forks like a repository, exports to every format a human or a machine could want, and is preserved outside the platform that made it. --- ## 03-mcp-surface.md # Deliverable 3c: MCP surface The Model Context Protocol server is the surface that makes datasetpapers a substrate other AI systems build on, rather than a website they scrape. It is the strongest differentiator (the competitive research found the machine-consumable, forkable output layer is unoccupied), so it is designed in from the start, not bolted on. The reference skeleton is in [mcp/](../mcp/). This document specifies the resources, tools, prompts, and the safety model. ## Design goals 1. An agent can discover, read, compare, verify, and fork datasetpapers without a human in the loop. 2. Every read carries verification_status and trust_distance, so a consuming agent cannot silently promote an unverified assertion to ground truth. This is the compounding-error guardrail expressed at the API layer. 3. Reads are open and cacheable. Writes (fork, propose, verify) are authenticated, rate-limited, and always routed through the quality gate. 4. The server is thin. It projects the Postgres store and the emitted surfaces; it holds no business logic that the pipeline does not already own. ## Transport and stack - Protocol: MCP over stdio (for local agents) and streamable HTTP (for hosted agents). - SDK: the official TypeScript MCP SDK. - Data access: read models over the Postgres schema in [db/schema.sql](../db/schema.sql), plus the object store for crate and emission bytes. - Interop: Code Ocean already ships an MCP server for executing capsules. The fork and re-execution tools can delegate actual compute to an execution backend rather than reimplementing it. ## Resources Resources are addressed by a `datasetpaper://` URI that mirrors the ARK tree, so a resource URI is derivable from any identifier. - datasetpaper://work/{concept_ark} — the lineage, its versions, and metadata. - datasetpaper://version/{version_ark} — a snapshot with its component list. - datasetpaper://component/{component_ark} — one part (figure, code, table, ...). - datasetpaper://claim/{claim_ark} — one atomic claim with evidence and verification. - datasetpaper://dataset/{source_ark_or_doi} — an upstream dataset with its Croissant. - datasetpaper://crate/{version_ark} — the RO-Crate metadata for a version. Resource reads support an Accept-like `format` hint: narrative, jsonld, ro_crate, croissant, nanopub. ## Tools Read tools (open): - search_datasetpapers(query, filters?, limit?, cursor?) filters: discipline, source_dataset, entity, claim_predicate, verification_status, min_trust_distance, max_trust_distance, status. Returns ranked works with snippets. - get_datasetpaper(ark, include?) include: [narrative, claims, figures, provenance, attribution]. Accepts concept or version ARK; a concept ARK returns the latest published version. - get_claim(claim_ark) Returns the assertion triple, assertion_text, supporting figures/tables, confidence, novelty_grade, verification_status, trust_distance, and the nanopub hash. - list_claims(ark, filters?) All claims in a work or version, filterable by verification_status or predicate. - get_provenance(version_ark) The PROV-O graph for the version (entities, activities, agents, edges), including the fork edge and figure-to-code lineage. - get_rocrate(version_ark) and get_croissant(ark) Return the RO-Crate metadata and the Croissant description (of a source dataset or a derived tabular output). - find_related(ark, relation) relation: forks, derivations, supports, contradicts, cites, same_dataset. Walks the knowledge graph and prov edges. - query_graph(query) A structured graph query (and, later, SPARQL) over claims, entities, and edges. Answers "every claim that used variable X" or "claims that contradict claim Y". - get_dataset(source_ark_or_doi) The upstream dataset descriptor plus Croissant plus the list of datasetpapers built on it. Write tools (authenticated, gated): - propose_analysis(source_dataset_ref, question, config?) Enqueues a new datasetpaper generation on an open dataset. Returns a job handle; the result must clear the gate before it is published. - fork_datasetpaper(parent_version_ark, question, config?) Creates a new work derived from a parent version, mints a concept ARK, records the wasDerivedFrom edge, and enqueues generation. This is the headline capability: an agent builds on top of an existing analysis. Returns a job handle. - submit_verification(target_ark, method, result, evidence_url?) Attaches a verification (re_execution, sensitivity, adversarial_review, manual) to a claim or version, advancing its verification_status. Open to accredited human and agent verifiers. - get_job(job_handle) Status of a propose or fork job, including the gate result when complete. ## Prompts Server-provided prompts standardize common agent workflows: - summarize_datasetpaper(ark) — a faithful, evidence-linked summary. - critique_claim(claim_ark) — an adversarial review prompt seeded with the claim and its evidence, for a verifier agent. - design_fork(parent_version_ark) — propose high-value questions to fork a given analysis toward. ## Safety and trust model - Trust labeling is mandatory on output. Every claim and version returned includes verification_status and trust_distance. Clients are expected to honor them; the default search ranking down-weights unverified and high-trust-distance results. - Writes never publish directly. propose_analysis and fork_datasetpaper always return drafts that must pass the preprints.ai gate (novelty, integrity markers, adversarial review) before they enter the public corpus and the read tools. - Rate limiting and quotas on write tools, because generation has real compute cost and an ungated fork tool is a slop-amplifier. The gate plus per-agent quotas are the throttle. - Provenance of agent actions. Every write records the calling agent as a prov:SoftwareAgent (or Person) via an activity and a wasAssociatedWith edge, so the corpus can always answer who or what produced a given object. - Licensing is enforced server-side. propose and fork check the source dataset license before generating a republishable derivative, and refuse or restrict output that the upstream license does not permit. ## Why this matters The read tools make the corpus queryable by any agent. The fork tool makes it extensible by any agent. The trust labels and the gate keep that openness from degrading into an unverified sludge that poisons downstream work. That triad, open read, gated write, explicit trust, is the whole product expressed as an API. --- ## 10-pipeline-design.md # Deliverable 2: end-to-end pipeline design How a datasetpaper gets made, from an open dataset to a published, forkable, machine-readable object. This is the build-ready design for the standalone datasetpapers.com stack. It owns orchestration and calls existing services rather than absorbing them. Companion files: the data model in [01-data-model.md](01-data-model.md), identifiers in [02-identifiers-ark.md](02-identifiers-ark.md), the agent API in [03-mcp-surface.md](03-mcp-surface.md), the standards rationale in [../research/competition-and-inspiration.md](../research/competition-and-inspiration.md). ## Services this stack calls - fairdata-ai: the first-batch supplier of open datasets and reusable FAIR engines. Also figshare and zenodo adapters. - An execution backend: the environment that runs the analysis notebook and returns figures, tables, and a captured container. Claude Science for generation; a Code Ocean style capsule (which already exposes an MCP server) for deterministic re-execution. - preprints.ai: the quality, novelty, and integrity gate (the six-model review plus the Opus advisor). - ARK resolver and N2T for identifiers; a DataCite path for promoted DOIs. - Object store for crate and emission bytes; Postgres for the model in db/schema.sql. The orchestrator is a queue plus workers (the same launchd worker shape already used for the preprints.ai pipeline), so batches of datasets flow through without a human babysitting each one. ## The pipeline at a glance ``` 0 ingest pin dataset, fetch Croissant, license + dedup check 1 frame generate candidate questions, novelty-score, pick 2 analyse run notebook, capture figures/tables + environment 3 decompose split into typed components, emit MyST AST 4 provenance build the PROV graph (figure->code, claim->figure, version->dataset) 5 claims extract atomic claims as nanopublications, compute trust distance 6 GATE preprints.ai: review + novelty + integrity markers -> pass/revise/fail 7 mint ARK concept/version/component/claim + content hashes 8 package RO-Crate, Croissant, JSON-LD, nanopubs, JATS, git commit 9 publish datasetpapers.com + knowledge graph + Dataset Search + notify creator 10 aftercare verification loop, forks, retraction propagation ``` Stage 6 is the only hard gate. Stage 1 is a soft gate (novelty) that exists to avoid spending compute on work the corpus does not need. Everything before stage 6 happens on a draft version; nothing reaches the read corpus or the MCP read tools until the gate passes. ## Stage detail ### 0. Ingest Input: a dataset reference (fairdata id, figshare or zenodo DOI). Output: a source_dataset row. - Pin to a specific version and record version_pinned_at. Never resolve "latest" at analysis time. - Fetch or generate the Croissant description and store it on source_dataset.croissant. - License gate: read the SPDX license. If it does not permit a republishable derivative, either stop or mark the output access-restricted. This is enforced here and again at the MCP write tools. - Dedup gate: hash the pinned distribution and check the corpus. If this exact dataset version already has datasetpapers, route to "propose a new question" rather than regenerate an existing analysis. Silent regeneration is how a corpus fills with near-duplicates. ### 1. Frame the question Input: source_dataset. Output: a chosen question plus a novelty score. - Generate candidate questions the dataset can actually answer given its columns (read the Table Schema, not just the title). - Novelty-score each candidate against the existing corpus and against OpenAlex and Semantic Scholar, so the analysis is not a restatement of known results. - Apply domain rules. For pharmacogenomics datasets, steer away from the heavily studied genes (the CYP family, TPMT and similar) toward genuinely underexplored targets; a well-worn hypothesis is low value even if correct. - This soft gate is a cost control: it is far cheaper to reject a low-novelty question here than to run the analysis and reject it at stage 6. ### 2. Analyse Input: dataset plus question. Output: figures, tables, intermediate outputs, and a captured environment. - Run the analysis in the execution backend (Claude Science generates the notebook and figures). - Capture the environment as a container digest plus a lockfile hash. Own this layer; the reproducible-document toolchains punt on it and that is why their outputs rot. Record environment_digest and lockfile_hash on the draft version. - Every figure and table is written with a pointer to the code cell that produced it. That pointer becomes a component.produced_by_component_id and a PROV edge. ### 3. Decompose Input: the analysis run. Output: typed component rows and a MyST AST. - Split the run into components: raw_data (a pinned reference, not a copy), code, environment, figure, table, and narrative. - Emit the narrative as a MyST JSON AST so it is structurally parseable and can be rendered to HTML, PDF, and JATS. The narrative is a build artifact, never hand-edited. - Fill the fixed slot schema borrowed from Scientific Data: Summary, Provenance/Methods, Data Records, Technical Validation, Usage Notes, Code Availability, Claims. Every object has the same slots, so objects are diffable and comparable. ### 4. Provenance Input: components and the run. Output: prov_edge rows and activity rows. - Record the run as an activity of kind analysis_run, with the AI engine as a prov:SoftwareAgent and the human requester (if any) as a prov:Person acting on behalf of an org. - Write the edges: each figure wasGeneratedBy the code activity and wasDerivedFrom its inputs; the version wasDerivedFrom the source dataset; a fork additionally wasDerivedFrom its parent version. - Compute trust_distance: 0 if the analysis is on raw source data, incremented if it builds on another datasetpaper's derived outputs. Store it on the version and copy it onto each claim. ### 5. Claims Input: the narrative and figures. Output: claim rows and nanopublications. - Extract atomic claims as subject-predicate-object triples with an assertion_text and a supported_by list of the figures or tables that back them. - Generate each claim as a nanopublication (assertion, provenance, publication-info) and compute its trusty-URI-style hash into claim.nanopub_hash. Claims are generated, never hand-authored in RDF. - Record model-reported confidence and a preliminary novelty grade. verification_status starts at unverified for every claim. ### 6. The gate (preprints.ai) Input: the draft version and all its parts. Output: a gate_result and a verdict. - Submit to the six-model review plus the Opus advisor. Store every model review and the advisor verdict on gate_result. - Novelty grade: confirm the analysis is not a low-value restatement. Watch for the inverted-novelty failure mode where genuinely novel work is scored as unoriginal; the grade must be sanity-checked against the stage 1 score. - Integrity markers: the pipeline writes a fixed set of machine-detectable provenance markers into the object (see the contract below), and the gate must detect them. A mismatch between markers written and markers detected is a gate failure, because it means the provenance is not machine-verifiable. - Verdict: pass proceeds to mint; revise loops back to stage 2 or 5 with the reviewer notes; fail stops and records why. The draft is retained either way. ### 7. Mint Input: a passed draft version. Output: identifier rows. - Mint the concept ARK (if new work), the version ARK, and register component and claim ARKs, each paired with its content hash. See the ARK scheme. - The version content_hash is the sha256 of the canonical RO-Crate produced in the next stage, so minting and packaging are ordered so the hash is final before the version is marked published. ### 8. Package and emit Input: the version and its parts. Output: emission rows and a git commit. - Assemble the RO-Crate (datasetpaper Profile Crate: schema.org plus PROV, Workflow Run Crate actions, the dataset as a detached reference, the pinned environment). - Emit Croissant for the dataset and derived tables, schema.org JSON-LD (Dataset and ScholarlyArticle), the nanopublications, and JATS from the MyST AST. - Commit the version to git; a fork is a git fork whose parent matches work.parent_version_id. - Emission is idempotent: keyed on (version, format, content_hash), so re-running the stage produces no duplicates. ### 9. Publish Input: emitted surfaces. Output: a live object and index entries. - Flip the version status to published and set work.current_version_id. - Index into the knowledge graph (claims, entities, edges) and expose the SPARQL or GraphQL endpoint and the MCP read tools. - Register schema.org JSON-LD for Google Dataset Search. - Notify the data creator: resolve the upstream depositor via the dataset and ORCID, write attribution rows, and send the "your dataset was analyzed" notification. This closes the credit loop and is the main organic growth channel. ### 10. Aftercare Ongoing, post-publish. - Verification loop: human and agent verifiers attach verifications via the MCP write tool, advancing claim verification_status from unverified toward re_executed or reviewed, or to refuted. A refuted claim is never deleted; its status is visible on every read surface. - Forks: a fork_datasetpaper call re-enters the pipeline at stage 1 with the parent version as context and a wasDerivedFrom edge already recorded. - Retraction propagation: poll upstream datasets for version changes and retractions. If a source is retracted, walk wasDerivedFrom to flag every affected version and claim and resolve their ARKs to a tombstone. ## Integrity markers (the contract with the gate) The recurring failure in the current openscience-to-preprints path is that the generator writes more provenance markers than the gate reliably detects. The fix is a written contract, versioned and shared by both sides. A marker is a machine-detectable statement that a specific integrity property holds. The proposed set: 1. source-pinned: the dataset is pinned to a version with a checksum. 2. env-captured: a container digest and lockfile are present. 3. figure-traced: every figure has a produced_by code pointer. 4. claim-supported: every claim has a non-empty supported_by list. 5. provenance-complete: the PROV graph covers dataset to output with no dangling entity. 6. license-cleared: the derivative is permitted by the source license. 7. trust-distance-declared: trust_distance is set and justified. The contract requires that each marker is emitted in one agreed location and format (a typed entry in the RO-Crate plus a mirrored field in the gate submission), and that the gate asserts detection of all seven. Detected-count must equal written-count or the gate fails. Version the contract so the two systems evolve together instead of drifting. ## Cost, throughput, and failure - Cost control is front-loaded. The stage 1 novelty gate and the stage 0 dedup gate exist to avoid spending analysis compute on work that will be rejected or duplicated. Route cheap, high-volume steps (dedup, framing, marker checks) to smaller models and reserve the strongest models for analysis and the advisor review. - Throughput is batch and queue driven. Phase one targets ten to fifty datasets; workers pull from the queue so a slow gate does not stall ingest. - Determinism and idempotency. Content-hash addressing at the version, component, and claim level means a re-run either reproduces the same hash (no-op) or produces a new version. Nothing is edited in place, so a failed stage is safe to retry. - Every stage writes its outputs to the draft version before advancing, so a crash resumes from the last completed stage rather than restarting the analysis. ## What reuse buys Four of the ten stages are existing infrastructure: ingest and FAIR engines from fairdata-ai, the gate from preprints.ai, execution from the Claude Science and capsule backends, and minting from the ARK resolver. The net-new build is the orchestration plus decomposition (stage 3), provenance capture (stage 4), claim-to-nanopub generation (stage 5), and the emission surfaces (stage 8). That is the smallest new surface that still yields a machine-first, forkable object.