Skip to content
DarkPrint
Aautogenschema-forge-etlPublic

Schema Forge ETL

Get blueprint

Download

schema-forge-etl-1.1.0.tgz

Every file of this release in one archive, ready to unpack into a folder.

Clone

npx -y darkprint clone autogen/schema-forge-etl

There is no repository and no history behind a release. Download hands you its files as they stand, and Clone fetches the same files by name.

release 1.1.0 · digest sha256:2456eac8 · 2 releases · published Sep 11, 2026

Extracts, normalizes and schema-validates messy documents, repairing anything that fails validation before it ever reaches the store.

Tool capabilities2
  • SQL
  • Messaging

Node raw, Document Intake. DOT line 5, card document-intake@1.0.0. 7 nodes in the graph.

Selectedrawtopology.dot:5document-intake@1.0.0

1 The graph

7 nodes · 7 edges
Hears from
nothing in this graph
Sends to
extract

2 The card skeleton

document-intake@1.0.0Open card →
iddocument-intakeline 1Filled.

The key a blueprint pins. A DOT node carries card="id@version", and this string is the only way a graph and a card find each other. It may be namespaced, berti/solver-a, so two authors can publish a card under the same short name.

nameDocument Intakeline 2Filled.

What a person calls the node. The drawing prints it and every listing leads with it. Nothing matches on it: that is the id's job, and the two are free to disagree.

typetoolline 3Filled.

Exactly one node-type term from the vocabulary. It says what kind of work the node does and whether a person acts at it: a type under human-in-the-loop, such as human-gate, holds the run until somebody acts; every other type lets it pass. The static analysis groups nodes by this term. Either way it is the author's design choice; nothing here is measured.

phaseoutside the fivenot writtenLeft empty by this card.

Which of the five phases the node stands in, any number of them. The phases describe a blueprint's shape rather than placing every node in one. Intake, retrieval and routing are real work that none of the five names, so declaring none is a valid answer.

actionOpen the run on a batch of unstructured source documents, stamp each one with its origin and content hash, and hand the batch downstream as the unit of work everything else is keyed to.lines 5–7Filled.

The operation, in one line, short enough to read off a drawing. It is prose for whoever opens the card. The agent is instructed by spec, and nothing parses this line.

specCollect the source documents waiting in the intake location and open a run over them, in batches of at most 100 documents. For each document record three things, the origin path it came from, its full content, and a stable hash of that content, and emit the batch on documents as one entry per document. Use that hash as the document's identity for the rest of the run: bytes you have seen before must hash to the same value, because that is what lets a re-run land on the record already written instead of a second copy of it. You are done when every document in the intake location appears in exactly one emitted batch.115 wordslines 8–15Filled.

The instructions handed to the agent when someone runs the graph on their own machine. They must stand on their own, because the agent never sees the rest of the graph. They must not carry what the graph withholds: if the text supplies information no edge brings to this node, the isolation the graph draws exists only on paper. One check reads this field, card/spec-too-thin, and it only measures length.

modelnot namednot writtenLeft empty by this card.

Which model the agent is instantiated with, written the way the provider writes the identifier. A default rather than a binding: a graph's model_stylesheet sets the model for every node matching a shape, an explicit field here outranks the sheet, and whoever runs the blueprint outranks both. Absent on most cards, which means the node takes whatever the graph or the runner supplies.

agentnot namednot writtenLeft empty by this card.

A label the card's author chose for the agent behind the node. Nothing checks it.

skillskills/document-intake.mdline 19Filled.

Where the written procedure for this agent lives, as a path inside the repository you run from. It is only a pointer: no skill document travels in a DarkPrint bundle, and you write the file it names. A node whose spec field holds the whole instruction declares none.

toolsnone requiredline 16Left empty by this card.

tool capability terms from the vocabulary: what the node is permitted to do. This does not say which server supplies the capability; the mcp field answers that. A node can carry either field without the other.

mcpfilesystemlines 17–18Filled.

The MCP servers the node reaches, under the names they are registered with on the machine that runs the graph. Free text by design: an MCP server is a process somebody installed, and the vocabulary has no term for one.

paramsbatch_size: 100lines 20–21Filled.

Nested configuration for the node, free-form but JSON-serializable. It travels with the card to whoever runs the graph; nothing on this site interprets a key of it.

inputsnothing arrives on a declared portline 23Left empty by this card.

The ports data arrives on, each with a name and a data-type term from the vocabulary. Every incoming edge is checked against them: an edge whose source produces nothing this node accepts is reported as bundle/type-mismatch.

outputsdocuments: jsonlines 24–27Filled.

The ports data leaves on. An output type makes an edge into the next node meaningful. The next node's declared inputs and prohibitions are checked against it.

documents: json. The batch, one entry per document, with its source path, content and hash.

dependenciesno upstream node is namedline 28Left empty by this card.

Cards this one expects to hear from, by id. This is the one card field that names topology, so it can point back at a line in the DOT on its own. The DOT still decides what is wired; this field says what the author expected to be wired.

cannotno type is refusedline 29Left empty by this card.

Data types that must never arrive, written as vocabulary term ids. Every incoming edge is checked against every entry, and an edge that can carry that type, or a narrower one, is refused as bundle/prohibition-violated. This is the half of the node's refusals that the checker enforces; will_not is the other half, which nothing checks automatically.

will_notemit the same document in two batcheslines 30–31Filled.

What the node promises never to do, in the author's own sentences. Nothing checks an entry here, and nothing can: a promise like "never opens a shell" cannot be read off a graph. It is addressed to whoever reads the card and to the agent instantiated from it, which is why it is a field of its own rather than a second kind of entry inside cannot.

risk_markersnone declaredline 33Left empty by this card.

risk-marker terms the author declares for the node. The static analysis subtracts each distinct marker's weight from the blueprint's static risk-exposure reading, once per blueprint however many nodes carry it. An empty list is a valid answer. Three markers are also read off the graph whether or not a card declares them: unbounded loops, unvalidated external access and criteria leaks.

notesThe content hash is what makes a re-run idempotent: a document that arrives twice resolves to the same record key at the store and upserts over itself rather than duplicating.30 wordslines 34–36Filled.

The author's commentary on the card, addressed to whoever reads it. Nothing checks it.

version1.0.0line 38Filled.

Semver of the card itself. A published version is never edited in place, so a pinned id@version means the same content forever and a change ships as a new version beside it.

authorautogenline 39Filled.

Who wrote the card. It is excluded from the card's digest, along with provenance. Two cards describing the same node are the same card, whoever typed them.

provenancenot statednot writtenLeft empty by this card.

Where the card came from when it did not start here, such as the blueprint it was forked from or the document behind it. Free text, and excluded from the card's digest.

filled, left empty; both are valid. The skeleton is the card’s fields with what each one is for: open a row to read it. Long values are cut at two lines until the row is opened. This card is pinned by raw at topology.dot line 5.

Files

AautogenExtracts, normalizes and schema-validates messy documents, repairing anything that fails validation before it ever reaches the store.sha256:2456ea…Sep 11, 2026
  • README.mdwhat this blueprint is, its digest, and how to run it
  • cards/7 pinned cards, one document each
  • topology.dotthe topology, as the author wrote it
9 entries · 7 pinned cards inside cards/Open README

README.md

Schema Forge ETL

Extracts, normalizes and schema-validates messy documents, repairing anything that fails validation before it ever reaches the store.

blueprint      schema-forge-etl
bundle digest  sha256:2456eac8705cc365d1e612d6413bb1eaafb11f5c111eab73ff0f06bdda209f2c
nodes          7
cards pinned   7

The digest is taken over topology.dot and the digest of every card version pinned in it. Recompute it to confirm these files are the ones DarkPrint read. One changed byte gives a different digest.

Run it

This runs on your machine. DarkPrint hands out the files and analyses them statically. It executes nothing and holds none of your provider keys.

This folder carries the topology and its pinned cards, nothing compiled. To compile them into a pipeline a graph runner takes, run darkprint export <dir> --attractor. It writes Attractor DOT to stdout, and that file opens with the same two lists this README carries under What these files leave to the runner. Adapting the result, or building the run yourself from these files instead, is your own harness's job.

4 of the 7 nodes name the model they run on, in their card's own model field. Read it off cards/<ref>.yaml; whether your harness honours it is yours to decide.

What is in the folder

topology.dot   node ids, edges, and the card version pinned on each node
cards/         the pinned cards, as the registry stores them; each carries the `spec` that becomes its node's prompt
README.md      this file

7 of the nodes in this bundle name a skill document. There is no skills/ directory above and there is not meant to be: DarkPrint stores the pointer and reads nothing at the other end of it. The paths are relative to the repository you run this blueprint from, and writing the documents is yours to do. Nothing here needs them to run, because every card carries its own spec inline.

raw         skills/document-intake.md
extract     skills/field-extractor.md
normalize   skills/field-normalizer.md
validate    skills/schema-gate.md
repair      skills/record-repairer.md
store       skills/record-store.md
publish     skills/dataset-publisher.md

What these files leave to the runner

Attractor reads more attributes than a DarkPrint blueprint has fields to set. Compile these files into a pipeline, by the command above or by hand, and the names below are the ones nothing in this folder sets. Write them in where your run needs them, and expect a later export of this blueprint to overwrite the whole compiled file. Appendix A of the Attractor spec tabulates most of them; the rest are named by the retry rules in §3.5, by the handler pseudocode in §4, and by §9.7's tool call hooks.

Left out, these fall to the runner and the pipeline still runs. The Attractor spec states a value or a behaviour for each one's absence, in Appendix A or in the handler pseudocode that reads it, so what you get is a choice nobody in this folder made:

  • graph: model_stylesheet, default_max_retries, default_max_retry, default_fidelity, retry_target, fallback_retry_target, stack.child_workdir, tool_hooks.pre, tool_hooks.post
  • node: goal_gate, retry_target, fallback_retry_target, fidelity, thread_id, timeout, llm_provider, reasoning_effort, auto_status, allow_partial, join_policy, max_parallel, manager.poll_interval, manager.max_cycles, manager.stop_condition, manager.actions, stack.child_autostart, tool_hooks.pre, tool_hooks.post
  • edge: fidelity, thread_id, loop_restart

Left out, these have nothing to fall to. The handler a node's shape selects reads each one directly, and with no value it refuses or goes round again while the rest of the compiled file reads as though the node would run. Read §4's handler section for the shape you are compiling before you leave one of these unset:

  • graph: stack.child_dotfile
  • node: human.default_choice

The nodes

nodecard
rawdocument-intake@1.0.0
extractfield-extractor@1.0.0
normalizefield-normalizer@1.1.0
validateschema-gate@1.1.0
repairrecord-repairer@1.1.0
storerecord-store@1.0.0
publishdataset-publisher@1.0.0

Exported from https://www.darkprint.io/blueprints/schema-forge-etl

History

each release is a frozen snapshot, addressed by its digest
  • 1.1.0sha256:2456ea…latest

    Extracts, normalizes and schema-validates messy documents, repairing anything that fails validation before it ever reaches the store.

    autogen · Sep 11, 2026

  • 1.0.0sha256:c2f47c…

    Extracts, normalizes and schema-validates messy documents, repairing anything that fails validation before it ever reaches the store.

    lupo · Aug 30, 2026

There is no repository behind a release, so there is nothing to pull.

Community notes (0)

No notes yet.

Nobody has posted about this blueprint yet.

Sign in to post a note.