Your ASTs were rows all along.

A family of extensions loads hierarchical data into ordinary DuckDB tables: sitting_duck (ASTs, 27 grammars), webbed (XML/HTML), panduck & duck_block_utils (Markdown, DOCX, PDF). No pointers, no nested types, no document store — rows. Here is the same pattern from two very different producers.

A Markdown file through read_markdown_blocks becomes a block per row: type, nesting level, source order, content — the duck_block table format that duck_block_utils, panduck, and the markdown extension all share. A Python file through sitting_duck becomes a node per row (output schema): type, depth, DFS order — plus native extras like descendant_count — and CSS selectors already query it. Compare the two schemas and the same skeleton appears in both: rows in traversal order, a nesting column, a type column. Everything else — ids, attributes, counts — varies by producer. That shared skeleton is not a coincidence, and taking it seriously is what the rest of this lesson does: first by hand (Lessons 1–3), then as a contract (Lesson 4), then as syntax (Lessons 6–8).

That is also the order the work happened in, and the order is the argument. We started with ASTs and CSS: sitting_duck put selectors on source code, and it worked. Then we went backwards, to where CSS came from — HTML — and brought along everything already modeled as duck_blocks: Markdown, DOCX, PDF, not just HTML, with no conversion to or from HTML at any point. The duck_block tree handled it as it stood. Your documents were rows too. Only then did we ask the question this lesson is organized around: what do ASTs and duck_blocks actually need to be trees, what does CSS need on top of that to have anything to say, and what is there only to make it fast? Those three answers are R, S, and O.

Start with the table function read_ast('src/**/*.py'): it returns your codebase as rows — node_id, parent_id, type, depth, descendant_count, semantic_type — across 27 tree-sitter grammars. The taxonomy is the point: semantic_type_to_string(semantic_type) = 'DEFINITION_FUNCTION' means function-definition however your grammar spells it, so one query spans Python, Rust, and JavaScript without editing a type name.

And ast_select puts thirty years of muscle memory on top: .fn is a semantic class, #name an id, :not/:has what you already know from CSS. Custom pseudo-classes are SQL macros — :docblock is a macro whose body is itself a selector, so grammar-specific structure hides behind one portable name.

Node IDs are depth-first order, so structure is adjacency: anchored at a body block, a documented function is three consecutive rows — block, expression_statement, string — and DuckDB's window machinery reads it (with parentage verified, not just position). First-child is an aggregate: min(node_id) per parent; arg_min(type, node_id) reads its type.

This encoding has a name and a history: pre/size/level, from the XML-database era — Grust's XPath accelerator, MonetDB/XQuery's Pathfinder. sitting_duck's node_id / descendant_count / depth is that table column for column, converged on independently. That history pays off directly: DuckDB's recently optimized recursive CTEs (USING KEY, from the same Tübingen group) are about to matter below.

The selector engine never needed pointer-linked trees. The tree is real; the pointers were optional — what it consumes is rows under three requirements. R1: order matters. R2: a structural basis of level | parent — either one, each derivable from the other. A level column obeys level[n+1] ≤ level[n]+1 (descend at most one, pop freely — that invariant alone characterizes a forest); a parent column derives (order, level) via one recursive CTE, needing only a sibling sort key — without one the source conforms to the sibling-free profile: +, ~, :first-child refused legibly, everything else intact. That's the whole of it: R1 + R2 are the minimal treeness set — everything else in the contract is about what you can say or how fast you can say it. A type accessor isn't required for trees at all, only for CSS-style selection — so it lives in S (as S0°, defaulting every node to type node), and a typeless tree still matches structurally. And R0°, spelled ROOT (col, …), is required iff the relation holds more than one tree: tree-identity columns, constant per tree — delimiting forests, keying tree-granular DML (INSERT OR REPLACE by ROOT = re-derivation; never row surgery), feeding ATTACH. Absent, the relation is one tree; per-tree order resets without ROOT are rejected outright. Accessors may be bare column reads.

First, the base slot: S0°, the type accessor — total, defaulting every node to type node, because selection needs a type dimension even when trees don't; declare TYPE and type selectors light up. S is the expressive surface: id accessors (#main), classes (.fn), and attributes in two tiers — named accessors, declared per attribute and typed (numeric and date operators live here; [start_line>100] needs a real INT), and a map catch-all for the long tail, string ops only. Named wins; the map serves undeclared names; neither declared → refused legibly. Rule of thumb: hot attributes earn names; the map is for the tail, not the hot path. And the extension mechanism: custom pseudo-classes as pure SQL macros — the macro is the semantics — with per-tree binding: a tree declares its pseudo map (or a prefix), shadowing a shared sel_* library tier, so :docblock can mean the right structure per grammar without collision. The pseudo name is the method, the tree is the type, the macro is the instance — and ambiguity is refused at registration, not discovered at match time. Some predicates are just named selectors: :docblock is a macro whose body is itself a selector, which is where grammar-specific structure goes to hide behind one portable name.

O accessors buy speed: the non-native basis member, subtree size, child count, next sibling (pre+size+1). Each has a default derivable from (order, level); native columns register as overrides — and every override must test-equal its default. An override that changes a match set is not a fast path; it is a corrupted encoding, flagged at that severity.

The refinement that turns this from rule into theory: the R/O split has a criterion. Type, id, classes are row-local — they depend on nothing but the row. Every O member is row-transcendent: its value summarizes other rows, and that dependence is exactly what scopes it. O values are namespaced to (taxon, load-watermark) — valid within their own tree's materialized extent, never across an attachment splice (a parent tree's descendant_count cannot see attached forests), never above an unloaded frontier. A namespace-checked read yields a Bound; a mismatched read yields an Estimate — estimates may inform join order, never results. The XML era fought this as the dynamic-update problem and answered with renumbering; the contract answers with locality.

Trees link to other trees, and the linkage has laws. W1: attachments are root-restricted foreign keys — child-tree roots reference parent-tree nodes, interior nodes never do (that restriction keeps ancestry a forest, not a graph), and the child root splices in at level(parent)+1. W2: loaders — demand materialization with a tri-state frontier (loaded / unloaded / absent; the distinction matters). Both are parent-parameterized: bindings are deterministic functions of the parent row's local columns — a code fence's language attr selecting the parser is the archetype. W3: epoch manifests, provenance keyed by (parent, params), refresh by selector.

The governing law, W-locality: within-tree match sets are invariant under any change to W. Attach, detach, load, refresh — a single tree's answers never move. W belongs to binding, planning, and provenance; never to semantics. And dangling references are drift telemetry, not errors: absence at a frontier is a fact.

Now read the four block letters back. We taught them in dependency order — R structure, S speech, O speed, W world — because each stratum leans only on the ones before it. Rearrange the initials and they spell the contract's name: ROWS — Required, Optimizations, World, Semantics. Nobody planned the acronym; it was found — which is how you know the factoring is right.

sitting_duck v2 is splitting into a pluggable stack — languages as packages over a sitting_duckling core — and the contract sits a level below even that, as its own extension (working names: duck_nest / duckent) that sitting_duck and duck_block_utils both build on. Documents conform by construction: blocks already ship as ordered, leveled rows, so section h2 + table is the same machinery pointed at markdown. And the parent-basis relaxation admits a third source: every adjacency list in your warehouse — org charts, category trees, BOMs, parent-pointer file hierarchies — conforms via one recursive CTE, which USING KEY just made fast.

DuckDB 2.0's runtime-extensible PEG parser lets extensions add syntax and reuse the rest of SQL — so the ROWS contract becomes DDL. The shape specification is the contract, slot for slot, and it scales from two lines to full dress. Minimal: one slot makes rows a tree — type defaults to node, so * and every structural combinator already work. Typical: an abstract tree (SHAPE ONLY — required whenever there's no AS, so sourcelessness is declared intent, never an accident) instantiated with LIKE, which copies semantics, never source — the universal LIKE contract. Full dress adds the S tiers and O declarations.

Read the full-dress figure as a tour of the blocks: ROOT is tree identity (R0); LEVEL/TYPE the required basis; ATTR the named/typed tier (bulk-bindable via COLUMNS('a_(.*)') AS '\1') over an ATTRS MAP catch-all; PSEUDO binds pseudo-classes per tree — the value is an ordinary expression in the tree's row scope, exactly like a generated column or CHECK constraint, so calling a macro is just calling it: docblock := has_docblock(file_path, node_id). No lambda syntax, no binding machinery — SQL already knows how to evaluate an expression per row. A PSEUDO PREFIX pulls in a whole shared library, shadowed by anything tree-local. And every OPTIMIZE annotation auto-generates its conformance assertion: an O override that diverges from its derivable default is flagged as corruption, not speed. Miss a capability and the refusal is legible at bind time: "tree org declares no CLASSES accessor."

Think of MATCH as Lesson 3, automated. The selector compiles into exactly the machinery you hand-wrote there — the QUALIFY windows, the grouped firsts, the anti-joins — generated against whatever a tree's shape declares. That's what "abstracts sitting_duck" means precisely: sitting_duck's ast_select is the special case where the basis is level, the O columns are native, and the vocabulary is the code taxonomy; MATCH is the same compilation, parameterized by any shape. Semantics consult declarations — R, S, and attachments — never state, so the answers can't depend on caches or load timing.

And the two bases earn their existence here. Level basis is what a producer emits when it already walks the structure — a parser streams DFS order and depth for free, which is why sitting_duck stores node_id/depth/descendant_count natively (pre/size/level: deep trees, structural queries daily, so the O columns are worth storing). Parent basis is how the world's data already sits — every org chart and category table is an adjacency list — and the recursive CTE that DFS-numbers it (fast under USING KEY) is the encoder. You can write it once to understand it; then the DDL internalizes it and you never write it again.

MATCH is a clause, not a function. Bare, it flows straight into the pipeline you already know — FROM … USING TREE … MATCH … SELECT … WHERE … GROUP BY — terminated by the next clause keyword, scanned outside balanced brackets. No outer string literal, so quote-doubling dissolves. And because the shape projection is the matcher's view, not the result type, MATCH emits raw rows: the WHERE after it is ordinary SQL filtering on everything the source knows, no attribute macros required.

When attribute operators run out — subqueries, BETWEEN, real expression trees — SQL does not go into CSS. CSS stays wholly portable: it extends by pseudo-classes, and minting one is a declaration, not a hack — PSEUDO (busy := child_count > 2) and :busy exists. The host door is a second language on the same matcher: MATCH USING TREEQL, one step per line, each a combinator keyword plus parenthesized clauses whose keywords are the S slots — TYPE, ID, CLASS, ATTR, PSEUDO — plus a native WHERE that is arbitrary SQL against that step's projection row (closed shapes still refuse undeclared columns: power added, visibility unchanged). AS y names a step, and a named step is a joined alias you can read dotted in the SELECT after the match. The boundary doctrine, quoted in every refusal: CSS extends by pseudo-classes; TREEQL integrates the host; never both in one language.

One last collision, from our own toolchain: the taxonomy's 28th grammar is SQL itself, whose trees contain select nodes. Keyword-typed selectors quote like identifiers — MATCH "select" > "from" — and the pipeline keeps flowing. And TREEQL is more than language number two: every registered language normalizes to it, EXPLAIN MATCH prints it, and cross-language differentials compare its normal forms.

Forests and DML. ROOT makes DML tree-granular: append trees, drop trees by key, and INSERT OR REPLACE is re-derivation — re-parse a file, replace its tree, epoch bumps. Within-tree row surgery doesn't exist; the 2004 renumbering problem is answered by granularity: renumbering never crosses a partition you didn't wholly replace.

Attachments. Trees link by root-restricted foreign key — child roots reference parent nodes, interior nodes structurally can't (their only cross-tree coordinate is ROOT, constant per tree) — splicing in at level(parent)+1, with bindings that are deterministic functions of the parent row: a code fence's language attr selecting the parser is the archetype. The first thing attachments buy is ontology mapping: shared reference trees — NCBI's tree of life, WordNet, HS tariff codes — attached many-to-one, every row that names a concept keying into one taxonomy, with subtree membership as a pseudo-class; if you've ever shipped a closure table, you already built that tree's O-block. The second is the analytics workhorse, hierarchy × facts: every subtree rollup today is the same three-part boilerplate — recursively collect descendant ids, semi-join the fact table, aggregate — rewritten per hierarchy, per question. Attach the facts to the tree once and the boilerplate becomes one selector plus one aggregate. (Chains extend arbitrarily far — network to service to file to AST — each boundary a loader; the details are in the field guide.)

Virtual trees. Trees too large to materialize whole: materialize two levels, load branches on demand — selector pushdown decides which branches, unloaded is distinct from absent, and matching must equal the fully-materialized ideal within the selector's scope (so demand loading is an evaluation strategy, never a semantics change). Refresh is by selector; every load carries an epoch manifest.

As of this revision, M0, M1, and M1½ run. The duckent repository holds them as SQL macros under sql/: a catalog of shapes and trees, the projection compiler onto the canonical columns, both basis derivations, forest DML with P13 checked on ingest, and a TREEQL match compiler with captures. Every mutating operation is a pure tree_compile_* macro that returns SQL text; a small Python runner executes it today, a bound table function will in the extension. Eleven sqllogictest files pass and nine planted mutants die, and the same .test files are meant to run unchanged against the C++ build.

Building it settled things the design could not. The derived _parent and _size equal sitting_duck's native parent_id and descendant_count on every row of both parquet fixtures, which is the O law holding on real data. DuckDB's query() refuses text from any macro whose body contains a SELECT, which is why the compilers that must read the catalog are executed by the runner rather than fed to it. A partitioned row_number() without ORDER BY does not preserve scan order, so frozen order captures the scan sequence first. And the harness did its job: two mutants survived their first run, and the answer was a multi-root test and a re-scoped mutant, never a manifest edit.

The review that gated the merge found the defect the tests had not: tree_ddl_alter rebuilt a tree from its create-time source and silently dropped every partition ingested since. It now refuses, naming the partitions and the way around. That is doctrine 3 and doctrine 4 in one finding, and it is the kind of thing the FINDINGS file exists to record.

Next is M2: nested HAS and NOT groups through a recursive fold over the selector IR, the CSS front-end lowering tree-sitter-css output to the same IR, and the differential against shipped ast_select on every selector both support. Then M3's O layer, M4's introspection and generated cheatsheet, and the TREEQL text parser. The C++ port replaces the derivations' recursive CTEs with one stack walk over pre-order, which is what "a tree is its own traversal" means operationally. If you want to help, the spec and the plan are in docs/superpowers/, the open decisions are where a second opinion is worth the most, and issues and pull requests are welcome; so is a note.

Text-to-ASTCSS. Text-to-SQL is a twenty-year benchmark field because SQL had the corpus; Text-to-XPath died with XML. Selectors arguably out-corpus SQL — every frontend tutorial since 2006 — and they fit in a grammar file, so a small local model can be constrained-decoded: it cannot emit an invalid selector, and spends its whole budget on which one. “Which functions are missing docstrings?” → .fn:not(:has(:docblock)). An execution-verified corpus (the engine as the verifier) is in progress. If you want to know more about Text-to-ASTCSS, message me: github.com/teaguesterling, or open an issue on duckent.

CSS is a policy engine. We've been using half of CSS: selectors are the address system, but CSS is selectors plus declarations plus a cascade — rules about what they select, conflicts resolved by specificity. Two lines make a permission system for code edits (see aside). The rest — worlds, audits-as-anti-joins, delegation — is in the companion field guide; this lesson only points at it. If you want to know more about CSS as a policy engine, look at umwelt, and at the selector surfaces it rides on: sitting_duck, pluckit, squackit. Or send me a note.