25 Polynomial Operations
panproto treats schemas as polynomial endofunctors and instances as their initial algebras (W-types). This interpretation yields a family of derived operations from the adjoint triple \(\Sigma \dashv \Delta \dashv \Pi\): fibers, sections, group-by, join, and the internal hom. These operations live in three crate modules: panproto-inst/src/poly.rs (fiber, section, group-by, join), panproto-inst/src/hom.rs (internal hom), and panproto-lens/src/graph.rs (Lawvere metric).
For the user-facing walkthrough of the internal hom and lens graph, see the tutorial.
25.1 Fiber operations
Fibers are the \(\Delta_f\) operation (pullback along a migration \(f: S \to T\)) applied to representable presheaves. Given a compiled migration and a target anchor, the fiber is the set of source nodes that map to that anchor.
25.1.1 fiber_at_anchor
pub fn fiber_at_anchor(
compiled: &CompiledMigration,
source: &WInstance,
target_anchor: &Name,
) -> Vec<u32>Iterates over source nodes, keeping those whose anchor remaps to target_anchor via compiled.vertex_remap. Returns source node IDs. This is \(\Delta_f(\delta_w) = \{v \in S \mid f(v) = w\}\).
25.1.2 fiber_decomposition
pub fn fiber_decomposition(
compiled: &CompiledMigration,
source: &WInstance,
) -> FxHashMap<Name, Vec<u32>>Computes fibers for all target anchors simultaneously. The result is a partition: every source node appears in exactly one fiber. Implemented as a single pass over source nodes, grouping by remapped anchor.
25.1.3 fiber_with_predicate
Combines fiber_at_anchor with a value predicate. Computes \(f^{-1}(a) \cap \{x \mid \mathrm{pred}(x)\}\). The predicate is evaluated against each node’s extra_fields using build_env_from_extra_fields, with additional bindings for _value (if present) and _anchor.
let predicate = Expr::Builtin(
BuiltinOp::Gt,
vec![Expr::Var("confidence".into()), Expr::Lit(Literal::Float(0.8))],
);
let filtered = fiber_with_predicate(&compiled, &inst, &Name::from("span"), &predicate, &config);25.1.4 fiber_at_node
pub fn fiber_at_node(
source: &WInstance,
target: &WInstance,
target_node_id: u32,
complement: &Complement,
) -> Vec<u32>Computes the fiber at a specific node ID in the restricted (target) instance. Finds source nodes in two ways: (a) direct preimage via anchor matching, and (b) nodes from the Complement that were contracted into the target node during ancestor contraction. Requires a Complement from restrict_with_complement.
25.2 Complement tracking
25.2.1 Complement struct
pub struct Complement {
pub dropped_nodes: Vec<DroppedNode>,
pub dropped_arcs: Vec<(u32, u32, Edge)>,
}Records nodes and arcs that were removed during restrict_with_complement. Each DroppedNode stores the original node ID, its anchor, and the surviving node it was contracted into (if any).
25.2.2 DroppedNode
pub struct DroppedNode {
pub original_id: u32,
pub anchor: Name,
pub contracted_into: Option<u32>,
}The contracted_into field records the nearest surviving ancestor. This is None only for unreachable nodes.
25.2.3 restrict_with_complement
pub fn restrict_with_complement(
instance: &WInstance,
src_schema: &Schema,
tgt_schema: &Schema,
migration: &CompiledMigration,
) -> Result<(WInstance, Complement), RestrictError>A BFS-based restrict that produces both the restricted instance and a Complement. The BFS traversal tracks the nearest surviving ancestor for each node. Surviving nodes are remapped and have field transforms applied; non-surviving nodes are recorded in the complement.
This differs from the Complement in panproto-lens/src/asymmetric.rs, which tracks dropped nodes, arcs, fans, contraction choices, and original parents for the bidirectional lens. The poly.rs complement is lighter-weight and focuses on fiber operations.
25.3 Section construction
A section of a projection \(p: S \to T\) is a morphism \(s: T \to S\) such that \(p \circ s = \mathrm{id}_T\). In instance terms: an annotation layer is a section that enriches a base document with additional structure that projects back to the original.
25.3.1 SectionEnrichment
pub struct SectionEnrichment {
pub base_node_id: u32,
pub anchor: Name,
pub edge: Edge,
pub value: Option<FieldPresence>,
pub extra_fields: FxHashMap<String, Value>,
}Each enrichment specifies a new node to attach at a specific position in the base instance. The anchor must be a vertex in the source schema but not in the target schema (otherwise it would already be present in the base).
25.3.2 section()
pub fn section(
base: &WInstance,
projection: &CompiledMigration,
enrichments: Vec<SectionEnrichment>,
) -> Result<WInstance, InstError>Constructs an S-instance from a T-instance (the base) plus enrichment nodes. The algorithm:
- Copies base nodes, inverse-remapping anchors from target to source schema.
- Remaps arcs to use source-schema anchors.
- Adds enrichment nodes with fresh IDs, connected to base nodes via the specified edges.
The roundtrip property holds: restrict(section(base, projection, enrichments), projection) ≅ base. The restricted section has the same nodes and anchors as the base.
25.4 Group-by and join
25.4.1 group_by
pub fn group_by(
compiled: &CompiledMigration,
source: &WInstance,
) -> FxHashMap<Name, WInstance>Partitions source nodes by their migration image, returning a sub-instance for each target anchor. Internal arcs between nodes in the same fiber are preserved. This is the dependent product \(\Pi_f\) computed explicitly on trees.
Implementation: calls fiber_decomposition, then extract_subinstance for each fiber. extract_subinstance copies the specified nodes and retains only arcs where both endpoints are in the fiber.
25.4.2 join
pub fn join(
left: &WInstance,
right: &WInstance,
left_compiled: &CompiledMigration,
right_compiled: &CompiledMigration,
) -> Vec<(u32, u32)>Given two instances A and B with migrations to a shared target C, computes the pullback \(A \times_C B\): all pairs \((a, b)\) where \(a\) and \(b\) map to the same target anchor.
Implementation: decomposes both instances into fibers via fiber_decomposition, then takes the Cartesian product of fibers at each shared anchor. If A has 2 nodes in the "span" fiber and B has 3, the join produces 6 pairs for that anchor.
25.5 Internal hom
The internal hom [S, T] is a schema whose instances represent structure-preserving maps from S to T. Implemented in panproto-inst/src/hom.rs.
25.5.1 hom_schema
pub fn hom_schema(source: &Schema, target: &Schema) -> SchemaFor each source vertex v:
- Creates
choice_v(kind"hom_choice") withmaps_toedges to all compatible target vertices (those sharing the same kind). - Creates
backward_v_e(kind"hom_backward") for each outgoing edgeefromv, with abackward_foredge fromchoice_v.
Target vertices are included in the hom schema so that maps_to edges have valid endpoints. Compatibility is determined by the kind field on vertices.
25.5.2 curry_migration
pub fn curry_migration(
compiled: &CompiledMigration,
source_schema: &Schema,
hom_schema: &Schema,
) -> WInstanceEncodes a compiled migration as an instance of [S, T]. Creates a synthetic hom_root node, then for each entry in vertex_remap:
- Creates a
choice_vnode. - Creates a target vertex node and connects it via a
maps_toarc. - For each outgoing edge from the source vertex, creates a
backward_v_enode with extra fields encoding the source and target edge metadata (src_src,src_tgt,src_kind,src_name,tgt_kind,tgt_name).
25.5.3 eval_hom
pub fn eval_hom(
hom_instance: &WInstance,
source_instance: &WInstance,
source_schema: &Schema,
target_schema: &Schema,
) -> Result<WInstance, InstError>Evaluates an instance of [S, T] against an instance of S to produce an instance of T. Three steps:
- Extract vertex remap from choice nodes: scan for nodes with
choice_-prefixed anchors, followmaps_toarcs to read the target anchor. - Extract edge remap from backward nodes: scan for
backward_-prefixed anchors, read source/target edge metadata from extra fields. - Build a
CompiledMigrationand applywtype_restrict.
The roundtrip property holds: eval_hom(curry_migration(m, S, [S,T]), x, S, T) = wtype_restrict(x, S, T, m).
25.6 Lawvere metric
The Lawvere metric on lens graphs is implemented across two modules in panproto-lens.
25.6.1 complement_cost (cost.rs)
pub fn complement_cost(complement: &ComplementConstructor) -> f64Assigns a numeric cost to each ComplementConstructor variant:
| Variant | Cost |
|---|---|
Empty |
0.0 |
DroppedSortData |
1.0 |
DroppedOpData |
1.0 |
AddedElement |
0.5 |
NatTransKernel |
10.0 |
Composite(children) |
sum(children) |
chain_cost sums the complement costs of all steps in a ProtolensChain.
25.6.2 LensGraph (graph.rs)
pub struct LensGraph {
schemas: Vec<Name>,
schema_index: FxHashMap<Name, usize>,
edges: FxHashMap<(usize, usize), (f64, ProtolensChain)>,
distances: Option<Vec<Vec<f64>>>,
next: Option<Vec<Vec<Option<usize>>>>,
}A weighted directed graph of schemas. Key methods:
add_schema(name): registers a schema, returns its index.add_lens(src, tgt, chain): adds an edge with costchain_cost(&chain). Auto-adds schemas. Keeps the cheaper edge when duplicates exist. Invalidates cached distances.compute_distances(): runs Floyd-Warshall. Initializes identity distances to 0, direct edges to their costs, then relaxes via the standard triple loop.preferred_path(src, tgt): reconstructs the shortest path from thenextmatrix. Returns(cost, ProtolensChain)by composing the chains along the path.distance(src, tgt): returns the shortest distance, orf64::INFINITYif unreachable or distances not yet computed.
The Floyd-Warshall implementation is \(O(n^3)\) where \(n\) is the number of schemas. This is acceptable because lens graphs are typically small (tens of schemas, not thousands).
25.7 WASM boundary
Five WASM exports in panproto-wasm/src/api.rs expose polynomial operations:
| Export | Input (MessagePack) | Output (MessagePack) | Calls |
|---|---|---|---|
fiber_at |
instance, migration, anchor name | Vec<u32> |
inst::fiber_at_anchor |
fiber_decomposition_wasm |
instance, migration | HashMap<Name, Vec<u32>> |
inst::fiber_decomposition |
poly_hom |
source schema, target schema | schema bytes | hom::hom_schema |
preferred_conversion_path |
edges, src name, tgt name | (cost, chain) |
LensGraph::preferred_path |
conversion_distance |
edges, src name, tgt name | f64 |
LensGraph::distance |
preferred_conversion_path and conversion_distance both construct a LensGraph from the provided edges and call compute_distances() before querying. The graph is not cached across calls; each invocation builds it from scratch.
25.8 TypeScript SDK
The TypeScript SDK wraps these WASM exports in three modules:
fiber.ts: exports fiberAt(instanceBytes, migrationBytes, anchor, wasm) and fiberDecomposition(instanceBytes, migrationBytes, wasm). Both serialize inputs as MessagePack, call the WASM function, and deserialize the result.
hom.ts: exports polyHom(sourceBytes, targetBytes, wasm), returning the serialized hom schema.
graph.ts: exports preferredPath(edges, src, tgt, wasm) returning { cost: number, chain: ProtolensChain }, and distance(edges, src, tgt, wasm) returning a number. The GraphEdge type describes an edge as { src: string, tgt: string, chain: ProtolensChain }.
All functions accept and return MessagePack-encoded bytes for the heavy data (instances, schemas, migrations). Schema and anchor names are passed as plain strings.
25.9 Source locations
crates/panproto-inst/src/poly.rs:
Complement,DroppedNode,SectionEnrichmentstructsfiber_at_anchor,fiber_decomposition,fiber_with_predicaterestrict_with_complement,fiber_at_nodegroup_by,join,sectionextract_subinstance(private)
crates/panproto-inst/src/hom.rs:
hom_schema,curry_migration,eval_homadd_maps_to_edges,add_backward_vertices,edge_label(private)
crates/panproto-lens/src/cost.rs:
complement_cost,chain_cost
crates/panproto-lens/src/graph.rs:
LensGraph,LensGraph::add_schema,LensGraph::add_lensLensGraph::compute_distances,LensGraph::preferred_path,LensGraph::distance
crates/panproto-wasm/src/api.rs:
fiber_at,fiber_decomposition_wasm,poly_hompreferred_conversion_path,conversion_distance
sdk/typescript/src/:
fiber.ts:fiberAt,fiberDecompositionhom.ts:polyHomgraph.ts:preferredPath,distance