pse-arrow

An Arrow-native process systems engineering core in Rust: typed relations as the only model authority, a relational math IR, DataFusion-based inference, native NLP solving, and a generated Pyomo backend. It is a clean-room re-implementation of core IDAES-PSE capabilities, parity-tested against idaes-pse 2.12.0, and it is not affiliated with IDAES — see Relationship to IDAES.

The distribution is pse-arrow on PyPI (the name pse is taken); the import name is pse and the crate prefix is pse-. Everything is licensed MIT OR Apache-2.0.

Where authority lives

When the code and a plan disagree, the code is what runs. When the code and the blueprint disagree, one of them is a bug, and the ADR says which.

Reading order

1. Orientation — this page, then Relationship to IDAES.
2. The design — blueprint §0 (purpose and reading guide), then §2 (the fourteen binding decisions D1–D14), then the section §0.3 points you at for what you are about to do.
3. The decisions — the ADR index. ADR-0004 … ADR-0017 are D1–D14 one record each; ADR-0018 … ADR-0038 are the repository, supply-chain and process decisions.
4. The evidence — the capability maps before believing any claim about what Arrow, DataFusion or a supporting crate does at the pinned version. Every claim in this book carries a charter §D evidence label; Proposed means exactly that.

Evidence vocabulary

Every claim in these documents is labelled with the charter §D vocabulary — Proposed, Interface-checked, Implemented, Tested, Measured, Formally established. The labels describe different claims, not one ladder: a measured implementation can still be incorrect, and an interface-checked design can still need substantial engineering. The definitions are in design_review/design_principles/DATA_MODEL_DESIGN_CHARTER.md §D.

Relationship to IDAES

Not affiliated

pse-arrow is not affiliated with, endorsed by, or sponsored by the IDAES project, the U.S. Department of Energy, or any of the institutions of the IDAES consortium. “IDAES” is used here only to name the project this one is compared against, which is nominative use; any trademarks are the property of their owners. Nothing in this repository is an IDAES release, an IDAES fork, or an IDAES-supported product, and questions about it should come here rather than to the IDAES project.

IDAES-PSE is Apache-2.0 licensed. pse-arrow is MIT OR Apache-2.0.

Clean room

The architecture is a re-implementation, not a translation:

• external/idaes-pse is a reading copy, checked out at the pinned parity tag by scripts/fetch-external.sh and gitignored. It is read to understand behaviour — what a control volume writes, which defaults a solver profile carries, what an initializer does in what order.
• No IDAES code is copied into this repository. No file, no function body, no docstring. The pull-request template carries a clean-room attestation, and external/ is on the agent configuration’s deny list for edits.
• The only coupling is the parity harness (python/pse/parity/), which imports idaes in a separate environment and compares results.
• Names are preserved where parity requires a two-way mapping — the property name enth_mol_phase, the enumerations listed in blueprint §6.14, the IDAES default thresholds in §15.1, and the external-function symbol names in §26. Those are interface facts, and each is enumerated in the blueprint rather than absorbed silently.

The architectural decisions are recorded in ADR-0003 (clean-room relationship and the parity pin) and ADR-0032 (external checkouts are fetched, never vendored).

The parity pin

Parity is measured against exactly one IDAES release:

idaes-pse == 2.12.0

It is declared once, in the parity dependency group of pyproject.toml, and it is the version scripts/fetch-external.sh checks out. A lint in governance / adr-lint asserts the two agree, so the reading copy can never drift from what the tests measure against.

The parity environment is a separate resolution (.venv-parity, CPython 3.13) because idaes-pse classifies 3.10–3.13 while the platform package targets ≥ 3.11 and is verified on 3.14. Keeping them separate is what stops IDAES’s transitive dependencies — pydantic, sympy, networkx, pandas — from becoming platform dependencies. The parity pre-flight fails rather than skips when the interpreter, the IDAES version or ipopt is not what the pin says.

How the pin moves

Moving it is a decision, not a dependency bump:

1. Open a PR labelled adr with a new ADR that supersedes the parity-pin part of ADR-0003, stating what changed upstream and what parity results moved.
2. Update the parity group pin and the tag in scripts/fetch-external.sh in the same PR; the lint checks they agree.
3. Re-run python / parity on every supported interpreter and record the differences in the ADR’s Consequences section with a charter §D evidence label.

Dependabot never moves it: idaes-pse is in the ignore list of the uv configuration for exactly this reason (ADR-0035).

Citing IDAES

When citing the project this one is compared against:

Lee, A., Ghouse, J. H., Eslick, J. C., Laird, C. D., Dowling, A. W., Bhattacharyya, D., Biegler, L. T., Burgard, A. P., Miller, D. C., & Omell, B. P. (2021). The IDAES process modeling framework and model library — Flexibility for process simulation and optimization. Journal of Advanced Manufacturing and Processing, 3(3), e10095. https://doi.org/10.1002/amp2.10095

CITATION.cff references the same paper as the parity reference.

What “core IDAES-PSE capabilities” means

The scope is the modeling framework, not the example libraries: the process block lifecycle, unit models and ports, control volumes, the physical and modular property frameworks, reactions, the generic unit model library, costing, initialization, scaling, diagnostics, solvers and DAE discretization, serialization and units, and parameter sweeps. Surrogates, the DMF, the UI, Pecos, the apps and the models_extra libraries are out of scope; surrogate embedding is covered by the kernel contract. Blueprint §0.2 carries the full table, and Appendix A maps every capability to its platform realization and its delivery phase.

Authoritative design

What “authoritative” means

blueprint.md is the design. When an implementation and the blueprint disagree, one of them is a bug, and an ADR says which. Nothing else in this repository has that status:

Section numbers are citation targets

ADRs, reviews, plans, the agent instructions and the lint all cite the blueprint by section: blueprint §14.3, §D7, §3.1. Those numbers are stable. New material is inserted as a sub-section (§14.3.1), never by renumbering, and scripts/adr.py lint resolves every § citation in every ADR against the headings of blueprint.md — so a renumbering fails governance / adr-lint rather than quietly rotting every record that cited it.

The decision behind that rule is ADR-0033.

The amendment rule

The blueprint changes only through a pull request that:

1. is labelled adr and titled adr: ADR-NNNN <title> (or is a named follow-up design: PR referenced by that ADR);
2. adds or changes exactly one decision record under ../adr/;
3. adds a row to the blueprint’s Revision history table naming the revision, the date, the change and the git tag or pull request;
4. adds an inline > Decision: ADR-NNNN line under the heading of every section the decision governs, so a reader of the section finds the record without searching;
5. keeps every existing section number exactly where it was.

An ADR may merge with status: proposed only if the PR is also labelled needs-review. Changes that alter D1–D14, add or remove a crate, add, drop or major-bump a dependency family, change the hashing contract, the Python boundary contract, the metadata conventions or the commit contract, or that deviate from a SHOULD, need an ADR and a design review with an Accept or Accept-scoped verdict before the record becomes accepted. The full table is in ../plans/01-repository-configuration.md §7 and in .codex/skills/adr/SKILL.md.

Revisions so far

Evidence labels are mandatory

Every claim in the blueprint, in an ADR, in a plan’s verification section and in a register row carries a charter §D label — Proposed, Interface-checked, Implemented, Tested, Measured, Formally established. Tested and Measured must name the test or benchmark and the conditions. Proposed is not a failure state; an unlabelled claim is.

Architecture blueprint

status: proposed revision: 6 date: 2026-09-13

Arrow-native IDAES core: detailed design architecture blueprint

Status: Proposed design blueprint, revision 6 (2026-09-13); implementation and numerical acceptance remain open. One file, revised in git: revisions 2 and 3 are the tags design-rev2 and design-rev3 (ADR-0033); the Revision history table below lists every revision. Reviews: revisions 1 and 2 have historical reviews under docs/design_review/reviews/. The revision-4 library-contract review overturned several earlier enforcement claims. Revision 5 incorporates R4-01–R4-13 and L1–L9; §26 records disposition and plan 02 names the remaining implementation gates. Library characterization is evidence about the pinned libraries, not acceptance of the skeletal platform. Follows: docs/authoritative_design/proposal.md (the proposal this document makes concrete) Governing doctrine: docs/design_review/design_principles/DATA_MODEL_DESIGN_CHARTER.md (DM-01–DM-60, gates G1–G7) applied through AGENT_DESIGN_DIRECTIVE.md. The P-numbered principles cited in §2.1 and §22.2 come from the doctrine texts this document was drafted against (semantic_design_principles_holistic, Inference-Complete Process Metamodel Design Principles, semantic_math_basis), which are not part of this repository. Library evidence: the capability maps under docs/capability-maps/, pinned DataFusion 55.1.0 / Arrow 59.3.0 source and rustdoc, and the retained review characterization probes. Context7 is a discovery aid; pinned interfaces and reproducible behavior determine capability claims. The external/idaes-pse reading copy at tag 2.12.0 is for behavior only (just fetch-external; never copy implementation or prose).

Contents

• Revision history
• 0. Purpose, scope, and how to read this document
• 1. Architectural summary
• 2. Architectural decisions
• 3. Workspace, crates, and dependency pins
• 4. The semantic schema registry
• 5. Identity, versions, snapshots, and the catalog
• 6. The canonical relation families
• 7. The mathematical IR
• 8. Physical typing
• 9. Material systems and the property framework
• 10. Balance laws and control volumes
• 11. The unit model library as templates
• 12. Connectivity
• 13. Flowsheets, time, and dynamics
• 14. The compiler
• 15. Structural analysis and diagnostics
• 16. Scaling as an explicit transformation
• 17. Initialization plans
• 18. Backends
• 19. Cases, results, and analytics
• 20. Persistence, provenance, and reproducibility
• 21. The Python boundary and the Pyomo adapter
• 22. Authoring and the extension model
• 23. Observability and failure semantics
• 24. Testing and acceptance
• 25. Delivery phases
• 26. Risks and open decisions
• Appendix A. IDAES capability coverage matrix
• Appendix B. Relation index
• Appendix C. Glossary

Revision history

0. Purpose, scope, and how to read this document

0.1 What this document is

The proposal established the thesis: make mathematical and physical semantics the data model, then compile every executable representation from it. This document is the blueprint that turns that thesis into things an implementer can build and test:

1. the exact relation families (Arrow schemas) that constitute the authoritative model;
2. the mathematical IR and its operator contracts;
3. the compiler passes, each with a declared contract;
4. the kernel contract for constitutive computation and how it generates DataFusion, native, and Pyomo adapters;
5. the backends (native evaluation program, Ipopt FFI, NL file, generated Pyomo);
6. the coverage map from every core IDAES-PSE capability to its platform realization;
7. the workspace layout, dependency pins, and delivery phases with acceptance tests.

It is written for engineers and programming agents who will implement the platform. Where a design decision could reasonably go two ways, the decision is stated with its rationale rather than left open.

This file is the authoritative design. Decisions that changed it are recorded under docs/adr/ (ADR-NNNN); design reviews under docs/design_review/reviews/ are evidence, not authority. Section numbers are stable citation targets: new material is inserted as a sub-section, never by renumbering.

0.2 What “core IDAES-PSE capabilities” means here

The scope is the modeling framework, not the example libraries. Concretely, the platform must reproduce the capability of these IDAES subsystems (source paths are relative to idaes-pse/idaes/):

Detailed capability-by-capability coverage appears in Appendix A.

0.3 Reading guide

0.4 Conventions used in this document

• Relation schemas are written as namespace.relation @version followed by columns as name : ArrowType [constraints] -- meaning. Arrow types use arrow-schema names. Semantic extension types are named pse.<name> and defined in §4.4.
• Equations are written in plain text; d/dt is the time derivative, Σ_j is a reduction over the named domain.
• “Authored” means written by a model author or package; “derived” means produced by a compiler pass and never authored; “reference” means shipped library data.
• IDAES names are quoted where the platform deliberately preserves them (for example the property name enth_mol_phase) so that parity tests can map both ways.

1. Architectural summary

1.1 The one-paragraph design

A process model is a versioned snapshot of typed Arrow relations organized into seven catalog namespaces (reference, authored, normalized, inferred, compiled, runtime, provenance). Authors write packages (material systems, property methods, unit templates, laws, costing methods) and cases (instances, connections, specifications) in a declarative authoring language that parses into authored relations. A staged compiler with contracted passes derives everything else: type and topology closure, property-demand closure, law expansion into equations, discretization, structural analysis, scaling and initialization plans, and finally a backend-neutral CanonicalMathProblem. DataFusion is the relational engine for the compiler’s set-oriented passes, the snapshot catalog, kernel batch evaluation, and all analytics. Native Rust numerics (an evaluation program with automatic differentiation, sparse linear algebra, and structural graph algorithms) execute the problem in-process with Ipopt, or lower it to the AMPL NL format or to a generated Pyomo model. Every artifact is content-addressed, every derived row carries a derivation, and every run records the exact inputs that produced it.

1.2 The three representations, made concrete

The runtime namespace holds results and evidence; the provenance namespace holds derivations and pass records. No pass writes backward into authored.

1.3 Where each library sits

authoring documents (YAML/TOML/Python builders)
        │  load (serde-saphyr, toml; both with source spans) + expression DSL parse (winnow) → authored relations
        ▼
 Semantic schema registry ──generates──► Arrow schemas · typed Rust views · serde_arrow adapters
        │                                 validators · docs · migrations · TableProviders
        ▼
 Snapshot catalog (DataFusion CatalogProvider/SchemaProvider/TableProvider over immutable artifacts)
        │
        ├─ Rule compiler: typed RulePlanSpec → LogicalPlanBuilder → RecordBatch   (closure, joins, selectors)
        ├─ Fixed-point executor (recursive closure with provenance)                (Rust over DataFusion plans)
        ├─ Native graph algorithms (petgraph + own matching/DM/BTD)                 (structural analysis)
        ├─ Math IR passes (Rust; memoized on declared-input content hashes)         (canonicalization, lowering)
        ├─ Kernel registry → generated ScalarUDFImpl wrappers                      (batch property evaluation)
        ▼
 CanonicalMathProblem
        ├─ Native evaluation program (instruction tape, per-equation reverse AD, faer sparse LA)
        │      └─ Ipopt C-API FFI in-process
        ├─ AMPL NL writer / SOL reader (ipopt, bonmin, couenne, cbc, scip, petsc_snes, petsc_ts, k_aug)
        └─ Generated Pyomo model via pyo3 + pyo3-arrow bundle (parmest, PyROS, GDPopt, parity)
        ▼
 runtime relations (solutions, residuals, duals, iterations, diagnostics) + provenance
        └─ DataFusion analytics: stream tables, KPIs, diagnostics reports, sweeps

2. Architectural decisions

Each decision below is binding for the implementation. “Because” states the rationale; “Consequence” states what it forces.

D1. Typed relations are the only authority

Decision: ADR-0004

Every durable fact about a model lives in a typed Arrow relation declared in the semantic schema registry (§4). There is no entity–attribute–value table, no JSON column that carries required behavior, and no Rust struct that is the “real” model with Arrow as an export.

Because the proposal’s central claim (§2.1 of the proposal) is that structure declared once removes five parallel maintenance surfaces. Consequence: Rust structs that hold model data are generated views over relations, never hand-written mirrors.

D2. Author causes, derive consequences

Decision: ADR-0005

The authored model holds only primitive facts (topology, materials, quantities, phenomena, laws, options, specifications). Participation, roles, boundary classification, property requirements, method selection, equations, and orderings are derived by contracted passes and materialized with derivation provenance.

Because the metamodel doctrine (Principles 1–3, 10–12, 20) and because IDAES’s largest maintenance burden is imperative build() methods that restate consequences. Consequence: an IDAES build() method becomes a template plus inference rules; a derived relation that an author tries to write is rejected or downgraded to an assertion.

D3. Three artifact levels, one schema system

Decision: ADR-0006

CanonicalModel, CanonicalMathGraph, and CanonicalMathProblem are stages in one catalog, share identity conventions, extension types, hashing, and provenance. They are not three applications.

D4. Semantic IDs, artifact-local ordinals, and content hashes are distinct

Decision: ADR-0007

Stable 128-bit semantic IDs identify authored and derived entities across revisions; UInt32/UInt64 ordinals index compiled artifacts; blake3 content hashes identify immutable artifact versions. Names, row positions, dictionary codes, and solver positions are never identity (§5). An authored entity’s identity is assigned when it is created and stored in its document; its qualified_name is an attribute that a rename change op may alter without touching any identity (§5.1, §22.2).

D5. Physical type is more than a unit string

Decision: ADR-0039

Every quantity-bearing column and every symbol carries a quantity_type_id resolving dimension vector, quantity kind, basis, reference state, affine semantics, and index shape (§8). Heterogeneous value columns require a per-row quantity_type_id. Null never means “unknown to be solved” (§7.6).

D6. The math IR is richer than DataFusion Expr

Decision: ADR-0009, ADR-0047

Expression nodes, arguments, and typed operator payloads are relations; indexed operators (SumOver, Gather, Broadcast, Derivative, Integral, ImplicitSystem, KernelCall) survive until a backend requires scalarization. DataFusion Expr is used to compute over the IR, not to be the IR (§7).

D7. Laws are templates over contributions

Decision: ADR-0010

Material, energy, momentum, element, and charge balances, and the costing and utility-minimization laws, are instances of law templates expanded over a contributions substrate by the generic compiler (§10). Control volumes are template compositions that declare which contributions exist.

D8. Property demand is resolved explicitly

Decision: ADR-0040

There is no lazy attribute construction. P6 closes demand extracted from normalized template, law, port, display and initializer declarations before P7–P9 realize equations or kernels (§9.6). Selection, unresolved demand and provision checks are explicit. Inspection never constructs physics.

D9. Kernels have one contract and generated adapters

Decision: ADR-0043

Every constitutive computation implemented in code is a KernelSpec with identity, signature (physical types), mathematical behavior, derivative availability, execution forms, failure behavior, and backend bindings. Scalar, batched Arrow, derivative, DataFusion UDF, NL external-function, and Pyomo adapters are generated only for its declared implementation-backed bindings (§18.5, §21.4). A signature does not supply an algorithm or prove support.

D10. DataFusion has four roles and no more

Decision: ADR-0013

Relational assembly and inference (rule plans), snapshot catalog (providers), batch kernel evaluation (generated UDFs), and analytics (queries over runtime). It does not run inside Newton iterations, and a solve is never a scalar function call.

D11. Native numerics own execution layouts

Decision: ADR-0047

The evaluation program, sparse Jacobian structures, and solver workspaces are compiled artifacts derived from relations. They borrow Arrow buffers when layouts permit and copy when they do not. Zero-copy is a preference, not an obligation.

D12. Pyomo is a generated, generic, coarse-grained backend

Decision: ADR-0015

One adapter consumes a CanonicalMathProblem bundle over the Arrow C stream interface and constructs Pyomo objects. No IDAES class hierarchy is recreated. The adapter exists for parity testing and for Pyomo-ecosystem tools (parmest, PyROS, GDPopt, DAE utilities) until native equivalents exist.

D13. Cases and results never mutate the model

Decision: ADR-0016

A model revision defines structure; a case revision defines values, bounds, fixed status, objectives, and overlays; a run records what happened. Initialization stages, homotopy steps, and “fix then release” operations are immutable case overlays, not mutations.

D14. Incrementality follows declared dependencies

Decision: ADR-0041, ADR-0042

Every pass declares its complete input ports; memoization is keyed by their logical contents, absence states, pass version, and — for passes that execute plans — the engine profile (§14.3). Initial reuse is at complete stage granularity; fine-grained invalidation is deferred until measured and qualified (§14.4, §14.5). No stage may read outside its declared input bundle.

2.1 Doctrine crosswalk

3. Workspace, crates, and dependency pins

3.1 Version anchors

Decision: ADR-0049

Engine and runtime (Rust). Every crate in both families is pinned with =; Cargo.lock is committed and CI asserts one version per family (cargo tree grouped by family) — = pins bind direct dependencies only, and cargo tree -d does not report a mixed family because it is not a duplicate.

Supporting crates (Rust). All pinned with =; the resolved set is the one the supporting-library capability map committed as a lockfile under docs/capability-maps/evidence/rust/.

Python. Every library pinned ==. Two interpreter ranges: the platform package targets CPython ≥ 3.11 (3.14 verified); the parity environment is 3.10–3.13 (3.13 verified) because idaes-pse classifies only those. The two are separate dependency sets, so what IDAES drags in (pydantic, sympy, networkx, pandas) cannot become a platform dependency.

Supply chain. cargo deny (advisories, bans, maintenance) runs in CI in addition to cargo audit, because a deprecation encoded only in semver build metadata (serde_yaml’s) is invisible to cargo audit.

The vendored arrow-rs/ checkout in this repository is on the 60.0.0 development line and is a reading copy only. The API facts extracted under build/facts/arrow59-default and build/facts/df55-default are the pinned surfaces to code against.

3.2 Cargo workspace layout

Crate names use the placeholder prefix pse- (process systems engineering); the prefix is a naming decision to confirm, the boundaries are not.

Cargo.toml                       workspace; [workspace.dependencies] carries every pin of §3.1 once; Cargo.lock committed; CI asserts one version per family
crates/
  pse-schema/                    RelationSpec/LogicalType/ExtensionType registry; codegen (syn/quote/prettyplease)
  pse-relations/                 generated typed relation views, builders, validators, migrations (build output committed)
  pse-ids/                       SemanticId, ContentHash, ordinals, derived-ID rules (blake3 derive_key), canonical IPC serializer and hashing; the only crate that depends on blake3
  pse-quantity/                  dimensions, units, quantity kinds/types, bases, reference states, conversions
  pse-material/                  species, elements, phases, phase-species, reactions, stoichiometry, material systems
  pse-authoring/                 package/case document loader (serde-saphyr, toml), expression DSL parser (winnow), change sets
  pse-mathir/                    expression graph, operator catalog and contracts, canonicalization, unit inference
  pse-templates/                 template instantiation, feature guards, law templates, contributions
  pse-rules/                     RuleSpec → DataFusion LogicalPlan compiler; fixed-point executor; provenance rows
  pse-catalog/                   snapshot catalog: CatalogProviderList/SchemaProvider/TableProvider, artifact store
  pse-compiler/                  pass registry, PassSpec contracts, artifact-hash memo (§14.3), pipeline driver
  pse-structural/                incidence, bipartite matching, Dulmage–Mendelsohn, SCC/BTD, DOF, tear selection
  pse-kernels/                   KernelSpec registry; ideal/cubic EOS, pure-component correlations, Henry, transport, reactions
  pse-kernels-ext/               optional providers: CoolProp FFI, Helmholtz external libraries; FeOs when its upstream tracks num-dual 0.15
  pse-numerics/                  evaluation program, AD (per-equation reverse; num-dual for kernels), faer sparse LA
  pse-plans/                     discretization, scaling transformations, initialization plans, solver profiles
  pse-backend-native/            Ipopt C-API FFI driver over the evaluation program
  pse-backend-nl/                AMPL NL writer, SOL reader, external-function library exporter
  pse-backend-pyomo/             bundle assembly for the Python adapter (schemas, ordering, source maps)
  pse-diagnostics/               structural/numerical diagnostics as queries and native analyses; findings relations
  pse-runtime/                   run controller, case overlays, sweeps, result ingestion, lifecycle/cancellation
  pse-py/                        pyo3 + pyo3-arrow extension module (cdylib)
python/
  pse/                           Python package: authoring builders, Pyomo adapter, parity harness (attrs/cattrs contracts)
packages/
  reference/                     shipped reference packages: units, elements, species, property methods, unit templates
tests/
  golden/                        golden relation snapshots and fingerprints per vertical slice (data, not a crate)
  governance/                    workspace member crate pse-tests-governance (§24.1 governance layer)
  engine/                        workspace member crate pse-tests-engine
  conformance/                   workspace member crate pse-tests-conformance
  lifecycle/                     workspace member crate pse-tests-lifecycle
  structural/                    workspace member crate pse-tests-structural
crates/pse-ipopt-sys/            links = "ipopt"; build.rs emits link directives only; bindgen output committed (§18.3)
crates/pse-buildinfo/            lockfile hashes, rustc version, profile, git sha; feeds the manifest and pse.build_info()
xtask/                           codegen, family-check, governance, doc-lint, probe-host, release — the logic behind the justfile
benches/                         crate pse-benches: criterion, harness = false, one bench per §24.3 group

Decision: ADR-0038 — the five tests/* crates, pse-ipopt-sys, pse-buildinfo, xtask and benches were added to this layout at revision 4; the phase-0 ipopt feature is default-on with pse-ipopt-sys optional.

Dependency direction is strictly downward in the list above within each layer; pse-mathir, pse-quantity, pse-material, and pse-ids depend on no engine crate; pse-catalog and pse-rules are the only crates that depend on DataFusion planning types; backends depend on compiled relation views only.

3.3 Supporting libraries and the boundary each must respect

Decision: ADR-0021, ADR-0022, ADR-0026, ADR-0037

Deferred with a stated trigger (not dependencies today): salsa — adopt only if sub-pass memoization inside P7 proves necessary after measurement (§14.3); LogicalPlan::Extension rule nodes — adopt when rule attribution inside EXPLAIN is required by the diff report or agent tooling (§14.2); datafusion-tracing/instrumented-object-store — when a release matching the pinned engine exists (§23.1); datafusion-ffi — only if §22’s extension model admits compiled third-party kernel packages.

Explicitly not added: a second dataframe engine, a graph database, a vector database, a second linear-algebra stack outside the trajectory backend, compile-time unit typing in kernels (uom was dropped: no kernel needs it — KernelSpec signatures are checked by P10’s unit inference and by the per-kernel parity suites — and its stock SI system cannot express the currency dimension), external-source TableProviders (a live database in the catalog is a second mutable authority), datafusion-spark/enable_ansi_mode (silently different arithmetic), Parquet virtual columns and input_file_name (position-based identity), FunctionFactory (SQL-defined kernels), AsyncScalarUDFImpl (I/O inside expression evaluation), parquet-variant (a typed hole in D1), parquet_derive/typed-arrow (they invert the registry→Rust direction), and any Python-side units library as an authority: pint is present because Pyomo’s units machinery is pint; it validates what the adapter emits, it never defines a quantity, and no relation is derived from a pint object (§21.2).

4. The semantic schema registry

4.1 What is declared once

Decision: ADR-0039, ADR-0040

A relation is declared exactly once as a RelationSpec. The registry itself is stored as relations in reference.schema_* so that the platform can query its own schema, diff versions, and generate migrations.

reference.schema_relations @1
  relation_id        : pse.semantic_id   PK
  namespace          : pse.enum(Namespace)      -- reference|authored|normalized|inferred|compiled|runtime|provenance
  name               : Utf8                     -- e.g. "stoichiometry"
  version            : UInt32
  authority          : pse.enum(Authority)      -- authored|reference|derived
  snapshot_class     : pse.enum(SnapshotClass)  -- model|case|derived|sidecar; explicit membership (§5.3)
  primary_key        : List<Utf8>               -- column names
  derivation_granularity : pse.enum(DerivationGranularity)  -- row|rule (§14.2 rule 4); required when authority = derived
  stability          : pse.enum(Stability)      -- stable|evolving|internal
  doc                : Utf8

reference.schema_columns @1
  relation_id        : pse.semantic_id   FK schema_relations
  ordinal            : UInt16
  name               : Utf8
  logical_type_id    : pse.semantic_id   FK schema_logical_types
  nullable           : Boolean
  quantity_type_id   : pse.semantic_id  [nullable]   -- one quantity contract for the whole column
  per_row_quantity   : Boolean                        -- true ⇒ a sibling column named <name>_quantity_type_id exists
  fk_relation_id     : pse.semantic_id  [nullable]
  fk_column          : Utf8             [nullable]
  role               : pse.enum(ColumnRole)           -- key|reference|measure|label|payload|provenance
  doc                : Utf8

reference.schema_logical_types @1
  logical_type_id    : pse.semantic_id   PK
  name               : Utf8                     -- "semantic_id", "content_hash", "dimension_vector", "enum:MaterialBalanceType", ...
  arrow_storage      : Utf8                     -- canonical Arrow DataType rendering
  extension_name     : Utf8            [nullable] -- ARROW:extension:name when an extension type
  metadata_schema    : Utf8            [nullable] -- JSON Schema for ARROW:extension:metadata

reference.schema_enums @1
  enum_id            : pse.semantic_id
  member_ordinal     : UInt16
  member             : Utf8
  idaes_name         : Utf8            [nullable] -- parity mapping (e.g. "componentPhase")
  deprecated         : Boolean
  doc                : Utf8

reference.schema_invariants @1
  invariant_id       : pse.semantic_id   PK
  relation_id        : pse.semantic_id
  kind               : pse.enum(InvariantKind)  -- unique|foreign_key|check|cardinality|domain|closure|acyclic
  rule_id            : pse.semantic_id          -- the typed rule plan (reference.rule_plan_nodes, §6.11) that returns violating keys
  severity           : pse.enum(Severity)       -- error|warning
  doc                : Utf8

reference.schema_migrations @1
  relation_id, from_version, to_version, plan_spec, doc

A relation schema has one meaning and may occur at several immutable compiler stages. Producer ownership belongs to pass_output_ports (§6.11), not to one relation-wide producing_pass_id. Each output port has one producer; every compiled read names its source stage and port. Runtime operations own their result bundles under the same rule. P1 stages authored changes; compiler output ports cannot target authored or reference.

4.2 What is generated from it

Decision: ADR-0051

pse-schema runs as a build step (a non-macro syn/quote/prettyplease pipeline; emitted source is committed under pse-relations/src/generated/ and diffed in CI) and produces, for every relation:

Two further artifacts are generated into committed trees: crates/pse-authoring/src/generated/documents.rs, the serde document structs the package loader parses into, generated from the registry’s DocumentSpec; and the docs/generated/ layout — README.md, relations/<ns>.md, enums.md, extension_types.md, passes.md, rules.md and schema/authoring.schema.json (ADR-0051).

Nothing hand-written may duplicate a column list. A hand-written struct that mirrors a relation is a governance failure (tests/governance/tests/no_shadow_structs.rs greps generated symbol names against hand-written code).

4.3 Metadata conventions

Decision: ADR-0039, ADR-0044, ADR-0050

Schema-level metadata keys (attached by the provider and preserved through IPC/Parquet):

pse.contract.id             relation_id
pse.contract.version        relation version
pse.contract.fingerprint    blake3 of the RelationSpec rows
pse.namespace               namespace
pse.snapshot_id             snapshot the batch belongs to
pse.producer_pass_id        pass that produced a derived relation

Field-level metadata keys:

pse.semantic.logical_type   logical type name
pse.semantic.quantity_type  quantity_type_id when the column carries one quantity contract
pse.semantic.role           ColumnRole
pse.semantic.fk             "<relation>.<column>"
pse.semantic.enum           enum_id for dictionary-encoded enums
ARROW:extension:name        for extension types (§4.4)
ARROW:extension:metadata    the extension's serialized metadata, mandatory beside the name when the type has metadata

pse.* and ARROW:extension:* metadata carry declarations. Neither arbitrary semantic keys nor extension registration automatically validates an ordinary query. One generated recursive validator interprets storage, extension name/version/metadata, nullability, field roles and the declared quantity contract. DataFusion’s extension registry supplies factories that the validator explicitly calls; Arrow’s try_extension_type supplies the typed non-panicking field check. P2 and batch admission separately check row values, cross-field references, keys and foreign keys. Constraints are planner information, never proof of validation.

Generated constructors reject unknown metadata and produce the same key/value set for the same contract. Sorted insertion into a Rust HashMap does not stabilize its iteration order or protobuf encoding. Canonical hashing serializes an ordered metadata representation under §5.3; ordinary Arrow fields retain their normal maps. SERDE_ARROW:* and other unregistered keys are errors, not silently discarded. Only the explicitly named contextual keys in §5.3 are excluded from logical identity.

4.4 Extension types

Decision: ADR-0039

All extension types have a standard storage type so that an unaware consumer reads them safely.

pse.enum metadata is {"v":1,"enum_id":"<32 hex>"} and pse.ordinal_ref metadata is {"v":1,"target_relation_id":"<32 hex>"}; every other type carries {"v":1}; a factory rejects any other shape.

Rust implementations use arrow_schema::extension::ExtensionType — NAME, typed Metadata, metadata, serialize_metadata, deserialize_metadata (which must error on unexpected metadata), supports_data_type, try_new, and the provided validate and try_new_from_field_metadata, which are the read path. The generated try_from(&RecordBatch) calls try_extension_type::<E>() on every extension column beside the schema-fingerprint check; Field::extension_type() (the panicking variant) is banned by a governance grep. Every type’s serialized metadata carries a v field, and the registration factory (below) accepts the metadata generations it knows and rejects the rest — that is the extension-type migration mechanism (DM-44, DM-51). One generated conformance test per type covers the metadata round trip, supports_data_type accept and reject, and reconstruction from a Field.

Engine registration and active admission. Every type in this table is registered in the DataFusion session’s ExtensionTypeRegistry at session construction (ExtensionTypeRegistration::new_arc(name, factory) on a MemoryExtensionTypeRegistry preloaded with the seven canonical arrow.* types), generated from this table; the generator asserts that no registration replaces an earlier one. Registration alone is passive: review E1 demonstrates that an ordinary SELECT can bypass its factories. The generated validator runs when a provider or bundle is admitted, on the complete logical plan before any rewrite can remove an invalid field, as the first and final platform AnalyzerRule around the ordered engine analyzers, after logical optimization before physical planning, and on every published output batch. Traversal includes subqueries (LogicalPlan::apply_with_subqueries), expressions and nested child fields. Rule expressions additionally use the platform’s quantity inference; an untyped computed quantity fails head admission (§14.2). A session exposes only these validated planning/execution entry points.

Native borrowing and Python ingestion invoke the same contract outside DataFusion (§18.2, §21). Metadata parsing can check the syntax/version of ordinal_ref.target_relation_id; only bundle admission can verify that the named target exists and every ordinal is in range. Unknown names/versions, malformed nested metadata, invalid values or incompatible quantities produce schema.* / rule.head_schema_mismatch before consumers assume validity. Structural Arrow validation is necessary and remains distinct from these semantic checks.

Rendering. One ArrayFormatterFactory (Arrow) that also serves DFExtensionType::create_array_formatter (DataFusion) renders every pse.* value meaningfully — a semantic ID as its short form and owner name, a bound as [lo, hi], a quantity value with its unit — in EXPLAIN, error messages and diagnostics, instead of raw storage bytes (§23.2).

Python. The platform package ships and registers the matching pyarrow.ExtensionType classes, generated from this table by the same generator, through an idempotent register-once helper (pyarrow.register_extension_type is process-global and raises on a second registration). An unregistered pse.* type in Python degrades silently to its storage type while ARROW:extension:name survives in the field metadata, so the contract layer asserts, per column, that every field claiming a pse.* extension resolved to a registered class (§21.1). The platform Rust admission layer errors on an unregistered name; an unaware Python consumer degrades — §21.1’s loss profile records that asymmetry.

4.5 Logical type catalog (physical scalars)

Dictionary encoding is applied at write time for enums and repeated labels; it never carries identity.

5. Identity, versions, snapshots, and the catalog

5.1 Three forms of identity

Decision: ADR-0045, ADR-0049, ADR-0050

Authored entity IDs. Identity is assigned when an entity is created and is stored in the document that declares it (id: on every entity); it is never computed from the entity’s name, so a rename changes an attribute and nothing else. Two policies exist, declared per package (authored.packages.id_policy):

• explicit (the default for authored packages): the authoring tool assigns a UUIDv7 when the entity is first written. P1 rejects an explicit-policy entity that lacks an id (parse.missing_id); pse authoring assign-ids inserts missing ones so that hand-written and agent-written documents stay valid.
• named (reference packages whose qualified names are the public contract: units, elements, constants, property kinds, the standard templates and methods): semantic_id = blake3_128("pse:named:v1" ‖ package_id ‖ qualified_name). Under this policy a rename is by definition a new entity; the old one is deprecated with a reference.aliases row, and change_ops.op = rename is rejected.

Both forms are 128-bit and opaque; the policy is recorded on the package, never in the ID. References between entities — template submodels, connections, case targets — are stored by identity (§6.10), so no authored fact depends on a name.

Derived entity IDs. Anything a pass creates has an ID computed from what it was created from, never from row position or numerical values:

symbol instance      = blake3_128("pse:symbol:v1" ‖ instance_id ‖ symbol_decl_id ‖ index_tuple)
equation instance    = blake3_128("pse:equation:v1" ‖ instance_id ‖ equation_decl_id ‖ index_tuple)
law-expanded term    = blake3_128("pse:term:v1" ‖ law_instance_id ‖ contribution_id ‖ index_tuple)
connection equation  = blake3_128("pse:conn:v1" ‖ connection_id ‖ member_ordinal ‖ index_tuple)
mesh node            = blake3_128("pse:node:v1" ‖ domain_id ‖ policy_id ‖ node_ordinal)
discretized symbol   = blake3_128("pse:symbol:v1" ‖ parent_symbol_id ‖ node_id)

blake3_128(context ‖ parts) denotes blake3::Hasher::new_derive_key(context), updated with each part length-prefixed, and the first 16 bytes of the extendable output — a defined 128-bit digest (BLAKE3 is an XOF), not a truncation of an unrelated hash. derive_key and a truncated plain hash produce different bytes, so this choice is frozen with the v1 context strings. pse-ids owns the blake3 dependency; no other crate hashes.

Consequence: re-running compilation on an unchanged snapshot reproduces identical IDs; changing mesh resolution changes only mesh-dependent IDs.

Ordinals. Passes assign ordinals by sorting on semantic ID (or on a declared stable order such as domain member ordinal) so that ordinals are a pure function of the snapshot. Ordinals are stored alongside semantic IDs in compiled.* relations; cross-artifact references always use the semantic ID.

5.2 Model revision, case revision, run

authored.model_revisions @1
  model_revision_id  : pse.semantic_id  PK
  parent_revision_id : pse.semantic_id  [nullable]
  snapshot_id        : pse.content_hash          -- of the authored+reference relations that define structure
  created_at, author, message

authored.case_revisions @1
  case_revision_id   : pse.semantic_id  PK
  model_revision_id  : pse.semantic_id  FK
  parent_case_id     : pse.semantic_id  [nullable]  -- overlay chain (§6.10)
  snapshot_id        : pse.content_hash
  created_at, author, message

runtime.runs @1  (see §6.13)

A change to a case never changes the model revision; a change to a template or connection creates a new model revision and invalidates compiled artifacts by dependency (§14.5), not wholesale.

5.3 Canonical serialization and hashing

Decision: ADR-0045, ADR-0030, ADR-0050

Logical content and physical encoding have different identities. pse-ids owns both hashing entry points. pse.canon.v2 is a new contract: v1’s ignored-null payload and stream/file ambiguity cannot be repaired while retaining its name. No implemented v2 canonicalizer is claimed by this document.

1. Admit and order. Validate the complete relation against its registry contract. Sort unique primary keys with arrow_row::RowConverter and explicit ASC NULLS FIRST; a null or duplicate required key is invalid. Preserve list element order and named domain order; dictionary codes and record-batch boundaries carry no identity.
2. Normalize a hashing copy recursively. Decode dictionaries to values after checking key bounds and declared enum membership. Rebuild zero-offset, exactly sized arrays; null primitive/Boolean/fixed-binary slots have zero payload, null variable-width slots have empty payload with repeated offsets, and masked struct/fixed-list descendants have recursively zero/empty placeholders with their own validity forced valid. This recursion replaces all hidden descendants, including nested child validity; it does not preserve an arbitrary hidden null pattern. Canonical validity is omitted when all values are valid; otherwise unused bitmap bits are zero. Rebuild offsets and drop unreferenced child/storage capacity. Only registered logical types and their declared canonical storage are supported; an unknown Arrow layout is rejected, never hashed opportunistically.
3. Preserve logical metadata separately. The canonical payload schema is generated from the declared schema with dictionaries decoded and all metadata removed. The original semantic metadata is a separate canonical metadata relation with columns (field_path, key, value), all non-null Utf8, no metadata, sorted by UTF-8 byte order. Field paths are zero-based registry column/child ordinals rendered in ASCII decimal without leading zeros, joined by / (for example 0/2); the empty path denotes the schema. (field_path, key) is unique. Keep every registered contract/semantic/extension key and its value; exclude only pse.snapshot_id and pse.producer_pass_id. This preserves the declared dictionary/extension contract without attaching incompatible extension storage claims to the decoded hashing copy. Registry fingerprint, relation ID and schema version also enter the hash frame. Runtime maps are never presumed ordered.
4. Freeze the physical hashing representation. Write the metadata relation and the normalized data relation as two separately length-framed, finished Arrow IPC streams, each with one batch (including a zero-row batch), IpcWriteOptions::try_new(64, false, MetadataVersion::V5) with its uncompressed default (compression = None); the compression mutator remains banned by the Rust rules. All padding bytes are zero. Every integer frame length is unsigned 64-bit little-endian; variable-length components have a preceding length, fixed-width IDs/hashes retain their declared width. The complete preimage is the framed version string pse.canon.v2, relation ID, schema version (u32 little-endian), registry fingerprint, metadata stream and data stream. There is no Debug/display serialization.
5. Valid floats retain the ADR-0030 policy. Before writing the data stream, map valid NaNs to bits 0x7fc00000 (Float32) or 0x7ff8000000000000 (Float64) and preserve valid -0.0; hidden null payload was already zeroed. Apply this only to the hashing copy. DataFusion grouping/distinct/hash joins collapse signed zero and treat NaN as self-equal, so rule keys remain non-floating (§14.2); analytics records that loss (§19.2).
6. Hash by role. logical_hash = blake3(framed canonical preimage) identifies the logical relation under its contract. encoding_checksum = blake3(finished stored bytes) checks one IPC-file, Parquet or evidence object. Neither checksum is substituted for the other; the manifest records both and the encoding descriptor (§20.2). Serialization helpers share the sole hashing owner and an explicit format version.
7. Name membership without self-reference. snapshot_id hashes a pse.snapshot.v2 frame: version string, registry fingerprint, snapshot kind, counted semantic-parent entries (role, snapshot_id) sorted by role, then counted (port, namespace, relation_id, schema_version, logical_hash) members sorted by that tuple’s first four fields. Text sorts by UTF-8 bytes, IDs by raw bytes and versions numerically. Strings/counts use the u64 little-endian framing above; schema versions use u32 and IDs/hashes retain fixed widths. Duplicate parent roles or member ports are rejected. Required empty relations appear explicitly. schema_relations.snapshot_class generates membership: structural authored/reference contracts are model; §6.10 case/observation relations are case; compiler/runtime outputs are derived; revision catalogs, change logs, mutable refs, stage catalogs and pass-attempt/plan-diagnostic records are sidecar. Registry admission requires one class for every relation; no runtime namespace wildcard or implicit omission chooses membership. Model/case snapshots include every relation of their class in the admitted package/case scope, including empties; their port names are namespace/ followed by the relation ID as 32 lowercase hexadecimal digits. A member is a complete relation artifact; any row-selection transformation is declared before snapshot assembly. A case has a model semantic parent and separately named overlay parents. Stage/run membership comes from complete declared output ports and semantic derivations, with parent roles naming their input bindings; those declarations and execution contracts enter the dependency environment. Sidecars and containing manifests never enter their own membership. Derivation snapshot_id names an input snapshot. A stage bundle_id is this output snapshot ID, not a hash column in its own payload. Initial reuse requires equal complete inputs; identical relation content can share storage, but changed lineage does not automatically backdate a whole stage snapshot.
8. Bound construction before allocation. Phase 1 supports at most 1,000,000 rows and 256 MiB of normalized value, offset and validity buffers per relation, and every Utf8/List offset must fit i32. Deployments may lower these limits. Preflight uses checked arithmetic and reserves all concurrently live input, normalized-copy and IPC buffers under §14.3; exceeding a bound is runtime.resource_limit, never partial publication. Raising the supported envelope or adopting deterministic fixed-batch streaming requires the R-23 measurement/format decision.

Equal declared logical content hashes identically across row/batch layout, null payload, dictionary encoding and supported file round trips. Renaming preserves entity IDs but changes relation hashes when label columns change. A changed reference convention changes its contract/reference relations and the dependent snapshot. Metamorphic fixtures must cover nested masked values, slices, all-valid/absent validity, empty relations and Parquet round trips; E4 refutes v1 assumptions and does not establish v2 correctness.

5.4 The snapshot catalog

Decision: ADR-0048

The catalog is a DataFusion CatalogProviderList whose catalogs are snapshots (named by snapshot_id, with mutable aliases such as head resolved to a snapshot at session creation), whose schemas are the seven namespaces, and whose tables are relations:

SELECT r.name, s.coefficient
FROM   "<snapshot_id>".authored.stoichiometry s
JOIN   "<snapshot_id>".authored.reactions r USING (reaction_id)

Provider contract (df §18, §51, §40A.5), stated against what DataFusion 55.1 was measured to do:

• schema() is the generated schema; cheap and stable; a session pins a snapshot so schemas cannot change mid-query. SchemaProvider::table() is async by signature but resolves from the manifest loaded at session creation and performs no I/O; registration methods reject calls after the session is sealed.
• scan_with_args is implemented; scan delegates. Projection is applied at read with Schema::project so metadata travels with the projected fields.
• Pushdown. Advertise Exact only for the registered, differentially tested predicate subset over key/enum columns; unsupported shapes return Unsupported. Equality, conjunction, disjunction, null semantics and supported InList forms are explicit contracts. A two-element IN becoming an OR in a probe is an observation, not a universal planner guarantee. supports_filters_pushdown is pure. Compile predicates using DataFusion physical expressions and evaluate with Arrow filter_record_batch; do not write a second SQL predicate evaluator. Projection retains filter columns until evaluation; limits apply after all required exact/residual filtering. Candidate Parquet pruning may use ProjectionMask, ArrowPredicate/RowFilter only under the same completeness contract and measured R-24 trigger. The test oracle compares complete values and row multiplicities against an otherwise identical Unsupported provider; an intentionally empty/over-pruning provider must fail. Rechecking returned rows alone cannot detect lost matches.
• Statistics are returned with explicit Precision per value: row counts from the manifest are Exact; min/max of key columns from the artifact footer are Exact for whole artifacts and Inexact for any filtered scan; everything not stored is Absent. ScanArgs::statistics_requests() is answered selectively from cached metadata; Sum and DistinctCount are Absent. Statistics::new_unknown is the starting point.
• Constraints. Constraints::new_unverified([PrimaryKey]) on every relation; Unique only where P2 validates a unique invariant; Constraints::project under projection. DataFusion does not validate these — P2 does.
• No mutation. None of insert_into, delete_from, update, truncate, merge_into (all defaulted on the trait, and the set grows) is implemented; a governance test asserts the set of implemented TableProvider methods. get_column_default is not implemented (defaults are authored facts and overlays). LogicalPlan::{Dml, Ddl, Copy, Statement} are never constructed.
• Semantic admission. Provider construction validates schema and batches, and the platform plan entry points execute §4.4 validation. Registration supplies factories; it supplies no automatic universal query check.
• Configuration. Session settings are assembled through ConfigOptions::set (typed errors); SessionConfig::set_str panics on an invalid value and is banned by a governance grep.
• The provider never prunes rows of a coupled mathematical problem for analytical convenience; analytical queries run over runtime/compiled relations, not over the problem the solver sees.

Artifacts behind providers are Arrow IPC files for compiled and runtime relations (hot path, zero-copy mmap) and Parquet for durable authored/reference/runtime history (§20).

6. The canonical relation families

This section is the schema catalog. Every relation is declared in the registry (§4) and appears here with its columns. Column types use the logical names of §4.4–§4.5; sid abbreviates pse.semantic_id, hash abbreviates pse.content_hash, enum(X) abbreviates pse.enum with enum X. [n] marks nullable. Enumerations are listed once, in §6.14.

6.1 Identity and packages (authored, reference)

authored.packages @1
  package_id : sid PK · name : text · version : text (semver) · kind : enum(PackageKind)
  id_policy : enum(IdPolicy)                      -- explicit|named (§5.1)
  dependencies : list<struct<package_id: sid, version_req: text>> · content_hash : hash · doc : text

authored.documents @1
  document_id : sid PK · package_id : sid · path : text · content_hash : hash

authored.entities @1
  entity_id : sid PK · package_id : sid · kind : enum(EntityKind) · name : text · qualified_name : text   -- attributes, never identity
  parent_entity_id : sid [n]                      -- containment (flowsheet ⊃ unit ⊃ control volume ⊃ state block)
  source_span : pse.source_span [n]

reference.aliases @1             -- deprecated qualified names of named-policy entities (§5.1)
  alias_id : sid PK · entity_id : sid · old_qualified_name : text · deprecated_in : text (package version)

authored.model_revisions @1, authored.case_revisions @1   (§5.2)

normalized.package_graph @1                     -- P0 output; exact version requirements only in phase 0
  package_id : sid PK · version : text · content_hash : hash · depth : u16
  dependency_package_ids : list<sid> · derivation_id : sid

Every other authored relation’s primary key is also present in authored.entities; the invariant closure:entity_registered enforces it.

6.2 Physical types (reference, extendable by packages)

Decision: ADR-0039

reference.dimensions @1
  ordinal : u16 PK · name : text                   -- length, mass, time, temperature, amount, current, luminous_intensity, currency

reference.units @1
  unit_id : sid PK · symbol : text · name : text · dimension : pse.dimension_vector
  scale_to_canonical : f64 · offset_to_canonical : f64 · is_affine : bool · system : text · doc : text
  -- canonical units are SI base units; USD_<year> units have dimension currency and scale by CE index (§19.5)

reference.unit_sets @1
  unit_set_id : sid PK · name : text
  time_unit_id, length_unit_id, mass_unit_id, amount_unit_id, temperature_unit_id,
  current_unit_id [n], luminous_intensity_unit_id [n], currency_unit_id [n] : sid
  -- IDAES UnitSet; derived units (pressure, energy, ...) are computed, not stored

reference.quantity_kinds @1
  quantity_kind_id : sid PK · name : text · dimension : pse.dimension_vector
  extensive : bool · addition_kind : enum(QuantityAdditionKind)   -- additive|origin_sensitive
  doc : text
  -- e.g. molar_flow, mass_flow, pressure, temperature, molar_enthalpy, molar_entropy,
  --      molar_density, mass_density, mole_fraction, phase_fraction, area, volume, length, heat_duty, work, ...

reference.bases @1
  basis_id : sid PK · kind : enum(BasisKind)       -- molar|mass|volume|energy|standard_volume|dimensionless
  composition_basis : enum(CompositionBasis) [n]   -- mole_fraction|mass_fraction|volume_fraction|molality|molarity
  rate_basis : enum(RateBasis) [n]
  reference_conditions_id : sid [n]

reference.reference_states @1
  reference_state_id : sid PK · kind : enum(ReferenceStateKind)
  -- elemental_at_conditions | compound_at_conditions | ideal_gas_at_conditions | custom
  temperature : f64 [n] (K) · pressure : f64 [n] (Pa) · include_enthalpy_of_formation : bool · phase_id : sid [n] · doc : text

reference.quantity_types @1
  quantity_type_id : sid PK · quantity_kind_id : sid · basis_id : sid [n]
  reference_state_id : sid [n] · scale_kind : enum(ScaleKind)      -- point|difference (§8.1); gauge pressure is a point quantity whose reference state names the datum
  shape : list<enum(DomainKind)>                                  -- e.g. [phase, species] for x_{p,j}
  subject_kind : enum(SubjectKind) [n]                             -- species|phase|element|reaction|none
  canonical_unit_id : sid · nominal_magnitude : f64 [n]            -- default scaling hint (§16.2)
  doc : text

reference.conversion_rules @1
  conversion_id : sid PK · from_quantity_type_id : sid · to_quantity_type_id : sid
  kind : enum(ConversionKind)                                     -- scale|affine|kernel
  kernel_id : sid [n]                                             -- e.g. molar↔mass via molecular weight
  required_parameters : list<text>                                -- e.g. ["mw"]

reference.quantity_operations @1  -- physical composition policy, distinct from dimension arithmetic (§8.3)
  operation_id : sid PK · opcode : enum(Opcode)
  input_kind_ids : list<sid> · result_kind_id : sid
  basis_rule : enum(BasisRule)               -- preserve|require_equal|registered_conversion|cancel|declared_result
  reference_rule : enum(ReferenceRule)       -- preserve|require_equal|registered_conversion|cancel|declared_result
  scale_rule : enum(QuantityScaleRule)       -- preserve|point|difference|addition|weighted_mean
  shape_rule : enum(QuantityShapeRule)       -- scalar|preserve|same_indices|explicit_broadcast|reduce_bound_index
  basis_source : u16 [n] · reference_source : u16 [n] · scale_source : u16 [n] · shape_source : u16 [n]
  subject_rule : enum(SubjectRule)           -- preserve|require_equal|declared_result
  subject_source : u16 [n] · result_subject_kind : enum(SubjectKind) [n]
  result_basis_id : sid [n] · result_reference_state_id : sid [n]
  input_conversions : list<struct<operand: u16, conversion_id: sid>> · precondition_invariant_ids : list<sid>
  -- *_source is an explicit zero-based operand ordinal, required for preserve; bounds are validated.
  -- require_equal checks all relevant operands and returns that common component; cancel requires
  -- matching non-absent tags under its invariant and returns no tag. declared_result uses only the
  -- explicit result field and a required invariant. Per-operand conversions occur once before matching.
  -- Every kind-changing operation resolves one rule; absent/ambiguous rules fail. No dimension-only fallback.

reference.constants @1
  constant_id : sid PK · name : text · idaes_name : text [n] · value : f64 · unit_id : sid · quantity_kind_id : sid · doc : text
  -- gas_constant R, boltzmann k_B, avogadro N_A, faraday F, gravity g, planck h, speed_of_light c, stefan_boltzmann σ, ...

6.3 Domains, index sets, coordinates

authored.domains @1
  domain_id : sid PK · owner_entity_id : sid · kind : enum(DomainKind)
  -- time|length|species|phase|phase_species|element|reaction|port_set|stage|cell|face|node|custom
  continuous : bool · unit_id : sid [n] · parent_domain_id : sid [n] · doc : text

authored.domain_members @1
  domain_id : sid · member_id : sid PK · ordinal : u32 · label : text
  coordinate : f64 [n]                                            -- for discrete points of a continuous domain
  ref_entity_id : sid [n]                                         -- species_id, phase_id, reaction_id, ...

authored.continuous_domains @1
  domain_id : sid PK · lower : f64 · upper : f64 · unit_id : sid
  initial_points : list<f64>                                      -- authored breakpoints (IDAES time_set / length_domain)
  discretization_policy_id : sid [n]                              -- FK authored.discretization_policies; cases may override (case_policies)

normalized.domain_products @1
  product_id : sid PK · domain_ids : list<sid>                    -- ordered factor domains of a symbol shape

inferred.valid_index_tuples @1
  product_id : sid · tuple : pse.index_tuple PK · derivation_id : sid
  -- e.g. valid (phase, species) pairs from phase_species; valid (time, length) grid after discretization

compiled.meshes @1
  mesh_id : sid PK · domain_id : sid · policy_id : sid · node_count : u32 · nodes : list<f64>
compiled.mesh_nodes @1
  node_id : sid PK · mesh_id : sid · ordinal : u32 · coordinate : f64 · kind : enum(NodeKind)   -- element_boundary|collocation|interior
compiled.stencils @1
  stencil_id : sid PK · mesh_id : sid · derivative_order : u8 · scheme : enum(DiscretizationScheme)
  node_id : sid · neighbor_node_id : sid · weight : f64
compiled.quadrature_rules @1
  rule_id : sid PK · mesh_id : sid · node_id : sid · weight : f64

6.4 Material systems (authored; elements are reference)

reference.elements @1
  element_id : sid PK · symbol : text · name : text · atomic_mass : f64 (kg/mol)

authored.species @1
  species_id : sid PK · package_id : sid · name : text · formula : text [n]
  mw : f64 [n] (kg/mol) · component_type : enum(ComponentType)
  -- Component|Solute|Solvent|Ion|Anion|Cation|Apparent (a platform enum mirroring the IDAES class hierarchy)
  charge : i64 · dissociation_species : list<struct<species_id: sid, coefficient: f64>> [n]   -- ions / apparent species
  valid_phase_types : list<enum(PhaseType)>                        -- IDAES valid_phase_types
  doc : text

authored.species_elements @1
  species_id : sid · element_id : sid · count : f64                -- elemental_composition

authored.phases @1
  phase_id : sid PK · package_id : sid · name : text · phase_type : enum(PhaseType)   -- liquid|vapor|solid|aqueous|undefined
  is_solvent_phase : bool · doc : text

authored.phase_species @1        -- explicit restriction (IDAES per-phase component_list); optional
  phase_id : sid · species_id : sid

authored.henry_declarations @1
  species_id : sid · phase_id : sid · henry_type : enum(HenryType)   -- Kpx|Kpc|Hxp|Hcp
  method_id : sid                                                     -- reference.method_specs (ConstantH, ...)

authored.material_systems @1
  material_system_id : sid PK · package_id : sid · name : text
  species_ids : list<sid> · phase_ids : list<sid> · doc : text

inferred.phase_species @1        -- closure: species valid in phase (valid_phase_types ∩ phase_type, restricted by explicit lists)
  material_system_id : sid · phase_id : sid · species_id : sid · henry : bool · derivation_id : sid

authored.reactions @1
  reaction_id : sid PK · package_id : sid · name : text
  kind : enum(ReactionKind)                                          -- rate|equilibrium|inherent
  basis : enum(BasisKind)                                            -- molar|mass (reaction_basis)
  concentration_form : enum(ConcentrationForm) [n]                   -- molarity|molality|moleFraction|massFraction|partialPressure|activity
  reaction_phase_id : sid [n] · doc : text

authored.stoichiometry @1        -- one row per (reaction, phase, species) with a non-zero coefficient; zero rows are not stored
  reaction_id : sid · phase_id : sid · species_id : sid · coefficient : f64
  -- invariant closure:stoichiometry_species_in_phase ; element balance check is a diagnostic, not an invariant (IDAES allows unbalanced authored data)

authored.reaction_methods @1
  reaction_id : sid PK · rate_form_method_id : sid [n] · rate_constant_method_id : sid [n]
  equilibrium_form_method_id : sid [n] · equilibrium_constant_method_id : sid [n] · heat_of_reaction_method_id : sid [n]

authored.reaction_packages @1
  reaction_package_id : sid PK · package_id : sid · property_package_id : sid · reaction_ids : list<sid>
  unit_set_id : sid · default_arguments : list<struct<key: text, value: text>> · doc : text

authored.parameter_values @1     -- parameter_data for species, phases, reactions, packages, methods
  owner_entity_id : sid · parameter_kind : text                     -- method parameter name, e.g. "cp_mol_ig_comp_coeff"
  index : pse.index_tuple                                            -- e.g. ["A"] or [phase, species]
  value : f64 · unit_id : sid · source : text [n] · std_dev : f64 [n] · estimable : bool

6.5 Property capability registry (reference, extended by packages)

reference.property_kinds @1
  property_kind_id : sid PK · idaes_name : text                      -- "enth_mol_phase", "dens_mass", "fug_phase_comp", ...
  quantity_kind_id : sid · basis_id : sid [n] · shape : list<enum(DomainKind)>
  category : enum(PropertyCategory)                                   -- state|thermo|transport|reaction|derived
  doc : text

reference.method_specs @1        -- a submodel that a package can select (IDAES "method" objects and EOS/state-definition modules)
  method_id : sid PK · family : enum(MethodFamily) · name : text · version : text
  -- MethodFamily: state_definition|eos|pure_component|phase_equilibrium_form|phase_equilibrium_state|bubble_dew|henry|
  --               transport_mixing|reaction_rate_form|rate_constant|equilibrium_form|equilibrium_constant|heat_of_reaction|
  --               enthalpy_transport|custom
  provides : list<sid>                                                -- property_kind_ids
  requires : list<sid>                                                -- property_kind_ids or state symbols needed
  parameter_kinds : list<struct<name: text, quantity_kind_id: sid, indexed_by: list<enum(DomainKind)>, required: bool>>
  realization : enum(MethodRealization)                               -- equation_template|kernel
  template_id : sid [n] · kernel_id : sid [n]
  validity : list<struct<input: text, lower: pse.bound, upper: pse.bound>>
  doc : text

authored.property_packages @1    -- IDAES GenericParameterBlock configuration, typed
  property_package_id : sid PK · package_id : sid · material_system_id : sid · unit_set_id : sid
  state_definition_method_id : sid                                    -- FTPx|FcTP|FpcTP|FPhx|FcPh|...
  temperature_ref : f64 (K) · pressure_ref : f64 (Pa) · include_enthalpy_of_formation : bool
  bubble_dew_method_id : sid [n] · doc : text

authored.state_bounds @1
  property_package_id : sid · state_symbol : text · lower : pse.bound · initial : f64 · upper : pse.bound · unit_id : sid

authored.phase_equilibrium_pairs @1
  property_package_id : sid · phase_a_id : sid · phase_b_id : sid
  state_method_id : sid                                                -- SmoothVLE|SmoothVLE2 (CubicComplementarityVLE)|...
  form_method_id : sid                                                 -- fugacity|log_fugacity per pair (may be overridden per species)

authored.method_selections @1    -- "which submodel computes what, for which scope"
  property_package_id : sid · scope_kind : enum(ScopeKind)             -- package|phase|species|phase_species|reaction
  scope_ids : pse.index_tuple · property_kind_id : sid [n] · family : enum(MethodFamily)
  method_id : sid · options : list<struct<key: text, value: text>>

authored.default_scaling @1
  property_package_id : sid · property_kind_id : sid · index : pse.index_tuple · scaling_factor : f64

inferred.property_requirements @1   -- explicit demand closure (§9.6)
  requirement_id : sid PK · state_scope_id : sid · property_kind_id : sid · index : pse.index_tuple
  requested_by : sid                                                    -- equation_id or requirement_id (transitive)
  derivation_id : sid

inferred.method_resolutions @1
  requirement_id : sid PK · method_id : sid · realization : enum(MethodRealization)
  template_id : sid [n] · kernel_binding_id : sid [n] · status : enum(ResolutionStatus)   -- resolved|unresolved|ambiguous
  derivation_id : sid

6.6 Templates (authored; reference packages ship the standard library)

A template is the declarative replacement for an IDAES *Data class with CONFIG and build(). One template kind covers unit models, control volumes, state blocks, reaction blocks, connection rules, costing methods, laws, initializers, and scalers.

authored.templates @1
  template_id : sid PK · package_id : sid · name : text · version : text · kind : enum(TemplateKind)
  -- unit|control_volume|state_block|reaction_block|connection_rule|costing_method|law|initializer|scaler|flowsheet|helper
  default_initializer_template_id : sid [n] · default_scaler_template_id : sid [n]
  idaes_class : text [n]                                               -- parity mapping, e.g. "idaes.models.unit_models.Heater"
  doc : text

authored.template_params @1      -- CONFIG options
  template_id : sid · name : text · logical_type_id : sid · enum_id : sid [n]
  default : text [n] · required : bool · domain_spec : text [n]        -- validator (e.g. "is_property_package", "positive")
  doc : text

authored.template_features @1    -- typed choice structures (dynamic, has_holdup, has_pressure_change, thermodynamic_assumption, ...)
  template_id : sid · name : text · kind : enum(FeatureKind)           -- bool|enum|choice
  enum_id : sid [n] · default : text [n] · inherit_from : text [n]     -- "parent.dynamic" style inheritance
  doc : text

authored.template_feature_rules @1
  template_id : sid · rule : enum(FeatureRuleKind)                     -- implies|excludes|requires
  antecedent : text · consequent : text                                -- predicates over features/params ("dynamic=true" ⇒ "has_holdup=true")

authored.template_guards @1
  guard_id : sid PK · template_id : sid · predicate : pse.expr_dsl · doc : text

authored.template_domains @1     -- local index sets (length_domain, inlet_list, outlet_list, element lists, ...)
  template_id : sid · name : text · kind : enum(DomainKind) · continuous : bool
  members_from : text [n]                                              -- param that supplies members ("inlet_list") or bound package domain ("phase_list")
  bounds : struct<lower: f64, upper: f64> [n] · unit_id : sid [n]

authored.template_symbols @1     -- Var / Param / Expression / DerivativeVar declarations
  template_id : sid · symbol_decl_id : sid PK · name : text · role : enum(SymbolRole)   -- variable|parameter|expression|derivative|reference
  quantity_type_id : sid · indexed_by : list<text>                     -- template domain names ("time", "phase", "species", "length")
  default_lower : pse.bound [n] · default_upper : pse.bound [n] · default_initial : f64 [n]
  reference_to : text [n]                                              -- for role=reference: path to aliased symbol (IDAES Reference)
  wrt_domain : text [n]                                                -- for role=derivative
  guard_id : sid [n] · idaes_name : text [n] · doc : text

authored.template_equations @1
  template_id : sid · equation_decl_id : sid PK · name : text
  indexed_by : list<text> · filter : pse.expr_dsl [n]                  -- e.g. "j in phase_species[p]"
  expression : pse.expr_dsl · sense : enum(Sense) · family_hint : enum(EquationFamily) [n] · role_hint : enum(EquationRole) [n]
  guard_id : sid [n] · idaes_name : text [n] · doc : text
  -- the text is the authored fact. Its parsed graph is normalized.template_expr_* (P3; same schema as compiled.math_* with
  -- template-scoped references; every node carries a derivation to the source span). No authored relation stores a parsed graph.

authored.template_submodels @1   -- composition (Heater ⊃ control_volume ⊃ properties_in/out ⊃ ...)
  template_id : sid · name : text · child_template_id : sid [n]        -- null ⇒ resolved from a param (property_package → its state_block template)
  child_from_param : text [n] · multiplicity_domain : text [n]          -- indexed submodels (state blocks over time × length)
  bindings : list<struct<child_param: text, value: pse.expr_dsl>>       -- e.g. dynamic ← parent.dynamic, has_phase_equilibrium ← ...
  guard_id : sid [n]

authored.template_ports @1
  template_id : sid · name : text · kind : enum(PortKind)              -- material|heat|work|signal
  direction : enum(Direction) · bound_to : text                        -- submodel path whose state defines members ("control_volume.properties_in")
  guard_id : sid [n] · doc : text

authored.template_contributions @1   -- declared phenomena (heat, work, mass transfer, pressure change, reaction, custom terms)
  template_id : sid · contribution_decl_id : sid PK · name : text
  law_family : enum(LawFamily)                                         -- material|energy|momentum|element|charge|cost|utility
  subject : pse.expr_dsl [n]                                           -- selector: species j, phase p, element e, "total"
  expression : pse.expr_dsl · orientation : enum(Orientation)          -- into_scope|out_of_scope|generation|accumulation
  scope : text                                                          -- control region / submodel path
  guard_id : sid [n] · doc : text

authored.template_law_instances @1
  template_id : sid · law_instance_decl_id : sid PK · law_template_id : sid
  scope : text · subject_selector : pse.expr_dsl · options : list<struct<key: text, value: text>>   -- balance_type, ...
  guard_id : sid [n]

authored.template_requirements @1   -- capabilities the bound packages must provide
  template_id : sid · requirement : enum(CapabilityRequirement) · detail : text [n]
  -- material_flow_terms|enthalpy_flow_terms|material_density_terms|energy_density_terms|diffusion_terms|reaction_rate_basis|phase_equilibrium|...

authored.template_display @1     -- _get_performance_contents, define_display_vars, stream-table membership
  template_id : sid · kind : enum(DisplayKind) · label : text · expression : pse.expr_dsl · display_unit_id : sid [n] · format : text [n]

Non-expression template consumers (initializer stages, opaque law operations and display adapters) declare required properties once:

authored.template_property_requirements @1
  requirement_id : sid PK · template_id : sid · operation_id : sid
  scope_selector_id : sid · property_kind_id : sid · guard : pse.expr_dsl [n]
  index_domain_bindings : list<struct<index_name: text, domain_id: sid>>
normalized.property_demand_seeds @1
  seed_id : sid PK · source_id : sid · source_kind : enum(DemandSource)
  scope_id : sid · property_kind_id : sid · index : pse.index_tuple
  guard_node_id : u64 [n] · derivation_id : sid

P3 derives expression demands directly from its parsed graphs and normalizes these opaque-operation declarations into the same seed schema. P4/P5 decide guards and bind scope/index identities before P6. The seed set contains no P7 equation or P15 stage identifier.

6.7 Instances, flowsheets, connectivity

authored.instances @1
  instance_id : sid PK · parent_instance_id : sid [n] · template_id : sid · name : text
  param_values : list<struct<name: text, value: text>> · feature_values : list<struct<name: text, value: text>>
  property_package_id : sid [n] · reaction_package_id : sid [n] · doc : text

authored.flowsheets @1
  instance_id : sid PK · time_domain_id : sid · dynamic : enum(TriState)   -- true|false|inherit
  default_property_package_id : sid [n]

authored.scopes @1                -- intensional selectors (metamodel P4); the selector is a typed tree, not text
  scope_id : sid PK · root_term_id : sid
authored.selector_terms @1
  term_id : sid PK · scope_id : sid · parent_term_id : sid [n] · ordinal : u16
  op : enum(SelectorOp)                              -- descendant_of|kind_is|tagged_with|union|intersection|difference|include|exclude
  entity_id : sid [n] · entity_kind : enum(EntityKind) [n] · tag : text [n]

authored.connections @1           -- Arcs
  connection_id : sid PK · from_port_id : sid · to_port_id : sid · rule_template_id : sid   -- default: equality connection rule
  tear_cost : f64 [n]                                        -- tear policy for sequential initialization (§12.5); null ⇒ tearable at unit cost
  doc : text

inferred.instance_tree @1         -- transitive containment
  ancestor_id : sid · descendant_id : sid · depth : u16 · derivation_id : sid

inferred.instance_features @1     -- resolved features after inheritance and rules (dynamic ⇒ has_holdup, useDefault resolution)
  instance_id : sid · name : text · value : text · derivation_id : sid

inferred.instances @1             -- expanded submodel instances (control volumes, state blocks per (t,x), reaction blocks)
  instance_id : sid PK · parent_instance_id : sid · template_id : sid · path : text · index : pse.index_tuple · derivation_id : sid

inferred.ports @1
  port_id : sid PK · instance_id : sid · name : text · kind : enum(PortKind) · direction : enum(Direction) · state_instance_id : sid

inferred.port_members @1
  port_id : sid · ordinal : u16 · symbol_group : text                 -- "flow_mol", "mole_frac_comp", "temperature", "pressure"
  symbol_decl_id : sid · quantity_type_id : sid · derivation_id : sid

inferred.scope_members @1         -- scope_id, entity_id, derivation_id
inferred.boundary_crossings @1    -- scope_id, connection_id, classification : enum(Crossing) (internal|external|inbound|outbound), derivation_id
inferred.topology_edges @1        -- unit-level directed graph: from_instance_id, to_instance_id, connection_id
inferred.tear_candidates @1       -- cycle-breaking edges for sequential initialization (§17.4)

inferred.undecided @1             -- rule outcomes that are unknown or in conflict (§14.2 rule 3); head relations hold only decided-true rows
  undecided_id : sid PK · rule_id : sid · head_relation_id : sid · key : pse.index_tuple · truth : enum(TruthValue)   -- unknown|conflict
  reason : text · supporting : list<struct<relation_id: sid, row_key: text>> · derivation_id : sid

6.8 Symbols (compiled)

compiled.symbols @1
  symbol_id : sid PK · ordinal : u64 · owner_instance_id : sid · symbol_decl_id : sid
  qualified_name : text                                                 -- fs.H101.control_volume.properties_out[0.0].temperature
  index : pse.index_tuple · quantity_type_id : sid · unit_id : sid       -- canonical unit of the package unit set
  role : enum(SymbolRole) · solver_type : enum(SolverVariableType)        -- continuous|binary|integer
  semantic_role : enum(VariableSemanticRole)                              -- state|design_capacity|allocation|slack|aux_reformulation|reporting_only|...
  lifecycle : enum(VariableLifecycle)                                     -- authored|generated_discretization|generated_reformulation|generated_relaxation|runtime_artifact
  default_lower : pse.bound · default_upper : pse.bound · default_initial : f64 [n]
  derivation_id : sid

compiled.symbol_references @1     -- alias links (IDAES Reference / VarLikeExpression)
  alias_symbol_id : sid · target_symbol_id : sid · kind : enum(AliasKind)   -- reference|display_alias

compiled.symbol_groups @1         -- named indexed families for ports, display, and lowering (flow_mol_phase_comp[p,j] over its product)
  group_id : sid PK · owner_instance_id : sid · name : text · product_id : sid
compiled.symbol_group_members @1  -- group_id, tuple, symbol_id

6.9 Mathematics (compiled.math_*)

compiled.math_expr_nodes @1
  node_id : u64 PK · opcode : enum(Opcode) · quantity_type_id : sid [n]   -- null until unit inference has run; never null after P10
  scope_instance_id : sid [n] · subtree_hash : hash                        -- hash-consing key (§7.4)

compiled.math_expr_args @1
  parent_node_id : u64 · argument_ordinal : u16 · child_node_id : u64

-- typed operator payloads (one relation per operator family that needs data beyond children)
compiled.math_symbol_refs @1        node_id, symbol_id
compiled.math_float_constants @1    node_id, value : f64, unit_id : sid
compiled.math_int_constants @1      node_id, value : i64
compiled.math_affine @1             node_id, constant : f64, terms : list<struct<coefficient: f64, child_node_id: u64>>  -- ordered; children need not be linear
compiled.math_weighted_means @1     node_id, pairs : list<struct<weight_node_id: u64, value_node_id: u64>>,
                                    normalization : enum(WeightNormalization) (divide_by_sum|certified_unit_sum), unit_sum_invariant_id : sid [n]
compiled.math_reductions @1         node_id, kind : enum(ReductionKind) (sum|prod|min|max), domain_id : sid,
                                    bound_index_id : sid, filter_node_id : u64 [n]
compiled.math_gathers @1            node_id, group_id : sid, coordinate_map : list<struct<bound_index_id: sid, position: u16>>
compiled.math_broadcasts @1         node_id, domain_id : sid, bound_index_id : sid
compiled.math_derivatives @1        node_id, wrt_domain_id : sid, order : u8
compiled.math_integrals @1          node_id, domain_id : sid, quadrature_policy_id : sid [n]
compiled.math_smooth_ops @1         node_id, eps : f64                     -- smooth_max, smooth_min, smooth_abs, safe_sqrt, safe_log
compiled.math_conditionals @1       node_id, guard_node_id : u64            -- piecewise with explicit guard; lowered per backend (§7.2)
compiled.math_kernel_calls @1       node_id, kernel_binding_id : sid, output_ordinal : u16
compiled.math_implicit_refs @1      node_id, implicit_system_id : sid, unknown_ordinal : u16
compiled.math_unit_converts @1      node_id, scale : f64, offset : f64, from_unit_id : sid, to_unit_id : sid
compiled.math_piecewise_linear @1   node_id, breakpoints : list<struct<x: f64, y: f64>>,
                                    input_quantity_type_id : sid, output_quantity_type_id : sid

compiled.math_quantity_selections @1  -- the P10 derivation of the §8.3 selection made at each node
  node_id : u64 · operation_id : sid [n] · builtin_rule : enum(BuiltinQuantityRule) [n]
  operand_permutation : list<u16> · conversions : list<struct<operand: u16, conversion_id: sid>>
  deferred_static_check : bool

compiled.math_indexed_equations @1   -- one row per (instance, equation declaration): the indexed form produced by P7–P9 and consumed by P12
  indexed_equation_id : sid PK · owner_instance_id : sid · equation_decl_id : sid [n] · qualified_name : text
  product_id : sid [n] · filter_node_id : u64 [n] · body_node_id : u64 · sense : enum(Sense) · lower_node_id : u64 [n] · upper_node_id : u64 [n]
  law_instance_id : sid [n] · derivation_id : sid

compiled.math_free_indices @1        -- the free indices of an indexed equation; P12 binds them to domain members
  indexed_equation_id : sid · bound_index_id : sid · domain_id : sid · position : u16

compiled.math_equations @1           -- scalar: one row per valid index tuple, produced by P12
  equation_id : sid PK · ordinal : u64 · owner_instance_id : sid · equation_decl_id : sid [n] · parent_indexed_equation_id : sid [n]
  qualified_name : text · index : pse.index_tuple
  body_node_id : u64 · sense : enum(Sense)                                -- eq|le|ge|range|definition
  lower_node_id : u64 [n] · upper_node_id : u64 [n]
  residual_quantity_type_id : sid
  family : enum(EquationFamily) · role : enum(EquationRole)               -- semantic_math_basis §1.2, §1.3
  differentiability : enum(Differentiability) · convexity : enum(Convexity) · monotonicity : enum(Monotonicity)
  default_active : bool · group_id : sid [n]
  law_instance_id : sid [n] · derivation_id : sid

compiled.math_objectives @1
  objective_id : sid PK · owner_instance_id : sid · body_node_id : u64 · sense : enum(ObjectiveSense) · quantity_type_id : sid · default_active : bool

compiled.math_implicit_systems @1
  implicit_system_id : sid PK · unknown_symbol_ids : list<sid> · equation_ids : list<sid>
  branch_policy : text · kernel_binding_id : sid [n]                        -- when encapsulated as an implicit kernel

compiled.math_complementarity @1
  pair_id : sid PK · expr_a_node_id : u64 · expr_b_node_id : u64 · formulation : enum(ComplementarityForm)   -- smooth_eps|binary|sos1

compiled.math_alternative_sets @1 / math_alternatives @1   -- typed optionality (mode, installed/absent, technology choice) for GDP lowering
compiled.math_dae_links @1          derivative_symbol_id : sid, state_symbol_id : sid, wrt_domain_id : sid

compiled.math_static_analysis @1    -- per-equation and per-variable static attributes (§18.7), computed by P14 on the case-bound view (§7.5)
  ... · parameter_dependence : list<sid>   -- fixed/parameter symbols whose substituted values the classification used

6.10 Cases, specifications, observations (authored)

Decision: ADR-0048

authored.cases @1
  case_id : sid PK · model_revision_id : sid · name : text · parent_case_id : sid [n]   -- overlay chain; child overrides parent
  kind : enum(CaseKind)                                                                -- base|overlay|initialization_stage|scenario|sweep_sample
  doc : text

authored.case_specs @1
  spec_id : sid PK · case_id : sid · target : pse.target_path   -- serialization of case_spec_targets (identity-based); re-rendered by rename
  treatment : enum(Treatment) [n]                             -- fixed|free|parameter
  value : f64 [n] · unit_id : sid [n]                          -- fixed or parameter value; null ⇒ untouched
  initial : f64 [n]                                            -- initial guess; null ⇒ no guess supplied
  lower : pse.bound [n] · upper : pse.bound [n]
  scaling_factor : f64 [n]
  priority : i32                                               -- overlay precedence within a case

authored.case_spec_targets @1    -- resolved by P1 at commit from case_specs.target; rename-stable because it references identities
  spec_id : sid · ordinal : u16 · instance_id : sid · member_kind : enum(TargetKind)   -- symbol|port|group|equation|instance_wildcard
  symbol_decl_id : sid [n] · port_decl_id : sid [n] · index : pse.index_tuple [n] · wildcard : bool

authored.case_activations @1     activation_id PK, case_id, target : pse.target_path, active : bool   -- resolved rows: case_activation_targets (same shape as case_spec_targets)
authored.case_objectives @1      case_id, objective_id, active : bool, weight : f64
authored.case_policies @1        case_id, discretization_policy_id [n] (FK authored.discretization_policies), scaler_template_id [n], numerical_policy_id [n] (FK reference.numerical_policies), initializer_template_id [n] (FK authored.templates), solver_profile_id [n] (FK authored.solver_profiles)

authored.datasets @1             dataset_id, name, source, content_hash
authored.observations @1
  observation_id : sid PK · dataset_id : sid · target : pse.target_path · value : f64 [n] · unit_id : sid   -- null means a missing measurement; resolved rows: observation_targets
  std_dev : f64 [n] · timestamp : ts [n] · tag : text [n]
authored.measurement_models @1   measurement_id, observation_id, model_expression : pse.expr_dsl, error_model : enum(ErrorModel), parameters
authored.scenarios @1            scenario_id, case_id, weight : f64, doc
authored.case_sets @1            case_set_id, base_case_id, generator_kind : enum(GeneratorKind) (grid|latin_hypercube|uniform_random|list), generator_params : list<struct<key: text, value: text>>, seed : u64 [n], sample_count
authored.case_set_samples @1     case_set_id, sample_ordinal, case_id

Tabular observation imports use a declared schema with Arrow CSV/JSON ReaderBuilder::new(schema), strict unknown-column handling and CSV header validation. Required columns, missing-value spellings, units, target resolution and source-row spans are specified by a versioned import descriptor; schema inference is disabled. Missing measurements remain rows with null values and never become zero. Structural/type/finite-value/FK validation precedes a change set. This applies to the §19.4 observation interface, not to the YAML/TOML modeling language or IDAES state-file format.

6.11 Numerical infrastructure (reference, authored, compiled)

Decision: ADR-0040, ADR-0043, ADR-0047, ADR-0048, ADR-0041

reference.kernel_specs @1        -- §18.5; generated registrations and adapters consume these fields
  kernel_id : sid PK · name : text · version : text · provider : text · artifact_digest : hash
  behavior : enum(KernelBehavior)                              -- explicit|implicit
  inputs : list<struct<name: text, quantity_type_id: sid, natural_unit_id: sid, nullable: bool, shape: list<enum(DomainKind)>>>
  outputs : list<struct<name: text, quantity_type_id: sid, natural_unit_id: sid, shape: list<enum(DomainKind)>>>
  parameters : list<struct<name: text, quantity_type_id: sid, natural_unit_id: sid, indexed_by: list<enum(DomainKind)>>>
  null_input_policy : enum(KernelNullPolicy)                   -- reject|propagate_missing; no null-as-zero mode
  failure_policy : enum(KernelFailurePolicy)                   -- typed_error; scalar and UDF default, no invalid-to-null fallback
  tolerant_batch : bool                                        -- true enables the separate mandatory outcome relation (§6.13)
  effects : enum(KernelEffects)                                -- pure; stateful/ambient kernels are unsupported by this contract
  argument_evaluation : enum(ArgumentEvaluation)               -- eager|guarded_expression; guarded requires an expression lowering
  smoothness : enum(Differentiability) · validity : list<struct<input: text, lower: pse.bound, upper: pse.bound>>
  monotonicity : list<struct<input: text, direction: enum(Monotonicity)>> · convexity : enum(Convexity)
  derivative_bindings : list<struct<kind: enum(DerivativeKind), implementation_id: sid, validity_invariant_ids: list<sid>>>
  execution_forms : list<struct<form: enum(ExecutionForm), implementation_id: sid>>  -- scalar|batch_arrow|dual|hyperdual
  thread_safe : bool · failure_classes : list<enum(KernelFailure)>
  bindings : list<struct<backend: enum(BackendBinding), implementation_id: sid, conformance_suite: text>>
  -- native|datafusion_udf|nl_external_function|pyomo_external_function|pyomo_expression; no row means unsupported
  nl_function_name : text [n] · doc : text · test_suite : text

reference.numerical_policies @1
  policy_id : sid PK · name : text · version : text
  reassociation : enum(ReassociationPolicy)                    -- preserve_order|canonical_order
  fma : enum(FmaPolicy)                                        -- disabled|explicit
  nonfinite : enum(NonfinitePolicy)                            -- evaluation_error
  rewrite_mode : enum(RewriteMode)                             -- guarded_floating; no unconditional real-algebra identity rewriting
  absolute_tolerance : f64 · relative_tolerance : f64 · signed_zero_equal : bool
  -- strict.default: preserve_order, disabled, evaluation_error, guarded_floating, 1e-12, 1e-10, false
  -- policy selection is case-bound (§7.3); package graph canonicalization always uses ordered strict semantics

compiled.kernel_bindings @1
  binding_id : sid PK · kernel_id : sid · scope_instance_id : sid
  parameter_bindings : list<struct<name: text, symbol_id: sid [n], value: f64 [n], unit_id: sid [n]>>
  input_bindings : list<struct<name: text, node_id: u64>>

reference.pass_specs @1          -- §14.1; ports supply the only input/output declarations
  pass_id : sid PK · name : text · version : text
  preconditions : list<sid> · postconditions : list<sid>       -- invariant_ids
  determinism : enum(Determinism) · diagnostics : list<enum(FailureClass)>
  -- every input port is keyed initially; no separate incomplete cache_key_inputs list
reference.pass_input_ports @1
  pass_id : sid · port : text · relation_id : sid
  source_pass_id : sid [n] · source_port : text [n]           -- both set for derived input, otherwise pinned snapshot input
  required : bool                                           -- optional absence is a typed input value and enters the key
reference.pass_output_ports @1
  pass_id : sid · port : text · relation_id : sid
  -- PK (pass_id, port); one producer per port; relation schemas can occur at successive stages
compiled.stage_bundles @1                                    -- sidecar catalog of completed immutable outputs
  bundle_id : hash PK · pass_id : sid · pass_version : text
  members : list<struct<port: text, relation_id: sid, schema_version: u32, logical_hash: hash>>
  -- bundle_id is the stage snapshot ID (§5.3), including semantic parents/derivations; this sidecar is excluded

reference.rule_specs @1          -- §14.2; the body is the typed plan below, never text
  rule_id : sid PK · version : text · stratum : u16 · head_relation_id : sid · root_node_id : sid
  negation : enum(NegationPolicy) · monotonic : bool · conflict_policy : enum(ConflictPolicy)

reference.rule_plan_nodes @1     -- the RulePlanSpec algebra as rows (one relation, typed payload columns)
  node_id : sid PK · rule_id : sid · op : enum(RulePlanOp)
  -- scan|filter|project|equi_join|anti_join|union|distinct|aggregate|unnest|kernel_call|recursive
  relation_id : sid [n]                                        -- required for scan
  input_port : text [n]                                       -- required for scan; names its pass_input_ports binding, never an implicit latest producer
  join_keys : list<struct<left: text, right: text>> [n]         -- equi/anti join; key columns must be key-role (never Float64, rule 7)
  predicate_expr_id : sid [n]                                  -- reference.rule_expr_* graph (same schema as math_* restricted to boolean ops)
  projection : list<struct<name: text, expr_id: sid>> [n] · kernel_binding_id : sid [n] · doc : text
reference.rule_aggregates @1    node_id, ordinal : u16, function : enum(RuleAggregate) (count|sum|min|max|collect_ordered),
                                 input_expr_id : sid [n], output_name : text, order_by : list<struct<column: text, ascending: bool>>,
                                 null_policy : enum(AggregateNullPolicy) (reject|skip_missing), empty_policy : enum(AggregateEmptyPolicy) (zero|empty_list|error)
reference.rule_group_keys @1    node_id, ordinal : u16, column : text
reference.rule_unnest @1        node_id, column : text, value_name : text,
                                 null_list_policy : enum(NullListPolicy) (reject|no_members), empty_list_policy : enum(EmptyListPolicy) (no_members)
reference.rule_plan_edges @1     parent_node_id : sid · ordinal : u16 · child_node_id : sid
reference.rule_dependencies @1   -- derived at registry build from rule_plan_nodes; the queryable dependency facts
  rule_id : sid · relation_id : sid · input_port : text [n] · mode : enum(DependencyMode)   -- read|negate|write; reads/negations retain the named port
  stratum : u16 · derivation_id : sid

authored.discretization_policies @1   -- a policy (DM-13): authored or shipped by the reference package; never produced by a pass
  policy_id : sid PK · package_id : sid · name : text · method : enum(DiscretizationMethod)   -- finite_difference|collocation
  scheme : enum(DiscretizationScheme)                                          -- BACKWARD|FORWARD|CENTRAL|LAGRANGE_RADAU|LAGRANGE_LEGENDRE
  finite_elements : u32 · collocation_points : u8 [n]

compiled.scaling_plans @1        plan_id, case_id, template_id (scaler), options
compiled.variable_scales @1      symbol_id, scale : f64, offset : f64, source : enum(ScaleSource), derivation_id
compiled.equation_scales @1      equation_id, scale : f64, scheme : enum(ConstraintScalingScheme), derivation_id

compiled.initialization_plans @1 plan_id, case_id, template_id (initializer), doc
compiled.init_stages @1
  stage_id : sid PK · plan_id : sid · ordinal : u16 · kind : enum(StageKind)
  -- solve_subset|solve_blocks|propagate|apply_overlay|continuation|call_plan|check|restore
  target_kind : enum(StageTargetKind)                          -- instance|equation_set|blocks|connection|all
  target_ids : list<sid>                                       -- identities (instances, equations, blocks, connections); never paths
  overlay_case_id : sid [n] · solver_profile_id : sid [n] · tolerance : f64 [n]
  failure_policy : enum(FailurePolicy)                         -- abort|continue|retry_with_profile
  doc : text

authored.solver_profiles @1      -- a policy: the reference package ships ipopt.default and ipopt.user_scaled (§18.3); users add their own
  profile_id : sid PK · package_id : sid · name : text · backend : enum(Backend)    -- native_ipopt|nl_ipopt|nl_bonmin|nl_couenne|nl_cbc|nl_scip|nl_petsc_snes|nl_petsc_ts|pyomo
  solver : text · options : list<struct<key: text, value: text>>
  scaling_mode : enum(ScalingMode)                             -- user|gradient_based|none
  derivative_mode : enum(DerivativeMode)                       -- exact_hessian|limited_memory
compiled.solve_plans @1          plan_id, problem_id, class : enum(SolvePlanClass), justification : text, modifiers : list<text>

reference.engine_profiles @1     -- the relational engine as a declared input (§14.2 rule 5)
  engine_profile_id : sid PK · datafusion_version : text · arrow_version : text
  analyzer_rules : list<text> · optimizer_rules : list<text> · physical_rules : list<text>   -- ordered; installed explicitly
  semantic_settings : list<struct<key: text, value: text>>                                  -- an explicit, versioned key allow-list (§14.2 rule 5), never a namespace match
  setting_allow_list_version : text                                                         -- bumped when the allow-list changes
  -- the engine-profile artifact logical hash is supplied by its bundle handle, not stored in its own row

6.12 Derived structure (compiled)

compiled.problems @1
  problem_id : sid PK · model_revision_id : sid · case_id : sid · discretization_policy_ids : list<sid>
  variable_count, equation_count, inequality_count, objective_count : u64 · degrees_of_freedom : i64 · input_bundle_hash : hash  -- P12 math plus resolved case/policy inputs; never this row's own hash

compiled.variable_order @1
  problem_id : sid · symbol_id : sid · position : u64 [n]      -- null when not in the solver vector (fixed/parameter/eliminated)
  treatment : enum(Treatment) · lower : pse.bound · upper : pse.bound · initial : f64 [n] · scale : f64 · offset : f64

compiled.equation_order @1
  problem_id : sid · equation_id : sid · position : u64 [n] · active : bool · scale : f64 · kind : enum(RowKind)   -- equality|inequality|objective

compiled.bound_values @1             -- fixed and parameter symbols, separate from free-variable initial guesses
  problem_id : sid · symbol_id : sid · treatment : enum(Treatment) · value : f64 · unit_id : sid · value_hash : hash · source_spec_id : sid [n]

compiled.case_bound_substitutions @1   -- fixed/parameter symbols substituted on the case-bound view (§7.4 step 3, §14.1 P13/P14)
  problem_id : sid · symbol_id : sid · value : f64 · unit_id : sid · value_hash : hash · source : enum(ValueSource)   -- case_spec|package_default|template_default

compiled.incidence @1
  problem_id : sid · equation_id : sid · symbol_id : sid · linear : bool · coefficient : f64 [n]   -- coefficient when affine in this symbol

compiled.dm_partition @1
  problem_id : sid · kind : enum(DMKind) (equation|variable) · id : sid
  block : enum(DMBlock)                                         -- underconstrained|square|overconstrained
  matched_id : sid [n]                                          -- matching partner; null ⇒ unmatched

compiled.blocks @1               problem_id, block_id, kind : enum(BlockKind) (scc|connected_component|bordered_diagonal), order : u32, size : u32
compiled.block_members @1        block_id, kind (equation|variable), id
compiled.sparsity_patterns @1    problem_id, kind (jacobian|hessian), row_ptr : list<u32>, col_idx : list<u32>, content_hash
compiled.evaluation_programs @1  problem_id, program_id, artifact_hash, instruction_count, workspace_size
compiled.backend_bindings @1     problem_id, backend, status : enum(BindingStatus) (supported|unsupported), unsupported_opcodes : list<text>, artifact_hash [n]

6.13 Execution and evidence (runtime, provenance)

Decision: ADR-0044

runtime.runs @1
  run_id : sid PK · problem_id : sid · case_id : sid · model_revision_id : sid
  backend : enum(Backend) · solver_profile_id : sid · plan_id : sid [n] · stage_id : sid [n] · parent_run_id : sid [n]
  attempt : u16                                     -- 1 for the first attempt of (plan run, stage); retry_with_profile creates a new row with attempt + 1
  resolved_options : list<struct<key: text, value: text>>   -- the options actually passed to the solver after capability resolution (§18.3)
  started_at, finished_at : ts · status : enum(TerminationStatus)
  -- optimal|locally_infeasible|infeasible|unbounded|max_iterations|max_time|restoration_failed|solver_error|cancelled|user_interrupt
  environment : struct<platform_version: text, compiler_version: text, solver_version: text, kernel_digests: list<hash>, host: text>
  wall_seconds : f64 · iterations : u32 [n]

runtime.solutions @1             run_id, symbol_id, value : f64, unit_id, bound_status : enum(BoundStatus) (interior|at_lower|at_upper|violated)
runtime.duals @1                 run_id, equation_id, dual : f64, bound_multiplier_lower : f64 [n], bound_multiplier_upper : f64 [n]
runtime.residuals @1             run_id, equation_id, residual : f64, scaled_residual : f64, relative_residual : f64 [n]
runtime.iterations @1            run_id, iteration : u32, objective, inf_pr, inf_du, mu, step_size, regularization, restoration : bool
runtime.solver_events @1         run_id, ordinal, kind : enum(SolverEvent), message : text
runtime.diagnostics_findings @1  finding_id, subject_snapshot : hash, run_id [n], check_id : sid, severity : enum(FindingSeverity), subjects : list<sid>, values : text (JSON), message, next_steps : list<text>
runtime.kernel_evaluations @1    evaluation_id, kernel_binding_id, input_hash, output_batch_hash  -- batch evaluations outside solves
runtime.kernel_evaluation_outcomes @1  -- mandatory for tolerant batch evaluation, one row per requested input/output coordinate
  evaluation_id : sid · row_ordinal : u64 · output_ordinal : u16
  outcome : enum(KernelOutcome) (success|missing_input|domain_failure|implementation_failure)
  value : f64 [n] · quantity_type_id : sid · unit_id : sid · reason_code : enum(KernelFailure) [n]
  -- success iff value is non-null; every non-success has a reason_code; failed values cannot feed ordinary aggregations

runtime.host_capabilities @1     -- written by the explicit `probe_host` operation (§18.3); read when a run resolves its profile
  host : text PK · probed_at : ts · ipopt_version : text · linear_solvers : list<text> · hsl_available : bool · petsc_version : text [n]
  python_env : struct<interpreter: text, pyomo: text, pint: text, pyarrow: text, numpy: text, scipy: text, idaes: text [n], pyomo_contrib: list<struct<name: text, version: text>>> [n]

provenance.derivations @1
  derivation_id : sid PK · relation_id : sid · row_key : text · rule_id : sid [n] · pass_id : sid [n]
  supporting : list<struct<relation_id: sid, row_key: text>> · snapshot_id : hash · fingerprint : hash

provenance.pass_records @1       pass_run_id, pass_id, version, snapshot_in : hash, snapshot_out : hash, engine_profile_hash : hash [n],
                                 plan_evidence : list<struct<encoding_checksum: hash, encoding: text, codec_version: text>> (noncanonical diagnostic bytes — §14.2 rule 5),
                                 plan_explain : list<text> (EXPLAIN in pgjson format), rules_fired : list<struct<plan_ordinal: u16, rule_name: text, ordinal: u16>>,
                                 duration_ms, finding_count, status
provenance.assertions @1         assertion_id, package_id, expected : text (typed JSON), status : enum(AssertionStatus) (pass|fail|obsolete), reason
provenance.refs @1               name : text PK, snapshot_id : hash, manifest_checksum : hash, updated_at : ts

6.14 Enumerations preserved from IDAES

These enumerations are closed dictionaries whose members carry idaes_name for parity. Member lists were taken from the vendored IDAES sources (2.10 line) and are re-verified against the pinned parity package, IDAES 2.12.0, by a governance test that compares schema_enums to the package’s enum classes before phase 1; where IDAES declares members without an implementation, the platform declares them too and reports a missing law binding rather than silently accepting them.

7. The mathematical IR

7.1 Design constraints

1. The expression graph is a DAG stored in compiled.math_expr_nodes and math_expr_args; the authoritative argument order is argument_ordinal. Packed adjacency arrays for evaluation are generated (§18.2), never maintained separately.
2. Indexed mathematics is preserved until a backend requires scalarization. A species balance over 1,000 cells is one indexed equation instance (compiled.math_indexed_equations) whose free index ranges over the cell domain until P12 (index expansion, §14.1) produces the scalar rows that structural analysis and the backends consume.
3. Every node carries a resolved quantity_type_id after P10. Unit inference is a pass, not a runtime check.
4. Every operator carries a contract (§7.3). A backend that cannot lower an operator fails explicitly at P16 with the unsupported opcodes listed in compiled.backend_bindings.
5. Free unknowns are symbols. A null in a value column never means “to be solved”.

7.2 Operator catalog

Phase 0 admits as a Conditional guard only a SymbolRef whose kind is the registered boolean kind or an IntConst 0/1 decided by feature selection; comparison and logical opcodes are a later amendment.

“✓ (expanded)” means the backend receives scalar forms after P12 (index expansion) and P16 (backend lowering). Conditional is admitted in authored templates only under a guard whose value is decided at compile time (feature flags) or when the case selects a backend that supports it; otherwise it must be reformulated (smooth approximation, complementarity, or disjunction).

7.3 Operator contract record

Decision: ADR-0047

Each opcode has a row in reference.operator_specs (generated from the pse-mathir operator table so that the code and the relation cannot drift):

reference.operator_specs @1
  opcode : enum(Opcode) PK · arity : enum(Arity) (fixed n|variadic|payload)
  quantity_operation_ids : list<sid> · shape_rule : text · derivative_rule : text
  argument_evaluation : enum(ArgumentEvaluation) · failure_classes : list<enum(KernelFailure)>
  domain_restrictions : list<struct<argument: u16, relation: enum(RelationOp), bound: f64>>
  smoothness : enum(Differentiability) · convexity_rule : text · monotonicity_rule : text
  sparsity_rule : text · rewrite_conditions : list<text>
  lowering : list<enum(BackendBinding)>

The operator table is the authority for evaluation order, quantity rules, derivatives and lowerings. P10 always preserves ordered guarded floating-point semantics, so the model graph is independent of case policy. x/x → 1 is not an unconditional rewrite: the relevant contract must establish finite nonzero inputs and preserve observable failure/signed-zero behavior. log(exp(x)) → x is disabled by default because intermediate overflow/underflow and rounding can change behavior; any future rule requires a checked interval/error bound under the selected policy.

P13 resolves case_policies.numerical_policy_id, defaulting to strict.default, and persists the selected policy in problem/preparation and run evidence. P16 may reorder terms or contract multiply-add only when that policy explicitly permits it; those choices enter preparation keys. Backend support includes numerical-policy support: an adapter that cannot preserve selected ordering, branch failures or comparison semantics returns backend.unsupported_numeric_policy, never silently relaxes them. Domain violations and non-finite intermediates are typed evaluation failures, not accepted approximate answers. Comparison of finite values is abs(a-b) <= atol + rtol*max(abs(a),abs(b)); policy determines signed-zero equivalence. NaNs never pass a finite comparison; non-finite/error outcomes must match their declared class. No backend gets a bitwise-equivalence claim from passing a tolerance test.

7.4 Canonicalization (pass P10)

Decision: ADR-0039, ADR-0047

1. Ordered hash-consing. Hash framed opcode, typed payload, quantity contract and child hashes through pse-ids; argument_ordinal and reduction domain order are significant. Do not sort commutative children under the default contract. Sharing a node does not authorize hoisting its evaluation outside a conditional region.
2. Preserve arithmetic structure. P3 emits ordered Add/Sub/Mul nodes. P10 does not flatten, merge coefficients, delete zero-weight evaluated terms or reassociate them just because the real expressions are algebraically equal. Explicit Affine nodes retain ordered terms; their children may be nonlinear. A case-bound optimization with a declared policy and complete assumptions belongs to P16, not the model graph.
3. Guarded literal folding. Fold literal-only subtrees only when domain, finite result, selected branch, signed zero and comparison behavior are preserved. Do not pre-evaluate a failing excluded branch. Never fold symbol references in P10, regardless of role; case-bound substitutions are separate P13 outputs.
4. Complete physical inference. Apply §8.3 and record the selected quantity operation/conversion. Infer kind, basis, reference convention, scale kind, shape, subject kind and canonical unit together. Dimension equality alone cannot attach a result type. WeightedMean has its own payload and checked normalization; it is not a free-form label on an arbitrary sum.
5. Acyclicity and indices. Reject cycles; resolve reduction/gather binders and retain the free-index domains. Deterministic topological numbering is storage order, not a command to execute excluded branches.

The pse.canon.v2 relation serializer preserves this ordered IR. Reordering arithmetic under an opted-in case policy changes the prepared program and its policy-qualified hash, not the authoritative graph.

7.5 Equation records

compiled.math_equations follows the EquationRecord shape of semantic_math_basis.md §1.1. Family and role are derived by P14 on the case-bound view: the canonical graph with fixed and parameter symbols replaced by recorded case_bound_substitutions values from bound_values. A coefficient is constant when it is a literal or a recorded fixed/parameter substitution; math_static_analysis.parameter_dependence records which fixed/parameter substitutions the classification used, so the same canonical graph may classify differently in a case that estimates a parameter (§19.4) than in one that fixes it, and the derivation says why:

Roles (HARD_FEASIBILITY, DEFINITION, LINKING for connection equations, DOMAIN_GUARD, REPORTING, APPROXIMATION) are assigned by the pass that created the equation and are part of the derivation.

7.6 Null, bound, and unknown semantics

Decision: ADR-0043

7.7 The expression DSL

Authored equations are written in a small textual language. The text, stored in pse.expr_dsl columns, is the authored fact and the only form a change set may edit; its parsed graph is a derived normalized.*_expr_* relation produced by P3 with a derivation to the source span (§6.6), and a parse → render → parse round trip is the identity on the graph (a test in §24.1).

expr        := affine ;
affine      := term { ("+" | "-") term } ;
term        := factor { ("*" | "/") factor } ;
factor      := unary { "^" unary } ;
unary       := ["-"] primary ;
primary     := number [unit]
             | path [index]
             | function "(" [expr {"," expr}] ")"
             | "sum" "(" binder "|" expr ")"
             | "prod" "(" binder "|" expr ")"
             | "d(" expr ")/d" ident                    (* derivative w.r.t. a continuous domain *)
             | "integral" "(" binder "|" expr ")"
             | "kernel" "." ident "(" [args] ")"
             | "if" expr "then" expr "else" expr        (* Conditional; guard must be compile-time or backend-supported *)
             | "(" expr ")" ;
binder      := ident "in" path [ "where" expr ] ;
path        := ident { "." ident } ;
index       := "[" expr { "," expr } "]" ;
unit        := "{" unit_expr "}" ;                      (* e.g. 320{K}, 2{bar}, 1{mol/s} *)
function    := "exp" | "log" | "log10" | "sqrt" | "abs" | "smooth_max" | "smooth_min" | "smooth_abs" | "safe_sqrt" | "safe_log"
             | "sin" | "cos" | "tan" | "tanh" | "erf" | "min" | "max" | "convert" | "weighted_mean" ;

Accepted superset (wave 1). The parser accepts a documented superset of the grammar above. There are three entry points: an expression; an equation, with ==, <= or >=; and a predicate, with and, or, not, != and in. expr where name = expr, … binds local names as in §9.3. eps= is the last, named argument of the smooth and safe functions. ^ is right-associative, and -x^2 is refused as ambiguous — write (-x)^2 or -(x^2). The comprehension form f(x_i for i in set) written in the mixer example below is not accepted; the chained smooth_min form of §11.1 is the authored form. min and max parse and are refused by P3 until their opcodes exist.

Examples (heater energy balance and a mixer’s smooth minimum pressure):

sum(p in phase | control_volume.properties_out[t].enth_flow_phase[p])
  - sum(p in phase | control_volume.properties_in[t].enth_flow_phase[p]) - heat_duty[t] == 0

mixed_state[t].pressure == smooth_min(inlet_state[t, i].pressure for i in inlet_list, eps=1e-3{Pa})

weighted_mean(w1, x1, w2, x2, ...) requires a nonempty even argument list and lowers to ordered pairs with divide_by_sum; only generated templates with an explicit unit-sum invariant may select certified_unit_sum. An empty set or zero denominator is a typed domain error.

The parser (winnow) produces nodes with unresolved paths; P3 (canonicalization) resolves paths against template symbols, submodels, and bound package domains. Any path that does not resolve is an authoring error with a source span.

8. Physical typing

8.1 QuantityType

A quantity_type_id resolves to the tuple (quantity kind, dimension vector, basis, reference state, scale kind, shape, subject kind, canonical unit, nominal magnitude). This is what allows the compiler to distinguish:

8.2 Units and unit sets

• Units are SI-anchored: scale_to_canonical, offset_to_canonical, and a dimension vector. Units with offsets (°C, °F, psig) carry is_affine = true: a point conversion applies the offset, while a difference conversion applies scale only. P3 normalizes authored constants to package canonical units. P9 explicitly converts to and from each kernel’s natural-unit ports; P10 checks those declared UnitConvert edges, including affine ports, instead of banning the conversion needed by the kernel.
• A property package selects a unit_set (IDAES UnitSet: time, length, mass, amount, temperature, plus optional current, luminous intensity, currency). Every symbol declared by templates bound to that package is stored in the canonical unit of the set; the unit_id column of compiled.symbols records it.
• Method natural units: a correlation kernel (Antoine in bar and °C, a Perry’s cp in J/(kmol·K)) declares its own input/output units in KernelSpec; P9 inserts UnitConvert nodes to and from the package canonical units. This reproduces the IDAES generic framework behavior of mixing methods written in different units, but every conversion is a visible node.
• Currency units (USD_2018, USD_2023, …) have dimension currency; conversion between years is a scale by the ratio of Chemical Engineering cost indices from reference.cost_indices, standardized through the CE=500 basis exactly as IDAES register_idaes_currency_units does (§19.5).

8.3 Unit inference algorithm (P10 step 4)

Decision: ADR-0039

Inference returns the complete QuantityType and the selected operation/conversion, never only a dimension vector. quantity_kinds.addition_kind distinguishes additive quantities (flows, transported energy, amounts) from origin-sensitive points (temperature, pressure, specific enthalpy/entropy). For an additive kind, signed sums retain its ordinary point representation; difference is reserved for origin-sensitive kinds. This avoids treating the sum of two material flows as an invalid affine-point sum.

The standard quantity package must declare and test at least flow times specific enthalpy to energy flow (preserving enthalpy reference state), flow times composition to component flow (preserving material basis), gas constant times temperature to molar energy, pressure divided by molar energy to molar density, temperature divided by an explicitly typed temperature scale to dimensionless, and activation energy divided by gas-constant-times-temperature to dimensionless. cancel policies require matching tagged basis/reference inputs; declared_result must name its result and invariant, never discard context by default. Package authors add a quantity rule for a genuinely new composition; adapters do not invent one.

A relation of operation selections/conversions is included in the typed stage’s derivations. Equation sides must have compatible complete types; the residual uses the applicable subtraction rule. Cross-package enthalpy reference changes and molar/mass conversions require a registered conversion with its parameter dependencies. Tests exercise composed expressions, weighted means, gauge/absolute pressure, scaled temperature differences and the standard package equations.

8.4 Bases and reference states

• Material flow basis (molar, mass, other) is a property of the state definition and of each reaction package; conservation-law expansion (§10) converts contributions to the law’s basis by inserting the conversion rule’s kernel (mw) rather than summing incompatible quantities.
• Reference states record T_ref, P_ref, include_enthalpy_of_formation, and phase. Enthalpy and entropy quantity types carry the reference state of the package that computed them; a Translator template between packages with different reference states must author the offset explicitly (the compiler refuses to equate them).

8.5 Nominal magnitudes

quantity_types.nominal_magnitude supplies the default scaling hint (ScaleSource = units) used when no better information exists (§16). Packages override per property in authored.default_scaling (IDAES default_scaling_factors).

9. Material systems and the property framework

This section maps the IDAES Generic (Modular) Property Framework (models/properties/modular_properties/**) and the property base classes onto relations, templates, and kernels. The governing change: IDAES resolves properties lazily by attribute access (build_on_demand in core/base/util.py); the platform resolves them once, explicitly, in pass P6, and persists the resolution.

9.1 From GenericParameterBlock configuration to relations

The IDAES rule that every component must be valid in at least one phase, and the derived phase_component_set, become the inference rule inferred.phase_species (§14.2) plus invariant closure:species_has_phase.

9.2 State definitions are templates

Each IDAES state-definition module (FTPx, FcTP, FpcTP, FPhx, FcPh, FpTPxpc) becomes a state_block template shipped in the reference package. Its declarations are exactly the IDAES ones:

Every state template declares the four interface contributions IDAES requires of a state block (get_material_flow_terms(p,j), get_enthalpy_flow_terms(p), get_material_density_terms(p,j), get_energy_density_terms(p)) as named expressions in template_requirements/template_display:

material_flow_term[p,j]     = flow_mol_phase_comp[p,j]
enthalpy_flow_term[p]       = flow_mol_phase[p] · enth_mol_phase[p]
material_density_term[p,j]  = dens_mol_phase[p] · mole_frac_phase_comp[p,j]
energy_density_term[p]      = dens_mol_phase[p] · energy_internal_mol_phase[p]

plus material_flow_basis = molar, default_material_balance_type = componentTotal, default_energy_balance_type = enthalpyTotal, define_port_members = state symbols, define_display_vars labels. Mole and phase fractions carry the IDAES bounds (mole_frac: [1e-20, 1.001], phase_frac: [0, ∞)) as template defaults.

The defined_state/always_flash rule of IDAES becomes one inference rule producing inferred.state_flash_required(state_instance):

flash_required(S) ← state_instance(S, pkg) ∧ has_phase_equilibrium_pairs(pkg)
                  ∧ ( ¬defined_state(S) ∨ always_flash(template_of(S)) )

Only state instances with flash_required = true receive _teq[pair], the phase-equilibrium constraints, and the smooth-VLE machinery.

9.3 The method registry

Every IDAES “method object” (a class exposing build_parameters and return_expression) becomes a reference.method_specs row. The parameter kinds it needs, in its natural units, are declared once; P9 inserts unit conversions to the package unit set (§8.2).

Two representative method templates in the authoring language:

method: pure.NIST.cp_mol_ig_comp@1
  family: pure_component
  provides: [cp_mol_ig_comp]
  parameters:
    - {name: A, quantity: heat_capacity_molar, unit: "J/mol/K"}
    - {name: B, unit: "J/mol/K/kK"}   # ... C, D, E, F, G, H as in NIST.py
  inputs: [{name: T, quantity: temperature, unit: K}]
  output_unit: "J/mol/K"
  expression: "A + B*t + C*t^2 + D*t^3 + E/t^2 where t = T/1000{K}"

method: rate_constant.arrhenius@1
  family: rate_constant
  provides: [k_rxn]
  parameters:
    - {name: arrhenius_const, unit: derived}      # unit derived by P9 from basis and concentration form
    - {name: energy_activation, unit: "J/mol"}
  expression: "arrhenius_const * exp(-energy_activation / (R * state.temperature))"

Unit derivation for arrhenius_const follows the IDAES rule exactly: for moleFraction, massFraction, activity forms the rate unit is amount·volume⁻¹·time⁻¹ (or mass basis); otherwise the concentration unit is raised to the total reaction order. This rule is a conversion_rules.kind = kernel entry, not code inside the template.

9.4 The equation-of-state contract

An eos template must provide expression declarations for the property kinds IDAES’s EoSBase enumerates. The reference eos.ideal template ships the complete equation set:

The reference eos.cubic template carries the Peng–Robinson and Soave–Redlich–Kwong parameterization (u, w, ΩA, Ωb), Soave alpha and its two temperature derivatives, the one-fluid mixing rules for am, bm, dam/dT, d²am/dT², delta[p,i], the dimensionless A[p], B[p], and the departure functions for enthalpy, entropy, internal energy, cp, cv, Gibbs energy, partial molar volume, speeds of sound, and the fugacity coefficient kernel ln φ_j = (b_j/bm)(Z − 1) − ln(Z − B) + (A/(B·p̂))·(b_j/bm − δ_j)·ln((2Z + B(u + p̂))/(2Z + B(u − p̂))), all as expressions over the IR. Equilibrium-state duplicates (_eq quantities at _teq) are the same expressions with T := _teq[pair], generated by template instantiation rather than hand-copied.

The compressibility root is the one non-algebraic step. It is a KernelSpec:

kernel cubic.compress_fact@1
  behavior: implicit                       -- z³ + b z² + c z + d = 0 with b, c, d from (A, B, u, w)
  inputs:  A (dimensionless), B (dimensionless), eos_type (enum)
  outputs: z_liq (lowest real root), z_vap (highest real root)
  branch_policy: {liquid: min_real_root, vapor: max_real_root, single_root: that_root}
  derivatives: jacobian (implicit function theorem on the selected root), hessian (available)
  bindings: native, datafusion_udf, nl_external_function ("cubic_root_l", "cubic_root_h"), pyomo_external_function

The native evaluation solves the cubic in closed form with Newton polishing and returns the branch and its derivatives; the NL binding ships a compiled external-function library with the same two symbols IDAES uses so that existing solver binaries work unchanged.

9.5 Phase equilibrium

Relations and generated mathematics per property package:

1. Pairs and species. phase_equilibrium_pairs lists (p1, p2); inferred.phase_equilibrium_species(pair, j) holds every species present in both phases (this is IDAES’s phase_equilibrium_list with keys PE<n>). The classification identify_VL_component_list becomes derived columns role ∈ {vl, henry, l_only, v_only}.
2. Equilibrium temperature. For each flash-required state instance and pair: symbol _teq[pair] (temperature, bounded like temperature).
3. Equilibrium constraint. For each pair and species with role vl or henry: equilibrium_constraint[pair, j] from the pair’s form (fugacity: fug_phase_comp_eq(p1) = fug_phase_comp_eq(p2); log_fugacity: the log form), family SMOOTH_TRANSCENDENTAL, role HARD_FEASIBILITY.
4. SmoothVLE (Burgard et al.): optional _t1[pair] with _t1 = smooth_max(T, temperature_bubble[pair], eps_1) when there are no vapor-only species, and _teq = smooth_min(_t1, temperature_dew[pair], eps_2) when there are no liquid-only species; otherwise the degenerate forms _teq = _t1 or _teq = T.
5. CubicComplementarityVLE (Dabadghao et al.): temperature slacks s[p] ≥ 0, _teq − T − s[vap] + s[liq] = 0, smooth_min(s[p]·(mol/s)/K, flow_mol_phase[p], eps_t) = 0; cubic second-derivative expression 6Z + 2b, split gp − gn, and smooth_min(gn·f, flow_mol_phase[vap], eps_z) = 0, smooth_min(gp·f, flow_mol_phase[liq], eps_z) = 0. Requires both phases to use the same cubic template (an invariant on method_selections).
6. Bubble and dew points. Symbols temperature_bubble[pair], temperature_dew[pair], pressure_bubble[pair], pressure_dew[pair] and the transition compositions _mole_frac_{tbub,tdew,pbub,pdew}[pair, j] (plus log forms) are created only when the property demand closure requests them (initialization plans do). IdealBubbleDew generates the Raoult/Henry sums (Σ_j z_j·Psat_j(Tbub) + Σ_henry z_j·H_j(Tbub) − P = 0, P·(Σ z_j/Psat_j(Tdew) + …) − 1 = 0, and the pressure forms); LogBubbleDew generates per-species log_fug_phase_comp_Tbub(liq, j) = log_fug_phase_comp_Tbub(vap, j) and Σ_j _mole_frac_tbub[j] = 1.
7. Henry’s law. HenryType value encodes direction (Hcp = 1, Hxp = 2 are constants; Kpc = 51, Kpx = 52 are volatilities). The concentration term is selected by type (conc_mol_phase_comp for Hcp/Kpc, mole_frac_phase_comp for Hxp/Kpx, with _true/_apparent suffix for electrolytes), and henry_pressure = c/H or c·K. Cubic EOS packages reject Henry species (invariant).

9.6 Property demand resolution (pass P6)

Decision: ADR-0040

P6 reads normalized demand seeds and the pinned method candidate universe, not equations produced by P7/P8 or plans produced by P15. P3 extracts member/property references from the authored expression ASTs; P4 resolves their feature guards; P5 binds finite scope/domain identities. Law contributions, ports and displays expose their property signatures at that same declarative boundary. Initializer and other opaque template operations declare template_property_requirements (§6.6); expression-derived requirements are generated rather than independently authored twice.

seeds := bound, decided requirements from normalized templates/laws/ports/displays/initializers
queue := seeds in semantic-key order
while an unprocessed requirement remains:
  form candidates from the complete pinned selections/defaults and state declarations
  select by the declared applicability and preference order, independent of arrival order
  zero candidates -> explicit unsupported; tied candidates -> explicit ambiguous
  one candidate -> persist its resolution and add its declared requires with provenance
stop when the finite set of (scope, property kind, bound index) keys is unchanged

P6 materializes inferred.property_requirements, inferred.method_resolutions and unresolved/ambiguous findings. Recursive requirements are monotone additions over the finite P5 scope; a declaration that creates an unbounded new scope or alternates selection is rejected. Negative selection reads the complete lower-stratum candidate set, including absence, and therefore invalidates when a new candidate appears. Default/explicit selection precedence is fixed in the registry, not chosen by execution order.

P7–P9 realize the selected state/method declarations into symbols, indexed equations and kernel bindings. Every advertised provision must produce its declared output; every property read by a realized equation or P15 plan must already have a resolution. A missing output is prop.incomplete_provider; late demand is prop.undeclared_requirement. Compilation fails rather than silently extending P6 or claiming that declarations make implementation mistakes impossible. The Property Interrogator remains a read-only projection of the requirement/resolution relations. Variable-plus-constraint versus expression provision is preserved by the templates and symbol roles.

A changed initializer selection changes its declarative demand input and reruns the affected stages before P15 binds a plan. This is one acyclic compiler schedule with a bounded internal P6 fixed point, not an implicit cycle across passes.

9.7 Reaction packages

GenericReactionParameterBlock configuration maps to reactions, stoichiometry, reaction_methods, parameter_values (including reaction_order[p,j] overrides; defaults: rate reactions order = −ν for reactants and 0 for products, equilibrium reactions order = +ν, zeroed for solid phases), and reaction_packages (which also carries reaction_basis). The reaction block template declares:

The “reaction package must match its property package’s unit set and required properties” check (_validate_property_parameter_units, _validate_property_parameter_properties) becomes two invariants on reaction_packages.

9.8 Provider kernels: Helmholtz, CoolProp, FeOs

• Helmholtz. IDAES computes pure-component Helmholtz properties through a compiled library that evaluates AMPL NL expression files (phi_ideal, phi_residual, their derivatives, saturation approximations) with a density root solve. The platform authors the same φ⁰(δ, τ) and φʳ(δ, τ) forms as expression templates (phi_ideal_type01..04, phi_residual_type01..05, surface_tension_type01), loads the per-fluid coefficient JSON into parameter_values, and exposes one implicit kernel helmholtz.state@1 whose unknowns are (δ, τ) given the chosen state pair (PH, PS, PU, TPX) and whose outputs are the mass-basis properties IDAES exposes (h, s, u, g, f, cv, cp, w, v, plus transport when available). Mole basis is a conversion rule with mw. Derivatives come from the implicit function theorem on the templates’ own derivatives, so no hand-coded derivative tables are needed. This removes the external-library dependency for the native backend; the NL backend keeps a generated external-function library with the IDAES symbol names.
• CoolProp. The CoolPropWrapper supplies coefficients for the exponential, non-exponential, and polynomial forms; the platform treats it as a parameter source producing parameter_values rows (with provenance), never as a runtime dependency of the solve.
• FeOs. Conditional provider crate under KernelSpec for SAFT and cubic models written over num-dual, giving derivatives natively. It is enabled only when feos-core tracks num-dual 0.15: feos-core 0.10.1 requires num-dual ^0.14, whose DualNum trait has a different shape from 0.15 (DualNum<f64> versus DualNum<Primitive = f64>), so one kernel source cannot serve both, and the platform writes every kernel against 0.15. FeOs also imports the quantity crate, a third units representation, which is never an authority for anything. Coverage (which property kinds, which conventions, which reference states) is declared per kernel and validated by parity tests; nothing is assumed.

9.9 Electrolytes and inherent reactions

The electrolyte extensions (true/apparent species bases, AqueousPhase, ENRTL, ion sets, apparent_inherent_reaction_extent, true_to_appr_species and appr_to_true_species constraints) are supported by the schema (component_type, charge, dissociation_species, StateIndex, _true/_apparent property variants) but are scheduled for phase 4 (§25). Inherent reactions are property-package-owned reactions (reactions.kind = inherent); their generation terms are emitted by law expansion exactly like rate and equilibrium reactions, but the extents live in the state scope.

9.10 Scaling defaults and validity

default_scaling rows (IDAES default_scaling_factors) and method_specs.validity intervals (IDAES metadata valid_range) are consumed by the scaling pass (§16) and by the diagnostics “values outside valid range” check (§15).

10. Balance laws and control volumes

10.1 The conservation law template

IDAES control volumes assemble balances procedurally (add_material_balances, add_energy_balances, add_momentum_balances and their variants in control_volume0d.py and control_volume1d.py). The platform replaces the procedure with one law template expanded over contributions (decision D7):

law_template conservation@1
  inputs:  scope (control region), subject selector (species j | element e | total | phase p | energy | momentum),
           basis, temporal policy (accumulation | none), balance options
  expansion:
    1. collect contributions whose law_family matches and whose subject matches the selector (semantic match, §8.4)
    2. classify each by orientation: into_scope (+), out_of_scope (−), generation (+), accumulation (−)
    3. convert each to the law's basis (insert conversion kernels, never sum incompatible quantities)
    4. group by the balance index (t; t,x; and p,j | j | e | none depending on balance type)
    5. emit one equation per index: Σ signed contributions = 0, family NETWORK_BALANCE or AFFINE_EQUALITY when affine,
       role HARD_FEASIBILITY, with derivation listing every contribution
    6. emit negative-completeness rows for candidate contributions that were excluded and why

Material, element, energy, and momentum balances are bindings of this template with different subjects and contribution kinds. The control-volume templates declare the contributions; the compiler does the bookkeeping.

10.2 Contribution kinds

The table lists every contribution IDAES control volumes can produce, the condition under which the template declares it, and its orientation. F(p,j) is the state’s material_flow_term, H(p) its enthalpy_flow_term.

Balance type selects the grouping and which contributions participate:

EnergyBalanceType.enthalpyTotal and isothermal (extended control volumes: T_in = T_out, or T[t, prev(x)] = T[t, x] in 1D) and MomentumBalanceType.pressureTotal are the implemented bindings; enthalpyPhase, energyTotal, energyPhase, pressurePhase, momentumTotal, momentumPhase are declared enum members whose law bindings are absent, so selecting them raises law.unsupported_binding at P8, exactly where IDAES raises BalanceTypeNotSupportedError.

10.3 The lumped control volume template (cv.lumped@1, IDAES ControlVolume0DBlock)

template: cv.lumped@1
  kind: control_volume
  params:
    property_package: {type: property_package, required: true}
    reaction_package: {type: reaction_package, default: null}
    property_package_args, reaction_package_args: {type: options}
  features:
    dynamic: {kind: bool, inherit_from: parent.dynamic}
    has_holdup: {kind: bool, default: "= dynamic"}      # rule: dynamic ⇒ has_holdup
    has_phase_equilibrium: {kind: bool, required_explicit: true}
    information_flow: {kind: enum, enum: FlowDirection, default: forward}
    has_rate_reactions, has_equilibrium_reactions, has_mass_transfer, has_heat_transfer,
    has_work_transfer, has_enthalpy_transfer, has_heat_of_reaction, has_pressure_change: {kind: bool, default: false}
    material_balance_type, energy_balance_type, momentum_balance_type: {kind: enum}
  domains:
    time: {from: flowsheet.time}
    phase, species, phase_species: {from: property_package}
  submodels:
    properties_in:  {template: "= property_package.state_block", multiplicity: time,
                     bindings: {defined_state: "= information_flow == forward", has_phase_equilibrium: "= has_phase_equilibrium"}}
    properties_out: {template: "= property_package.state_block", multiplicity: time,
                     bindings: {defined_state: "= information_flow == backward", has_phase_equilibrium: "= has_phase_equilibrium"}}
    reactions:      {template: "= reaction_package.reaction_block", multiplicity: time, guard: "reaction_package != null",
                     bindings: {state: properties_out, has_equilibrium: "= has_equilibrium_reactions"}}
  symbols:
    volume[t]                       : {quantity: volume, guard: has_holdup, default_initial: 1}
    phase_fraction[t,p]             : {quantity: phase_fraction, role: "= n_phases > 1 ? variable : expression(1)", default_initial: 1/n_phases}
    material_holdup[t,p,j]          : {quantity: amount (or mass by basis), guard: has_holdup}
    material_accumulation[t,p,j]    : {role: derivative, of: material_holdup, wrt: time, guard: dynamic}
    rate_reaction_extent[t,r]       : {quantity: molar_flow (reaction basis), guard: has_rate_reactions}
    rate_reaction_generation[t,p,j] : {guard: has_rate_reactions}
    equilibrium_reaction_extent[t,r], equilibrium_reaction_generation[t,p,j]: {guard: has_equilibrium_reactions}
    inherent_reaction_extent[t,r], inherent_reaction_generation[t,p,j]:       {guard: "properties_out.include_inherent_reactions"}
    phase_equilibrium_generation[t,pe]: {guard: "has_phase_equilibrium and material_balance_type == componentPhase"}
    mass_transfer_term[t,p,j]       : {guard: has_mass_transfer}
    energy_holdup[t,p], energy_accumulation[t,p]: {guard: has_holdup / dynamic}
    heat[t], work[t], enthalpy_transfer[t]: {quantity: power, guards: has_heat_transfer / has_work_transfer / has_enthalpy_transfer}
    heat_of_reaction[t]             : {role: expression, guard: has_heat_of_reaction}
    deltaP[t]                       : {quantity: pressure_difference, guard: has_pressure_change}
    element_holdup[t,e], element_accumulation[t,e], elemental_flow_in[t,p,e], elemental_flow_out[t,p,e], elemental_mass_transfer_term[t,e]:
                                      {guard: material_balance_type == elementTotal (and holdup/dynamic/mass-transfer as applicable)}
  equations:
    sum_of_phase_fractions[t]:            "1 == sum(p in phase | phase_fraction[t,p])"            guard: n_phases > 1 and has_holdup
    material_holdup_calculation[t,p,j]:   "material_holdup[t,p,j] == volume[t]*phase_fraction[t,p]*properties_out[t].material_density_term[p,j]"
    rate_reaction_stoichiometry[t,p,j]:   "rate_reaction_generation[t,p,j] == sum(r in rate_reactions | stoich[r,p,j]*rate_reaction_extent[t,r])"
    equilibrium_reaction_stoichiometry, inherent_reaction_stoichiometry: analogous
    energy_holdup_calculation[t,p]:       "energy_holdup[t,p] == volume[t]*phase_fraction[t,p]*properties_out[t].energy_density_term[p]"
    elemental_holdup_calculation[t,e]:    "element_holdup[t,e] == volume[t]*sum((p,j) in phase_species | conv(j)*phase_fraction[t,p]*material_density_term[p,j]*element_comp[j,e])"
  contributions: (as in §10.2, each guarded by its feature)
  law_instances:
    material: {law: conservation, subject: "= material_balance_type", scope: self, options: {basis: properties_out.material_flow_basis}}
    energy:   {law: conservation, subject: energy,   guard: energy_balance_type != none}
    momentum: {law: conservation, subject: momentum, guard: momentum_balance_type != none}
  requirements: [material_flow_terms, enthalpy_flow_terms, material_density_terms (if has_holdup), energy_density_terms (if has_holdup)]

The generated equations reproduce IDAES’s forms exactly, for example the componentPhase material balance:

accum[t,p,j] == F_in[t,p,j] − F_out[t,p,j] + rate_gen[t,p,j]·rxn_conv(t,j) + equil_gen[t,p,j] + inherent_gen[t,p,j]
                + phase_equilibrium_term(t,p,j) + mass_transfer_term[t,p,j] + user_mol(t,p,j) + user_mass(t,p,j)

with accum = convert(material_accumulation → flow units) when dynamic and 0 otherwise; the total enthalpy balance Σ_p accum_E[t,p] == Σ_p H_in[t,p] − Σ_p H_out[t,p] + heat + work + enthalpy_transfer + heat_of_reaction + custom; and the pressure balance 0 == P_in − P_out + deltaP + custom.

Units follow the IDAES derivations (flow_units from the flow basis, acc_units = amount/time or mass/time, holdup_units, rxn_flow_units from the reaction basis); a mismatch between flowsheet time units and package time units is invariant cv.time_units_consistent.

10.4 The distributed control volume template (cv.distributed_1d@1, IDAES ControlVolume1DBlock)

Additional parameters and features: area_definition (DistributedVars.uniform ⇒ scalar area; variant ⇒ area[t,x]), transformation_method (dae.finite_difference | dae.collocation), transformation_scheme (BACKWARD, FORWARD for finite difference; LAGRANGE_RADAU, LAGRANGE_LEGENDRE for collocation), finite_elements, collocation_points, flow_direction, length_domain_set, optional external length_var.

Structural facts preserved from IDAES:

• length_domain is a normalized, dimensionless continuous domain on [0, 1]; the physical length is the scalar symbol length (or an external reference). The mathematics is written per unit ξ, so every per-unit-length source term is multiplied by length.
• _flow_direction_term = −1 for forward, +1 for backward; ports bind to properties[:, first] / properties[:, last] accordingly (§12.1).
• One indexed state block properties[t, x]; defined_state = true only at the inlet end (first for forward, last for backward), which is an idx_map in IDAES and a per-member binding rule here.
• Linking symbols _flow_terms[t,x,p,j] = properties[t,x].material_flow_term[p,j] and _enthalpy_flow[t,x,p] = properties[t,x].enthalpy_flow_term[p] exist so that the spatial derivatives material_flow_dx, enthalpy_flow_dx, elemental_flow_dx, pressure_dx are derivatives of symbols, not of expressions.
• Per-unit-length quantities: material_holdup (amount/length), rate_reaction_generation (flow/length), heat, work, enthalpy_transfer (power/length), deltaP (pressure/length).
• Boundary skip rule. Balance and stoichiometry equations are skipped at x = first unless the scheme is FORWARD, in which case they are skipped at x = last. This is a filter on the equation template index (filter: "x != boundary(scheme)"), evaluated by P12 (index expansion) against the mesh that P11 produced.

Generated equations (componentPhase form):

material_flow_linking[t,x,p,j]:   _flow_terms[t,x,p,j] == properties[t,x].material_flow_term[p,j]
material_balances[t,x,p,j]:       L·convert(material_accumulation[t,x,p,j]) == fdt·material_flow_dx[t,x,p,j]
                                     + L·rate_gen·rxn_conv + L·equil_gen + L·inherent_gen + L·pe_term + L·mass_transfer + L·custom
material_holdup_calculation:      material_holdup[t,x,p,j] == area(t,x)·phase_fraction[t,x,p]·material_density_term[p,j]
enthalpy_balances[t,x]:           L·convert(Σ_p energy_accumulation[t,x,p]) == fdt·Σ_p enthalpy_flow_dx[t,x,p] + L·(heat + work + enthalpy_transfer + heat_of_reaction + custom)
pressure_balance[t,x]:            0 == fdt·pressure_dx[t,x] + L·deltaP[t,x] + L·custom
element_balances[t,x,e]:          L·convert(element_accumulation) == fdt·elemental_flow_dx[t,x,e] + L·elemental_mass_transfer + L·custom

The IDAES apply_transformation() step is the discretization policy attached to length_domain (§13.4); the transformation must be applied to a domain owned by the control volume (external domains are rejected, invariant cv.length_domain_owned).

10.5 What the template model changes structurally

The unit-model survey identified five mechanisms IDAES uses that a declarative template must express without code:

The recurring IDAES initialization idiom (fix a transfer term to a guess, deactivate its defining equation, solve, then release) is a two-stage initialization_plan (§17).

11. The unit model library as templates

11.1 Catalog

Every IDAES generic unit becomes a reference template. The table records composition, the features that switch structure, extra symbols and equations beyond the control volume, ports, and the default plans. Equations are summarized; full forms are in the survey catalogs and are reproduced in the reference package sources.

Costing methods (FlowsheetCostingBlock, UnitModelCostingBlock, SSLW) are templates of kind costing_method (§19.5).

11.2 Worked template: the heater

template: unit.heater@1
  kind: unit
  idaes_class: idaes.models.unit_models.Heater
  params:
    include: [param_group.unit_base, param_group.heater_side]   # dynamic, has_holdup; balance types, has_phase_equilibrium, has_pressure_change, property_package(+args)
  submodels:
    control_volume:
      template: cv.lumped@1
      bindings:
        dynamic: "= dynamic"; has_holdup: "= has_holdup"
        property_package: "= property_package"; property_package_args: "= property_package_args"
        has_phase_equilibrium: "= has_phase_equilibrium"
        material_balance_type: "= material_balance_type"; energy_balance_type: "= energy_balance_type"; momentum_balance_type: "= momentum_balance_type"
        has_heat_transfer: true; has_pressure_change: "= has_pressure_change"
        geometry: "= has_holdup"
  symbols:
    heat_duty[t]: {role: reference, reference_to: control_volume.heat[t], quantity: power}
    deltaP[t]:    {role: reference, reference_to: control_volume.deltaP[t], guard: "has_pressure_change and momentum_balance_type != none"}
  ports:
    inlet:  {kind: material, direction: inlet,  bound_to: control_volume.properties_in}
    outlet: {kind: material, direction: outlet, bound_to: control_volume.properties_out}
  display:
    performance: [{label: "Heat Duty", expr: heat_duty[t]}, {label: "Pressure Change", expr: deltaP[t], guard: has_pressure_change}]
  plans:
    initializer: init.single_control_volume@1
    scaler: scale.heater@1

Nothing in this template computes; the compiler instantiates the control volume, resolves the property package’s state template into properties_in[t] and properties_out[t], expands the three law instances, derives port members from the state template, and emits the equations of §10.3 with derivations back to this file.

11.3 Worked template fragment: isentropic pressure change

template: unit.pressure_changer@1
  features:
    compressor: {kind: bool, default: true}
    thermodynamic_assumption: {kind: enum, enum: ThermodynamicAssumption, default: isothermal}
    support_isentropic_performance_curves: {kind: bool, default: false}
  submodels:
    control_volume: {template: cv.lumped@1, bindings: {has_work_transfer: true, has_pressure_change: true, ...}}
    properties_isentropic: {template: "= property_package.state_block", multiplicity: time, guard: "thermodynamic_assumption == isentropic",
                            bindings: {defined_state: false, has_phase_equilibrium: "= has_phase_equilibrium"}}
    performance_curve: {template: helper.isentropic_performance_curve@1, guard: support_isentropic_performance_curves}
  symbols:
    work_mechanical[t]: {role: reference, reference_to: control_volume.work[t]}
    ratioP[t]: {quantity: dimensionless, default_initial: 1}
    efficiency_isentropic[t]: {quantity: dimensionless, default_initial: 0.8, guard: "thermodynamic_assumption == isentropic"}
    work_isentropic[t]: {quantity: power, guard: "thermodynamic_assumption == isentropic"}
  equations:
    ratioP_calculation[t]: "ratioP[t] * control_volume.properties_in[t].pressure == control_volume.properties_out[t].pressure"
    isothermal[t]:        {expr: "properties_in[t].temperature == properties_out[t].temperature", guard: "thermodynamic_assumption == isothermal"}
    zero_work_equation[t]:{expr: "control_volume.work[t] == 0", guard: "thermodynamic_assumption == adiabatic"}
    isentropic_pressure[t]: {expr: "properties_isentropic[t].pressure == control_volume.properties_out[t].pressure", guard: isentropic}
    isentropic[t]:          {expr: "properties_isentropic[t].entr_mol == control_volume.properties_in[t].entr_mol", guard: isentropic}
    isentropic_energy_balance[t]:
      {expr: "work_isentropic[t] == sum(p in phase | properties_isentropic[t].enthalpy_flow_term[p]) - sum(p in phase | control_volume.properties_in[t].enthalpy_flow_term[p])", guard: isentropic}
    actual_work[t]:
      {expr: "if compressor then work_isentropic[t] == work_mechanical[t]*efficiency_isentropic[t] else work_mechanical[t] == work_isentropic[t]*efficiency_isentropic[t]", guard: isentropic}
  law_instances:
    isentropic_state_material: {law: state_material_balance, subject: "= material_balance_type", states: [properties_isentropic, control_volume.properties_out], guard: isentropic}

if compressor then … else … is a compile-time conditional on a feature, so P7 selects one branch and no Conditional node reaches the backend.

11.4 Derived templates

template: unit.turbine@1
  derives_from: unit.pressure_changer@1
  feature_rules:
    - {rule: requires, antecedent: "true", consequent: "compressor == false"}
    - {rule: requires, antecedent: "true", consequent: "thermodynamic_assumption == isentropic"}
  plans: {initializer: init.isentropic_pressure_changer@1}

A derived template narrows features and parameters; it cannot add code. The three IDAES wrappers (Compressor, Pump, Turbine) and the valve are four such rows.

12. Connectivity

12.1 Ports

A port is a typed interface derived from a state instance, never a hand-maintained list of variables.

• Members come from the bound state template’s define_port_members declaration (default: the state symbols). For an IDAES-compatible FTPx state a material port has members flow_mol, mole_frac_comp[j], temperature, pressure; each member row records quantity_type_id and the symbol group.
• Kinds. material (state-defined members), heat (a single power quantity), work, signal (a dimensionless or typed scalar). Heat and signal ports are how heat integration and controllers connect without inventing pseudo-streams.
• Direction is declared (inlet, outlet, bidirectional) and used only for topology inference and stream tables; equations never depend on it.
• Distributed control volumes bind ports to properties[:, first] for the inlet and properties[:, last] for the outlet when flow_direction = forward, reversed for backward. The binding is a Gather with a fixed coordinate on the length domain.
• Expression members (IDAES VarLikeExpression, used by the ideal separator and the stream scaler) are legal: a member may reference an expression symbol; the connection rule then produces an equation between an expression and a variable, which is still an affine equality when the expression is affine.

12.2 Connection rules

authored.connections records an arc; rule_template_id selects the physics of the junction. The reference package ships:

A connection whose ports have mismatched member sets or incompatible quantity types (different reference states, bases, or shapes) is invariant conn.member_mismatch; IDAES surfaces this only at arc expansion as a Pyomo error. Pyomo’s Port.Extensive rule (automatic splitting and mixing at multi-arc ports) is intentionally not provided: mixers and separators are templates with explicit physics (proposal §5.3).

12.3 Arc expansion is a pass, not a transformation call

P8 generates inferred.connection_equations (one row per generated equation with connection_id, member_ordinal, index, equation_id) and the corresponding compiled.math_indexed_equations rows, which P12 expands to scalar math_equations. There is no separate “expand arcs” step to forget: a snapshot with connections always compiles to a problem that contains them. Deactivating a connection for an initialization stage is a case_activations overlay (§17).

12.4 Value propagation is a case operation

IDAES propagate_state(arc, direction, overwrite_fixed) copies port member values from source to destination. The platform defines it as a runtime operation producing an overlay case:

propagate(connection, direction = forward | backward, overwrite_fixed = false):
  for each member m and index i of the connection's ports:
    if destination symbol is free, or overwrite_fixed:
      case_specs.initial[destination symbol] := current value of source symbol

The operation writes rows, never mutates a model; initialization plans (§17) sequence such operations explicitly.

12.5 Topology closure

Pass P5 materializes:

• inferred.instance_tree (transitive containment) and inferred.instances (expanded submodels, including per-time and per-node state instances);
• inferred.topology_edges: a directed multigraph of unit instances from connections (built in petgraph, persisted as rows);
• inferred.scope_members and inferred.boundary_crossings for every authored.scopes row, using the cut-set rule (internal, external, inbound, outbound) of the metamodel doctrine; flowsheet-level element and energy audits and the diagnostics “external variables” check are queries over these relations;
• inferred.tear_candidates: for each cycle family of the unit graph, the edge set that breaks all cycles, computed as a feedback arc set: heuristic uses petgraph::algo::greedy_feedback_arc_set over the unit multigraph (it returns exactly the recycle edges on flowsheet-shaped graphs, and reproduces the FOQUS cycle-count choice under the platform’s edge weights), and mip solves the minimum feedback arc set through the NL backend and a MILP solver. The search is library work; the tear policy — which connections may be torn and at what cost — is authored data on connections.tear_cost. Tears feed the sequential-modular initialization plan (§17.4).

12.6 Translators, state junctions, and scalers

• Translator. The template declares two states and no equations; the author supplies mapping equations in the instance’s instance_equations (an authored relation of the same shape as template_equations but instance-scoped). The compiler’s unit inference refuses enth_mol_in == enth_mol_out across packages with different reference states unless the author converts explicitly, which is the failure mode IDAES leaves to the user.
• StateJunction. One state, two ports; connection equations on both sides make it a named stream. Stream tables treat it as a stream anchor.
• StreamScaler. Outlet members are expressions multiplier · inlet member for flow-like members (members whose quantity kind is extensive) and pass-through otherwise; the “flow-like” test is by quantity kind (extensive = true), not by name substring as in IDAES.

13. Flowsheets, time, and dynamics

13.1 The flowsheet template and time domain

unit.flowsheet@1 (IDAES FlowsheetBlock) declares:

The time domain is an authored.domains row of kind time, continuous = dynamic, with its members in domain_members (steady state: one member t = 0; dynamic: the breakpoints, extended by discretization in P11). Every state instance and time-indexed symbol is indexed by this domain even in steady state, so state[0.0] addressing survives unchanged.

13.2 Dynamic and holdup inference

IDAES _setup_dynamics becomes four rules in inferred.instance_features:

dynamic(I)     ← authored dynamic(I) = true
dynamic(I)     ← authored dynamic(I) = inherit ∧ parent(I, P) ∧ dynamic(P)
has_holdup(I)  ← authored has_holdup(I) = inherit ∧ dynamic(I)            -- default follows dynamic
error(I, dynamic_in_steady_parent) ← dynamic(I) ∧ parent(I, P) ∧ ¬dynamic(P)
error(I, holdup_required)          ← dynamic(I) ∧ authored has_holdup(I) = false

13.3 Accumulation terms and derivative symbols

When an instance is dynamic, the control-volume templates declare material_accumulation[t,(p,j)], energy_accumulation[t,p], element_accumulation[t,e] with role = derivative, wrt = time. P7 creates the symbol and a math_dae_links row linking it to its state (material_holdup, …). The balance equations reference the derivative symbol; nothing else changes. Steady-state instances substitute 0 for the accumulation contribution (a compile-time guard, not a runtime branch).

The IDAES steady-state initial condition convention (fix_initial_conditions: fix every accumulation at t0 to zero) is the reference case overlay overlay.steady_initial_conditions@1.

13.4 Discretization is a lowering pass (P11)

An authored.discretization_policies row (selected per domain by continuous_domains.discretization_policy_id or overridden by case_policies) attaches to a continuous domain (time or a control volume’s length domain). P11 implements Pyomo DAE’s semantics as data:

1. Mesh. finite_elements (nfe) equal-width elements over the domain bounds, retaining authored breakpoints; collocation adds ncp interior points per element (Radau roots include the element end, Legendre roots do not). Rows in compiled.meshes, compiled.mesh_nodes.
2. Symbol expansion. Every symbol indexed by the domain is expanded from its authored members to all mesh nodes (lifecycle = generated_discretization, IDs from §5.1).
3. Discretization equations. For every derivative symbol d of state v with respect to the domain, one equation per node per the scheme, named <d>_disc_eq[...]:

Second derivatives use adotdot (collocation) or the standard three-point stencils (finite difference); mixed derivatives are unsupported (diagnostic disc.mixed_derivative, as in IDAES). Stencil weights are stored in compiled.stencils so that the equations are data-generated Affine nodes.

4. Integrals. Integral nodes lower to trapezoid sums over the mesh (Pyomo Integral semantics) unless a quadrature rule is attached.
5. Boundary filters on 1D balances (§10.4) are evaluated against the mesh.
6. Scaling propagation. The IDAES post-discretization step (scale_time_discretization_equations) is the rule sf(dv/dt) = sf(v) / sf(t) and sf(disc_eq) = sf(dv/dt), sf(cont_eq) = sf(v), applied by the scaling pass because P11 has already produced the rows; the “propagate after build” failure cannot occur.

13.5 Dynamic operations as case operations

13.6 Trajectory backends

• Fully discretized NLP (default). The problem after P11 is an algebraic system; any NLP backend solves it. Consistent initial conditions are a first stage of the dynamic plan (solve at t0 with everything else inactive).
• PETSc TS through the NL backend. The IDAES petsc_dae_by_time_element workflow is reproduced as a plan: per time interval, the NL writer emits the subsystem at the target node with the dae_suffix (ALGEBRAIC = 0, DIFFERENTIAL = 1, DERIVATIVE = 2, TIME = 3) and dae_link suffixes generated from math_dae_links, runs petsc_ts with --ts_init_time/--ts_max_time, reads the trajectory, and writes runtime.solutions for the interval, interpolating skipped nodes. A fixed derivative with a non-zero value and a fixed differential variable paired with a free derivative are the same errors IDAES raises (dae.fixed_derivative, dae.fixed_state_free_derivative).
• diffsol (optional). A native DAE integrator fed directly from the evaluation program (residual and Jacobian-vector-product callbacks; the evaluation program’s sparse rows supply the product). It is a backend binding; the IR is unchanged; DiffSL, its own model language, is never used. diffsol carries its own linear-algebra stack (diffsol-la/diffsol-nl, not faer), accepted for the trajectory backend only; whether every discretized process model of §13 is a semi-explicit DAE that diffsol accepts is verified per model by P16 (backend.unsupported_dae_class).

14. The compiler

14.1 Pass pipeline and contracts

Decision: ADR-0040, ADR-0041

Every pass is a reference.pass_specs row with pass_input_ports and pass_output_ports. The driver derives its DAG from named stage/port edges; reading a relation schema without an input binding is invalid. P0 resolves package headers from an input document bundle; P1 stages authored rows and change operations; P2 validates the candidate snapshot for atomic commit (§22.2). P1 does not mutate an existing committed snapshot. Compilation of a committed P2-valid snapshot starts at P3; P3 onward cannot output authored/reference relations.

An output bundle contains all declared output ports, including explicit empty relations. One port has one producer. The same compiled.math_* schema can appear in P7’s template bundle, P8’s law-completed bundle, P9’s method-completed bundle and P10’s typed bundle. Each stage reads its predecessor explicitly, preserves unaffected rows and publishes a complete immutable replacement bundle; it does not overwrite another stage’s artifact. Sidecar pass records and stage catalog rows describe these outputs and are excluded from the bundle they describe. Runtime operations similarly publish a complete named result bundle. All read dependencies, including registry and reference data, enter initial stage keys.

The useDefault resolution of balance types (representative state block’s defaults), IDAES’s _get_representative_property_block, is part of P4.

P12 is the pass that scalarizes indexed equation domains: P7–P11 keep every equation in its indexed form (one row per instance and declaration), and P12 is where the compact form is expanded, why, and with what IDs (doctrine P18 of the metamodel: expansion rules, domains, and generated identities are inspectable).

14.2 The rule compiler

Decision: ADR-0039, ADR-0044, ADR-0048

Set-oriented inference (P4–P6, P8’s contribution matching, P5’s scope closure) is written as reference.rule_specs, not as Rust loops over rows. A rule body is a typed RulePlanSpec (a bounded algebra: scan, filter, project, equi-join, anti-join for stratified negation, union, distinct, aggregate and unnest) compiled to a DataFusion LogicalPlan through LogicalPlanBuilder, optimized, and executed against the snapshot session. Example (rule for valid phase–species pairs):

rule phase_species_valid @1  stratum 1  head inferred.phase_species(material_system_id, phase_id, species_id, henry)
  ← material_system_species(M, J) ⋈ material_system_phases(M, P)
    ⋈ species(J, valid_phase_types) ⋈ phases(P, phase_type)
    where phase_type ∈ valid_phase_types            -- or valid_phase_types is null and phase is not aqueous
    ▷ authored.phase_species restricts (P, J) when the phase declares an explicit list
    left ⋈ henry_declarations(J, P) → henry := present

Execution rules:

1. Strata run in order; within a stratum rules are evaluated to a least fixed point. Recursive closures (containment, subtype, reachability) use the generic fixed-point executor: seed, evaluate delta plans, deduplicate by semantic key, append, repeat until no new rows. DataFusion’s recursive CTE support may be used for closures whose provenance can be reconstructed from the result; where per-row derivations are required the external loop is used.

2. Negation is stratified: a rule may negate only relations fully computed in a lower stratum.

3. Four-valued predicates. Matching predicates return true | false | unknown | conflict. true rows are written to the head relation. unknown and conflict rows are written to inferred.undecided with the rule, the key, a reason and the supporting rows, and produce diagnostics; a candidate is never silently dropped (metamodel P18), and no consumer can mistake an undecided fact for a true one because head relations hold only decided-true rows. Inside a plan, Expr::IsUnknown/IsNotUnknown cover the three SQL truth values; conflict (two rules asserting incompatible values for one key under conflict_policy = reject) is detected by the executor after the stratum’s fixed point.

4. Every head row carries a derivation (provenance.derivations: rule id, supporting rows, fingerprint) at the granularity the registry declares for the head relation (schema_relations.derivation_granularity): row for inference, law expansion and method resolution, where negative completeness is the deliverable; rule for mechanically expanded relations (index expansion, discretization, connection equations), where the derivation is the rule plus the ID formula of §5.1 and is reconstructed on demand rather than stored per row. A benchmark in §24.3 measures derivation rows and bytes per equation so the granularity assignment is evidence-based.

5. Determinism, and the engine as a declared input. Output relations are sorted by primary key before hashing (arrival order across partitions is nondeterministic and never assumed). The DataFusion session is built from a reference.engine_profiles row (§6.11): the DataFusion and Arrow crate versions; the ordered analyzer rule list installed with Analyzer::with_rules and the ordered optimizer rule list installed with Optimizer::with_rules (never the engine defaults, whose pipeline changes between releases; Analyzer::add_function_rewrite is never called after construction); the physical-optimizer rule list; and the semantic settings as an explicit, versioned key allow-list — every datafusion.optimizer.* and datafusion.sql_parser.* key, datafusion.execution.time_zone, datafusion.execution.skip_physical_aggregate_schema_check (asserted false) and datafusion.execution.enable_ansi_mode (asserted false) — read back from information_schema.df_settings; a namespace match would miss the two execution.* keys. Platform policy that can change a plan (bound kernel outcome-policy identifiers and the null policy of §14.2 rule 3) is exposed as a ConfigExtension under datafusion.pse.*, so it appears in df_settings and enters the same hash. These entries are generated views of the bound contract values; a session setting cannot independently override kernel null/strictness semantics. The profile artifact’s logical hash, the function registry hash (kernel digests) and the catalog snapshot are part of every rule-executing pass’s memo key (§14.3). The plan fingerprint is not part of the memo key: the plan is a pure function of the rule plan, the catalog snapshot and the engine profile, all of which are already in the key, so hashing the plan adds no information to reuse validity. Plan bytes are noncanonical diagnostic evidence under ADR-0044. Store logical_plan_to_bytes_with_extension_codec output with its physical encoding_checksum, engine version and platform codec version. The codec identifies snapshot providers by (snapshot_id, relation_id, schema_version, logical_hash) and UDFs by the complete kernel binding digest; decoding resolves only admitted pinned objects. Sorted HashMap insertion does not make encoding deterministic, and no two-process byte-equality assertion is allowed for equivalent plans. pgjson EXPLAIN and ordered analyzer/optimizer observations accompany the artifact for attribution. They are outside semantic stage outputs and memo inputs. Round-trip and attribution tests, rather than byte-equality tests, validate the diagnostic route.

6. Head semantic admission. Before execution the rule compiler infers complete field/quantity contracts for the typed expression graph, including UDF arguments and returns. Losing semantic metadata is not permission to reconstruct meaning from storage alone: the compiler must retain the source-to-expression typing derivation. At output admission compare the inferred contract and actual arrays with the generated head schema. Exact type, nullability and semantics are the default. A declared numeric conversion may use Arrow casting only after exact representability checks (including fractional-to-integer and large-integer-to-float cases); physical unit/basis/reference conversion invokes its named conversion and dependencies. can_cast_types and CastOptions { safe: false } do not prove losslessness. Attach the destination metadata only after these checks; validate nested values and required-field nulls, then execute the head invariants. Failure is rule.head_schema_mismatch, never silent relabeling.

7. Key discipline. Distinct, union-dedup and join keys in a rule plan are key-role columns (semantic IDs, ordinals, enums) — never Float64, because the engine’s value equality merges -0.0 with +0.0 and treats NaN as self-equal (§5.3). Null-bearing key comparisons use distinct/not_distinct semantics and Kleene booleans (and_kleene/or_kleene) so that a null-versus-null candidate is neither dropped nor matched by accident. The rule compiler rejects a violating rule_plan_nodes row (rule.float_key).

8. Rule nodes in plans (deferred). Wrapping each rule body in a LogicalPlan::Extension node that carries rule_id survives the optimizer intact and would make rule attribution visible in EXPLAIN; it costs the 14-method UserDefinedLogicalNode contract and an ExtensionPlanner. It is not adopted now: derivations and rules_fired already answer “which rule produced this row”. Trigger for adoption: the §22.4 diff report or agent tooling needs per-rule attribution inside a plan rendering.

9. Aggregate and unnest. LogicalPlanBuilder::aggregate groups P8 contribution descriptors by declared law/scope/domain keys; collect_ordered supplies terms in stable contribution-ID/index order, not physical arrival order. The generated operation includes an explicit order expression and required provenance inputs. Compiler sum is exact checked integer/count arithmetic initially; floating value reductions require a separately supported numerical policy, while P8 groups expression descriptors instead of summing solver values. count, min, max and collection have explicit null and empty-group rules from §6.11; missing input is not an evaluation failure. Unnest one declared list at a time through unnest_columns_with_options, retaining the parent key; reject or produce no members for null lists as declared, and no members for empty lists. Where member order is semantic, the input carries explicit (ordinal, value) structs; unnest does not invent stable identities from physical row positions. Duplicate memberships and domain validity are checked by invariants. New operations require a real consumer and an independent expansion fixture.

Generated ScalarUDFImpl wrappers for kernels (§18.5) are registered in the session so that rule bodies can evaluate kernels in batch (for example, estimating bubble-point temperatures for every state instance in one plan during initialization-plan generation).

14.2.1 Optional predicate preimages

Decision: ADR-0048

ScalarUDFImpl::preimage can provide a contiguous input range for supported output comparisons. It is disabled for platform UDFs until R-25 qualifies a concrete consumer. A replacement predicate must be equivalent, including half-open endpoints, nulls, selected branch and floating representability; monotonicity alone is insufficient. A conservative range may be used only as candidate pruning with the original residual predicate retained. Start with exact discrete/affine cases, compare full query results with pruning disabled, then measure avoided decoding and total cost. No general thermodynamic inverse is inferred from this hook.

14.3 The pass engine

Decision: ADR-0041, ADR-0042, ADR-0044, ADR-0046, ADR-0050

• Complete stage memo. Key a pass by ID/version, every named input port’s schema and logical hash or explicit absence, registry/operator/quantity contracts, relevant package/domain/binding/provider-candidate artifacts, kernel implementation digests, and selected policies. Plan-executing passes also key the complete engine profile and generated function registry. The driver binds this immutable environment; undeclared file/catalog/config reads fail the contract. A changed candidate universe or absent optional relation is a changed input. Initially P7 and every other stage are cached as complete bundles, not per-instance tuples. Attempt metadata is excluded from semantic output membership; changed semantic lineage remains in the stage snapshot. Reusing equal relation storage does not authorize skipping a dependent stage whose complete inputs changed.
• Measured refinement. A complete per-instance input bundle may be considered only when representative cold/warm measurements show whole-stage recomputation dominates preparation and the candidate passes full incremental-versus-clean comparisons (R-22). salsa remains deferred to the further R-01 trigger; neither a second dependency engine nor automatic read tracking is needed for initial execution.
• Snapshot sessions, shared runtime. Each snapshot has a sealed catalog and its own SessionContext, extension/UDF registries and ordered engine profile. The process driver injects a shared RuntimeEnv with SessionStateBuilder::with_runtime_env; compatible snapshots share one deployment FairSpillPool and DiskManager. Limits, spill settings, time provider and thread budget are explicit. Passes obtain sessions from the driver and cannot create private full-budget pools.
• Accounted allocation. Platform canonicalization, binding, transfer and result buffers register a MemoryConsumer and call MemoryReservation::try_grow before allocation; estimates include temporary/coexisting copies and serialization buffers. Shared immutable buffers have one reservation owner for their lifetime, so readers do not double-charge them. Reserve before replacement, release with the final owner, and release temporary reservations on failure/cancellation. TrackConsumersPool and PeakRecordingPool provide attribution; Arrow tracking pools have infallible reserve/resize and cannot enforce a rejection limit. DataFusion grow and unregistered/native/solver allocations are outside the fallible guarantee. Pool accounting is reported alongside measured process peaks; no whole-process OOM immunity is claimed.
• Failure and parallelism. Independent DAG stages may run concurrently within the shared memory/thread budget (§18.8). Reservation exhaustion becomes typed ResourcesExhausted/runtime.resource_limit naming the affected consumer and configuration; the supported relation envelope is §5.3. A pass publishes all its output ports after validation or none. Failed/cancelled attempts record terminal findings separately and cannot be mistaken for an empty successful result.

14.4 Incrementality

Decision: ADR-0041, ADR-0042

The initial contract promises complete dependency correctness and reuse at stage granularity. It does not promise minimal per-row or per-instance recomputation. Every reused bundle must equal a clean execution under the same complete inputs.

Fixed and parameter values are in compiled.bound_values; free initial guesses are in variable_order.initial. Binding refresh and program-topology reuse are separate decisions. P10 never substitutes case values. P14 records every fixed/parameter substitution or value-bound assumption used in classification, including zeros that alter incidence; derivatives and prepared programs retain the corresponding dependencies. A policy or value change cannot reuse stale bound segments simply because opcode topology is unchanged.

14.5 Closure proofs

At the end of P14 the compiler writes provenance.closure_report (one row per closure category of the metamodel doctrine §31: type, topology, scope, quantity, method, balance, optionality, equation, runtime) with status ∈ {closed, open} and the list of open items. A category is open while any inferred.undecided row belongs to it. A snapshot is publishable as a compiled problem only when every category is closed or the case explicitly declares the open items as intentional (unconnected optional ports, free decision variables).

15. Structural analysis and diagnostics

IDAES’s diagnostics (core/util/diagnostics_tools/**, core/util/model_statistics.py) become (a) predicates and aggregates over compiled and runtime relations, (b) graph algorithms over compiled.incidence, (c) linear-algebra analyses over the evaluation program’s Jacobian, (d) walks over the expression graph, and (e) auxiliary optimization problems built from the Jacobian or the problem. Every check writes runtime.diagnostics_findings rows; reports are views over findings. Thresholds are a relation (reference.diagnostic_thresholds) whose defaults are the IDAES DiagnosticsToolbox.CONFIG values.

15.1 Thresholds (defaults preserved from IDAES)

15.2 Structural checks (no numerical values required)

15.3 Native structural algorithms (pse-structural)

• Incidence construction. From compiled.incidence restricted to active equalities and free variables (IDAES IncidenceGraphInterface(include_inequality=False, include_fixed=False)). Linear-only participation is known from the canonical form: a symbol occurring only in Affine terms with a non-zero coefficient participates linearly; coefficients that fold to exactly zero after canonicalization do not create an edge (the standard_repn rule).
• Maximum matching: Hopcroft–Karp, own implementation. petgraph ships only general-graph matching (maximum_matching, Gabow’s algorithm, O(|V|³), input treated as undirected, and it panics on a degenerate node bound); the incidence graph is bipartite, so Hopcroft–Karp’s O(|E|√|V|) is required. is_bipartite_undirected is asserted on the graph in debug builds. In CI the matching cardinality and structural rank are cross-checked against scipy.sparse.csgraph.maximum_bipartite_matching / structural_rank and Pyomo’s IncidenceGraphInterface over the golden snapshots (§24.1).
• Coarse Dulmage–Mendelsohn: projected digraphs over the matching, alternating-path reachability from unmatched nodes on both sides, giving the row partition (unmatched, overconstrained, underconstrained, square) and column partition (unmatched, underconstrained, overconstrained, square), with the matching order preserved.
• Block triangularization: requires a perfect matching; Tarjan SCC on the matching-projected digraph (reversed), condensation DAG, topological order made deterministic by ordering ties on semantic ID. Output compiled.blocks with kind = scc and order.
• Connected components for independent subsystems.
• Tear selection (§12.5).

15.4 Numerical checks (require values)

The Jacobian is produced by the evaluation program (§18.2) at the current point and scaled as IDAES does: J_scaled = diag(sf_equation) · J · diag(1/sf_variable); an “Ipopt gradient-based” emulation (max_grad = 100, min_scale = 1e-8) is available for comparison.

15.5 Advanced analyses

• SVD toolbox. Smallest k singular values of the scaled equality Jacobian: dense faer SVD below svd_dense_limit (default 2,000 rows); above it, shift-invert Lanczos over faer::matrix_free with the operator (JᵀJ)⁻¹ applied through a sparse LU factorization of J (partial_svd alone targets the largest singular values and cannot be used for this); problems whose LU factorization fails fall back to a declared svd.unavailable finding rather than a dense attempt; rank deficiency count below singular_value_tolerance (1e-6); for each small singular value, the variables and equations whose singular-vector components exceed size_cutoff_in_singular_vector (0.1).
• Degeneracy Hunter (Dowling and Biegler). Stage 1 candidate MILP: Jᵀν = 0, ν ∈ [−M − m_small, M + m_small], binaries y_pos, y_neg forcing |ν| ≥ m_small when set, Σ(y_pos + y_neg) ≥ 1, minimize Σ|ν|; columns with Σ|J_ij| ≤ 1e-6 skipped. Stage 2 irreducible degenerate set MILP per candidate: fix ν_k = 1, Jᵀν = 0, −M·y ≤ ν ≤ M·y, minimize Σy; members are equations with y > 0.9. Both are problems built as compiled artifacts and solved through the NL backend with a MILP solver profile (scip default, cbc).
• Ill-conditioning certificate (Klotz): LP Jᵀy = res, Σy_pos − Σy_neg = 1, minimize ‖res‖₁, then minimize ‖y‖₁ at fixed residual norm; report components by |y| down to ratio_cutoff of the largest.
• Infeasibility explanation (minimal intractable system). Elasticize in three rounds (variable bounds; inequalities; everything) with slack variables, recording violated items and re-fixing their slacks; then a deletion filter over the elastic set (deactivate each; if the subproblem becomes feasible the item is essential). Output: the MIS and the guard set. Implemented as a plan over case overlays and slack symbols, not as model cloning.
• Convergence analysis. A case set (§19.3) solved with Ipopt; per sample record iterations, iterations in restoration, iterations with regularization, CPU time, and whether the numerical checks raise warnings; compare to a baseline with rel_tol = 0.1, abs_tol = 1.

15.6 Reports

report_structural_issues and report_numerical_issues are projection views: model statistics, warnings (ordered as IDAES orders them: DOF, units, structural singularity, evaluation errors; residuals, bounds, extreme Jacobian norms at warning thresholds, parallel components), cautions (fixed-to-zero, unused; near bounds, near zero, extreme values, none values, mismatched, canceling, no free variables, extreme Jacobian at caution thresholds), and next-step check ids. assert_no_structural_warnings and assert_no_numerical_warnings are the same views with an error policy, usable as CI gates.

16. Scaling as an explicit transformation

16.1 Model

Scaling is data in compiled.variable_scales (scale, offset, source) and compiled.equation_scales (scale, scheme), both with derivations. The transformation is

x = x_ref + D_x x̂           (D_x = diag(1/scale_v), x_ref = offset)
r̂(x̂) = D_r r(x_ref + D_x x̂)  (D_r = diag(scale_e))
Ĵ = D_r J D_x

Backends consume it in one of two ways, recorded in solver_profiles.scaling_mode: the native and NL backends pre-scale the problem (equivalent to Pyomo’s scale_model = true writer option) and unscale results on read-back; the NL backend can alternatively emit the scaling_factor suffix and set nlp_scaling_method = user-scaling for Ipopt. Expression scaling hints (IDAES scaling_hint on named expressions) are variable_scales rows for expression symbols and serve only the nominal-value algebra.

Validity: a scale is strictly positive and finite; values are clamped to [min, max] per component kind (1e-10, 1e10) with a debug finding when clamping occurred.

16.2 Sources of scaling factors, in precedence order

16.3 Nominal value algebra (constraint scaling without a solved point)

get_sum_terms_nominal_values(expr) returns one signed nominal value per additive term. It is an operator-indexed evaluation over the IR (a native visitor, generated from the operator table so it cannot drift from the opcode catalog):

Constraint schemes over the non-zero nominal terms n_i (default inverse_maximum):

Nominal derivative norm (scale_constraint_by_nominal_derivative_norm(norm)): for each variable v in the equation, set every other variable to its nominal value 1/sf_w, set v to 1 (its perfectly scaled value), evaluate ∂body/∂v · (1/sf_v) with the evaluation program’s derivative; sf = 1/‖(pjac_v)‖_norm. The Jacobian-norm autoscaler is the same computation at the current point, composing with existing factors: sf_new = sf_old/‖row‖.

16.4 Scaler templates

A scaler template declares the four-stage pipeline IDAES’s CustomScalerBase.scale_model runs (variable routine, first fill-in, constraint routine, second fill-in), the defaults, and the submodel dispatch order (user-supplied scaler for the submodel → package default state or reaction scaler → the template’s default scaler). Reference scaler templates exist for every unit template of §11 with the IDAES routines (for example scale.cstr: control-volume routine, deltaP by default, heat[t] by 1/(0.1·Σ_p nominal enthalpy inflow), volume forced to 1e3, reaction rates 1e-3, constraints by inverse_maximum; scale.gibbs: lagrange_mult[t,e] by 1/(R·T_nominal), constraints by inverse_sum).

16.5 Profiling and persistence

The scaling profiler is a plan that runs the case under a matrix of scaler methods (variables only; each nominal scheme; nominal L1 and L2 norms; actual L1 and L2 norms), with perfect (magnitude-based) and user variable scaling, and records the condition number and the Ipopt iteration statistics (iterations, restoration, regularization) per cell in runtime.profile_results. Scaling factors are relations; the IDAES JSON scaling-file format is an import/export adapter.

17. Initialization plans

17.1 Plan model

An initialization plan is a compiled.initialization_plans row with ordered init_stages. Stage kinds:

State snapshot and restore follow IDAES StoreState semantics: at the end of a plan the fixed status of every variable and the active status of every equation are restored to the pre-plan case, values of variables that were fixed are restored, and values of free variables keep their solved values. Because overlays are immutable rows, “restore” is discarding overlays, not replaying mutations.

Every stage run is a runtime.runs row with parent_run_id = the plan run and stage_id; failures carry the stage.

17.2 Standard plan templates

The postcheck (check(residuals ≤ constraint_tolerance·sf; no missing values)) uses scaled residuals exactly as IDAES’s large_residuals_set does. Status values are Ok, Failed, DoF, PrecheckFailed, Error.

17.3 Plug-ins and initialization order

inferred.initialization_order(instance, ordinal) lists the instance followed by its plug-ins (costing blocks register themselves). Plans run prepare in order and finalize in reverse, as ModularInitializerBase does.

17.4 Flowsheet and dynamic plans

• plan.sequential_modular@1: order units topologically with tears from inferred.tear_candidates; for each unit in order run its plan then propagate along outgoing connections; iterate tear streams by direct substitution or Wegstein (accel clamp [−5, 0], iterLim = 40, tol = 1e-5, absolute or relative); finish with solve_subset(all).
• plan.time_element_march@1 (IDAES initialize_by_time_element): supported schemes LAGRANGE_RADAU and BACKWARD (others rejected); overlay deactivating all but t0, solve consistent initial conditions; deactivate time-unindexed equations and fix time-unindexed variables; for each element, reactivate its nodes, fix the differential variables and derivatives at the element start (or every variable at or before it when fix_diff_only = false), copy values forward, solve, then deactivate again; restore.
• PETSc time stepping and consistent-initial-condition solves (§13.6) are plan templates over the NL backend.

17.5 Continuation (homotopy)

stage.kind = continuation reproduces the IDAES homotopy meta-solver: for fixed symbols v_i with targets t_i, the continuation parameter n runs from 0 to 1 with v_i = t_i·n + v_i,0·(1 − n); parameters step_init = 0.1, step_cut = 0.5, iter_target = 4, step_accel = 0.5, max_step = 1, min_step = 0.05, max_eval = 200 with the IDAES validity ranges; after a successful step s ← s·(1 + step_accel·(iter_target/iterations − 1)) clamped to [min_step, max_step]; on failure roll back to the last solved overlay and s ← max(min_step, s·step_cut); terminations optimal, minStepLength, maxEvaluations, infeasible, other (converged only with regularization). Each step is a runtime.runs row; the progress fraction is recorded.

18. Backends

18.1 The CanonicalMathProblem

A problem (compiled.problems plus variable_order, equation_order, canonical math_*, incidence, sparsity_patterns, variable_scales, equation_scales, kernel_bindings, solve_plans) is what every backend consumes. Ordinals are assigned by sorting on semantic ID; positions in the solver vectors are variable_order.position and equation_order.position. Fixed and parameter symbols are not in the solver vector; their values are bound parameters of the evaluation program.

18.2 The native evaluation program

Decision: ADR-0047, ADR-0046

pse-numerics lowers the ordered expression DAG into control-flow regions with owned numerical workspaces:

1. Compile reachable expressions in deterministic order, but represent every runtime Conditional as a guard and separate then/else regions with a result merge. Only the selected region executes for a scalar; batched execution carries the corresponding row mask. A shared expression is computed once within a valid execution region, or reused across regions only with proof that doing so preserves guard/error behavior. A flat unconditional topological sweep is not a conditional evaluator.
2. Instructions name opcode, output/input slots and typed payload. Preserve operation and reduction order by default; Affine evaluates its ordered children and coefficients without incidental FMA. Opted-in reordering/contraction follows the persisted numerical policy and enters the program key (§7.3). Never use target-cpu=native to choose semantics implicitly.
3. Residual, inequality and objective segments use the declared complete quantity contracts. Fixed/parameter values come from the separately refreshable bound_values segment, not free initial guesses.
4. Reverse-mode Jacobian sweeps and forward-over-reverse Hessian sweeps follow only the executed branch; branch-boundary nonsmoothness and implicit-system singularities use declared outcomes. Sparse patterns may conservatively include both branches, but excluded branches cannot execute merely to fill derivative slots. Kernel derivative implementations are selected from derivative_bindings and validated under their conditions. Exact-Hessian requests fail when unavailable unless the profile explicitly selects a limited-memory fallback.
5. Domain violations or non-finite intermediate/results produce a recoverable typed evaluation error carrying equation, node and selected policy. Ipopt callbacks translate it without unwinding (§18.3). Invalid output is never an ordinary null or a successful last iterate.
6. Batch evaluation uses the same operation/branch contracts. DataFusion Expr::Case/CaseExpr provides guarded relational execution; opaque eager UDFs are called only within selected branches. Tolerant analysis uses the explicit outcome relations in §18.5.

Borrow null-free contiguous primitive arrays through PrimitiveArray::values and an owning ScalarBuffer/array handle whose lifetime covers the compiled consumer. These safe views incorporate the slice offset: an eligible one-chunk slice is not rejected solely for a nonzero offset. Multiple chunks, coercion/layout requirements or invalid values take a checked, reserved copy path; null fixed/parameter values are rejected, not silently copied into a numerical slot. Never construct the view by indexing a raw base pointer. Retain immutable owners through callbacks, cancellation and Python stream draining. Measure the benefit against the copy path before claiming zero-copy performance.

Implicit-kernel derivatives may use the pinned num-dual implicit-derivative APIs behind a declared implementation binding and independent fixtures. Arrow supplies storage and views; it does not own solver mutation or certify the derivative contract.

18.3 In-process Ipopt

Decision: ADR-0028

pse-backend-native binds the Ipopt C API (CreateIpoptProblem, IpoptSolve, AddIpoptNumOption/StrOption/IntOption) with callbacks for eval_f, eval_grad_f, eval_g, eval_jac_g, eval_h over the evaluation program. Options come from solver_profiles; the IDAES defaults are shipped as profiles ipopt.default (tol = 1e-6, max_iter = 200, nlp_scaling_method = gradient-based) and ipopt.user_scaled (nlp_scaling_method = user-scaling, or pre-scaled problem with nlp_scaling_method = none), with linear_solver resolved when a run starts from runtime.host_capabilities (§6.13), which the explicit runtime operation probe_host writes (Ipopt version, available linear solvers, HSL, PETSc, the Python environment); a profile naming ma57 on a host without HSL fails with capability.backend unless the profile declares fallback_linear_solver = mumps, in which case the fallback is a selected policy, and the options actually passed to CreateIpoptProblem are recorded in runs.resolved_options so that a reproduction on a different host is either identical or visibly different. Results: primal values (unscaled), duals, bound multipliers (zL, zU), termination status mapped to TerminationStatus, iteration statistics taken as data inside the intermediate callback through GetIpoptCurrentIterate and GetIpoptCurrentViolations (Ipopt ≥ 3.14; probe_host records the version and an older Ipopt degrades to parsing the solver’s text output, recorded as such) into runtime.iterations (objective, primal and dual infeasibility, mu, step size, regularization, restoration flag). The -sys crate is generated with bindgen from IpStdCInterface.h, with the generated bindings committed and diffed in CI as §4.2 treats generated relation code; index_style is C (0) and asserted.

Interruption across the C boundary: the driver checks the cancellation token inside the intermediate callback and returns false to stop the solve. No Rust panic and no unwind ever crosses an Ipopt callback; a callback catches the failure, records it in the driver state, and returns false, and the driver converts the recorded failure into solve.evaluation_error or runtime.cancelled after IpoptSolve returns. The last iterate is written to runtime.solutions under the run’s terminal status, so a cancelled or failed run is distinguished from a converged one by runs.status, never by the absence of rows.

18.4 The NL backend

pse-backend-nl writes the AMPL NL format directly from the problem (linear and nonlinear parts, common subexpressions as defined variables, bounds, objective, suffixes) and reads SOL files. It is the solver-neutral route and the only route to ipopt_sens, k_aug, dot_sens, bonmin, couenne, cbc, clp, scip, petsc_snes, and petsc_ts. Suffixes emitted: scaling_factor (when the profile asks for user scaling), dae_suffix and dae_link (PETSc), dual, ipopt_zL_out, ipopt_zU_out on read-back. Symbolic labels (.row, .col files) are always written so that solver messages and PETSc trajectories map back to semantic IDs.

Subprocess lifecycle: a solver process belongs to the run that started it. Cancellation or a wall-time limit kills the process group; a SOL file is ingested only when the process exited normally and the file’s trailer is complete, otherwise it is discarded and the run ends runtime.cancelled or solve.solver_error. A partial SOL file is never parsed.

Kernels that are not expression-expandable reach NL solvers only through an AMPL external-function library. The platform exports one compiled library per kernel package (funcadd entry points generated from KernelSpec, including the IDAES-compatible symbols cubic_root_l, cubic_root_h, cbrt, and the Helmholtz function names) so that the same binaries IDAES uses work unchanged; KernelSpec.bindings must list nl_external_function for the binding to be valid, and P16 refuses otherwise.

18.5 Kernel adapters generated from KernelSpec

Decision: ADR-0043, ADR-0047

The generator derives adapters only for KernelSpec.bindings entries with a compatible implementation and named conformance suite. It validates natural units, null policy, effects, argument evaluation and derivative routes before emitting code. There is no independent adapter default that can change mathematical behavior.

Constant simplification must obey §7.3 and excluded-branch behavior. Ordering/interval hooks are generated only from an explicit sound bound/ordering contract, not from an unqualified monotonicity label. Kernel bodies read no ambient session configuration. A kernel’s natural-unit conversions appear in P9 once; adapters consume those bindings and must not apply the same conversion again. Scalar and batch output admission rechecks the generated contract.

The operator and aggregate capability tables are human-readable views of these declarations. Generated conformance checks require every supported entry to name its implementation; no entry is marked Tested until its actual backend route passes. ImplicitRef itself has no direct DataFusion/NL/Pyomo binding; an explicit implicit-system KernelCall may expose a separately declared and tested UDF.

18.6 Solver-neutral results

Every backend writes the same relations (runtime.runs, solutions, duals, residuals, iterations, solver_events). Residuals are always recomputed natively from the returned primals so that the record does not depend on the solver’s reporting.

18.7 Solve-plan selection

Static attributes only (semantic math basis §0.2): problem class from math_static_analysis and problems.degrees_of_freedom.

The selected class, its justification, and the modifiers are rows in compiled.solve_plans.

18.8 Threading

Decision: ADR-0046

One configuration owns the thread budget: the DataFusion tokio runtime and target_partitions, the rayon pool for batch kernels and pass parallelism, artifact hashing (blake3::update_rayon), diagnostics linear algebra (faer’s Par), and the solver’s linear-algebra threads (HSL, MUMPS, PETSc). Defaults leave the solver single-threaded while a solve is running and give the pool to DataFusion otherwise; hashing and diagnostics never take pool threads during a solve; target_partitions is validated against the pool size when the session is built; oversubscription is a configuration error, not a runtime surprise.

18.9 Backend capability matrix

Decision: ADR-0043, ADR-0047

Entries are Proposed contract scope, conditional on the declared implementation, selected numerical policy and conformance suite; they are not implementation certification. Generate this view from the operation/binding declarations when codegen exists.

19. Cases, results, and analytics

19.1 Cases and overlays

A case is a chain of overlays over a model revision. Precedence: the child’s case_specs rows override the parent’s for the same target; within a case, higher priority wins; targets are the identity-based case_spec_targets rows resolved at commit (§6.10), so a rename between two case revisions changes nothing; P13 expands each target to symbols, and a wildcard expands over the port’s or group’s members at that time. treatment = fixed with a value, free, or parameter decides variable_order.treatment; initial supplies guesses; bounds override template defaults. A specification that names a symbol with role = expression is an error (IDAES VarLikeExpression semantics: expressions cannot be fixed or bounded; use an equation instead).

19.2 Results and reporting

• Stream tables. A view joining inferred.ports on connections (streams named by connection, plus additional ports for feeds and products), the state’s display declaration (or the state symbols when true_state), runtime.solutions, and the reporting unit set (IDAES reporting_units config): output columns are streams, rows are display quantities with a units column, missing rows shown as -. The interactive variant adds the variable kind (fixed, unfixed, parameter, expression) per cell.
• Unit reports. template_display.performance rows per instance joined to solutions; the IDAES report() layout is a projection.
• KPI tables. generate_table is a DataFrame query over any attribute paths.
• Tags. authored.tags (name, expression, display_unit_id, format, doc, group) with group defaults (str_include_units, set_in_display_units); reading a tag converts to the display unit; setting a tag writes a case_specs row (in display units when the group says so). Tag tables (table_heading, table_row) and SVG label substitution are projections.
• Model statistics and diagnostics reports are the views of §15.
• Float semantics in analytics. GROUP BY, DISTINCT and joins on a Float64 column merge -0.0 with +0.0 and treat NaN as self-equal (§5.3); ordering does not. Analytics over runtime values that group on a float record this as a selected loss in the report’s metadata; the platform’s own views never group on floats.

19.3 Sweeps and convergence studies

authored.case_sets hold a generator (grid, latin_hypercube, uniform_random, explicit list; the IDAES PySMO sampling methods map to these) and produce sample cases; a runner (sequential, parallel over the pool) executes each sample with rebuild_model semantics replaced by immutable snapshots, records per-sample outputs and solver statistics, and stores results in runtime.sweep_results. Baseline comparison and the convergence-summary report (successes, failures, restoration, regularization, numerical issues) are queries.

19.4 Parameter estimation and data reconciliation

observations and measurement_models bind measured values (with std_dev) to model expressions. The reference problem templates:

• problem.weighted_least_squares@1: objective Σ_k ((expr_k − obs_k)/σ_k)² over the bound observations; estimated parameters are symbols whose treatment is switched from parameter to free by the case; ports left free where reconciliation requires it.
• problem.multi_scenario_estimation@1: stacks one problem instance per scenario (shared θ symbols, scenario-local everything else) into one problem via scenarios; this is the parmest experiment model expressed as data.

Covariance and confidence regions use reduced-Hessian sensitivities from k_aug/sIpopt through the NL backend or the native Hessian. Until native estimation is complete, the Pyomo adapter exposes the stacked problem to parmest.

19.5 Costing

The costing framework (core/base/costing_base.py, models/costing/SSLW.py) is templates and reference data:

• costing.flowsheet@1 (IDAES FlowsheetCostingBlock): parameters base_currency (a currency unit), base_period (default year); flow types registered as flow_types rows with a fixed cost symbol <flow>_cost per type; aggregate symbols and equations aggregate_capital_cost = Σ_u convert(u.capital_cost), aggregate_fixed_operating_cost, aggregate_variable_operating_cost (per period), aggregate_flow_<f> = Σ convert(flows), aggregate_flow_costs[f] = convert(aggregate_flow_f · f_cost), only for flow types with at least one registered flow. Plug-in ordering registers each unit costing instance in the unit’s initialization order.
• costing_method templates (IDAES UnitModelCostingBlock methods): applied to a unit instance, creating capital_cost, fixed_operating_cost, variable_operating_cost symbols (each must be non-negative; the compiler warns otherwise) and their constraints; unit_mapping is a relation costing_method_defaults(unit_template_id, method_template_id) with inheritance-aware lookup.
• Currency. USD_CE500 is the base currency unit (dimension currency); USD_CE394 = (500/394)·USD_CE500; USD_<year> = (500/CE(year))·USD_CE500 for 1990 onward from reference.cost_indices; location factors are reference data.
• SSLW methods (cost_heat_exchanger, cost_vessel with platforms, ladders, and trays, cost_fired_heater, cost_compressor, cost_fan, cost_blower, cost_turbine, cost_pump with motor, and the cost_pressure_changer dispatcher) are templates with their enumerations (HXType, HXMaterial, HXTubeLength, VesselMaterial, TrayType, TrayMaterial, HeaterMaterial, HeaterSource, CompressorType, CompressorDriveType, CompressorMaterial, PumpMaterial, PumpType, PumpMotorType, FanType, FanMaterial, BlowerType, BlowerMaterial) as reference enums, the correlation coefficients as parameter_values, and the cost-basis quantities (area in ft², work in hp, pressure in psig, duty in BTU/hr, flow in gpm or ft³/min) as UnitConvert nodes. number_of_units is an integer symbol (solver_type = integer when the integer option is set, otherwise continuous and fixed to 1). Two IDAES defects noted by the survey (the vertical and horizontal vessel wrappers discarding their arguments; the blower dispatch routing to the pump method) are corrected in the reference package with parity notes.
• Costing initialization is the plan stage “solve each cost symbol from its defining constraint” (1×1 blocks).

19.6 Utility minimization (Duran–Grossmann)

law.heat_integration@1 takes the heating and cooling unit instances, DTmin, and eps, and generates: Tin[i], Tout[i] expressions from the units’ control volumes; Theta[i] (the smoothed Q/ΔT); pinch candidates T_[p] = Tin[p] + dT[p] (dT = DTmin for heating units, 0 for cooling); QAh[p], QAc[p] (bounded below by 1e-8) with the smooth-max heat-content equations; Qs, Qw; heating_utility[p]: Qs ≥ QAc[p] − QAh[p]; cooling_utility: Qw = −Σ_i Q_i + Qs. Initial values come from the pinch calculation kernel. Composite curves are an analytics query.

19.7 Optionality

math_alternative_sets (installed or absent, mode selection, technology choice) lower to disjunctions. Phase 3 lowers through the Pyomo adapter (Big-M, hull, GDPopt); native reformulation to indicator constraints or binaries through the NL backend follows.

19.8 Uncertainty

First-order propagation (IDAES uncertainty_propagation) is sensitivities ds/dp from k_aug/sIpopt (NL) or the native reduced Hessian, combined with the parameter covariance from estimation: σ²_f = (df/dθ)Σ_θ(df/dθ)ᵀ. Robust optimization (PyROS) is reached through the Pyomo adapter.

20. Persistence, provenance, and reproducibility

20.1 Artifact store

Decision: ADR-0045, ADR-0046

<store root>/
  refs/<name>.json                             -- {snapshot_id, manifest_checksum, updated_at}; conditional publication
  manifests/<manifest_checksum>.json          -- immutable encoded manifest; snapshot identity is a field, not its byte checksum
  relations/<namespace>/<relation>@<version>/<encoding_checksum>.arrow
  relations/<namespace>/<relation>@<version>/<encoding_checksum>.parquet
  evidence/<encoding_checksum>                 -- noncanonical plan/diagnostic encodings
  kernels/<kernel_id>@<version>/<digest>/       -- versioned implementation artifacts
  documents/<document_id>/<encoding_checksum> -- exact authored source bytes

Publication is a local single-writer protocol over atomic object creation and conditional ref updates. Prepare and semantically validate complete output bundles, compute logical hashes (§5.3), serialize each selected physical encoding with an explicit successful finish, and verify its size/format/checksum. Store the encoded object with PutMode::Create. A physical checksum verifies physical bytes; it cannot substitute for the logical relation hash.

Before accepting AlreadyExists, verify the existing object’s checksum, length and encoding contract against the intended manifest member. A cached verified-object handle may supply that proof only while its immutability/ownership guarantee remains valid; a restored or externally writable store requires rereading. Truncated, corrupt or incompatible existing bytes are an error requiring explicit repair, never a successful idempotent write. Untrusted imports/restores additionally decode, restore only the declared encoding projection, validate the complete semantic contract and recompute the logical hash. Missing or conflicting metadata is not repaired by guessing units.

Write the complete manifest under its own encoded checksum, then publish (snapshot_id, manifest_checksum) with PutMode::Update using the ref’s prior version (or Create for an absent ref). Readers pin both values, verify the manifest checksum, recompute its semantic snapshot identity, and resolve only its complete member set. Two valid encodings of the same logical snapshot may have different manifest checksums. No self-hashing JSON object or timestamp-dependent logical identity is needed.

An interrupted operation leaves the old ref valid and at most unreachable complete immutable objects; incomplete local staging is never a committed artifact. A CAS conflict re-reads the head and applies the declared conflict/retry policy, never merges arbitrary members. Use the pinned *_opts object-store methods, not rename_opts or copy-then-delete as a commit. Multi-writer/cloud qualification stays behind R-10. Reservations cover staging and serialization (§14.3); successful puts do not excuse an incomplete logical output bundle.

20.2 Manifest

Decision: ADR-0044, ADR-0045, ADR-0049

The versioned manifest is the physical envelope around the explicit semantic membership of §5.3. Its own checksum is held by the ref, not embedded recursively in itself. A representative member is:

{
  "manifest_version": "pse.manifest.v2",
  "snapshot_kind": "stage",
  "snapshot_id": "blake3:…",
  "membership_profile": "pse.snapshot.v2",
  "created_at": "…",
  "schema_registry_fingerprint": "blake3:…",
  "relations": [{
    "port": "typed_math", "namespace": "compiled", "relation_id": "…",
    "name": "math_expr_nodes", "version": 1, "logical_hash": "blake3:…", "rows": 12,
    "encodings": [{"format": "arrow_ipc_file", "writer_version": "…", "encoding_checksum": "blake3:…", "bytes": 1234, "path": "…"}]
  }],
  "packages": [{"package_id": "…", "version": "1.2.0", "logical_hash": "…"}],
  "compiler": {"version": "…", "passes": [{"pass_id": "…", "version": "…"}]},
  "engine_profile": {"engine_profile_id": "…", "content_hash": "blake3:…"},
  "numerical_policy": {"policy_id": "…", "content_hash": "blake3:…"},
  "toolchain": {"lockfile_hash": "blake3:…", "canonicalization": "pse.canon.v2"},
  "kernels": [{"kernel_id": "…", "version": "…", "digest": "…"}],
  "semantic_parents": [{"role": "input.typed_math", "snapshot_id": "blake3:…"}],
  "evidence": [{"kind": "datafusion_proto", "canonical": false, "encoding_checksum": "blake3:…", "codec_version": "…", "path": "…"}]
}

Schema, member uniqueness/completeness, path containment, row counts, format versions, checksums and registry compatibility are validated before exposure. Evidence, timestamps and alternative encodings are excluded from logical membership; every model/case/policy/kernel dependency that changes meaning is either a semantic member or a declared semantic parent. The snapshot-kind profile, not optional JSON fields, decides which dependencies are required.

20.3 What a run references

runtime.runs.environment, runs.resolved_options, runtime.host_capabilities and the run’s manifest capture: model and case revisions; property, reaction, and template package versions; compiler and pass versions; kernel digests; discretization, scaling, and initialization plan artifacts; solver profile, the options actually used, and backend versions (Ipopt, HSL, PETSc; Pyomo, pint and every pyomo.contrib.* package used, numpy and scipy versions, and the IDAES version for parity runs); host and thread configuration; the lockfile hash; result and diagnostic artifact hashes. The questions in the proposal (§10.2) are joins: “which correlation produced this enthalpy term” walks math_equations → derivations → method_resolutions → method_specs.

20.4 Reproduction

Decision: ADR-0045

pse reproduce <run_id> verifies the pinned manifest and encoded objects, validates decoded logical identities, then re-executes the declared pass DAG and solve under recorded policies. Deterministic semantic outputs must reproduce their logical hashes; physical encodings, pass-attempt records and plan evidence need not be byte-identical. Solver values and failures use §7.3 comparison rules and the profile tolerances; unsupported runtime capabilities are explicit refusals. A cached result is accepted only with its complete dependency key and validated artifact contract.

20.5 Compatibility and migration

Decision: ADR-0045

Relation and canonicalization versions are explicit. Unknown canonicalization/manifest versions are rejected; v1 artifacts require a separately declared importer that decodes, validates and rehashes into v2, never relabels or copies old hashes. No v1 migration is claimed implemented. Loaders apply declared generated relation migrations forward; a snapshot whose schema registry fingerprint is unknown is rejected with schema.unknown_registry. IDAES JSON state files (to_json format version 4) and scaling JSON files are importable through adapters that map local names to semantic IDs by qualified path.

21. The Python boundary and the Pyomo adapter

21.1 Extension module

Decision: ADR-0024, ADR-0039, ADR-0048

pse-py (pyo3 0.29, pyo3-arrow 0.19) exposes a small API; every table crosses as an Arrow C stream (__arrow_c_stream__) that the consumer drains as a RecordBatchReader, never as row objects and never as a materialized Table on both sides at once. The capsule protocol, not pyarrow, is the contract: any Arrow implementation (pyarrow, polars, datafusion-python at whatever version) can consume it, and none of the typed classes of §21.5 names a pyarrow type.

snap = pse.open(store_path).head()                      # SnapshotHandle
snap.table("compiled", "math_equations")                 # object exposing __arrow_c_stream__ → pyarrow / polars / datafusion-python
problem = pse.compile(snap, case="base")                 # ProblemHandle
bundle  = problem.bundle()                               # ProblemBundle: dict of named Arrow streams + manifest
result  = pse.solve(problem, profile="ipopt.default")    # RunHandle with .tables()

Bundle contents: manifest (problem id, ordering hashes, unit set, and the per-column loss profile: for every table and field, whether a pse.* extension type is carried and whether the consumer registered it — platform Rust admission rejects an unregistered extension while an unaware Python consumer may expose the storage type, so the declaration is asymmetric and says which side enforces what), variables (ordinal, semantic id, qualified name, quantity_type_id, unit_id, bounds, initial, treatment, scale), equations (ordinal, sense, body node, bounds, scale, active, residual_quantity_type_id), objectives, expr_nodes, expr_args, the typed payload tables, kernel_bindings with their Pyomo binding descriptors, source_map (ordinal → qualified name and source span). Transfer is one stream per table, not per-node calls. Both ingress and egress invoke generated recursive semantic validation; a storage-only consumer cannot re-enter through a checked API without restoring a verified declared contract. Stream owners retain immutable buffer and reservation lifetimes. BatchCoalescer may bound small transfer batches under R-24 measurements; it never collects an entire dataset merely to cross the boundary.

21.2 Adapter algorithm (Python)

Decision: ADR-0047

1. Pre-flight: SolverFactory(name).available(False) for the profile’s solver and, for every kernel bound as pyomo_external_function, that the exported library resolves on this host — both before any model object is built; a failure is capability.backend naming the solver or kernel id (Pyomo itself only warns when an ExternalFunction library is missing and fails later from the solver). Create ConcreteModel; one Var per free variable (indexed by ordinal), one Param(mutable=True) per parameter (a generated-code invariant, not a convention), bounds and initial values from the bundle; units attached from the bundle’s unit set through pyomo.units, which is backed by pint. pyomo.util.check_units.identify_inconsistent_units (not a bare assert_units_consistent) runs on the built model and names the offending components in a compile.math finding; the registry is the authority for every unit — pint validates what the adapter built and, if the two disagree, the adapter has a bug. No relation is ever derived from a pint object.
2. Lower the ordered graph with numerical-policy and capability preflight. Use LinearExpression only for a body proved affine in active decision variables after recorded substitutions; coefficients must be decision-independent and children actual variable terms. Verify the expected degree (0 for constants, 1 for a nonconstant affine expression), not an unconditional degree-one assertion. An Affine combination containing nonlinear children lowers as ordinary ordered expressions; its opcode does not imply a linear optimization problem. Preserve Add/Sub structure unless the selected policy permits changing it. WeightedMean expands with the declared normalization/domain guard; smooth functions and reductions follow their contracts. Conditional constructs Expr_if, but evaluation support is certified for the actual writer/solver route; building a Pyomo node alone is not proof of lazy solver evaluation. Expand declared kernels or bind the exported ExternalFunction; ImplicitRef remains equations. Refuse unsupported numerical policies or guarded routes before solve.
3. Constraints and objectives from the equation rows; scaling_factor suffix from the bundle when the profile uses user scaling.
4. Solve through SolverFactory with the profile’s options; results (values, dual, ipopt_zL_out, ipopt_zU_out, termination) are returned as Arrow tables keyed by ordinal; residuals are recomputed natively when ingested. The versions of Pyomo, pint and every pyomo.contrib.* package touched are recorded in runs.environment.

The adapter builds no IDAES classes. Engineering names live in the source map.

21.3 Uses of the adapter

Parity testing (§24), and the Pyomo ecosystem: parmest, PyROS, GDPopt, pyomo.dae utilities, pyomo.contrib.iis as a cross-check oracle for §15.5’s minimal intractable system, and IncidenceGraphInterface-based cross-checks of the native structural results, run as standing CI parity tests over the golden snapshots. Five of these six live in pyomo.contrib, the least stable surface in the dependency set; §18.9’s matrix records that standing, and every result obtained through one records the package version (§20.3).

21.4 Opaque kernels and ASL

Decision: ADR-0043

A kernel bound as pyomo_external_function requires the exported AMPL external-function library; the same library serves the NL backend. Python callback functions are not used for solves because ASL-based solvers cannot execute them. Every kernel with this binding has an end-to-end test that solves a tiny problem through Ipopt from Pyomo and compares to the native result.

21.5 Python contracts

The Python package follows the contract discipline (formerly docs/library_ref/contract_substrate_discipline.md, now stated here so the reference cannot dangle):

• Contract classes are generated. The attrs classes for every relation and bundle table are emitted by the §4.2 generator from RelationSpec — the same projection as the Rust views, in another language — and shipped in python/pse/contracts/. A hand-written class that mirrors a relation’s column list is the governance failure §4.2 names; the no_shadow_structs check covers Python too.
• Two class systems, one contract each. msgspec.Struct(frozen=True, forbid_unknown_fields=True) with Meta constraints is the codec for the §20.2 manifest and for every JSON or TOML file the Python package reads or writes (strict decode, located errors, JSON Schema emitted only for these wire types — never for authoring documents, whose schema the Rust registry generates). attrs + cattrs structure Arrow rows (already-decoded material) into the generated classes. No type is defined in both systems.
• Strict structuring. Every converter is built with the hook factory register_structure_hook_factory(has, …make_dict_structure_fn(…, _cattrs_forbid_extra_keys=True)) so an unknown key raises ForbiddenExtraKeysError (cattrs ignores extra keys by default, which would turn a version drift between Rust and Python into silent data loss); detailed_validation is never disabled; ClassValidationError groups are rendered with cattrs.transform_error into §23.2 findings rather than reduced to their first message. Classes are attrs.frozen.
• No Any. A governance lint walks attrs.fields() of every contract class at import time and fails on Any, bare dict or bare list; nothing at runtime forbids them otherwise.
• Extension types. The generated pyarrow.ExtensionType classes of §4.4 are registered once, idempotently, when the package is imported; the contract layer asserts per column that every field claiming a pse.* extension resolved to a registered class, using the ARROW:extension:name key that survives even when the type does not.
• No Python-side model classes duplicate relations; engineering names live in the source map.

21.6 The array boundary (numpy)

A nullable Arrow column converted to a bare ndarray turns null into NaN, collapsing the distinction §4.4 exists to protect. The adapter therefore converts with zero_copy_only=True by default (which refuses nulls and chunked columns rather than copying silently), asserts null_count == 0 before any conversion (true for solver vectors by construction), and carries the validity mask alongside where a nullable column must cross. A copy is requested explicitly (zero_copy_only=False) only where the call site records why. -0.0 survives the conversion. numpy and scipy are pinned because the parity harness compares numbers through them; neither is used on a production path.

22. Authoring and the extension model

22.1 Package layout

<package>/
  package.toml            -- id, version, kind, dependencies
  materials/*.yaml        -- species, phases, elements, reactions, stoichiometry, parameter values
  methods/*.yaml          -- method_specs for equation-template methods (kernels are Rust and only referenced)
  properties/*.yaml       -- property_packages, method_selections, state bounds, equilibrium pairs
  templates/*.yaml        -- unit, control volume, state, reaction, connection rule, helper, initializer, scaler templates
  laws/*.yaml             -- law templates and law instances
  costing/*.yaml          -- costing methods and reference cost data
  cases/*.yaml            -- instances, connections, cases, observations, case sets
  assertions/*.yaml       -- expected derived facts (tests, not truth)

P1 parses every document into authored rows via a change set; the document’s content hash is recorded on each row’s source span. YAML documents are parsed by serde-saphyr (typed errors with line and column, hostile input refused without a panic, a parsing budget on nesting, aliases and allocation) and TOML by toml with Spanned<T>, so every row’s pse.source_span comes from the parser. Every entity declared under id_policy = explicit carries an id: field; pse authoring assign-ids inserts missing ones and P1 rejects a document that still lacks one (§5.1). YAML is one surface; the same change sets can be produced by Python builders (pse.authoring) and by agents.

22.2 The change-set model

Decision: ADR-0027

authored.change_sets @1     change_set_id, base_revision_id, author, message, created_at
authored.change_ops @1      change_set_id, ordinal, op : enum (insert|update|delete|rename), relation_id, row_key,
                            row : struct<staged_port: text, staged_ordinal: u64>, precondition : text [n]
                            -- row points into the change set's per-relation staged members, never at JSON text (D1)
                            -- rename: the staged row carries entity_id and the new name/qualified_name; rejected for named-policy entities (§5.1)

A change set is validated (P2 on the resulting snapshot), applied atomically, and yields a new model or case revision. The commit contract is P2-level validity: a committed revision is structurally and referentially valid (every invariant of reference.schema_invariants holds, every target resolves to an identity, every expression parses); it may still fail P3–P10 (an unresolvable template path, a unit inconsistency). Publication as a compiled problem is gated separately by the closure report (§14.5), and every revision carries last_closure_status so a reader can tell a committed revision from a compilable one. There is no other write path into authored (decision D2 and doctrine P20). Attempts to write a derived relation are rejected; an “expected” derived fact goes to provenance.assertions.

rename changes an entity’s name and qualified_name attributes and re-renders every retained pse.target_path and pse.expr_dsl text that serialized a reference to the old path from the identity-based rows (case_spec_targets and kin, normalized.*_expr_*); references by identity are untouched, every derived ID is unchanged, and the §22.4 diff report shows one rename row. A change op that edits a retained target text without the corresponding identity rows is rejected by P2 (case.target_text_mismatch): the text is a serialization, not a second authority.

22.3 What is data and what is code

This is the litmus test of the doctrine: new scope enters as semantics and plug-ins, not as runtime branches.

22.4 Agent change sets

An agent proposes a change set (for example: replace the heat-transfer correlation on one exchanger, regenerate, compare). The platform compiles the resulting snapshot, produces a diff report (added and removed equations, changed sparsity, changed scaling, changed DOF, closure report), and the change is accepted or rejected as a whole. Agents never edit build()-style code.

23. Observability and failure semantics

23.1 Observability

tracing spans are opened per pass run, per rule evaluation, per plan stage, and per solve; events carry the semantic IDs involved. Fields that are known only at the end of a span (output hashes, row counts, finding counts, cache hit or miss) are declared as field::Empty when the span opens, because a later record of an undeclared field is silently discarded; #[instrument] is used with skip(...) and explicit fields(...), never with its default of recording every argument. DataFusion emits no spans of its own: rule-level spans are platform code around plan execution, operator-level visibility comes from the EXPLAIN pgjson output and rules_fired stored in the pass record (§14.2 rule 5), and JoinSetTracer may be installed for per-task tracing when needed (datafusion-tracing is adopted only when a release matching the pinned engine exists). Solver output is captured as data through the Ipopt iterate API (§18.3) with text capture only for messages that have no structured equivalent. Metrics (pass durations, row counts, cache hits, solver iterations) are emitted with the same identifiers. Provenance (provenance.*) explains derivation; observability explains execution; they are linked by pass run and run IDs but never merged.

23.2 Failure taxonomy

Failures carry the relation rows and source spans involved; they are never flattened into “run failed”. Every pse-* crate returns concrete error enums (thiserror with miette::Diagnostic derived, the class names above as #[diagnostic(code(...))] in Rust path form); miette::Result and the graphical reporter exist only in the CLI and the driver. Findings are relations; a rendered diagnostic is a projection of a finding, never its storage. pse.* values inside a diagnostic are rendered by the formatter factory of §4.4, not as raw storage bytes.

DataFusion errors map into the taxonomy at one place, never per call site:

24. Testing and acceptance

24.1 Test layers

Decision: ADR-0048, ADR-0051

24.2 Vertical slices and acceptance criteria

Slice A — steady-state heater and mixer (phase 1). Benzene–toluene ideal package with FTPx, feed, heater, mixer, product; equality connections; case fixing feed state and heater duty, and a second case fixing outlet temperature instead. Accept when: the compiled problem has the same variable and equation counts and DOF as the IDAES model; native and Pyomo residuals and Jacobians agree; Ipopt converges to IDAES values under a recorded solution comparison (default atol 1e-8, rtol 1e-6; non-finite/error outcomes checked separately); the stream table matches; reload from the artifact store reproduces identical hashes; the block-triangularization plan initializes both cases; renaming the heater between the two cases changes no derived ID and no case binding.

Slice B — recycle with flash (phase 2). Peng–Robinson package, SmoothVLE, LogBubbleDew, flash, separator, compressor (isentropic), heat exchanger (LMTD), recycle with tear stream; modular-properties initialization plan; full diagnostics including SVD and Degeneracy Hunter on a deliberately degenerate variant. Accept when: cubic root kernel derivatives agree with finite differences and the NL external function; the sequential-modular plan converges the recycle; structural and numerical reports match IDAES on the same flowsheet; homotopy moves the feed specification across a phase boundary with the IDAES step statistics; a value-only case change recompiles only P13 onward.

Slice C — distributed dynamic reactor with costing and estimation (phase 3). CSTR and PFR (backward finite difference and Radau collocation) with a generic reaction package (Arrhenius, power law), dynamic flowsheet with time discretization, PID controller, SSLW costing, utility minimization, a parameter-estimation case over observations, and a parameter sweep. Accept when: discretization equations equal Pyomo DAE’s; the time-element plan and the PETSc plan reproduce IDAES trajectories; costing aggregates match SSLW values; estimated parameters and covariance match parmest; sweep statistics match the IDAES convergence analysis format.

24.3 Benchmarks

Decision: ADR-0042, ADR-0046

Measure the same semantic workload and report cold/warm preparation separately: parse, admission, demand closure, each stage, rule optimization, canonicalization, physical persistence, restore/recovery, Python transfer/construction, residual/Jacobian/Hessian evaluation, solve, inspection and total wall time. Include derivation rows/bytes per equation, accounted consumer peaks, whole-process peak, temporary copies, spill bytes, scanned/decoded rows and buffer copy rate. Record pins, profiles, numerical policy, thread/memory/size limits, dataset and case hashes and result equivalence.

Compare complete stage reuse against any proposed finer input bundle before R-22/R-01 adoption. Compare shared predicates with storage pruning, scalar UDF fast paths with materialization, and safe borrows/coalescing with checked copies before claiming improvement (R-24). Predicate preimages require full-result equivalence before decoding savings are measured (R-25). No timing gate or performance gain is established by the design or by the standalone characterization probes.

25. Delivery phases

Each phase ends with the parity harness green against IDAES 2.12.0 (the pinned parity package, §3.1) for its scope and with the doctrine conformance tests passing. The current phase-0 skeleton and environment preflight do not satisfy these behavioral exits; revision 5 is Proposed design, not a claim that its mechanisms exist.

26. Risks and open decisions

Decision: ADR-0042, ADR-0048

Revision-4 review disposition in revision 5 (Proposed contract corrections; implementation acceptance is open):