Design review: blueprint revision 4 and library contracts

Historical assessment of revision 4, identified by the digest below. The revision-5 follow-up reviews the resulting amendment; the findings and probe receipts here remain unchanged.

1. Decision and scope

Decision: Revise. Retain the Arrow relation authority, separate mathematical IR, explicit case overlays, and native numerical execution. The largest immediate improvement is to make existing library mechanisms enforce the proposed contracts. Several mechanisms currently credited with enforcement provide only an interface or a declaration. Adding more infrastructure before correcting that distinction would increase the number of places that can disagree.

Proposal: blueprint revision 4. Standard: Data Model–Based Design Charter, version 1.0, including DM-01–DM-60 and independent gates G1–G7. Reviewer: Codex. Date: 2026-09-13. Evidence status: the platform design is Proposed. Selected library routes are Interface-checked; six standalone characterization groups are Tested under the conditions in §9. This is not an implementation acceptance review.

Observable outcome: a compiler whose model meaning survives validation, inference, numerical preparation, storage, and backend transfer; ordinary process model extensions add declarations and any genuinely new algorithm, with mechanical adapters derived from those declarations.

Baseline and constraints: the repository is skeletal. Existing foundation checks do not establish a functioning modeling or solving lifecycle. The review covers the proposed local snapshot/compiler/native/NL/Pyomo architecture and its stated delivery phases. It does not require a distributed runtime or a new query engine. Scientific parity with IDAES 2.12.0 remains a future behavioral gate.

Method and coverage

The analysis reconstructed authority, pass dependencies, invariants, and failure paths from blueprint §§2, 4–9, 12–24, then compared them with the charter. The heater and lumped control-volume examples, property-demand algorithm, and phase exits provided concrete consumers for the extension and proportionality tests. Blueprint §§10–11 and Appendix A were sampled for architectural integration; individual thermodynamic correlations, all unit templates, and the entire IDAES coverage matrix were not independently certified.

The Arrow, DataFusion, and supporting Rust libraries maps supplied discovery leads. Their relevant claims were compared with the current blueprint, existing probe source, and pinned upstream source. Earlier reviews were consulted for context; their acceptance statements were not reused as current proof. The Python contract in blueprint §21 was inspected as a design; the Python capability map and its behavioral probes were not re-audited.

Context7 was used for DataFusion provider contracts, Arrow extension validation, and UDF predicate inversion. Versioned upstream documentation supplemented it. Retrieved examples were treated as leads: Context7 returned some historical or generic snippets that cannot establish a 55.1.0 contract. Final library findings rest on the pinned source and executable probes, not snippet ranking.

Local evidence included just metadata, Cargo resolution of the scratch probe, Arrow 59.3.0 and DataFusion 55.1.0 source checkouts, and cached rustdoc JSON whose crate_version and format version 61 were checked. The cached JSON was not regenerated; no completeness claim is made for that extraction. The source checkouts matched tags 59.3.0 and 55.1.0. The probe resolved 292 packages, with zero third-party versions absent from the repository lockfile.

The inspected blueprint has SHA-256 fa3e959a0005f97c284818eaeae4af94720fa6f001d4b9f10f98c4a44f5b4cca. Review began at 4becdebe856cd52cac944c5114dd48f1e0e5048e; unrelated concurrent commits advanced HEAD to fb0717d9151ada4884bd7ccb5efc27422bb92f7a during probing. The blueprint, charter, Cargo manifest, and lockfile were confirmed unchanged across that movement. Probe receipts bind the actual execution inputs.

Adversarial checks attacked extension admission, conditional evaluation, casting, canonical bytes, protobuf determinism, floating-point rewrites, pass scheduling, cache dependencies, and exact-filter verification. Crash recovery, solver FFI, numerical derivatives, actual provider scans, and end-to-end performance remain Proposed guarantees, not behavior verified by this review.

2. Authority and lifecycle map

All enforcement descriptions in this table are proposed platform mechanisms.

Deliberately opaque behavior: constitutive kernels, numerical solvers, structural algorithms, and external-function libraries can remain ordinary code. Their selection, units, domains, derivatives, effects, failure policy, and source digests must be explicit. KernelSpec does not yet contain every policy that its generated adapters are instructed to derive (R4-10).

Identity behavior: creation-time IDs and explicit rename semantics are a real strength: an ordinary rename cannot retarget an old case through a reused name. Artifact ordinals remain local. Hashes, however, still require corrections for physical encodings and ignored null payloads. Stable entity identity should not be confused with identical whole-relation hashes when labels or provenance change.

3. Semantic contracts and invariants

Absence and uncertainty: free unknowns, absent initial guesses, and unbounded values are deliberately distinct. Four-valued inference also has an explicit undecided relation. Preserve those choices. Two remaining ambiguities need closure: blueprint §18.5 permits invalid kernel evaluations to become null while diagnostics are optional; §7.6 permits missing observations as null although §6.10 declares observations.value without its nullable marker. A consumer must not infer invalidity, missing input, or failed evaluation from the same unqualified null.

Equivalence: maintain separate contracts for entity identity, canonical logical content, physical artifact integrity, symbolic mathematical equivalence, and floating-point agreement. Blueprint §24.1’s 1e-10 relative backend comparison and §24.2’s 1e-6 relative solution comparison do not authorize arbitrary overflow-eliminating rewrites, precision-losing casts, or an unstated fused multiply-add policy. Near-zero and non-finite cases also need explicit comparison rules rather than an unspecified relative-only check.

4. Derivation and execution design

There is a concrete cycle in the documented stage dependencies:

P6 property-demand closure -> P7 template equations -> P8 law equations
          ^                       |                        |
          +---- equation/contribution demand per §9.6 -----+
          ^
          +---- initialization-plan requirements from P15

P15 itself consumes the problem produced downstream of P6. This is different from the specified fixed point inside a rule stratum. A bounded correction is to derive demand seeds from normalized templates, laws, ports, displays, and declared initializer requirements, then realize their closure. Alternatively, explicitly contract a mutually recursive resolution stage with monotonicity and termination conditions. The generic DAG driver cannot infer either decision from §14.1.

The shared compiled.math_* names also need stage-qualified artifact inputs and outputs. P7–P10 produce or replace overlapping relations, while RelationSpec offers one producing_pass_id. A pass-local artifact bundle or versioned stage handle is enough; this does not require a new database or a new physical layout.

Relationship structures: containment, process connectivity, expression dependencies, structural incidence, scheduling, and provenance remain distinct. The use of petgraph for specific derived graphs is appropriate. A process recycle must not be rejected because its scheduling projection needs a tear or SCC.

Provider and boundary contract: the catalog correctly pins a manifest; the numerical problem remains whole; unsupported backend bindings fail before solve. Add active semantic validation before DataFusion optimization and at output admission. Preserve the complete input row set when pushing filters: physical pruning may exclude only rows proved irrelevant to the selected query.

Coherent publication: retain immutable object creation, a complete manifest, and one conditional ref update. Specify separately the checksum of each stored object and the canonical relation hash it represents. An existing content-named object can be reused only under a stated integrity/trust contract; its name alone is not an integrity check after corruption or incomplete external restoration.

5. Representative journeys

Ordinary extension: add a property method

A new equation-based heat-capacity correlation should add its method declaration, parameter quantities and natural units, expression template, and independent value/derivative fixtures. P6 selects it; P9 instantiates it; generated boundary code follows the same contract. No new provider framework is needed.

For a genuinely new opaque method, one Rust algorithm is legitimate additional work. Its descriptor must also declare the null/domain policy, purity, branch semantics, derivative conditions and actual backend routes. Merely setting Immutable, is_strict, monotonicity, or Hessian availability in an adapter would introduce an independent semantic decision. A method claiming a property but omitting its realization must fail conformance even if demand closure terminates.

Meaningful change: add a species to a bound package

Keep the instance ID, template contents and Boolean features unchanged; extend the material package with a third species. Its state symbols and balance domains must expand. The explicit per-instance P7 memo tuple in §14.3 is unchanged, although the required output differs. A fresh compile and an incremental compile therefore have no specified reason to agree. Include the resolved domain/package and binding artifacts in the expansion input contract, or use a conservative complete-stage key until finer invalidation is justified.

A related value journey must distinguish a fixed value from its initial guess. Blueprint §18.1 binds fixed and parameter symbols outside the solver vector, but §14.4 describes a fixed-value change as changing only variable_order.initial. The bound-value segment must be refreshed explicitly even when its tape and sparsity can be reused. Conservatively rebuilding that segment is inexpensive compared with silently solving the prior case.

Boundary: rule output and stored artifact

An engine expression emits Float64(1.9) for an integer target. Both can_cast_types and safe: false accept the conversion; the result is 1 (E3). Reattaching the target field metadata cannot restore the lost fraction. The same storage type is even less informative for quantities: attaching a Pa field to a bar-valued result performs no factor-of-100000 conversion.

For storage, the same two-row relation has canonical-option IPC stream, IPC file, and Parquet encodings of 648, 954, and 773 bytes in E4. The file and Parquet digests both differ from the canonical stream digest. Blueprint §§5.3 and 20.1 currently ask that one target hash mean both things. These are compatible only after their roles are separated or storage verification explicitly decodes and canonicalizes.

Interruption and guarded failure

The cancellation design is useful: a failed native solve retains its last iterate under a terminal failure status, and a killed NL process contributes no partial SOL result. That is Proposed, not exercised here.

An earlier failure occurs without cancellation: a conditional kernel selects a valid branch, yet the generic DataFusion UDF evaluator evaluates the other branch first. E2 reproduces this with a failing argument and both conditional flags set. Built-in CASE succeeds on the same input. The native tape needs the corresponding control-flow contract; an unconditional topological sweep cannot implement lazy guards simply because the opcode is named Conditional.

6. Acceptance gates

These are document-stage verdicts. A pass means the inspected design has an identified authority/mechanism, not that a skeleton implementation has passed it.

An unresolved gate is not a pass. No maturity total is computed.

7. Principle findings

Rows are grouped by structural cause. P1 denotes correctness work required before implementing the affected contract; P2 denotes verification or operational work that must precede the associated acceptance claim.

Applicability and principle verdicts

All twelve groups apply because this proposal defines a semantic model, compiler, execution lifecycle, persistence protocol, interchange boundaries and extension system. That broad scope is the reason for the coverage below. Distributed coordination, an untrusted binary-plugin service and an accelerator backend are not added to scope. Their absence is not a defect.

“Satisfied” means the inspected Proposed design identifies the mechanism, boundary and rejection behavior for the stated scope. It does not upgrade the platform to Implemented or Tested. “Unresolved” identifies a decision or evidence gap; it is not an implicit pass. The grouped rows provide a verdict for each applicable principle without using a numerical score.

8. Alternatives and architectural leverage

The simpler alternative preserves the stated observable outputs and future capability scope. It changes internal granularity and ordering of investment. It does not replace the semantic model with Pyomo objects or force Newton iterations through a relational engine. An initially coarser memo is a deliberate DM-33 tradeoff and should be recorded if adopted, with refinement triggered by measured rebuild cost and complete dependency fixtures.

Library capabilities to use more extensively

The following recommendations are Proposed. “Interface-checked” establishes the available mechanism; it does not establish its benefit or complete domain semantics. Most use already pinned families and do not call for another engine.

Already adopted and worth retaining: RowConverter for composite key ordering, Schema::project, extension metadata and generated Arrow/Python registrations, explicit engine rule lists and configuration, immutable snapshot providers, structured DataFusion error translation, native graph algorithms, and kernel-local num-dual. Registration, method availability, and selected configuration should be described at exactly the strength they establish.

Do not add Flight, distributed query orchestration, a second dataframe engine, or a general plugin service to solve these findings. salsa, egglog, FeOs and additional physical encodings should retain their explicit consumer/measurement triggers. Arrow RunEndEncoded, view arrays and custom allocation can be valuable later, but none repairs lost meaning at an admission boundary.

What remains ordinary code

Physical quantity algebra, semantic admissibility, canonicalization, four-valued inference and provenance, demand closure, sparse derivative preparation, matching, branch-aware numerical execution, solver callbacks and the snapshot commit protocol remain platform responsibilities. Arrow supplies representations and kernels; DataFusion supplies plans and execution machinery. Neither library defines process-engineering meaning on behalf of the registry.

9. Verification and measurement plan

The retained probe source, runner and receipt make the library counterexamples reproducible. The full regeneration recipe would rewrite unrelated capability evidence and Python exports; no recipe isolates these characterization questions. The bounded runner therefore creates an ephemeral crate from workspace pins and a copied lockfile, checks resolution against that lock, and runs it offline with the pinned Rust toolchain.

Executed command:

bash docs/design_review/evidence/blueprint-rev4-2026-09-13/run.sh

Mode and baseline: Rust 1.98.1; dev profile; Arrow 59.3.0 with arrow/force_validate explicitly enabled by the scratch force-validate feature; DataFusion 55.1.0; offline locked execution; failure baseline zero. Final result: 6 characterization groups completed, 0 assertion failures. This proves the named library behavior, including failures of the proposed assumptions. It does not mean six platform acceptance gates passed.

The initial scratch run stopped at the prototype’s missing protobuf provider codec after E1–E4; that harness defect was corrected before recording the final successful receipt. No full Rust suite, Python suite, solver parity, or benchmark was run as part of this documentation review.

Document and evidence checks: just docs, just lint-typos and just lint-license completed with exit status 0 in documentation/static mode (failure baseline: zero). The pinned Rust formatter’s rustfmt --check --edition 2024 accepted probe.rs; /usr/bin/shellcheck accepted run.sh. A focused check found 0 failures across the five review/evidence files, ten local links, eleven required sections and three recorded input digests; git diff --check also succeeded. The recipes emitted the existing host-shell diagnostic /etc/bash.bashrc: line 7: PS1: unbound variable before succeeding; host startup configuration was outside this review’s edit scope.

10. Exceptions and unresolved decisions

No MUST-level exception is accepted by this review. The gate failures require revision or explicit narrowing of the affected supported behavior. Existing deferrals for distributed coordination, optional kernels and additional engines remain outside this review’s implementation demand.

Owner entries identify the responsible role for a future decision; this review does not assign a person or authorize a dependency/design change. Adoption that changes D1–D14, identity, boundary or commit contracts must follow the repository’s ADR and design-review process. The blueprint and accepted ADRs were not amended.

11. Decision and implementation changes

Decision: Revise. G1 has a coherent proposed authority model; G2, G3, G4, G6 and G7 fail on concrete contract or capability evidence, and G5 remains unresolved. The architecture’s central choices are useful, but the current blueprint is not ready to serve as an unambiguous specification for its affected boundaries.

The recommended next design revision should settle the priority-1 contracts first, then express their library bindings and acceptance tests. It should also correct the capability-map claims identified by E1, E2, E3 and E5; otherwise future agents will inherit the same unsupported assumptions. More library use is valuable when it removes a duplicated mechanism while leaving one explicit semantic authority.