case study · memory safety · Apr 19, 2026
inih · C → Rust
cargo check Three-arm ablation on a 522-LOC C INI parser. The ADR-as-contract closed 5 of 6 compile errors on its own; the memory-safe domain schema closed the last one. Total spend: $0.15.
What the test is
Three-arm head-to-head on the same 522-LOC C codebase. Each arm tests a different level of scaffolding, so we can measure where the architecture’s value actually comes from.
- Arm A — Symmetric pipeline with
--domain memory-safe(full scaffolding): produces an ADR with an Ownership Decisions table, then a Rust crate that honors it. - Arm B — One-shot baseline (zero scaffolding): the LLM-only control. Sees the full source, emits a Rust crate in one call.
- Arm C — Plain symmetric pipeline, no
--domainflag (medium scaffolding): runsdecompose_v2with--cross-languagebut no domain schema. The ADR-as-contract only, no ownership-decisions table.
All three arms run at temperature=0 on the same LLM (gpt-4.1). Measurement is cargo check compile status.
Results
| Arm | Scaffold | Output LOC | cargo check | Notes |
|---|---|---|---|---|
A — sym + --domain memory-safe | Full (ADR + ownership schema) | 175 | ✅ 0 errors, 18 warnings | Clean compile; safe Rust throughout |
| B — one-shot (no pipeline) | None | 365 | ❌ 6 compile errors | Requires libc dep; *mut libc::FILE; libc::fgets FFI |
C — plain sym (no --domain) | Medium (ADR only) | 93 | ❌ 1 compile error | Missing helper ini_parse_stream_internal; otherwise clean |
What the ablation tells us
| Source of value | Errors closed | % of total win |
|---|---|---|
| ADR-as-contract alone (Arm B → Arm C) | 5 | 83% |
| Memory-safe domain schema (Arm C → Arm A) | 1 | 17% |
The ADR carries ~5/6 of the architecture’s value on this case study. The domain schema is a useful enterprise-tier enhancement — additive polish, not load-bearing.
What the diff says
Arm A’s output followed the ADR’s Ownership Decisions table:
input: borrowed (const char*)→ Rust&str(safe)handler: borrowed (callback)→ Rust&dyn Fn(...)(safe)user: borrowed (void*)→ Rust generic&T(safe)buffers: owned heap→ RustString/Vec<u8>(safe)
Arm C made similar choices implicitly from the ADR’s plain Decisions section alone. Arm B (one-shot) transliterated C types literally — *mut libc::FILE for FILE* — producing unsafe-laden FFI code instead of idiomatic Rust.
Honest takeaways
- The ADR-as-contract alone closed 5 of 6 compile errors. Going from one-shot to plain symmetric captured 83% of the architecture’s measurable value — without any domain-specific schema fields. The reviewable ADR is the load-bearing piece.
- The memory-safe domain schema closed the last error. Useful polish, not the headline.
- Reviewable ownership contract is a bonus, not a prerequisite. Arm C picked idiomatic Rust (
&File,&str) from the plain ADR alone.
Caveats
- OpenAI-only run; Anthropic retry pending (transient connectivity).
cargo checkmeasures compile, not behavior. Behavioral parity is Phase B+ work.- 522 LOC is within the validated scale envelope. Larger files (2500+ LOC) are not yet measured.