case study · framework modernization · Apr 19, 2026
Nancy NancyContext.cs · .NET Framework → .NET 8
dotnet build (net8.0) Three-arm ablation on a 148-LOC .NET Framework class. The plain ADR closed all 14 compile errors on its own. The framework-migration domain schema added zero compile-cleanness value but produced 15 explicit API contracts and 6 dependency decisions — value shifts from code to documents.
What the test is
Three-arm head-to-head on the same 148-LOC C# file, same LLM (gpt-4.1), same temperature=0. Each arm represents a different level of scaffolding.
- Arm A —
kaizen migrate-plan --from dotnet-framework --to dotnet8(full scaffolding,--domain framework-migration) - Arm B —
oneshot_baseline.py(zero scaffolding) - Arm C —
kaizen migrate-plan --from dotnet-framework --to dotnet8 --plain(medium scaffolding, ADR only)
Measurement: dotnet build on a minimal net8.0 csproj. Pass = 0 errors; fail = N errors.
Results
| Arm | Scaffold | LOC (output) | dotnet build | Notes |
|---|---|---|---|---|
A — sym + --domain framework-migration | Full (ADR + API-contract schema) | 86 | ✅ 0 errors, 0 warnings | Plan includes 15 API contracts, 6 dependency decisions |
| B — one-shot (no pipeline) | None | 144 | ❌ 14 errors | Preserved Nancy-internal type references that don’t resolve standalone |
C — plain sym (no --domain) | Medium (ADR only) | 92 | ✅ 0 errors, 0 warnings | Plan: 2 identifiers, 4 decisions, 0 API contracts |
What the ablation tells us
| Source of value | Errors closed | % of compile-cleanness win |
|---|---|---|
| ADR-as-contract alone (B → C) | 14 | 100% |
| Framework-migration schema (C → A) | 0 | 0% |
The ADR carries 100% of the compile-cleanness win. The domain schema’s value is in plan-document richness, not in output code quality.
Where the domain schema did add value
| Plan Content | Arm A (domain) | Arm C (plain) |
|---|---|---|
| Key Identifiers (migration candidates) | 16 | 2 |
| Architectural Decisions | 3 | 4 |
| API Contracts (public surface) | 15 | 0 |
| Dependency Upgrade Decisions | 6 | 0 |
The schema didn’t change whether the code compiled — both arms compiled cleanly. It produced a much richer plan document. For a compliance officer or program manager, that richer content is the artifact they want. For an engineer who just needs code that builds, plain mode is sufficient.
Honest takeaways
- The plain ADR is sufficient for compile-clean output on classic C# → .NET 8. No framework-migration schema required for the code-quality axis.
- The
--domain framework-migrationschema shifts value from code to documents. 15 API contracts + 6 dependency decisions are artifacts an auditor wants. They don’t make the code compile better. - The one-shot baseline continues to fail “will it build?” 14 unresolved type references — the exact class of problem the ADR’s stub-missing-types decision avoids.
Caveats
- Single-file case. NancyContext is one module out of a 200+-file framework. Multi-file cases with cross-module state are next.
dotnet buildmeasures compile, not behavior. Running Nancy’s nunit tests against the outputs is Phase B+ infrastructure.- NancyContext is mostly a data-bag. Real ASP.NET MVC controllers with session state and routing attributes would test the API-contract schema’s value more directly.