/ 
Test-FitNesse-Slim-1.00
/ docs/fitnesse-native-fixture-gap-triage.md

Native-suite fixture gap triage

Executive findings

The audit examined every one of the 93 native-suite pages whose baseline has at least one wrong, ignored, or exception result: 24 pages. No protocol-engine defect was demonstrated. The native suite’s 321 exception assertions are not 321 defects: 2 are deliberate v0.5 ignore-control responses, while the remaining 319 are execution failures caused by unavailable fixture classes/methods, Java-only FitNesse utilities, or Java runtime features. The 51 wrong assertions group into known FitNesse expression behavior (21), nested-request context loss (5), Java reflection/conversion fixtures (14), harness startup preconditions (6), and one unavailable Table Table fixture (5).

The highest-value gap from this triage has since been implemented: a dedicated ordinary TableTableIncFirstCol adapter for SymbolsInTableTable. It remains separate from the consumer adapter, which manages symbol changes inside the fixture. The implementation also prefers a more-specific imported package over its parent, so the managed-symbol mode remains selectable when both namespaces appear in the page context.

The discrepancy inventory and root-cause group counts below describe the original 525/51/4/321 baseline. The implementation results section records the verified post-change state.

This inventory separates fixture availability from protocol compatibility. A Java class failing to load proves that the page cannot reach its intended test, not that the Perl wire implementation is wrong. The full per-page data, estimated gains, confidence, and evidence references are in fitnesse-native-fixture-gap-triage.json.

Scope and evidence

The native baseline is pinned to FitNesse v20251025 and contains 93 full page paths, 525 right, 51 wrong, 4 ignored, and 321 exceptions. Its runner validates exact per-page result counts and confirms external Perl request/response traffic. This triage selects all baseline pages with nonzero wrong, ignored, or exception counts; it does not mistake intentional ignore results for failures.

The direct Perl backend was verified from the runner’s execution logs and balanced listener trace. For ambiguous fixture and Java-client pages, I ran selected leaves using a fresh FitNesse process with /opt/fitnesse/fitnesse.jar; every captured execution log named java -cp /opt/fitnesse/fitnesse.jar fitnesse.slim.SlimService 1. The Java and Perl counts below are separate observations, not claims that their nested fixture contexts are equivalent. Java direct references were run for 15 pages, including the three nested-context pages documented in fitnesse-rendering-comparator-compatibility.md. SystemExitIsPrevented was not run against Java: its own page says it should be removed or pruned on Java 21+, and the environment runs OpenJDK 25.

Complete discrepancy inventory

Counts are right/wrong/ignored/exceptions. — means a Java comparison was not run in this triage. A Java page passing does not mean every corresponding Perl failure is a protocol defect; several Java runs have fixtures or test-system functions unavailable to Perl.

| Full upstream page path | Perl baseline | Java reference | Root cause and interpretation | |---|---:|---:|---| | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.BaseLineDecisionTable.HowtoIgnoreResultValues | 1/0/2/0 | — | The two ignored values are intentional decision-table outcomes. No wrong result or exception. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.ChainTest | 3/0/0/1 | 4/0/0/0 | TestSlim.echo_boolean is called but missing from the Perl TestSlim fixture. One focused fixture-method gap; not a transport or dispatch defect. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.CustomTableRenderingAddedByPluginTest | 5/1/0/0 | 6/0/0/0 | Nested Perl PageDriver HTTP request loses the active FitNesse plugin context. Java passes with its in-process context. Harness limitation. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.DefineTableTypeTest | 0/0/0/6 | 4/0/0/0 | Missing TableOrDecisionFixture prevents the test reaching FitNesse’s table-type parsing assertions. The main behavior is FitNesse-side parsing, not SLiM server functionality. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.DisgracerTest | 0/0/0/22 | 22/0/0/0 | Tests FitNesse Java Disgracer and Java method/class name normalization. Not a Perl server responsibility. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.HybridDecisionTable | 14/0/0/46 | 40/0/1/0 | Properties cannot load on Perl; setter/getter calls cascade. A direct Perl run had 44 exceptions versus 46 in suite context. Java passes using Java Properties; hybrid-table interpretation is FitNesse-side. A Perl map fixture could unblock it, but imitating java.util.Properties solely for this page is not justified yet. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.HybridDecisionTable.MethodExtractorRuleTest | 0/0/0/85 | 35/0/0/0 | Tests FitNesse Java method-extractor and configuration helper classes. The 85 exceptions cascade from unavailable helpers; this is not general Perl dispatch evidence. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.IgnoredTestsShouldColorGray | 5/2/0/0 | 7/0/0/0 | Stop/ignore handling works, but the nested Perl HTTP request lacks the active scenario context needed for expected markup. Harness limitation; see the nested-context report. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.QueryTableSuite.SetContextValue | 4/0/2/0 | — | The two ignored rows are expected query-table outcomes. No wrong assertions or exceptions. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.SlimSymbols.SlimSymbolsCanBeExpressionsToEvaluate | 9/21/0/0 | 30/0/0/0 | FitNesse’s Java VariableStore evaluates Nashorn expressions. Backtick expression evaluation is not a SLiM protocol requirement and was previously investigated; do not add an arbitrary code evaluator to the Perl server. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.SystemExitIsPrevented | 0/0/0/12 | — | Missing Java SystemExitTable helpers; the page itself says to remove/prune on Java 21+. Current JDK is 25. JVM exit interception has no Perl analogue. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.SystemUnderTestTest | 0/0/0/8 | 4/0/0/0 | The named Java fixture MySystemUnderTestDriver is absent. Perl already has generic SUT delegation and focused tests (t/sut-delegation.t); this page does not show a core SUT defect. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.TableTableSuite.SymbolsInTableTable | 0/5/0/4 before; 13/0/0/0 after | 13/0/0/0 | The ordinary fixture was missing. The Perl adapter now matches Java v20251025. Ordinary substitutions happen before the table call; output-cell assignments do not alter inputs for later rows. The separate managed-fixture page remains 15/0/0/0. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.TableTableSuite.TestReturnsNull | 1/0/0/2 before; 1/0/1/0 after | 1/0/1/0 | The page's table fixture was absent. A minimal adapter now returns undef, which existing serialization carries as SLiM null; FitNesse marks the empty table ignored, matching the Java reference. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.TestCustomComparators | 1/2/0/0 | 3/0/0/0 | Nested Perl request lacks the configured custom comparator. Harness limitation, not comparator protocol behavior. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.TestExceptionIsDisplayed.ParameterConversionExceptions | 3/14/0/0 | 17/0/0/0 | Nested page invokes Java static methods and Java types (java.time.*, String.valueOf) and asserts Java reflection/converter diagnostics. Perl’s standard converter behavior is covered independently; these failures do not isolate a protocol-level conversion defect. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.TestExceptionIsDisplayed.SlimExceptionResult | 0/0/0/126 | 24/0/0/0 | Directly unit-tests FitNesse Java SlimExceptionResult via a Java helper/scenario. The 126 exceptions are cascades from trying to run Java-side helper behavior in the Perl fixture environment. Not server exception-serialization evidence. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.TestIgnoreAll | 4/0/0/1 | — | The one exception is the intended IGNORE_ALL_TESTS control marker; retain it as passing v0.5 ignore-semantics coverage. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.TestIgnoreInScriptTable | 4/0/0/1 | — | The one exception is the intended IGNORE_SCRIPT_TEST control marker; retain it as passing v0.5 ignore-semantics coverage. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.TestSystemSlimSuite.ErrorSituationsDuringSlimServerStartup.A_TimeoutAsSlimHeaderNotReceived | 2/2/0/0 | — | Requires a specific nested test-system/port setup that the shared runner does not establish. Harness startup condition, not fixture compatibility. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.TestSystemSlimSuite.ErrorSituationsDuringSlimServerStartup.B_TimeoutAsSlimServerDiesBeforeConnecting | 2/1/0/0 | — | The already-running Perl service changes nested startup behavior, bypassing the page’s invalid child runner. Known harness precondition limitation. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.TestSystemSlimSuite.ErrorSituationsDuringSlimServerStartup.C_TimeoutAsNoServerAtGivenSocket | 3/1/0/0 | — | The intended unused port is occupied by the external Perl listener. Known harness/environment limitation. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.TestSystemSlimSuite.JavaSlimServer.CallStaticJavaMethod | 0/0/0/7 | 2/0/0/0 | Uses Java static reflection and java.time factories. Java reference passes; class loading/static invocation are not SLiM protocol operations. | | FitNesse.SuiteAcceptanceTests.SuiteSlimTests.TestSystemSlimSuite.ManualStartofSlimService | 2/2/0/0 | — | Demonstrates a manually started service on a page-configured port. Shared service configuration prevents the page’s expected invalid-header condition. Harness/configuration example, not a Perl fixture gap. |

The seven pages with context/startup issues are already documented boundaries: three nested context pages and four startup configuration pages. The two ignore-control exceptions are intentional. These classifications should not be used to expand the Perl engine scope.

Root-cause groups

The categories cover 24 pages, 51 wrong assertions, 4 ignored assertions, and 321 exception results. The only exception results established as intentional controls are the 2 ignore-marker exceptions. The other 319 exception results are not equivalent to 319 defects; they are cascades or unsupported Java/test-harness cases, with the small fixture gaps identified above.

Prioritized implementation opportunities

| Rank | Capability | Verified cause and affected pages | Potential gain | Layer / effort / risk | Practical value and recommendation | |---:|---|---|---|---|---| | 1 | Add a dedicated ordinary Table Table fixture and verify standard symbol behavior | TableTableSuite.SymbolsInTableTable; its requested default fixture is absent. The existing consumer fixture handles symbol mutation itself and would violate this page’s row-isolation behavior if aliased. | Up to 1 page and 9 assertion outcomes (5 wrong + 4 exceptions); fixture-load root cause is certain, while post-load behavior needs focused verification. | Test fixture adapter, with core changes only if tests isolate a symbol-boundary defect; medium; medium risk because default substitution and assignments must remain distinct from fixture-managed state. | Best benefit-to-effort, provided the next increment tests semantics rather than adding a class alias. It exercises meaningful Table Table/symbol behavior and has an adjacent 15-assertion managed-mode test for contrast. | | 2 | Add an idiomatic null-returning Table Table test fixture | TableTableSuite.TestReturnsNull; the named fixture was absent; existing executor and serializer already support null. | One page changed from 1/0/0/2 to 1/0/1/0, matching Java v20251025; the two fixture cascade exceptions are gone and the expected ignored table result is visible. | Test fixture adapter; small; low risk. | Implemented in this follow-up with no protocol-engine change. | | 3 | Add a Perl SUT-delegation acceptance fixture matching the upstream page | SystemUnderTestTest; 8 exceptions are all downstream of missing MySystemUnderTestDriver; generic core and unit tests already exist. | Up to 1 page / 8 exception outcomes; Java reference passes 4 assertions. | Test fixture adapter; small; low risk. | Could provide an acceptance-level smoke test of actor/SUT precedence, but does not add core semantics beyond t/sut-delegation.t. Defer unless acceptance coverage is desired. | | 4 | Add a Perl property-map fixture for hybrid decision tables | HybridDecisionTable; Properties is unavailable, causing 46 native (44 direct) exceptions. | At most 1 page and 44–46 exception outcomes; Java reference is 40 right/1 ignored. Full page parity is not established by merely removing fixture-load failures. | Optional fixture adapter; medium; medium risk due hybrid header rules, property semantics, and context-dependent exception count. | A map-backed fixture could be useful for Perl users, but copying java.util.Properties only to improve this page is poor value. Investigate real fixture use cases before implementing. | | 5 | Add missing echo_boolean to the Perl TestSlim acceptance fixture | ChainTest; one specific missing method. | 1 page / 1 exception outcome. | Test fixture adapter; small; low risk. | Trivial but narrow. Add only if the upstream fixture is maintained as a broad compatibility fixture; otherwise current declared Boolean converter tests already cover the behavior. |

These potential gains are page-local and not additive claims of protocol conformance. In particular, 46 Hybrid exceptions are an upper bound on results that may change, not 46 confirmed recoverable protocol assertions. DefineTableTypeTest and the three Java utility pages have high raw exception counts but low Perl-server value, so they are not ranked as implementation opportunities.

Investigate and implement only the TableTableSuite.SymbolsInTableTable fixture-adapter gap. First establish the exact class import, SLiM instruction exchange, and where each table-cell symbol is substituted or assigned; then add a default-namespace Perl adapter for the page’s ordinary symbol behavior and regression coverage for row isolation and post-table values. Do not alias the existing consumer-managed fixture, change general import lookup, or alter the upstream page. The 15-assertion managed-symbol page is a contrast case, not a substitute implementation.

This outranked the larger exception counts because its failure had a precise fixture-resolution cause, its adjacent implementation already existed, and success exercises meaningful SLiM table/symbol traffic. Hybrid Properties remains a tempting 46-exception target, but that number mostly reflects an absent Java standard-library fixture and is not a protocol defect. Java method extraction, Disgracer, static invocation, exception-result parsing, and JVM exit prevention belong to FitNesse/Java. SUT delegation already has direct Perl unit coverage. The single ChainTest method is easy but offers only one assertion. The three nested-context pages and four startup pages remain separate harness work.

Table Table implementation evidence

The pinned Java fixture is fitnesse.slim.test.TableTableIncFirstCol, constructed with no arguments. Its doTable(List<List<?>>) iterates input rows in order and returns "no change" plus "pass:" + (Integer.parseInt(firstCell.toString()) + 1). It keeps no row or session state. The bundled class signature/constants match the upstream source; a clean Java SlimService run on FitNesse v20251025 reports 13 right, 0 wrong, 0 ignored, 0 exceptions.

The target page assigns A=99, B=5, C=14, constructs the ordinary fixture, and calls doTable once with $A/$resultA=, $B/$resultB=, and $C/$C= rows. Both Java and Perl return a successful table result; the following checks observe 100, 6, and 15. Its included same-table example also passes with Z=45, not 201. The existing executor substitutes the full nested input before the one fixture call. FitNesse applies output-cell symbol assignments after the complete returned table, so later row inputs cannot see assignments made by earlier output cells.

The Java and Perl v20251025 XML instruction logs agree on the relevant exchange. Both report callAndAssign(A, scriptTableActor, echoInt, 99) → $A<-[99], then B=5 and C=14 similarly; make(tableTable_44, TableTableIncFirstCol) → pass(table: Table Table Inc First Col); and one call(tableTable_44, doTable, [[$A,$resultA=],[$B,$resultB=],[$C,$C=]]) → pass(). The following calls echoInt($resultA), echoInt($resultB), and echoInt($C) return pass(100), pass(6), and pass(15) in both runs. In the included row-isolation example, the returned checks are echoInt($Z) → pass(45) and the running page name → pass(13). FitNesse's Table Table implementation guide documents the list-of-rows doTable contract and how returned cells are evaluated and assigned by the client.

The managed page's trace stays distinct: after X=99 and Y=44, it calls a fixture from statementexecutorconsumer with six $X rows. Its first result is 100, the fixture updates X within each row, and the final values are X=104 and Y=105. This page returns 15 right in Java and Perl. The Java execution logs positively identify java -cp /opt/fitnesse/fitnesse.jar fitnesse.slim.SlimService 1; the Perl native-suite run verifies its runner-owned loopback service and 724 balanced request/response exchanges.

The Perl adapter is Fitnesse::Slim::Test::TableTableIncFirstCol in t/lib/; it has no executor hook or mutable table state. It follows Java signed-int parsing/increment boundaries, including null and malformed/empty input failures. StatementExecutor tries a more-specific imported package before its parent, with later imports first among equally specific prefixes. This keeps Statementexecutorconsumer::TableTableIncFirstCol independent when the parent namespace is also imported. Focused instruction tests cover construction, dispatch, symbol substitution, row order, repeated and independent symbols, isolation, null/empty values, integer boundaries, and the managed-mode contrast.

Null-returning Table Table fixture

The pinned fixture is fitnesse.slim.test.DummyTableTableReturnsNull: it has a no-argument constructor and doTable(List<List<String>>) returns Java null for the complete table. FitNesse sends call tableTable_42 doTable [[]]; the result is ignore(No results from table), not an empty list, empty string, or exception. Java SlimService positively identified from the execution log reports 1/0/1/0. Perl initially reported 1/0/0/2 because its matching fixture adapter was absent: construction failed with Could not find class Fitnesse::Slim::Test::DummyTableTableReturnsNull, and the following call failed because that instance did not exist. The new Fitnesse::Slim::Test::DummyTableTableReturnsNull returns undef; the existing StatementExecutor and Test::FitNesse::Slim::List encode the protocol null without changes. The external Perl execution now also reports 1/0/1/0. Focused coverage in t/null-table-fixture.t exercises serialized make/call instructions, result serialization, blank and empty-list differences, missing methods, repeated calls, and executor isolation.

Imported fixture-class selection now has direct regression tests in t/import-resolution-precedence.t: the longest package import prefix wins regardless of order, later imports win on equal prefix length, reversed ties select the reversed class, and an absent class candidate falls through to a later valid candidate. These tests drive StatementExecutor and use fixture methods with distinguishable results.

The first ordinary-symbol fixture run detected only that target-page delta: right 0→13, wrong 5→0, exceptions 4→0. The later null-fixture run changed only TestReturnsNull: 1/0/0/2→1/0/1/0; the other 92 page results stayed exact. Current native totals are 538 right, 46 wrong, 5 ignored, and 315 exceptions across 93 pages. FitNesse's suite final counts are 72 right, 9 wrong, 1 ignored, and 11 exceptions. The separate 50-page individual baseline remains unchanged at 45 conformant, 373 right, 7 wrong, 2 ignored, and 0 exceptions.

Deferred functionality and limitations

Verification scope for this triage

No upstream page or expected result changed. The native baseline change is restricted to the verified target page; rerun bin/test-dev, bin/test-fitnesse, and bin/test-fitnesse-suite, validate the JSON inventories, and run git diff --check before committing. The 50-page individual results remain 45 conformant / 373 right / 7 wrong / 2 ignored / 0 exceptions. The updated native totals are 93 pages / 538 right / 46 wrong / 4 ignored / 317 exceptions.