Skip to content

Development and Validation

Jump to bottom
issuelens[bot] edited this page Oct 10, 2026 · 1 revision

Development and Validation

Java Pack: Java 27 UI validation

These are source-backed test assertions and CI configuration in microsoft/vscode-java-pack, not independently observed test passes, released Java support, or a language-server implementation change. Verified at full merged/source-tip commit eb203464c082d7aa98d15202236a8bdeb91b3b56, PR #1767. The existing Test Plan remains the manual feature checklist; Java Pack owns this automated-validation entry point.

Coverage and runtime boundaries

Five AutoTest plans share test-fixtures/java27. Both Maven and Gradle build files target release 27 with preview enabled; each plan selects its intended importer. Plans register the project runtime as JavaSE-27 without setting java.jdt.ls.java.home: the project toolchain is distinct from the language-server runtime. The fixture also includes JDT preview preferences, separate from Gradle compiler arguments. [1][2]

Plan in test-plans/ Configured checks
java-maven-java27.yaml Maven import, generated JDT source level 27, zero diagnostics, clean compilation and execution.
java-gradle-java27.yaml Gradle 9.8.1 import/toolchain, source level 27, zero diagnostics, clean compilation and execution; Maven importer disabled.
java27-api.yaml LazyConstant and Set.ofLazy fixture; completion for Set.ofL includes ofLazy; compile/run assertions.
java27-preview.yaml Maven preview enabled → disabled → enabled; diagnostics and compiler failure while disabled, then recovery.
java27-primitive-patterns.yaml switch, instanceof, and nested record patterns; intentional unresolved type, deletion/save back to zero errors, compile/run assertions.

Coverage sources: the five named plans [1] and fixture/build files [2].

verify.mjs runs from the VS Code terminal, requires JAVA27_HOME with Java specification version 27, and sets the child build/runtime environment. It checks actual clean-build exit status, class-file major version 71 (also preview minor 65535 for non-project scenarios), and exact application output before emitting a success marker absent from the sent command. Preview-disabled verification requires both compilation failure and the expected primitive-pattern preview-disabled diagnostic, not an arbitrary failure. [3]

The primitive-patterns plan alone loads the test-only java27-autotest-support extension. Its java27.autotest.insertUnresolvedTypeAfterClose command rejects dirty or mismatched text, waits for actual document close, performs a redhat.java project-settings round trip checking source level 27, writes one unresolved-type insertion, then reopens a new document lifecycle and checks text, another round trip, and the specific fresh diagnostic. It rejects duplicate-field errors and incremental insertion events; the Java 27 AutoTest channel reports REOPEN_INSERTION_CONFIRMED or REOPEN_FAILED. This covers full-file reloading, not continuous editor edits or a production JDT fix. The fixture guide explains the former disk-write/Revert race; external JDT processing guarantees are not established here. [1][4][6]

CI toolchain and reproduction

The E2E workflow runs on Windows, Linux, and macOS. For Java 27 plans it provisions JDK 27, sets JAVA27_HOME, and adjusts plan paths per platform. Gradle preparation verifies 9.8.1 and exports the resolved absolute JAVA27_CI_GRADLE. With GITHUB_ACTIONS=true or CI=true, the verifier requires that absolute executable for the primary Gradle build; it neither falls back to PATH nor retries a failed CI build with another Gradle. Local execution retains PATH-based Gradle; a diagnostic control build, when configured, cannot replace the original failure or emit its success marker. Gradle diagnostics use --stacktrace --info. [3][5]

For reproduction, the fixture guide describes Maven/JDK 27/Gradle 9.8.1 prerequisites, JAVA27_HOME, matching the plan's project-runtime path, and a platform-specific Java extension VSIX. CI manual inputs test_plan, pre_release, and vsix_urls allow a selected-plan/exact-release comparison; retain the failed run's extension release when isolating a toolchain problem. No particular external extension release's support or CI success is inferred from these instructions. [5][6]

E2E artifacts and failure diagnosis

Discovery resolves @vscjava/vscode-autotest@latest once, then matrix and analysis jobs install that exact resolved release; there is no permanent 0.7.34 pin. Discovery uses nonrecursive test-plans/*.yaml and excludes java-fresh-import, so test-plans/config/artifacts.yaml is configuration, not an executable plan. [5]

Each case writes test-results/<plan>/ with file logging and case analysis. Pre-run logs and redirected CLI output are staged separately in RUNNER_TEMP/autotest-<plan> because startup clears the output directory; JAVA27_DIAGNOSTICS_DIR points to its terminal subdirectory. The run uses --logs, case-local --log-output, --artifacts-config, and --analysis-mode case. A separate unconditional autotest collect step supplements artifacts, followed by unconditional upload under results-<plan>-<os> with 30-day retention. The original test exit code is preserved, and collection has its own status. Aggregate organization skips artifact roots lacking results.json with a warning: inspect the original uploaded artifact for setup failures. [5]

The consumer artifact configuration declares these optional sources and destinations; the fixture guide documents the root-relative archive layout and artifacts/manifest.json. Missing optional sources are not evidence of a clean run. Consult collection status as well as test results. [6][7]

Diagnostic question Configured artifact destination
Which toolchain/command ran, and how did compilation or execution fail? diagnostics/ci/ (version probes, toolchain JSON, and terminal/ logs/exit metadata); diagnostics/workspace/.autotest/.
Did the IDE, extension host, or JDT LS fail? logs/ide/logs/ and logs/jdtls/User/workspaceStorage/, sourced from persistent .vscode-test/user-data.
What did the CLI or display process report? logs/console/console.log; Linux logs/display/xvfb.log.
Was there a native macOS failure? logs/crashes/, restricted by configuration to matching Node/Electron/Code/Java reports modified since run start.

Workspace, IDE, JDT LS, and CI diagnostic sources request evidence: tail; full diagnostic archives and bounded analysis evidence serve different purposes. Console, display, and native reports use the collect phase. Product-specific paths live in this repository's configuration; collector internals and package-release guarantees are outside this source verification. The webview-migration plan also enables local logging at ../test-results/java-webview-migration/logs. [6][7][8]

Sources

All references below are in microsoft/vscode-java-pack at full source commit eb203464c082d7aa98d15202236a8bdeb91b3b56, PR #1767; source identity microsoft/vscode-java-pack#1767:eb203464c082d7aa98d15202236a8bdeb91b3b56. The complete 21-file PR inventory was inspected, PR identities rechecked after paging, and final file identities/content verified at this merged revision.

Clone this wiki locally