ci: run autotest-based tests on Windows using Git Bash#823
Merged
yutaro-sakamoto merged 9 commits intoApr 15, 2026
Conversation
- Introduce PATHSEP variable (`:` on Unix, `;` on Windows) in atlocal files
- Use ${PATHSEP} for CLASSPATH separators in m.at and jar.at
- Skip debug.at on Windows pending cobj.exe -debug segfault fix
cobj.exe generates malformed jar paths when using -java-package option on Windows, producing "Parameter format not correct" errors.
- Skip COB_DATE CURRENT-DATE time test (requires GNU date -d and +%s) - Skip SEARCH KEY IN RHS test (cobj.exe segfaults on Windows)
- Skip subscripts out-of-bounds tests (2): cobj.exe segfault - Skip ref-mod tests (3): cobj.exe segfault - Skip MOVE with refmod test (1): cobj.exe segfault - Skip PROGRAM-ID case tests (2): class/file name case mismatch
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends the CI pipeline to run the autotest-generated test suites on Windows (via Git Bash), by generating the test runner scripts on Ubuntu and executing them on Windows with an installed cobj.exe/libcobj.jar test environment.
Changes:
- Added Ubuntu workflow to generate autotest scripts and upload them as artifacts for Windows jobs.
- Added Windows workflow/matrix job to download artifacts, configure
atconfig/atlocal, and run multiple test suites under Git Bash. - Updated many autotest sources to use
${COBJ}(and introducedPATHSEP) plus Windows-only skips for knowncobj.exeissues; updated Windows install script to includecopy/.
Reviewed changes
Copilot reviewed 51 out of 51 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| win/make-install.ps1 | Installs copy/*.cpy into the Windows install prefix. |
| tests/run.src/subscripts.at | Skips two Windows-crashing subscript tests via AT_SKIP_IF. |
| tests/run.src/ref-mod.at | Skips Windows-crashing ref-mod tests via AT_SKIP_IF. |
| tests/run.src/miscellaneous.at | Skips a Windows-crashing refmod test via AT_SKIP_IF. |
| tests/run.src/extensions.at | Skips case-insensitive filesystem tests on Windows. |
| tests/README.md | Updates list of generated autotest scripts. |
| tests/README_JP.md | Updates list of generated autotest scripts (JP). |
| tests/misc.src/read_prev_after_start.at | Replaces cobj with ${COBJ}. |
| tests/misc.src/move-sign-leading-separate-to-signed-comp3.at | Replaces cobj with ${COBJ}. |
| tests/misc.src/java-interface.at | Replaces cobj with ${COBJ} across cases. |
| tests/misc.src/index-file-status.at | Replaces cobj with ${COBJ}. |
| tests/misc.src/high-low-value.at | Replaces cobj with ${COBJ}. |
| tests/misc.src/evaluate-switch.at | Replaces cobj with ${COBJ} (including in embedded scripts). |
| tests/misc.src/current-date.at | Replaces cobj with ${COBJ}. |
| tests/misc.src/compare-national-diff-size.at | Replaces cobj with ${COBJ}. |
| tests/misc.src/compare-large-number.at | Replaces cobj with ${COBJ}. |
| tests/misc.src/comp3-is-numeric.at | Replaces cobj with ${COBJ}. |
| tests/jp-compat.src/search-key-in-rhs.at | Skips a Windows-crashing test via AT_SKIP_IF. |
| tests/jp-compat.src/job-date.at | Skips a GNU date-dependent test on Windows. |
| tests/i18n_sjis.src/user-defined-word.at | Replaces cobj with ${COBJ}. |
| tests/i18n_sjis.src/pic-x.at | Replaces cobj with ${COBJ} (including commented checks). |
| tests/i18n_sjis.src/pic-n.at | Replaces cobj with ${COBJ} (including commented checks). |
| tests/i18n_sjis.src/national.at | Replaces cobj with ${COBJ}. |
| tests/file-lock2.src/same-process.at | Replaces cobj with ${COBJ}. |
| tests/file-lock2.src/release-lock.at | Replaces cobj with ${COBJ} (script + checks). |
| tests/file-lock2.src/open-start-write-rewrite.at | Replaces cobj with ${COBJ} in generated scripts. |
| tests/file-lock2.src/open-input.at | Replaces cobj with ${COBJ} in generated scripts. |
| tests/file-lock2.src/lock-mode-clause.at | Replaces cobj with ${COBJ} in generated scripts. |
| tests/file-lock2.src/lock-mode-automatic.at | Replaces cobj with ${COBJ} in generated scripts. |
| tests/file-lock2.src/input-mode.at | Replaces cobj with ${COBJ} in generated scripts. |
| tests/file-lock.src/lock-file.at | Uses ${COBJ} and propagates compiler value into run.sh. |
| tests/file-lock.src/access-same-record.at | Replaces cobj with ${COBJ} in generated scripts. |
| tests/file-lock.src/access-different-record.at | Replaces cobj with ${COBJ} in generated scripts. |
| tests/command-line-options.src/Wconstant.at | Replaces cobj with ${COBJ}. |
| tests/command-line-options.src/m.at | Uses PATHSEP for CLASSPATH portability on Windows. |
| tests/command-line-options.src/jar.at | Adds Windows skip + uses PATHSEP for CLASSPATH portability. |
| tests/command-line-options.src/fsyntax-only.at | Replaces cobj with ${COBJ}. |
| tests/command-line-options.src/debug.at | Skips Windows -debug segfault test. |
| tests/cobol_utf8.src/pic-x.at | Replaces cobj with ${COBJ}. |
| tests/cobol_utf8.src/pic-n.at | Replaces cobj with ${COBJ}. |
| tests/cobol_utf8.src/national.at | Replaces cobj with ${COBJ}. |
| tests/cobol_utf8.src/java-interface.at | Replaces cobj with ${COBJ} across cases. |
| tests/cobol_utf8.src/compare-national-diff-size.at | Replaces cobj with ${COBJ}. |
| tests/atlocal.in | Adds PATHSEP and exports it for test portability. |
| tests/atlocal-win.env | Adds Windows-specific atlocal equivalent for CI. |
| README.md | Documents new copy/ install destination and updates line reference. |
| README_JP.md | Documents new copy/ install destination and updates line reference (JP). |
| .github/workflows/windows-test-other.yml | New reusable Windows workflow to run autotest suites under Git Bash. |
| .github/workflows/windows-generate-test-scripts.yml | New reusable Ubuntu workflow to generate and upload autotest scripts for Windows. |
| .github/workflows/push.yml | Wires the new Windows autotest generation + execution into push CI. |
| .github/workflows/pull-request.yml | Wires the new Windows autotest generation + execution into PR CI. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
tsh-hashimoto
approved these changes
Apr 15, 2026
This was referenced Apr 20, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Previously, autotest-based tests (command-line-options, syntax, run, misc, etc.) were only executed on Linux CI. This PR enables them on Windows CI as well, using Git Bash (
shell: bash) to run the autotest-generated shell scripts.Approach
./configure && cd tests && make) and uploads them as artifacts.cobj.exeandlibcobj.jar, sets up the test environment (atlocal, atconfig, CLASSPATH), and runs each test suite via Git Bash.cobj.exeare skipped with `AT_SKIP_IF`.Results
All 8 test suites pass on Windows:
Skipped tests (Windows-specific bugs)
The following tests are skipped on Windows due to `cobj.exe` bugs, not test-side issues. Issues will be filed after this PR is merged.
Other changes
cobjwith${COBJ}in all test cases for portabilityPATHSEPvariable (:on Unix,;on Windows) for CLASSPATH separatorcopy/directory towin/make-install.ps1概要
これまで Linux CI でのみ実行していた autotest ベースのテスト（command-line-options, syntax, run, misc 等）を、Windows CI でも実行できるようにしました。autotest が生成するシェルスクリプトは Git Bash (
shell: bash) を使って実行しています。アプローチ
./configure && cd tests && make）し、artifact としてアップロード。cobj.exe/libcobj.jarをダウンロードし、テスト環境（atlocal, atconfig, CLASSPATH）を設定後、各テストスイートを Git Bash で実行。cobj.exeバグで失敗するテストはAT_SKIP_IFでスキップ。結果
8 テストスイート全てが Windows で成功:
スキップしたテスト（Windows 固有バグ）
以下のテストは
cobj.exeのバグによりスキップしています。テスト側の不具合ではありません。PR マージ後に Issue として起票します。その他の変更
cobjを全て${COBJ}に置換（ポータビリティ向上）PATHSEP変数を追加（Unix::、Windows:;）win/make-install.ps1にcopy/ディレクトリのインストールを追加