Skip to content

fix windows support - #701

Merged
candytaco merged 12 commits into
mainfrom
windows
Aug 21, 2026
Merged

fix windows support#701
candytaco merged 12 commits into
mainfrom
windows

Conversation

@candytaco

Copy link
Copy Markdown
Contributor

Fix: Windows compatibility

Turns out that the issue was simple: the C math libraries are integrated in Windows, so asking for m.lib during opemctm compilation failed on that line in Windows. Added a guard that only asks for m.lib if not Windows.

A few other things to ensure compatibility:

  • explicitly enforce the javascript mimetype when opening viewers
  • account for Windows locking files when they are opened
  • the direct path to the inkscape executable can now be specified; falls back to just calling inkscape
  • removed legacy Windows config resolution

@candytaco
candytaco marked this pull request as ready for review August 21, 2026 20:09
Copilot AI lite review requested due to automatic review settings August 21, 2026 20:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to make Pycortex run more reliably on Windows by removing the hard dependency on the Unix libm during compilation, fixing MIME type serving for WebGL viewers, and addressing Windows file-locking constraints around temporary files and external executables (e.g., Inkscape).

Changes:

  • Skip linking against m on Windows when building OpenCTM.
  • Force .js to be served with a JavaScript MIME type to avoid browser execution refusal on Windows.
  • Update multiple temp-file workflows (and dependency path resolution) to better tolerate Windows file locking and PATH differences.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
setup.py Avoid linking m on Windows builds.
README.md Remove Windows “unsupported” note; add build-from-source note.
cortex/webgl/serve.py Force JavaScript MIME type mapping for .js.
cortex/volume.py Avoid Windows temp-file locking by closing NamedTemporaryFile and cleaning up explicitly.
cortex/utils.py Use resolved Inkscape executable path constant for launching.
cortex/tests/test_testing_utils.py Update mocks to align with new Inkscape path resolution helper.
cortex/tests/test_quickflat.py Adjust Inkscape detection and temp-file handling for Windows.
cortex/tests/test_dataset.py Close temp files before reopening via filename (Windows) and update Inkscape detection.
cortex/testing_utils.py Add config-aware Inkscape path resolution used by tests and runtime.
cortex/svgoverlay.py Avoid /dev/stdin, build Inkscape command as argv list, and use temp files for Windows.
cortex/options.py Update user config directory resolution for the new appdirs behavior.
cortex/freesurfer.py Close temp files before reuse on Windows; explicitly unlink temp outputs.
cortex/defaults.cfg Document Windows-friendly Inkscape configuration behavior.
cortex/dataset/dataset.py Write overlay content to a named temp file for Windows compatibility.
cortex/brainctm.py Adjust OpenCTM temp-file handling for Windows.
cortex/blender/init.py Replace context-managed temp file usage with explicit close/unlink flow.
cortex/appdirs.py Remove legacy Windows appauthor-based directory structure; use environment variables directly.
Suppressed comments (5)

cortex/tests/test_dataset.py:40

  • This test creates a delete=False HDF temp file but never removes it, so it will accumulate across repeated test runs. Wrap the body in try/finally and os.unlink(tf.name) in the finally block.

This issue also appears in the following locations of the same file:

  • line 163
  • line 184
  • line 197
  • line 209
def test_dataset():
    vol = np.random.randn(*volshape)
    stack = (np.ones(volshape[::-1]) * np.linspace(0, 1, volshape[0])).T
    mask = db.get_mask(subj, xfmname, "thick")

    ds = dataset.Dataset(randvol=(vol, subj, xfmname), stack=(stack, subj, xfmname))
    ds.append(thickstack=ds.stack.masked["thick"])
    tf = tempfile.NamedTemporaryFile(suffix = ".hdf", delete = False)
    tf.close()
    ds.save(tf.name)

    ds = dataset.Dataset.from_file(tf.name)
    assert len(ds["thickstack"].data) == mask.sum()
    assert np.allclose(ds["stack"].data[mask], ds["thickstack"].data)

cortex/tests/test_dataset.py:194

  • This test creates a delete=False temp file but never deletes it, which can leave behind many *.hdf files. Add a try/finally cleanup to unlink the temp file.
def test_mask_save():
    tf = tempfile.NamedTemporaryFile(suffix = ".hdf", delete = False)
    tf.close()
    ds = cortex.Dataset(test=(np.random.randn(*volshape), subj, xfmname))
    ds.append(masked=ds.test.masked["thin"])
    data = ds.masked.data
    ds.save(tf.name)

    ds = cortex.load(tf.name)
    assert ds.masked.shape == volshape
    assert np.allclose(ds.masked.data, data)

cortex/tests/test_dataset.py:207

  • This test creates a delete=False temp file but never deletes it. Use try/finally so the temp file is unlinked even if ds.save() or assertions fail.
def test_overwrite():
    # the handle is closed so that h5py can open the file by name, which
    # Windows forbids while the original handle is still open
    tf = tempfile.NamedTemporaryFile(suffix = ".hdf", delete = False)
    tf.close()
    ds = cortex.Dataset(test=(np.random.randn(*volshape), subj, xfmname))
    ds.save(tf.name)

    ds.save()
    assert ds.test.data.shape == volshape

cortex/tests/test_dataset.py:227

  • This test creates a delete=False temp file but never deletes it (and the test is long enough that failures are plausible), so it can leave behind many *.hdf files. Add a try/finally cleanup that unlinks the temp file at the end.
def test_pack():
    tf = tempfile.NamedTemporaryFile(suffix = ".hdf", delete = False)
    tf.close()
    ds = cortex.Dataset(test=(np.random.randn(*volshape), subj, xfmname))
    ds.save(tf.name, pack=True)

    ds = cortex.load(tf.name)
    pts, polys = cortex.db.get_surf(subj, "fiducial", "lh")
    dpts, dpolys = ds.get_surf(subj, "fiducial", "lh")
    assert np.allclose(pts, dpts)

    overlay_db = cortex.db.get_overlay(subj, None, modify_svg_file=False)
    rois_db = overlay_db.rois.labels.elements.keys()
    # keep the temporary file object in memory to avoid the file being deleted
    temp_file = ds.get_overlay(subj, "rois")
    overlay_ds = cortex.db.get_overlay(subj, temp_file.name, modify_svg_file=False)
    rois_ds = overlay_ds.rois.labels.elements.keys()
    assert rois_db == rois_ds

cortex/tests/test_dataset.py:173

  • This test creates a delete=False temp file but never deletes it. Add cleanup (preferably try/finally) so the temp file is removed even when assertions fail.
def test_dataset_save():
    tf = tempfile.NamedTemporaryFile(suffix = ".hdf", delete = False)
    tf.close()
    mrand = np.random.randn(2, *volshape)
    rand = np.random.randn(*volshape)
    ds = cortex.Dataset(test=(mrand, subj, xfmname))
    ds.append(twod=cortex.Volume2D(rand, rand, subj, xfmname))
    ds.append(rgb=cortex.VolumeRGB(rand, rand, rand, subj, xfmname))
    ds.append(vert=cortex.Vertex.random(subj))
    ds.save(tf.name)


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread cortex/blender/__init__.py Outdated
Comment thread cortex/brainctm.py
Comment thread cortex/svgoverlay.py Outdated
Comment thread cortex/dataset/dataset.py
Comment on lines +221 to 224
tf = tempfile.NamedTemporaryFile(delete = False)
tf.write(group['rois'][0])
tf.seek(0)
tf.close()
return tf
Comment thread README.md
Comment on lines +37 to +38
Note you will need a C compiler installed to build from scratch because there are cython code.

Comment on lines 13 to 18
@pytest.mark.skipif(no_inkscape, reason='Inkscape required')
def test_quickflat():
tf = tempfile.NamedTemporaryFile(suffix=".png")
tf = tempfile.NamedTemporaryFile(suffix = ".png", delete = False)
tf.close()
view = cortex.Volume.random("S1", "fullhead", cmap="hot")
cortex.quickflat.make_png(tf.name, view)
Comment thread cortex/svgoverlay.py
@kroq-gar78

Copy link
Copy Markdown
Contributor

What's going on with this commit? e67eb94

candytaco and others added 2 commits August 21, 2026 13:50
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@candytaco

candytaco commented Aug 21, 2026

Copy link
Copy Markdown
Contributor Author

That's one of the file locking fixes. Windows locks a file when a process opens it to prevent multiple threads from writing to it simultaneously. So we have to close the handlers when passing files between processes.

Because of this, we have the tempfiles not be delete immediately after the opening process closes it, and we track filename inside the scope. unlink then deletes it as the tempfile scope exits.

candytaco and others added 2 commits August 21, 2026 14:59
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
AbhroneelGhosh added a commit to AbhroneelGhosh/pycortex that referenced this pull request Aug 21, 2026
self.tfname = tfname.encode('ascii') raises UnicodeEncodeError if
%TEMP% contains a non-ASCII character (e.g. a non-ASCII Windows
username) -- same class of bug as the options.cfg/webgl-export fixes,
just in a spot those didn't cover. Reproduced directly: encoding a
path containing 'café' with .encode('ascii') raises; os.fsencode()
(which uses the actual filesystem encoding) does not.

Windows branch only; POSIX branch (bytes(self.tf.name, 'ascii')) is
unchanged -- it has the same latent issue but touching it is a
different, non-Windows concern out of scope for this branch.

Found via comparison with gallantlab#701, which hit the same
bug (via Copilot Autofix) in their ungated version of this code.
AbhroneelGhosh added a commit to AbhroneelGhosh/pycortex that referenced this pull request Aug 21, 2026
Same root cause as the earlier svgoverlay.py/brainctm.py Hemi fixes:
NamedTemporaryFile holds an exclusive handle on Windows, so a second
opener of the same path fails.

- brainctm.read_pack(): the OpenCTM C library's fopen() can't open a
  path NamedTemporaryFile still holds (mirrors Hemi.save()). Write via
  mkstemp and close before CTMfile opens it.
- Dataset.get_overlay(): returns tf for a caller to reopen tf.name by
  path (e.g. get_overlay(modify_svg_file=False) -> shutil.copy2 reading
  it as a source). Reproduced directly: shutil.copy2 raises
  PermissionError(13) reading a path NamedTemporaryFile still holds.
  delete=False + close() releases the handle while keeping the file on
  disk for the caller. POSIX branches unchanged in both functions.

Found by comparing with gallantlab#701, which hit both of
these in its own (ungated) version of the same fix.
AbhroneelGhosh added a commit to AbhroneelGhosh/pycortex that referenced this pull request Aug 21, 2026
Every test that creates a NamedTemporaryFile and passes its .name to
ds.save() (h5py) or quickflat.make_png() (PIL) hits the same reopen-
while-open PermissionError as the source-tree bugs fixed earlier in
this branch -- h5py/PIL independently open the path while the test's
own handle is still open.

Adds windows_safe_named_tempfile() to testing_utils.py (close the
handle immediately on Windows via delete=False, unchanged on POSIX)
rather than repeating the same gated block at each of the 6 call
sites. Confirms the gap flagged earlier this session, and matches
gallantlab#701's equivalent (ungated) test fixes.
AbhroneelGhosh added a commit to AbhroneelGhosh/pycortex that referenced this pull request Aug 21, 2026
mimetypes.guess_type() reads the extension->type mapping from the
Windows registry, where .js is frequently (machine-dependently)
registered as text/plain rather than a JS mimetype. Browsers refuse
to execute a <script> tag served with a non-JS Content-Type, which
would silently break the WebGL viewer's static JS assets.

Registry-state-dependent, so I could not reproduce it on this
particular machine (.js already resolves to application/javascript
here) -- found via comparison with gallantlab#701, which
documents hitting it directly and fixes it the same way, ungated.
Windows-gated here per this branch's POSIX-parity requirement; POSIX
mimetypes are untouched.
@candytaco
candytaco merged commit 31aa4d6 into main Aug 21, 2026
13 checks passed
@AbhroneelGhosh

Copy link
Copy Markdown

Thanks for this — tested it thoroughly on native Windows (MSVC build, conda env), including installing Inkscape and running the full test suite plus the example scripts under examples/. With Inkscape configured, pytest cortex/tests (excluding the headless-browser tests) passes cleanly: 71 passed, 4 skipped (playwright/Chromium and FreeSurfer, both legitimately unavailable here), 0 failed. The /dev/stdin → argument-list rewrite in svgoverlay.py works correctly end-to-end.

Found two remaining UnicodeEncodeError gaps, both from the same root cause (open() with no explicit encoding=, so Windows falls back to the locale codepage — cp1252 on this machine — instead of UTF-8). Both are quick, one-line-per-call-site fixes.

1. cortex.webgl.make_static() crashes on Windows — 100% reproducible via your own example

Running examples/webgl/static.py verbatim (cortex.Volume.randomcortex.webgl.make_static(outpath=..., data=volume, recache=True)) crashes every time:

UnicodeEncodeError: 'charmap' codec can't encode character '\ufeff' in position 0: character maps to <undefined>
  File "cortex\webgl\view.py", line 279, in make_static
    htmlembed.embed(html, desthtml, rootdirs)
  File "cortex\webgl\htmlembed.py", line 131, in embed
    htmlfile.write(line)

Root cause: htmlembed.py:122

with open(outfile, "w") as htmlfile:

opens the output file with no encoding=, so it inherits the Windows locale codepage. The html5lib serializer's output isn't guaranteed to be codepage-safe — here it's a leading \ufeff, but any character outside cp1252 would trigger the same crash. This isn't an edge case tied to unusual subject/colormap names — it's unconditional, since html_embed=True (the default) always routes through this exact code path. Static export — one of the most basic ways to share a viewer — is currently broken on Windows out of the box.

The same missing encoding= exists in the sibling code path at webgl/view.py:281 (make_static(..., html_embed=False)):

with open(desthtml, "w") as htmlfile:

Fix (both locations):

with open(outfile, "w", encoding="utf-8") as htmlfile:   # htmlembed.py:122
with open(desthtml, "w", encoding="utf-8") as htmlfile:  # webgl/view.py:281

2. Non-ASCII Windows usernames crash on first import cortex

options.py:20 writes the user's options.cfg the same unencoded way:

with open(usercfg, 'w') as fp:
    config.write(fp)

config.write() embeds the resolved filestore path, which is derived from appdirs.user_data_dir() — i.e. it contains the Windows username. I reproduced the exact failure mode by running this same code (unmodified) against a representative non-Latin-script value, since I can't create a differently-scripted Windows account on this machine to trigger it natively:

>>> c.set('basic', 'filestore', 'C:\\Users\\王伟\\AppData\\Local\\pycortex\\db')
>>> with open(path, 'w') as fp: c.write(fp)
UnicodeEncodeError: 'charmap' codec can't encode characters in position 21-23: character maps to <undefined>

(Note: a simple accented character like café won't reproduce this — é happens to be directly encodable in cp1252. It takes a genuinely non-Western-European script — Chinese, Japanese, Korean, Cyrillic, Arabic — which is a realistic username on any non-US/Western-European Windows install.)

Practical impact: any user whose Windows account name isn't in cp1252's repertoire hits this on the very first import cortex, before they've done anything else.

Fix:

with open(usercfg, 'w', encoding="utf-8") as fp:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants