Skip to content

Allow building with cmake on Windows - #2090

Open
gabrieljreed wants to merge 5 commits into
AcademySoftwareFoundation:mainfrom
gabrieljreed:fix-cmake-on-windows
Open

Allow building with cmake on Windows#2090
gabrieljreed wants to merge 5 commits into
AcademySoftwareFoundation:mainfrom
gabrieljreed:fix-cmake-on-windows

Conversation

@gabrieljreed

@gabrieljreed gabrieljreed commented Apr 15, 2026

Copy link
Copy Markdown

Resolves #1321
Resolves #2123

Premise

Path normalization

On Windows, rez applies path normalization (converting forward slashes to back slashes) to any environment variable whose name matches the pattern in the pathed_env_vars config setting, which defaults to *PATH. This is correct behavior for variables like PATH, PYTHONPATH, and LD_LIBRARY_PATH, which need native Windows path separators. However, CMAKE_MODULE_PATH also matches *PATH, but Cmake requires forward slashes in this variable.

Unterminated string literal in rez_install_python

When building rez packages on Windows that use rez_install_python (with nmake), the build fails with a Python SyntaxError: unterminated string literal

File "<string>", line 1
    'import
    ^
SyntaxError: unterminated string literal (detected at line 1)

The root cause is in InstallPython.cmake - the add_custom_command that compiles .py files into .pyc uses single quotes around the Python -c argument.

COMMAND ${py_bin} -c 'import py_compile \; py_compile.compile(...)'

On Unix, make dispatches commands via /bin/sh where single quotes are valid string delimiters. On Windows, nmake dispatches via cmd.exe, where single quotes are literal characters, so Python receives 'import as the start of an unterminated string literal.

Changes

  • Add a new configuration setting, non_pathed_env_vars, which is an exclusion list that takes priority over pathed_env_vars.
    • Any variable matching a pattern in this list is except from path normalization, even if it also matches pathed_env_vars
    • Supports the same fnmatch wildcards as pathed_env_vars
    • The default value ships with CMAKE_MODULE_PATH out of the box, since this is a known case where we don't want normalization
  • Fix install_python macro
    • Switch from single quoted shell args to cmake double quoted string for the -c argument
    • Add VERBATIM keyword to add_custom_command so Cmake handles all platform-specific argument quoting correctly

Tests

  • I'm able to build the hello_world example package on both Windows and Linux after setting build_system = "cmake"
  • Add new tests for ActionInterpreter._is_pathed_key and shell.set_env
  • I was unable to write tests for the rez_install_python cmake macro, but I verified that it worked manually

@gabrieljreed
gabrieljreed requested a review from a team as a code owner April 15, 2026 23:27
@linux-foundation-easycla

linux-foundation-easycla Bot commented Apr 15, 2026

Copy link
Copy Markdown

CLA Signed

The committers listed above are authorized under a signed CLA.

@herronelou

Copy link
Copy Markdown
Contributor

This would also help/fix: #1923 #1909

@maxnbk maxnbk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nicely done contribution.

Apart from this PR, we could use actual tests that use/generate the hello_world package, but I won't consider that a blocker for this particular PR merge.

I made a few suggestions are mostly cosmetic, or affect the way docs generate.

The only open question is the CMAKE_PREFIX_PATH one, or CMAKE_*_PATH, alternatively. I suspect the right thing to do would be the wildcard, since this is a new config value, and if someone needs to override the default config value, the result is effectively the same. I'm not just sure if there are any CMAKE path vars that are supposed to have the same treatment.

Comment thread src/rez/rezconfig.py
# This setting identifies environment variables that should NOT have path
# normalization applied, even if they match a pattern in ``pathed_env_vars``.
# This is useful for variables like ``CMAKE_MODULE_PATH`` which end in ``PATH``
# but require forward slashes regardless of the shell. Wildcards are supported.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# but require forward slashes regardless of the shell. Wildcards are supported.
# but require forward slashes regardless of the shell. Wildcards are supported.
# Takes priority over :data:`pathed_env_vars`.

Comment thread src/rez/rex.py
@@ -558,6 +558,8 @@ def escape_string(self, value, is_path=False):

@classmethod
def _is_pathed_key(cls, key):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def _is_pathed_key(cls, key):
def _is_pathed_key(cls, key):
"""Return True if ``key`` is a path-like env var subject to normalization."""

Comment thread src/rez/rezconfig.py Outdated
]

# This setting identifies environment variables that should NOT have path
# normalization applied, even if they match a pattern in ``pathed_env_vars``.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# normalization applied, even if they match a pattern in ``pathed_env_vars``.
# normalization applied, even if they match a pattern in :data:`pathed_env_vars`.

Comment thread src/rez/rezconfig.py Outdated
# This is useful for variables like ``CMAKE_MODULE_PATH`` which end in ``PATH``
# but require forward slashes regardless of the shell. Wildcards are supported.
non_pathed_env_vars = [
"CMAKE_MODULE_PATH"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we consider "CMAKE_*_PATH" or an additional "CMAKE_PREFIX_PATH"? It feels to me like it makes sense to do so.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my testing, I've only needed to set the one variable, but I'm happy to change it to your suggestion for flexibility!

Comment thread src/rez/rezconfig.py Outdated
"*PATH"
]

# This setting identifies environment variables that should NOT have path

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# This setting identifies environment variables that should NOT have path
# This setting identifies environment variables that should not have path

Comment thread docs/source/package_commands.rst Outdated
``CMAKE_MODULE_PATH`` requires forward slashes regardless of shell, and backslash conversion will
break CMake. You can exclude variables from normalization using the :data:`non_pathed_env_vars`
config setting, which takes priority over :data:`pathed_env_vars`. By default,
``CMAKE_MODULE_PATH`` is already excluded. Both settings support ``fnmatch``-style wildcards.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
``CMAKE_MODULE_PATH`` is already excluded. Both settings support ``fnmatch``-style wildcards.
``CMAKE_MODULE_PATH`` is already excluded. Both settings support :func:`fnmatch.fnmatch`-style wildcards.

@codecov

codecov Bot commented Jul 30, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 66.66667% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 61.29%. Comparing base (a4160c2) to head (585c224).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/rez/rezconfig.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2090   +/-   ##
=======================================
  Coverage   61.29%   61.29%           
=======================================
  Files         164      164           
  Lines       20568    20571    +3     
  Branches     3575     3576    +1     
=======================================
+ Hits        12607    12609    +2     
- Misses       7089     7090    +1     
  Partials      872      872           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

- Introduced `non_pathed_env_vars` configuration to exclude specific
variables from path normalization
- Updated `_is_pathed_key` method to respect the new configuration
- Added tests to verify behavior of non-pathed environment variables in
Windows shell

Signed-off-by: Gabriel Reed <gabrieljreed@gmail.com>
Signed-off-by: Gabriel Reed <gabrieljreed@gmail.com>
Signed-off-by: Gabriel Reed <gabrieljreed@gmail.com>
Signed-off-by: Gabriel Reed <gabrieljreed@gmail.com>
- Documentation improvements
- Replace default value of non_pathed_env_vars from `CMAKE_MODULE_PATH`
to `CMAKE_*_PATH`

Signed-off-by: Gabriel Reed <gabrieljreed@gmail.com>
@gabrieljreed
gabrieljreed force-pushed the fix-cmake-on-windows branch from e73f789 to 585c224 Compare July 30, 2026 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Windows Cmake path issue CMake breaks due to windows penchant for escape characters as path separators

4 participants