.. sectnum::
README - SCons.Tool.MSCommon
############################
.. contents:: **Table of Contents**
:depth: 2
:local:
Design Notes
============
* Public, user-callable functions and exception types are available via
the ``SCons.Tool.MSCommon`` namespace.
* Some existing code has been moved from ``MSCommon/vc.py`` to the appropriate
``MSCommon/MSVC/<modulename>``.
* No functions from the MSVC module or its child modules are intended to be invoked directly.
All functions of interest are made available via the ``SCons.Tool.MSCommon`` namespace.
It is anticipated that more code may be moved in the future as new features are added.
By exposing the public API through ``SCons.Tool.MSCommon`` there should not be a problem
with code movement.
* Additional helper functions primarily used for the test suite were added to
``MSCommon/vc.py`` and are available via the ``SCons.Tool.MSCommon`` namespace.
Known Issues
============
The following issues are known to exist:
* Using ``MSVC_USE_SCRIPT`` and ``MSVC_USE_SCRIPT_ARGS`` to call older Microsoft SDK
``SetEnv.cmd`` batch files may result in build failures. Some of these batch files
require delayed expansion to be enabled which is not usually the Windows default.
One solution would be to launch the MSVC batch file command in a new command interpreter
instance with delayed expansion enabled via command-line options.
* The code to suppress the "No versions of the MSVC compiler were found" warning for
the default environment was moved from ``MSCommon/vc.py`` to ``MSCommon/MSVC/SetupEnvDefault.py``.
There very few, if any, existing unit tests. Now that the code is isolated in its own
module with a limited API, unit tests may be easier to implement.
Experimental Features
=====================
msvc_query_version_toolset(version=None, prefer_newest=True)
------------------------------------------------------------
The experimental function ``msvc_query_version_toolset`` was added to ``MSCommon/vc.py``
and is available via the ``SCons.Tool.MSCommon`` namespace. This function takes a version
specification or a toolset version specification and a product preference as arguments and
returns the msvc version and the msvc toolset version for the corresponding version specification.
This is a proxy for using the toolset version for selection until that functionality can be added.
Example usage:
::
for version in [
'14.3',
'14.2',
'14.1',
'14.0',
'14.32',
'14.31',
'14.29',
'14.16',
'14.00',
'14.28.29333', # only 14.2
'14.20.29333', # fictitious for testing
]:
for prefer_newest in (True, False):
try:
msvc_version, msvc_toolset_version = msvc_query_version_toolset(version, prefer_newest=prefer_newest)
failed = False
except MSVCToolsetVersionNotFound:
failed = True
if failed:
msg = 'FAILED'
newline = '\n'
else:
env = Environment(MSVC_VERSION=msvc_version, MSVC_TOOLSET_VERSION=msvc_toolset_version)
msg = 'passed'
newline = ''
print('{}Query: {} version={}, prefer_newest={}'.format(newline, msg, version, prefer_newest))
Example output fragment
::
Build: _build003 {'MSVC_VERSION': '14.3', 'MSVC_TOOLSET_VERSION': '14.29.30133'}
Where: C:\Software\MSVS-2022-143-Com\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64\cl.exe
Where: C:\Software\MSVS-2022-143-Com\Common7\Tools\guidgen.exe
Query: passed version=14.2, prefer_newest=True
Build: _build004 {'MSVC_VERSION': '14.2', 'MSVC_TOOLSET_VERSION': '14.29.30133'}
Where: C:\Software\MSVS-2019-142-Com\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64\cl.exe
Where: C:\Software\MSVS-2019-142-Com\Common7\Tools\guidgen.exe
Query: passed version=14.2, prefer_newest=False
Undocumented Features
=====================
set SCONS_CACHE_MSVC_FORCE_DEFAULTS=1
-------------------------------------
The Windows system environment variable ``SCONS_CACHE_MSVC_FORCE_DEFAULTS`` was added. This variable is only
evaluated when the msvc cache is enabled and accepts the values ``1``, ``true``, and ``True``.
When enabled, the default msvc toolset version and the default sdk version, if not otherwise specified, are
added to the batch file argument list. This is intended to make the cache more resilient to Visual Studio
updates that may change the default toolset version and/or the default SDK version.
Example usage:
::
@echo Enabling scons cache ...
@set "SCONS_CACHE_MSVC_CONFIG=mycachefile.json"
@set "SCONS_CACHE_MSVC_FORCE_DEFAULTS=True"
End-User Diagnostic Tools
=========================
Due to the proliferation of user-defined msvc batch file arguments, the likelihood of end-user build
failures has increased.
Some of the options that may be employed in diagnosing end-user msvc build failures are listed below.
msvc_set_scripterror_policy('Warning') and MSVC_SCRIPTERROR_POLICY='Warning'
----------------------------------------------------------------------------
Enabling warnings to be produced for detected msvc batch file errors may provide additional context
for build failures. Refer to the documentation for details.
Change the default policy:
::
from SCons.Tool.MSCommon import msvc_set_scripterror_policy
msvc_set_scripterror_policy('Warning')
Specify the policy per-environment:
::
env = Environment(MSVC_VERSION='14.3', MSVC_SPECTRE_LIBS=True, MSVC_SCRIPTERROR_POLICY='Warning')
set SCONS_MSCOMMON_DEBUG=mydebugfile.txt
----------------------------------------
The traditional method of diagnosing end-user issues is to enable the internal msvc debug logging.
set SCONS_CACHE_MSVC_CONFIG=mycachefile.json
--------------------------------------------
On occasion, enabling the cache file can prove to be a useful diagnostic tool. If nothing else,
issues with the msvc environment may be readily apparent.
vswhere.exe
-----------
On occasion, the raw vswhere output may prove useful especially if there are suspected issues with
detection of installed msvc instances.
Windows command-line sample invocations:
::
@rem 64-Bit Windows
"%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -all -sort -prerelease -products * -legacy -format json >MYVSWHEREOUTPUT.json
@rem 32-Bit Windows:
"%ProgramFiles%\Microsoft Visual Studio\Installer\vswhere.exe" -all -sort -prerelease -products * -legacy -format json >MYVSWHEREOUTPUT.json
Visual Studio Implementation Notes
==================================
Batch File Arguments
--------------------
Supported MSVC batch file arguments by product:
======= === === ======= =======
Product UWP SDK Toolset Spectre
======= === === ======= =======
VS2022 X X X X
------- --- --- ------- -------
VS2019 X X X X
------- --- --- ------- -------
VS2017 X X X X
------- --- --- ------- -------
VS2015 X X
======= === === ======= =======
Supported MSVC batch file arguments in SCons:
======== ====================================== ===================================================
Argument Construction Variable Script Argument Equivalent
======== ====================================== ===================================================
UWP ``MSVC_UWP_APP=True`` ``MSVC_SCRIPT_ARGS='store'``
-------- -------------------------------------- ---------------------------------------------------
SDK ``MSVC_SDK_VERSION='10.0.20348.0'`` ``MSVC_SCRIPT_ARGS='10.0.20348.0'``
-------- -------------------------------------- ---------------------------------------------------
Toolset ``MSVC_TOOLSET_VERSION='14.31.31103'`` ``MSVC_SCRIPT_ARGS='-vcvars_ver=14.31.31103'``
-------- -------------------------------------- ---------------------------------------------------
Spectre ``MSVC_SPECTRE_LIBS=True`` ``MSVC_SCRIPT_ARGS='-vcvars_spectre_libs=spectre'``
======== ====================================== ===================================================
**MSVC_SCRIPT_ARGS contents are not validated. Utilizing script arguments that have construction
variable equivalents is discouraged and may lead to difficult to diagnose build errors.**
Additional constraints:
* ``MSVC_SDK_VERSION='8.1'`` and ``MSVC_UWP_APP=True`` is supported only for the v140
build tools (i.e., ``MSVC_VERSION='14.0'`` or ``MSVC_TOOLSET_VERSION='14.0'``).
* ``MSVC_SPECTRE_LIBS=True`` and ``MSVC_UWP_APP=True`` is not supported (i.e., there
are no spectre mitigations libraries for UWP builds).
Default Toolset Version
-----------------------
Side-by-side toolset versions were introduced in Visual Studio 2017.
The examples shown below are for Visual Studio 2022.
The msvc default toolset version is dependent on the installation options
selected. This means that the default toolset version may be different for
each machine given the same Visual Studio product.
The msvc default toolset is not necessarily the latest toolset installed.
This has implications when a toolset version is specified using only one minor
digit (e.g., ``MSVC_TOOLSET_VERSION='14.3'`` or ``MSVC_SCRIPT_ARGS='-vcvars_ver=14.3'``).
Explicitly defining ``MSVC_TOOLSET_VERSION=None`` will return the same toolset
that the msvc batch files would return. When using ``MSVC_SCRIPT_ARGS``, the
toolset specification should be omitted entirely.
Local installation and summary test results:
::
VS2022\VC\Auxiliary\Build\Microsoft.VCToolsVersion.v143.default.txt
14.31.31103
VS2022\VC\Auxiliary\Build\Microsoft.VCToolsVersion.default.txt
14.32.31326
Toolset version summary:
::
14.31.31103 Environment()
14.31.31103 Environment(MSVC_TOOLSET_VERSION=None)
14.32.31326* Environment(MSVC_TOOLSET_VERSION='14.3')
14.32.31326* Environment(MSVC_SCRIPT_ARGS=['-vcvars_ver=14.3'])
14.31.31103 Environment(MSVC_TOOLSET_VERSION='14.31')
14.31.31103 Environment(MSVC_SCRIPT_ARGS=['-vcvars_ver=14.31'])
14.32.31326 Environment(MSVC_TOOLSET_VERSION='14.32')
14.32.31326 Environment(MSVC_SCRIPT_ARGS=['-vcvars_ver=14.32'])
VS2022\\Common7\\Tools\\vsdevcmd\\ext\\vcvars.bat usage fragment:
::
@echo -vcvars_ver=version : Version of VC++ Toolset to select
@echo ** [Default] : If -vcvars_ver=version is NOT specified, the toolset specified by
@echo [VSInstallDir]\VC\Auxiliary\Build\Microsoft.VCToolsVersion.v143.default.txt will be used.
@echo ** 14.0 : VS 2015 (v140) VC++ Toolset (installation of the v140 toolset is a prerequisite)
@echo ** 14.xx : VS 2017 or VS 2019 VC++ Toolset, if that version is installed on the system under
@echo [VSInstallDir]\VC\MSVC\Tools\[version]. Where '14.xx' specifies a partial
@echo [version]. The latest [version] directory that matches the specified value will
@echo be used.
@echo ** 14.xx.yyyyy : VS 2017 or VS 2019 VC++ Toolset, if that version is installed on the system under
@echo [VSInstallDir]\VC\MSVC\Tools\[version]. Where '14.xx.yyyyy' specifies an
@echo exact [version] directory to be used.
@echo ** 14.xx.VV.vv : VS 2019 C++ side-by-side toolset package identity alias, if the SxS toolset has been installed on the system.
@echo Where '14.xx.VV.vv' corresponds to a SxS toolset
@echo VV = VS Update Major Version (e.g. "16" for VS 2019 v16.9)
@echo vv = VS Update Minor version (e.g. "9" for VS 2019 v16.9)
@echo Please see [VSInstallDir]\VC\Auxiliary\Build\[version]\Microsoft.VCToolsVersion.[version].txt for mapping of
@echo SxS toolset to [VSInstallDir]\VC\MSVC\Tools\ directory.
VS2022 batch file fragment to determine the default toolset version:
::
@REM Add MSVC
set "__VCVARS_DEFAULT_CONFIG_FILE=%VCINSTALLDIR%Auxiliary\Build\Microsoft.VCToolsVersion.default.txt"
@REM We will "fallback" to Microsoft.VCToolsVersion.default.txt (latest) if Microsoft.VCToolsVersion.v143.default.txt does not exist.
if EXIST "%VCINSTALLDIR%Auxiliary\Build\Microsoft.VCToolsVersion.v143.default.txt" (
if "%VSCMD_DEBUG%" GEQ "2" @echo [DEBUG:ext\%~nx0] Microsoft.VCToolsVersion.v143.default.txt was found.
set "__VCVARS_DEFAULT_CONFIG_FILE=%VCINSTALLDIR%Auxiliary\Build\Microsoft.VCToolsVersion.v143.default.txt"
) else (
if "%VSCMD_DEBUG%" GEQ "1" @echo [DEBUG:ext\%~nx0] Microsoft.VCToolsVersion.v143.default.txt was not found. Defaulting to 'Microsoft.VCToolsVersion.default.txt'.
)
Empirical evidence suggests that the default toolset version is different from the latest
toolset version when the toolset version immediately preceding the latest version is
installed. For example, the ``14.31`` toolset version is installed when the ``14.32``
toolset version is the latest.
Visual Studio Version Notes
============================
SDK Versions
------------
==== ============
SDK Format
==== ============
10.0 10.0.XXXXX.Y
---- ------------
8.1 8.1
==== ============
BuildTools Versions
-------------------
========== ===== ===== ========
BuildTools VCVER CLVER MSVCRT
========== ===== ===== ========
v143 14.3 19.3 140/ucrt
---------- ----- ----- --------
v142 14.2 19.2 140/ucrt
---------- ----- ----- --------
v141 14.1 19.1 140/ucrt
---------- ----- ----- --------
v140 14.0 19.0 140/ucrt
---------- ----- ----- --------
v120 12.0 18.0 120
---------- ----- ----- --------
v110 11.0 17.0 110
---------- ----- ----- --------
v100 10.0 16.0 100
---------- ----- ----- --------
v90 9.0 15.0 90
---------- ----- ----- --------
v80 8.0 14.0 80
---------- ----- ----- --------
v71 7.1 13.1 71
---------- ----- ----- --------
v70 7.0 13.0 70
---------- ----- ----- --------
v60 6.0 12.0 60
========== ===== ===== ========
Product Versions
----------------
======== ===== ========= ============
Product VSVER SDK BuildTools
======== ===== ========= ============
2022 17.0 10.0, 8.1 v143 .. v140
-------- ----- --------- ------------
2019 16.0 10.0, 8.1 v142 .. v140
-------- ----- --------- ------------
2017 15.0 10.0, 8.1 v141 .. v140
-------- ----- --------- ------------
2015 14.0 10.0, 8.1 v140
-------- ----- --------- ------------
2013 12.0 v120
-------- ----- --------- ------------
2012 11.0 v110
-------- ----- --------- ------------
2010 10.0 v100
-------- ----- --------- ------------
2008 9.0 v90
-------- ----- --------- ------------
2005 8.0 v80
-------- ----- --------- ------------
2003.NET 7.1 v71
-------- ----- --------- ------------
2002.NET 7.0 v70
-------- ----- --------- ------------
6.0 6.0 v60
======== ===== ========= ============
SCons Implementation Notes
==========================
Compiler Detection Logic
------------------------
**WARNING: the compiler detection logic documentation below is likely out-of-date.**
In the future, the compiler detection logic documentation will be updated and integrated
into the current document format as appropriate.
::
This is the flow of the compiler detection logic:
External to MSCommon:
The Tool init modules, in their exists() routines, call -> msvc_exists(env)
At the moment, those modules are:
SCons/Tool/midl.py
SCons/Tool/mslib.py
SCons/Tool/mslink.py
SCons/Tool/msvc.py
SCons/Tool/msvs.py
env may contain a version request in MSVC_VERSION, but this is not used
in the detection that follows from msvc_exists(), only in the later
batch that starts with a call to msvc_setup_env().
Internal to MSCommon/vc.py:
+ MSCommon/vc.py:msvc_exists:
| vcs = cached_get_installed_vcs(env)
| returns True if vcs > 0
|
+-> MSCommon/vc.py:cached_get_installed_vcs:
| checks global if we've run previously, if so return it
| populate the global from -> get_installed_vcs(env)
|
+-> MSCommon/vc.py:get_installed_vcs:
| loop through "known" versions of msvc, granularity is maj.min
| check for product dir -> find_vc_pdir(env, ver)
|
+-> MSCommon/vc.py:find_vc_pdir:
| From the msvc-version to pdir mapping dict, get reg key base and value
| If value is none -> find_vc_pdir_vswhere(ver, env)
|
+-> MSCommon/vc.py:find_vc_pdir_vswhere:
| From the vc-version to VS-version mapping table get string
| Figure out where vswhere is -> msvc_find_vswhere()
| Use subprocess to call vswhere, return first line of match
/
| else get product directory from registry (<= 14.0)
/
| if we found one -> _check_cl_exists_in_vc_dir(env, pdir, ver)
|
+-> MSCommon/vc.py:_check_cl_exists_in_vc_dir:
| Figure out host/target pair
| if version > 14.0 get specific version by looking in
| pdir + Auxiliary/Build/Microsoft/VCToolsVersion/default.txt
| look for pdir + Tools/MSVC/{specver}/bin/host/target/cl.exe
| if 14.0 or less, "do older stuff"
All of this just got us a yes-no answer on whether /some/ msvc version
exists, but does populate __INSTALLED_VCS_RUN with all of the top-level
versions as noted for get_installed_vcs
Externally:
Once a module's exists() has been called (or, in the case of
clang/clangxx, after the compiler has been detected by other means -
those still expect the rest of the msvc chain but not cl.exe)
the module's generate() function calls -> msvc_setup_env_once(env)
Internally:
+ MSCommon/vc.py:msvc_setup_env_once:
| checks for environment flag MSVC_SETUP_RUN
| if not, -> msvc_setup_env(env) and set flag
|
+-+ MSCommon/vc.py:msvc_setup_env:
| set ver from -> get_default_version(env)
|
+-+ MSCommon/vc.py:get_default_version:
| if no version specified in env.MSVC_VERSION:
| return first entry from -> cached_get_installed_vcs(env)
| else return requested version
/
| get script from MSVC_USE_SCRIPT if set to a filename
| -> script_env(script)
|
+-+ MSCommon/vc.py:script_env:
| return (possibly cached) script variables matching script arg
/
| else -> msvc_find_valid_batch_script(env, version)
|
+-+ MSCommon/vc.py:msvc_find_valid_batch_script:
| Build a list of plausible target values, and loop through
| look for host + target -> find_batch_file(env, ver, host, target)
|
+-+ MSCommon/vc.py:find_batch_file:
| call -> find_vc_pdir (see above)
| use the return to construct a version-biased batfile path, check
/
| if not found, try sdk scripts (unknown if this is still useful)
Problems:
- For VS >= 2017, VS and VS are not 1:1, there can be many VC for one VS
- For vswhere-ready versions, detection does not proceed beyond the
product level ("2019") into individual "features" (individual msvc)
- As documented for MSVC_VERSION, compilers can only be requested if versions
are from the set in _VCVER, so 14.1 but not 14.16 or 14.16.27023
- Information found in the first pass (msvs_exists) isn't really
available anywhere except the cached version list, since we just
return true/false.
- Since msvc_exists chain of calls does not look at version, we
can proceed to compiler setup if *any* msvc was found, even if the
one requested wasn't found.
