Skip to content

Pull Upstream to devshFixes 2 #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 90 commits into from
Apr 18, 2025

Conversation

AnastaZIuk
Copy link
Member

No description provided.

pow2clk and others added 30 commits February 17, 2025 21:49
Remove errors in Sema diagnostics for vectors longer than 4 in 6.9.
Test for failures using long vectors in unspported contexts and for correct codegen in
supported contexts. Verify errors persist in pre-6.9 shader models

The type buffer cache expects a max vector size of 4. By just skipping the cache for longer vectors, we don't overrun and store float7 vectors in the double3 slot or retrieve the double3 in place of float7.

Testing is for acceptance, mangling and basic copying that takes place
at the high level to ensure they are being accepted and recognized
correctly. The intent is not to tully test the passing of data as that
requires enabling vector operations to do properly. This test is used to
verify that these same constructs are disallowed in 6.8 and earlier.

A separate test verifies that disallowed contexts produce the
appropriate errors

Fixes #7117
Disallow long vectors, and arrays or structs containing long vectors in
cbuffers, entry functions, node records, tessellation patchs, or special intrinsic parameters with
user-defined struct parameters.
Expand resource attribute to all resource types by adding reskind and
resclass arguments indicating the specific resource type. Change detection
in HlslTypes to use these attribute arguments. Similarly add vertex
number arguments to output stream attribute and a boolean indicator of
input or output for tessellation patches.

Add geomstream attr to detect those objects

Use attribute to detect tesselation patches
Removes template arg counts and startswith stirngs to identify
tesslations patches and distinguish them from multisampled textures
Add setting for max vec size.

Determine long vector presence using DefinitionData bit?
OR
Rename testing for long vecs function?

Add attribute for geometry streams, produce and test errors for long vectors there.

Add and test errors for > 1024 element vectors.

Add vector size to error messages

good test changes
Go for consistent test filename formatting. most LLVM tests have dashes,
so dashes it is. Remove redundant sm68 test
Expand existing tests to different target and contexts. Add thorough
testing for geometry streams and tessellation patches.

Add toolong vector test. Verify that vectors that are over the maximum
for 6.9 fail.

Add subobjects and template classes to tests. These are unfortunately
disabled because the code to make them work causes other tests to fail.
Use RequireCompleteType to force specialization of templates encountered
in global and other scopes where finding long vectors is necessary where
possible. This populates the definitiondata which contains the base
class chain needed to detect when a base class has disqualifying long
vectors. It was also needed to detect when dependent types in a template
class result in long vectors.

Work graph node types didn't check their base classes for failures. This
affects base classes with longvectors that have sub classes used for
node objects which should fail for having long vector members.

Respond to feedback about iterating through fields in clunky manner
which got left out of the last reviewer feedback response
I guess it was about time. Should simplify some things later as well as at present and it was too easy to not do. Specifically, I was going to need to add another string check to the template instantiation code to identify longvectors. This is cleaner.

Incidentally convert another feedback texture string check to use attribs.

Incidentally resort the recently-added attribs to not break up the node shader attribs.
Vector types can be cached in a 2D array that has a column for lenghts 1-4. This uses the added contant to indicate the length and for the checks that confirm it isn't exceeded.
By setting the bit when the vector template is instantiated and then propagating it when members, be they standard members or base classes, the bit will be set correctly for any struct or struct-like type. For arrays, the arrays are pealed away in a utility function to get at the elements.

Decided to separate the check for completeness from the check for long vectors. Even though the latter almost always requires the former, they are separate concepts and embedding the first in the second would be unexpected
Output Streams, Tessellation patches, and global variables should be complete when receiving other correctness checks. If they cannot be made complete, they should produce an error. This was omitted for various of these including non-template globals, which was fine, but it meant that redundant errors were produced for templates, but not standard globals likely just because that was what was tested. This removes that distinction and adds testing for all of the above to the existing incomplete-type.hlsl test.
This adds a new `-metal` flag to DXC which can be used to generate
Metal's IR directly from DXC after compilation. There are some
limitations in this flag which are worth noting:

1) It does not support library shaders (yet)
2) It does not support disassembly (yet)
3) It is _wildly_ under tested because wtihout (2) we can't do anything
to really verify correct output (yay?)
This is a bunch of small changes to improve the quality of the time
traces. This mostly adds new timers breakign down dxcompilerobj and the
always inliner code.
remove some stale elements. Add some HLSL type helper functions and add some new ones. Make resource type retreiveals type-safe. Add some parameter comments and names to make clearer what the effect of them are. Pass resource attribute to cbuffer/tbuffer creation. Clean up and clarify error messages. Remove redundant type canonization from type queries. Correct resclass of tbuffers. Use multimatch utility of verify to condense checks
Correct UAVness of consume/append buffers.

Add HLSL notes for changes to DeclCXX

Share more code in IsHLSLVecMatType
Remove errors in Sema diagnostics for vectors longer than 4 in 6.9.
Test for failures using long vectors in unspported contexts and for
correct codegen in
supported contexts. Verify errors persist in pre-6.9 shader models

The type buffer cache expects a max vector size of 4. By just skipping
the cache for longer vectors, we don't overrun and store float7 vectors
in the double3 slot or retrieve the double3 in place of float7.

Testing is for acceptance, mangling and basic copying that takes place
at the high level to ensure they are being accepted and recognized
correctly. The intent is not to tully test the passing of data as that
requires enabling vector operations to do properly. This test is used to
verify that these same constructs are disallowed in 6.8 and earlier.

A separate test verifies that disallowed contexts produce the
appropriate errors

Fixes #7117
DxilContainerValidation.cpp has some int / bool comparisons that cause
default error warnings in some of the private builds.
This needs to be addressed. This PR changes the comparisons by
converting the numerical expressions into the appropriate boolean, then
comparing the booleans.
Polishing up changes made to improve dxc buildability in different
environments.
…#7199)

HLSL_INTRINSIC will need to be updated for SM 6.9, specifically:
- to add a new flag
- to encode minimum shader model version for an availability attribute

Changing this structure is a breaking change to the internal intrinsic
table protocol, which is used for the extension mechanism.

This change separates out the breaking change with no functional changes
for simpler review and testing.

For the new flag, this change switches to using a UINT Flags field to
make flags extensible without breaking the table format. For the
version, a UINT MinShaderModel will be the encoded version format used
elsewhere: (Major << 4) | (Minor & 0xF)

Commented code for using the MinShaderModel is provided for when a
subsequent change will implement the availability attribute checks.
…nline, and add Availability Attributes (#7156)

This PR addresses the front end part of OMM, defining the new flags
defined in the spec, and implementing the relevant diagnostics should
the flags be incompatible. It also adds the second template argument to
the RayQuery object, which is set to have a default value of 0 if no
explicit template argument is provided.
Fixes #7145

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
This is all about updating urls and tables in docs/SPIR-V.rst. I'm
currently working on a demo & tutorial about how to work with
Vulkan+HLSL+dxc toolchain so I made this PR. Thanks a lot for your
contribution to the ecosystem, and glad to see your feedback!

Signed-off-by: lumina37 <starry.qvq@gmail.com>
bob80905 and others added 25 commits April 2, 2025 16:19
…ests to spec. (#7281)

This PR adds 2 tests that were mentioned in the spec that haven't yet
been added.
1. A test that makes sure that restricted flags are diagnosed in DXR
entry shaders.
2. A test that makes sure that no diagnostics are emitted when a
restricted flag is used for a subobject in a non-library shadaer target.

Fixes #7282
If `"SPV_KHR_quad_control"` can be used, uses
`OpGroupNonUniformQuadAnyKHR` and `OpGroupNonUniformQuadAllKHR`. If not,
falls back to constructing the value using
`OpGroupNonUniformQuadSwap`.

Fixes #7247
Fixes #7248

Fix Clang Compilation on Linux without Microsoft extensions enabled.

## Rationale
Clang support depends on the `-fms-extensions` compiler flag.
[[1]](https://clang.llvm.org/docs/UsersManual.html#microsoft-extensions)
If enabled, the `_MSC_EXTENSIONS` macro is defined.
[[2]](https://github.com/llvm/llvm-project/blob/19a319667b567a26a20f9829a0ae7e6a5c259cba/clang/lib/Basic/Targets/OSTargets.cpp#L248)
This change seeks to address some recent questions about how the LLLVM
Coding Standards are applied in DXC.

---------

Co-authored-by: Ashley Coleman <ascoleman@microsoft.com>
…fferPointer proposal) (#7306)

#ifdef ENABLE_SPIRV_CODEGEN was omitted in several places.
Capitalize all the variables and rename a few in DxilValidation.cpp in
keeping with
https://llvm.org/docs/CodingStandards.html#name-types-functions-variables-and-enumerators-properly

As this file was easily mistaken for applying to the golden rule:
https://llvm.org/docs/CodingStandards.html#name-types-functions-variables-and-enumerators-properly
it is at serious risk of receiving changes that will get hung up by
requirements to follow the LLVM coding guidelines. This brings the cases
where variable capitalization is not in line with the coding standards
to avoid such pitfalls in the future.
Enables the declaration of long vector types for raw buffers, the
lowering of those and traditional vectors in loads and stores maintaining
the native types with new dxil ops along with validation and testing
support of the same.

Allow declaring long vector rawbuffer resources.
Previously disallowed along with other global types, this provides a
mechanism for indicating which buffers are raw and allowing them to
contain long vectors, continuing to produce an error for other resource
types verified by existing tests

Introduce native vector DXIL load/store intrinsics.
Add new raw buffer vector load/store intrinsics using the new vector
overload types.
Include them in validation associated with similar load/stores

Lower native vector raw buffers load/stores into new ops.
When the loaded/stored type is a vector of more than 1 element, the
shader model is 6.9 or higher, and the operation is on a raw buffer,
enable the generation of a native vector raw buffer load or store.

Incidental removal of unused parameter in load translation and some
refactoring
of the lowering to flow better with the new resret types.

add validation and compute shader tests

Vector to scalar raw buffer load lowering pass
Native vector loads and stores are generated for 6.9 targets and above.
This includes the 6.x target used when compiling to libraries. This adds
a pass run when linking that will lower the vector operations to scalar
operations for shader models that don't have native vector support. This
allows libraries compiled for supportive shader models to be linked to
targets without support.

Validate native vector loads and stores for properly defined parameters
of the correct type. Add tests for both vector load/stores and the
original scalar load/stores since they share a lot of validation code.

Fixes #7118
I provided feedback during code review that this function should be
named following LLVM conventions. That feedback did not account for the
fact that SemaHLSL is otherwise consistent using CamelCase instead of
camelCase naming.

This corrects my error by renaming to match the consistent style in
SemaHLSL.h.

I've also updated the parameter naming in the source file to conform to
LLVM style since I was in the area anyways.
… into a UAV (#7272)

This is a pass to add instructions to determine missing usage of the
NonUniformResourceIndex qualifier when dynamically indexing resources.
The instruction numbers will be written out to a UAV for later ingestion
by PIX to present a view of the output.
Bumps [cryptography](https://github.com/pyca/cryptography) from 43.0.1
to 44.0.1.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst">cryptography's
changelog</a>.</em></p>
<blockquote>
<p>44.0.1 - 2025-02-11</p>
<pre><code>
* Updated Windows, macOS, and Linux wheels to be compiled with OpenSSL
3.4.1.
* We now build ``armv7l`` ``manylinux`` wheels and publish them to PyPI.
* We now build ``manylinux_2_34`` wheels and publish them to PyPI.
<p>.. _v44-0-0:</p>
<p>44.0.0 - 2024-11-27
</code></pre></p>
<ul>
<li><strong>BACKWARDS INCOMPATIBLE:</strong> Dropped support for
LibreSSL &lt; 3.9.</li>
<li>Deprecated Python 3.7 support. Python 3.7 is no longer supported by
the
Python core team. Support for Python 3.7 will be removed in a future
<code>cryptography</code> release.</li>
<li>Updated Windows, macOS, and Linux wheels to be compiled with OpenSSL
3.4.0.</li>
<li>macOS wheels are now built against the macOS 10.13 SDK. Users on
older
versions of macOS should upgrade, or they will need to build
<code>cryptography</code> themselves.</li>
<li>Enforce the :rfc:<code>5280</code> requirement that extended key
usage extensions must
not be empty.</li>
<li>Added support for timestamp extraction to the
:class:<code>~cryptography.fernet.MultiFernet</code> class.</li>
<li>Relax the Authority Key Identifier requirements on root CA
certificates
during X.509 verification to allow fields permitted by
:rfc:<code>5280</code> but
forbidden by the CA/Browser BRs.</li>
<li>Added support for
:class:<code>~cryptography.hazmat.primitives.kdf.argon2.Argon2id</code>
when using OpenSSL 3.2.0+.</li>
<li>Added support for the
:class:<code>~cryptography.x509.Admissions</code> certificate
extension.</li>
<li>Added basic support for PKCS7 decryption (including S/MIME 3.2) via

:func:<code>~cryptography.hazmat.primitives.serialization.pkcs7.pkcs7_decrypt_der</code>,

:func:<code>~cryptography.hazmat.primitives.serialization.pkcs7.pkcs7_decrypt_pem</code>,
and

:func:<code>~cryptography.hazmat.primitives.serialization.pkcs7.pkcs7_decrypt_smime</code>.</li>
</ul>
<p>.. _v43-0-3:</p>
<p>43.0.3 - 2024-10-18</p>
<pre><code>
* Fixed release metadata for ``cryptography-vectors``
<p>.. _v43-0-2:</p>
<p>43.0.2 - 2024-10-18
</code></pre></p>
<ul>
<li>Fixed compilation when using LibreSSL 4.0.0.</li>
</ul>
<p>.. _v43-0-1:</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pyca/cryptography/commit/adaaaed77db676bbaa9d171175db81dce056e2a7"><code>adaaaed</code></a>
Bump for 44.0.1 release (<a
href="https://redirect.github.com/pyca/cryptography/issues/12441">#12441</a>)</li>
<li><a
href="https://github.com/pyca/cryptography/commit/ccc61dabe38b86956bf218565cd4e82b918345a1"><code>ccc61da</code></a>
[backport] test and build on armv7l (<a
href="https://redirect.github.com/pyca/cryptography/issues/12420">#12420</a>)
(<a
href="https://redirect.github.com/pyca/cryptography/issues/12431">#12431</a>)</li>
<li><a
href="https://github.com/pyca/cryptography/commit/f299a48153650f2dd87716343f2daa7cd39a1f59"><code>f299a48</code></a>
remove deprecated call (<a
href="https://redirect.github.com/pyca/cryptography/issues/12052">#12052</a>)</li>
<li><a
href="https://github.com/pyca/cryptography/commit/439eb0594a9ffb7c9adedb2490998d83914d141e"><code>439eb05</code></a>
Bump version for 44.0.0 (<a
href="https://redirect.github.com/pyca/cryptography/issues/12051">#12051</a>)</li>
<li><a
href="https://github.com/pyca/cryptography/commit/2c5ad4d8dcec1b8f833198bc2f3b4634c4fd9d78"><code>2c5ad4d</code></a>
chore(deps): bump maturin from 1.7.4 to 1.7.5 in /.github/requirements
(<a
href="https://redirect.github.com/pyca/cryptography/issues/12050">#12050</a>)</li>
<li><a
href="https://github.com/pyca/cryptography/commit/d23968adddd79aa8508d7c1f985da09383b3808f"><code>d23968a</code></a>
chore(deps): bump libc from 0.2.165 to 0.2.166 (<a
href="https://redirect.github.com/pyca/cryptography/issues/12049">#12049</a>)</li>
<li><a
href="https://github.com/pyca/cryptography/commit/133c0e02edf2f172318eb27d8f50525ed64c9ec3"><code>133c0e0</code></a>
Bump x509-limbo and/or wycheproof in CI (<a
href="https://redirect.github.com/pyca/cryptography/issues/12047">#12047</a>)</li>
<li><a
href="https://github.com/pyca/cryptography/commit/f2259d7aa0d134c839ebe298baa8b63de9ead804"><code>f2259d7</code></a>
Bump BoringSSL and/or OpenSSL in CI (<a
href="https://redirect.github.com/pyca/cryptography/issues/12046">#12046</a>)</li>
<li><a
href="https://github.com/pyca/cryptography/commit/e201c870b89fd2606d67230a97e50c3badb07907"><code>e201c87</code></a>
fixed metadata in changelog (<a
href="https://redirect.github.com/pyca/cryptography/issues/12044">#12044</a>)</li>
<li><a
href="https://github.com/pyca/cryptography/commit/c6104cc3669585941dc1d2b9c6507621c53d242f"><code>c6104cc</code></a>
Prohibit Python 3.9.0, 3.9.1 -- they have a bug that causes errors (<a
href="https://redirect.github.com/pyca/cryptography/issues/12045">#12045</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/pyca/cryptography/compare/43.0.1...44.0.1">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=cryptography&package-manager=pip&previous-version=43.0.1&new-version=44.0.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/microsoft/DirectXShaderCompiler/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This enables the generation of native vector DXIL Operations
that are "trivial", meaning they take only a single DXOp Call
instruction to implement as well as a few others that either only took
such a call and some llvm operations or were of particular interest for
other reasons.

This involves allowing the overloads by adding the vector indication in
hctdb, altering the lowering to maintain the vectors instead of
scalarizing them, and a few sundry changes to fix issues along the way.

The "trivial" dxil operations that return a different value from the
overload type had to be moved out of the way and given their own
lowering function so that the main function could generate vectors
conditional on the version and vector type. These will be added in a
later change.

While the long vector supporting intrinsics that weren't given this
treatment will continue to generate scalarized operations, some of them
needed some work as well. The dot product for float vectors longer than
4 had to take the integer fallback path, which required some small
modificaitons and a rename.
Additionally, a heuristic for pow that malfunctioned with too many
elements had to have a limit placed on it.

Since the or()/and()/select() intrinsics translate directly to LLVM ops,
they can have their lowering scalarization removed and what future
scalarization might be needed by the current version can be done by
later passes as with other LLVM operators.

An issue with a special value used to represent unassined dimensions had
to be addressed since new dimensions can exceed that value. It's now
MAX_INT.

Contributes to #7120, but I'd prefer to leave it open until all
intrinsics are covered

Primary work by @pow2clk

Fixes #7297 & #7120

---------

Co-authored-by: Greg Roth <grroth@microsoft.com>
- HLSL REORDER_SCOPE flag (available from SM6.9)
- Make validator accept REORDER_SCOPE from DXIL 1.9
…ests (#7277)

Add the DXIL operations and a passing validation test for:
- HitObject_FromRayQuery
- HitObject_FromRayQueryWithAttrs

DXC SER implementation tracker: #7214
Add the DXIL operations and a passing validation test for:
- HitObject_IsMiss, HitObject_IsHit, HitObject_IsNop
- HitObject_RayFlags, HitObject_RayTMin, HitObject_RayTCurrent
- HitObject_GeometryIndex, HitObject_InstanceIndex, HitObject_InstanceID
- HitObject_PrimitiveIndex, HitObject_HitKind, HitObject_ShaderTableIndex
- HitObject_WorldRayOrigin, HitObject_WorldRayDirection,
- HitObject_ObjectRayOrigin, HitObject_ObjectRayDirection
- HitObject_ObjectToWorld3x4, HitObject_WorldToObject3x4
- HitObject_SetShaderTableIndex, HitObject_LoadLocalRootTableConstant
- HitObject_Attributes

Closes #7310
DXC SER implementation tracker: #7214
Add the DXIL operations and a passing validation test for:
- HitObject_TraceRay
- HitObject_Invoke

DXC SER implementation tracker: #7214
- DXIL opcodes for MaybeReorderThread
- Validator rules
- DXV validation test (passing & expected failures)

Specification:
https://github.com/microsoft/hlsl-specs/blob/main/proposals/0027-shader-execution-reordering.md

DXC SER implementation tracker: #7214
…er::Get(). (#7313)

[spirv] Handles rvalue as implicit object argument of
vk::BufferPointer::Get().
Fixes #7302.
…7331)

Constructors are now properly attached to the template class declaration
instead of a specialization.
Closes #6489 (again).

---------

Co-authored-by: Nathan Gauër <github@keenuts.net>
Simplify merging the SER lowering PRs by declaring all missing SER HLSL
intrinsics up front. This reserves stable HLSL opcodes similar to what
was done for the DXIL opcodes before.

Specification:
https://github.com/microsoft/hlsl-specs/blob/main/proposals/0027-shader-execution-reordering.md
DXC SER implementation tracker: #7214
…#7346)

HctGen of DXIL.rst should happen on every ordinary build, and be updated
with other HctGen modified files.

This isn't about building the doc, it's about updating it when
definitions change in hctdb.py.

We've been missing updates to DXIL.rst for quite a while due to this
issue, introduced
[here](https://github.com/microsoft/DirectXShaderCompiler/pull/6715/files#diff-1e7de1ae2d059d21e1dd75d5812d5a34b0222cef273b7c3a2af62eb747f9d20aR768-R770).

This also brings DXIL.rst up to date.
Fix OOB accesses for debug info vector field names for vectors of length >4.
This PR fixes
#7187.
gl_TessCoord is not a per-patch builtin and therefore the SPIRV should
not be decorated with Patch. This is clear in the GLSL specification,
and a SPIRV spec clarification is online here:
https://gitlab.khronos.org/spirv/SPIR-V/-/issues/819
…ests (#7262)

- HLSL -> DXIL lowering
- ast, hlsl->dxil, dxilgen, and ScalarReplAggregatesHLSL tests

SER implementation tracker (#7214)
Copy link

PR description is empty, please add some valid description

@AnastaZIuk AnastaZIuk merged commit 5ab4d36 into Devsh-Graphics-Programming:devshFixes Apr 18, 2025
4 of 5 checks passed
AnastaZIuk added a commit to Devsh-Graphics-Programming/Nabla that referenced this pull request Apr 18, 2025
AnastaZIuk added a commit to Devsh-Graphics-Programming/Nabla that referenced this pull request Apr 18, 2025
keptsecret pushed a commit to Devsh-Graphics-Programming/Nabla that referenced this pull request May 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.