-
Notifications
You must be signed in to change notification settings - Fork 595
Implement global shader registry #2222
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/2222
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New Failure, 1 Unrelated FailureAs of commit eb3a381 with merge base 2a42737 ( NEW FAILURE - The following job has failed:
BROKEN TRUNK - The following job failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This pull request was exported from Phabricator. Differential Revision: D54447700 |
Summary: X-link: pytorch/pytorch#121088 ## Context This changeset updates Vulkan SPIR-V codegen to introduce a global SPIR-V shader registry and register shaders dynamically at static initialization time. This change makes it possible to define and link custom shader libraries to the ATen-Vulkan runtime. Before: * `gen_vulkan_spv.py` generated two files, `spv.h` and `spv.cpp` which would contain the definition and initialization of Vulkan shader registry variables. After: * Introduce the `ShaderRegistry` class in `api/`, which encapsulates functionality of the `ShaderRegistry` class previously defined in the generated `spv.h` file * Introduce a global shader registry (defined as a static variable in the `api::shader_registry() function` * Define a `ShaderRegisterInit` class (taking inspiration from `TorchLibraryInit`) that allows for dynamic shader registration * `gen_vulkan_spv.py` now only generates `spv.cpp`, which defines a static `ShaderRegisterInit` instance that triggers registration of the compiled shaders to the global shader registry. Benefits: * Cleaner code base; we no longer have `ShaderRegistry` defined in a generated file, and don't need a separate implementation file (`impl/Registry.*`) to handle shader lookup. All that logic now lives under `api/ShaderRegistry.*` * Makes it possible to compile and link separate shader libraries, providing similar flexibility as defining and linking custom ATen operators Differential Revision: D54447700
This pull request was exported from Phabricator. Differential Revision: D54447700 |
Summary: X-link: pytorch/pytorch#121088 ## Context This changeset updates Vulkan SPIR-V codegen to introduce a global SPIR-V shader registry and register shaders dynamically at static initialization time. This change makes it possible to define and link custom shader libraries to the ATen-Vulkan runtime. Before: * `gen_vulkan_spv.py` generated two files, `spv.h` and `spv.cpp` which would contain the definition and initialization of Vulkan shader registry variables. After: * Introduce the `ShaderRegistry` class in `api/`, which encapsulates functionality of the `ShaderRegistry` class previously defined in the generated `spv.h` file * Introduce a global shader registry (defined as a static variable in the `api::shader_registry() function` * Define a `ShaderRegisterInit` class (taking inspiration from `TorchLibraryInit`) that allows for dynamic shader registration * `gen_vulkan_spv.py` now only generates `spv.cpp`, which defines a static `ShaderRegisterInit` instance that triggers registration of the compiled shaders to the global shader registry. Benefits: * Cleaner code base; we no longer have `ShaderRegistry` defined in a generated file, and don't need a separate implementation file (`impl/Registry.*`) to handle shader lookup. All that logic now lives under `api/ShaderRegistry.*` * Makes it possible to compile and link separate shader libraries, providing similar flexibility as defining and linking custom ATen operators Differential Revision: D54447700
This pull request was exported from Phabricator. Differential Revision: D54447700 |
Summary: X-link: pytorch/pytorch#121088 ## Context This changeset updates Vulkan SPIR-V codegen to introduce a global SPIR-V shader registry and register shaders dynamically at static initialization time. This change makes it possible to define and link custom shader libraries to the ATen-Vulkan runtime. Before: * `gen_vulkan_spv.py` generated two files, `spv.h` and `spv.cpp` which would contain the definition and initialization of Vulkan shader registry variables. After: * Introduce the `ShaderRegistry` class in `api/`, which encapsulates functionality of the `ShaderRegistry` class previously defined in the generated `spv.h` file * Introduce a global shader registry (defined as a static variable in the `api::shader_registry() function` * Define a `ShaderRegisterInit` class (taking inspiration from `TorchLibraryInit`) that allows for dynamic shader registration * `gen_vulkan_spv.py` now only generates `spv.cpp`, which defines a static `ShaderRegisterInit` instance that triggers registration of the compiled shaders to the global shader registry. Benefits: * Cleaner code base; we no longer have `ShaderRegistry` defined in a generated file, and don't need a separate implementation file (`impl/Registry.*`) to handle shader lookup. All that logic now lives under `api/ShaderRegistry.*` * Makes it possible to compile and link separate shader libraries, providing similar flexibility as defining and linking custom ATen operators Differential Revision: D54447700
This pull request was exported from Phabricator. Differential Revision: D54447700 |
Summary: X-link: pytorch/pytorch#121088 ## Context This changeset updates Vulkan SPIR-V codegen to introduce a global SPIR-V shader registry and register shaders dynamically at static initialization time. This change makes it possible to define and link custom shader libraries to the ATen-Vulkan runtime. Before: * `gen_vulkan_spv.py` generated two files, `spv.h` and `spv.cpp` which would contain the definition and initialization of Vulkan shader registry variables. After: * Introduce the `ShaderRegistry` class in `api/`, which encapsulates functionality of the `ShaderRegistry` class previously defined in the generated `spv.h` file * Introduce a global shader registry (defined as a static variable in the `api::shader_registry() function` * Define a `ShaderRegisterInit` class (taking inspiration from `TorchLibraryInit`) that allows for dynamic shader registration * `gen_vulkan_spv.py` now only generates `spv.cpp`, which defines a static `ShaderRegisterInit` instance that triggers registration of the compiled shaders to the global shader registry. Benefits: * Cleaner code base; we no longer have `ShaderRegistry` defined in a generated file, and don't need a separate implementation file (`impl/Registry.*`) to handle shader lookup. All that logic now lives under `api/ShaderRegistry.*` * Makes it possible to compile and link separate shader libraries, providing similar flexibility as defining and linking custom ATen operators Differential Revision: D54447700
This pull request was exported from Phabricator. Differential Revision: D54447700 |
Summary: X-link: pytorch/pytorch#121088 ## Context This changeset updates Vulkan SPIR-V codegen to introduce a global SPIR-V shader registry and register shaders dynamically at static initialization time. This change makes it possible to define and link custom shader libraries to the ATen-Vulkan runtime. Before: * `gen_vulkan_spv.py` generated two files, `spv.h` and `spv.cpp` which would contain the definition and initialization of Vulkan shader registry variables. After: * Introduce the `ShaderRegistry` class in `api/`, which encapsulates functionality of the `ShaderRegistry` class previously defined in the generated `spv.h` file * Introduce a global shader registry (defined as a static variable in the `api::shader_registry() function` * Define a `ShaderRegisterInit` class (taking inspiration from `TorchLibraryInit`) that allows for dynamic shader registration * `gen_vulkan_spv.py` now only generates `spv.cpp`, which defines a static `ShaderRegisterInit` instance that triggers registration of the compiled shaders to the global shader registry. Benefits: * Cleaner code base; we no longer have `ShaderRegistry` defined in a generated file, and don't need a separate implementation file (`impl/Registry.*`) to handle shader lookup. All that logic now lives under `api/ShaderRegistry.*` * Makes it possible to compile and link separate shader libraries, providing similar flexibility as defining and linking custom ATen operators Differential Revision: D54447700
Summary: X-link: pytorch/executorch#2222 ## Context This changeset updates Vulkan SPIR-V codegen to introduce a global SPIR-V shader registry and register shaders dynamically at static initialization time. This change makes it possible to define and link custom shader libraries to the ATen-Vulkan runtime. Before: * `gen_vulkan_spv.py` generated two files, `spv.h` and `spv.cpp` which would contain the definition and initialization of Vulkan shader registry variables. After: * Introduce the `ShaderRegistry` class in `api/`, which encapsulates functionality of the `ShaderRegistry` class previously defined in the generated `spv.h` file * Introduce a global shader registry (defined as a static variable in the `api::shader_registry() function` * Define a `ShaderRegisterInit` class (taking inspiration from `TorchLibraryInit`) that allows for dynamic shader registration * `gen_vulkan_spv.py` now only generates `spv.cpp`, which defines a static `ShaderRegisterInit` instance that triggers registration of the compiled shaders to the global shader registry. Benefits: * Cleaner code base; we no longer have `ShaderRegistry` defined in a generated file, and don't need a separate implementation file (`impl/Registry.*`) to handle shader lookup. All that logic now lives under `api/ShaderRegistry.*` * Makes it possible to compile and link separate shader libraries, providing similar flexibility as defining and linking custom ATen operators Test Plan: CI Differential Revision: D54447700
This pull request was exported from Phabricator. Differential Revision: D54447700 |
This pull request has been merged in 6c9880c. |
Align `OperatorRegistry` with the style of `ShaderRegistry` in #2222 This means - Improve comments and comment formatting. - Use snake case, even if it deviates from the original registry I was following. Snake case is more consistent with the Vulkan backend code. https://www.internalfb.com/code/fbsource/[a97f9ed1a715231bb61b05942273f1e8f8631503]/fbcode/executorch/runtime/kernel/operator_registry.h?lines=208%2C213 - Move `using` declarations and member variables to top of class definition. - Place static `OperatorRegistry` instance declaration in a global function `operator_registry()` instead of in member function `getInstance()`. - Use macros to wrap `OperatorRegistry` functions instead of global functions. - For simplicity, remove unneeded ctor and assignment operator deletion/hiding. Note users can now create their own non-static `OperatorRegistry` instance and we can consider hiding this again later. Differential Revision: [D54640160](https://our.internmc.facebook.com/intern/diff/D54640160/) [ghstack-poisoned]
This change follows 1. in the footsteps of #2222 for static initialization and 2. the popular `TorchLibraryImpl` for wrapping with macros. https://www.internalfb.com/code/fbsource/[b6860acf0fd7a95224f2ed3f6fe48f699a9a45c0]/fbcode/caffe2/torch/library.h?lines=1004%2C1012-1026 Contributors can now write their operator and register them within the same file using `REGISTER_OPERATORS` + `VK_REGISTER_OP()`, as shown in `Arithmetic.h/cpp`. Typically in Linux/Android C++ environments, the symbols corresponding to `OperatorRegisterInit` static instances are discarded since they aren't used for anything other than static initialization. Hence, we need to `link_whole = True` for the `vulkan_graph_runtime` library. We update our Compute API tests to verify we can go through `OperatorRegistry` with proper static initialization. Differential Revision: [D54641117](https://our.internmc.facebook.com/intern/diff/D54641117/) [ghstack-poisoned]
This change follows 1. in the footsteps of #2222 for static initialization and 2. the popular `TorchLibraryImpl` for wrapping with macros. https://www.internalfb.com/code/fbsource/[b6860acf0fd7a95224f2ed3f6fe48f699a9a45c0]/fbcode/caffe2/torch/library.h?lines=1004%2C1012-1026 Contributors can now write their operator and register them within the same file using `REGISTER_OPERATORS` + `VK_REGISTER_OP()`, as shown in `Arithmetic.h/cpp`. Typically in Linux/Android C++ environments, the symbols corresponding to `OperatorRegisterInit` static instances are discarded since they aren't used for anything other than static initialization. Hence, we need to `link_whole = True` for the `vulkan_graph_runtime` library. We update our Compute API tests to verify we can go through `OperatorRegistry` with proper static initialization. Differential Revision: [D54641117](https://our.internmc.facebook.com/intern/diff/D54641117/) ghstack-source-id: 217857140 Pull Request resolved: #2305
This change follows 1. in the footsteps of #2222 for static initialization and 2. the popular `TorchLibraryImpl` for wrapping with macros. https://www.internalfb.com/code/fbsource/[b6860acf0fd7a95224f2ed3f6fe48f699a9a45c0]/fbcode/caffe2/torch/library.h?lines=1004%2C1012-1026 Contributors can now write their operator and register them within the same file using `REGISTER_OPERATORS` + `VK_REGISTER_OP()`, as shown in `Arithmetic.h/cpp`. Typically in Linux/Android C++ environments, the symbols corresponding to `OperatorRegisterInit` static instances are discarded since they aren't used for anything other than static initialization. Hence, we need to `link_whole = True` for the `vulkan_graph_runtime` library. We update our Compute API tests to verify we can go through `OperatorRegistry` with proper static initialization. Differential Revision: [D54641117](https://our.internmc.facebook.com/intern/diff/D54641117/) [ghstack-poisoned]
Pull Request resolved: #2305 This change follows 1. in the footsteps of #2222 for static initialization and 2. the popular `TorchLibraryImpl` for wrapping with macros. https://www.internalfb.com/code/fbsource/[b6860acf0fd7a95224f2ed3f6fe48f699a9a45c0]/fbcode/caffe2/torch/library.h?lines=1004%2C1012-1026 Contributors can now write their operator and register them within the same file using `REGISTER_OPERATORS` + `VK_REGISTER_OP()`, as shown in `Arithmetic.h/cpp`. Typically in Linux/Android C++ environments, the symbols corresponding to `OperatorRegisterInit` static instances are discarded since they aren't used for anything other than static initialization. Hence, we need to `link_whole = True` for the `vulkan_graph_runtime` library. We update our Compute API tests to verify we can go through `OperatorRegistry` with proper static initialization. ghstack-source-id: 217867140 @exported-using-ghexport Differential Revision: [D54641117](https://our.internmc.facebook.com/intern/diff/D54641117/)
This change follows 1. in the footsteps of #2222 for static initialization and 2. the popular `TorchLibraryImpl` for wrapping with macros. https://www.internalfb.com/code/fbsource/[b6860acf0fd7a95224f2ed3f6fe48f699a9a45c0]/fbcode/caffe2/torch/library.h?lines=1004%2C1012-1026 Contributors can now write their operator and register them within the same file using `REGISTER_OPERATORS` + `VK_REGISTER_OP()`, as shown in `Arithmetic.h/cpp`. Typically in Linux/Android C++ environments, the symbols corresponding to `OperatorRegisterInit` static instances are discarded since they aren't used for anything other than static initialization. Hence, we need to `link_whole = True` for the `vulkan_graph_runtime` library. We update our Compute API tests to verify we can go through `OperatorRegistry` with proper static initialization. Differential Revision: [D54641117](https://our.internmc.facebook.com/intern/diff/D54641117/) [ghstack-poisoned]
Pull Request resolved: #2305 This change follows 1. in the footsteps of #2222 for static initialization and 2. the popular `TorchLibraryImpl` for wrapping with macros. https://www.internalfb.com/code/fbsource/[b6860acf0fd7a95224f2ed3f6fe48f699a9a45c0]/fbcode/caffe2/torch/library.h?lines=1004%2C1012-1026 Contributors can now write their operator and register them within the same file using `REGISTER_OPERATORS` + `VK_REGISTER_OP()`, as shown in `Arithmetic.h/cpp`. Typically in Linux/Android C++ environments, the symbols corresponding to `OperatorRegisterInit` static instances are discarded since they aren't used for anything other than static initialization. Hence, we need to `link_whole = True` for the `vulkan_graph_runtime` library. We update our Compute API tests to verify we can go through `OperatorRegistry` with proper static initialization. ghstack-source-id: 217871298 @exported-using-ghexport Differential Revision: [D54641117](https://our.internmc.facebook.com/intern/diff/D54641117/)
This change follows 1. in the footsteps of #2222 for static initialization and 2. the popular `TorchLibraryImpl` for wrapping with macros. https://www.internalfb.com/code/fbsource/[b6860acf0fd7a95224f2ed3f6fe48f699a9a45c0]/fbcode/caffe2/torch/library.h?lines=1004%2C1012-1026 Contributors can now write their operator and register them within the same file using `REGISTER_OPERATORS` + `VK_REGISTER_OP()`, as shown in `Arithmetic.h/cpp`. Typically in Linux/Android C++ environments, the symbols corresponding to `OperatorRegisterInit` static instances are discarded since they aren't used for anything other than static initialization. Hence, we need to `link_whole = True` for the `vulkan_graph_runtime` library. We update our Compute API tests to verify we can go through `OperatorRegistry` with proper static initialization. Differential Revision: [D54641117](https://our.internmc.facebook.com/intern/diff/D54641117/) [ghstack-poisoned]
Align `OperatorRegistry` with the style of `ShaderRegistry` in #2222 This means - Improve comments and comment formatting. - Use snake case, even if it deviates from the original registry I was following. Snake case is more consistent with the Vulkan backend code. https://www.internalfb.com/code/fbsource/[a97f9ed1a715231bb61b05942273f1e8f8631503]/fbcode/executorch/runtime/kernel/operator_registry.h?lines=208%2C213 - Move `using` declarations and member variables to top of class definition. - Place static `OperatorRegistry` instance declaration in a global function `operator_registry()` instead of in member function `getInstance()`. - Use macros to wrap `OperatorRegistry` functions instead of global functions. - For simplicity, remove unneeded ctor and assignment operator deletion/hiding. Note users can now create their own non-static `OperatorRegistry` instance and we can consider hiding this again later. Differential Revision: [D54640160](https://our.internmc.facebook.com/intern/diff/D54640160/) [ghstack-poisoned]
This change follows 1. in the footsteps of #2222 for static initialization and 2. the popular `TorchLibraryImpl` for wrapping with macros. https://www.internalfb.com/code/fbsource/[b6860acf0fd7a95224f2ed3f6fe48f699a9a45c0]/fbcode/caffe2/torch/library.h?lines=1004%2C1012-1026 Contributors can now write their operator and register them within the same file using `REGISTER_OPERATORS` + `VK_REGISTER_OP()`, as shown in `Arithmetic.h/cpp`. Typically in Linux/Android C++ environments, the symbols corresponding to `OperatorRegisterInit` static instances are discarded since they aren't used for anything other than static initialization. Hence, we need to `link_whole = True` for the `vulkan_graph_runtime` library. We update our Compute API tests to verify we can go through `OperatorRegistry` with proper static initialization. Differential Revision: [D54641117](https://our.internmc.facebook.com/intern/diff/D54641117/) [ghstack-poisoned]
Align `OperatorRegistry` with the style of `ShaderRegistry` in #2222 This means - Improve comments and comment formatting. - Use snake case, even if it deviates from the original registry I was following. Snake case is more consistent with the Vulkan backend code. https://www.internalfb.com/code/fbsource/[a97f9ed1a715231bb61b05942273f1e8f8631503]/fbcode/executorch/runtime/kernel/operator_registry.h?lines=208%2C213 - Move `using` declarations and member variables to top of class definition. - Place static `OperatorRegistry` instance declaration in a global function `operator_registry()` instead of in member function `getInstance()`. - Use macros to wrap `OperatorRegistry` functions instead of global functions. - For simplicity, remove unneeded ctor and assignment operator deletion/hiding. Note users can now create their own non-static `OperatorRegistry` instance and we can consider hiding this again later. Differential Revision: [D54640160](https://our.internmc.facebook.com/intern/diff/D54640160/) [ghstack-poisoned]
Pull Request resolved: #2305 This change follows 1. in the footsteps of #2222 for static initialization and 2. the popular `TorchLibraryImpl` for wrapping with macros. https://www.internalfb.com/code/fbsource/[b6860acf0fd7a95224f2ed3f6fe48f699a9a45c0]/fbcode/caffe2/torch/library.h?lines=1004%2C1012-1026 Contributors can now write their operator and register them within the same file using `REGISTER_OPERATORS` + `VK_REGISTER_OP()`, as shown in `Arithmetic.h/cpp`. Typically in Linux/Android C++ environments, the symbols corresponding to `OperatorRegisterInit` static instances are discarded since they aren't used for anything other than static initialization. Hence, we need to `link_whole = True` for the `vulkan_graph_runtime` library. We update our Compute API tests to verify we can go through `OperatorRegistry` with proper static initialization. ghstack-source-id: 217884083 @exported-using-ghexport Differential Revision: [D54641117](https://our.internmc.facebook.com/intern/diff/D54641117/)
Summary: bypass-github-export-checks Pull Request resolved: #2304 Align `OperatorRegistry` with the style of `ShaderRegistry` in #2222 This means - Improve comments and comment formatting. - Use snake case, even if it deviates from the original registry I was following. Snake case is more consistent with the Vulkan backend code. https://www.internalfb.com/code/fbsource/[a97f9ed1a715231bb61b05942273f1e8f8631503]/fbcode/executorch/runtime/kernel/operator_registry.h?lines=208%2C213 - Move `using` declarations and member variables to top of class definition. - Place static `OperatorRegistry` instance declaration in a global function `operator_registry()` instead of in member function `getInstance()`. - Use macros to wrap `OperatorRegistry` functions instead of global functions. - For simplicity, remove unneeded ctor and assignment operator deletion/hiding. Note users can now create their own non-static `OperatorRegistry` instance and we can consider hiding this again later. ghstack-source-id: 217884084 exported-using-ghexport Reviewed By: SS-JIA Differential Revision: D54640160 fbshipit-source-id: 298e19b07e505f37326f216e6fa4eb8e6513fec6
Summary: bypass-github-export-checks Pull Request resolved: #2305 This change follows 1. in the footsteps of #2222 for static initialization and 2. the popular `TorchLibraryImpl` for wrapping with macros. https://www.internalfb.com/code/fbsource/[b6860acf0fd7a95224f2ed3f6fe48f699a9a45c0]/fbcode/caffe2/torch/library.h?lines=1004%2C1012-1026 Contributors can now write their operator and register them within the same file using `REGISTER_OPERATORS` + `VK_REGISTER_OP()`, as shown in `Arithmetic.h/cpp`. Typically in Linux/Android C++ environments, the symbols corresponding to `OperatorRegisterInit` static instances are discarded since they aren't used for anything other than static initialization. Hence, we need to `link_whole = True` for the `vulkan_graph_runtime` library. We update our Compute API tests to verify we can go through `OperatorRegistry` with proper static initialization. ghstack-source-id: 217884083 exported-using-ghexport Reviewed By: SS-JIA Differential Revision: D54641117 fbshipit-source-id: f7a01b3e028c05d258fdb1390cda31daed7c5f16
Differential Revision: D54447700
Context
This changeset updates Vulkan SPIR-V codegen to introduce a global SPIR-V shader registry and register shaders dynamically at static initialization time. This change makes it possible to define and link custom shader libraries to the ATen-Vulkan runtime.
Before:
gen_vulkan_spv.py
generated two files,spv.h
andspv.cpp
which would contain the definition and initialization of Vulkan shader registry variables.After:
ShaderRegistry
class inapi/
, which encapsulates functionality of theShaderRegistry
class previously defined in the generatedspv.h
fileapi::shader_registry() function
ShaderRegisterInit
class (taking inspiration fromTorchLibraryInit
) that allows for dynamic shader registrationgen_vulkan_spv.py
now only generatesspv.cpp
, which defines a staticShaderRegisterInit
instance that triggers registration of the compiled shaders to the global shader registry.Benefits:
ShaderRegistry
defined in a generated file, and don't need a separate implementation file (impl/Registry.*
) to handle shader lookup. All that logic now lives underapi/ShaderRegistry.*