hwdec_cuda/vulkan: fix partial failure edge cases in error paths#18017
Open
philipl wants to merge 2 commits into
Open
hwdec_cuda/vulkan: fix partial failure edge cases in error paths#18017philipl wants to merge 2 commits into
philipl wants to merge 2 commits into
Conversation
The slopbot spam did find valid issues in the error paths in this code, and as part of verifying them, I have additional fixes. * ensure that the return value -1 from `cuda_init()` if `ext_init()` fails * zero alloc the `ext_vk` and `ext_gl` structs so that we have a known initial state * explicitly initialise the `sem_handle.fd` to -1 to ensure we don't accidentally close stdin in the error path. But also don't do this on windows as an invalid Handle is 0. * Don't do a double `cuCtxPopCurrent()` in the `ext_gl` init failure path Fixes mpv-player#17976, mpv-player#17988, mpv-player#17989
This is another file where the slopbot identified a problem, but I was surprised to see it fix it in a poor way and also miss an adjacement problem. `mapper_map` calls `mapper_unmap` with the intention of using it to clean up partial failure after some planes are wrapped and released while others are not. But the current code would actually segfault in that situation. There are two fixes required: * Ensure `p->vkf` is set by `mapper_map` early so that its available to `mapper_unmap` in the error path * Don't call `vkfc->unlock_frame` in the `mapper_map` error path, as this will be called by `mapper_unmap` Fixes mpv-player#17977
826d640 to
7858bf8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
These changes were inspired by the slopbot, but I went through and manually verified them, identifying additional problems, and better solutions along the way.
hwdec_cuda: fix partial failure edge cases in error paths
The slopbot spam did find valid issues in the error paths in this code, and as
part of verifying them, I have additional fixes.
cuda_init()ifext_init()failsext_vkandext_glstructs so that we have a known initialstate
sem_handle.fdto -1 to ensure we don'taccidentally close stdin in the error path. But also don't do this on
windows as an invalid Handle is 0.
cuCtxPopCurrent()in theext_glinit failure pathFixes #17976, #17988, #17989
hwdec_vulkan: fix partial failure edge cases in error paths
This is another file where the slopbot identified a problem, but I was
surprised to see it fix it in a poor way and also miss an adjacement
problem.
mapper_mapcallsmapper_unmapwith the intention of using it to cleanup partial failure after some planes are wrapped and released while others
are not. But the current code would actually segfault in that situation.
There are two fixes required:
p->vkfis set bymapper_mapearly so that its available tomapper_unmapin the error pathvkfc->unlock_framein themapper_maperror path, asthis will be called by
mapper_unmapFixes #17977