[WIP] Added support for converting LayoutLMv3 to CoreML #78
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
salmanmaq
force-pushed
the
layoutlmv3_coreml
branch
from
cba732f to
f1135dc
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a WIP implementation of converting a multimodal model (LayoutLMv3) to CoreML. While I was able to adapt the library to convert the model, the resulting model doesn't predict correctly (testing in both Swift on mobile as well as with
coremltoolsin Python). I would appreciate any feedback to make it work.I had to patch two
pytorchops:clip: As it's an alias ofclamp, I reused the same code fromcoremltoolshere.one_hot: I used some help from Copilot and LLMs to write this and make it work. However, I am not sure if it's the correct implementation. I suspect that the error might lie in this one.