Skip to content

fix: Add support for truncate_long_and_double in Dynamo #1969

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from

Conversation

gs-olive
Copy link
Collaborator

@gs-olive gs-olive commented Jun 1, 2023

Description

  • Add default, setting, and function arguments for truncate_long_and_double in Dynamo
  • Add utilities for repairing long/double inputs to TRT engines, including support for autocasting back to long/double after the computation completes
  • Add multiple helper functions to enable easy testing and diagnosis of long/double IO to TRT engines
  • Add necessary compiler code to enable usage of the truncate_long_and_double argument as a switch for the feature
  • Add Dynamo compile support for truncate_long_and_double compilation argument by intercepting long/double type inputs and casting them to their 32-bit counterparts prior to usage in TRT-accelerated subgraphs, then casting back if necessary
  • Add robust logic to handle 64-bit inputs and outputs
  • Add test cases for long and double scenarios

Fixes #1964
Addresses #1740

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Checklist:

  • [ x ] My code follows the style guidelines of this project (You can use the linters)
  • [ x ] I have performed a self-review of my own code
  • [ x ] I have commented my code, particularly in hard-to-understand areas and hacks
  • [ x ] I have made corresponding changes to the documentation
  • [ x ] I have added tests to verify my fix or my feature
  • [ x ] New and existing unit tests pass locally with my changes
  • [ x ] I have added the relevant labels to my PR in so that relevant reviewers are notified

gs-olive added 3 commits May 31, 2023 14:47
- Add default, setting, and function arguments for
`truncate_long_and_double` in Dynamo
- Add utilities for repairing long/double inputs to TRT engines,
including support for autocasting back to long/double after the
computation completes
- Add multiple helper functions to enable easy testing and diagnosis of
long/double IO to TRT engines
- Add necessary compiler code to enable usage of the `truncate_long_and_double`
argument as a switch for the feature
- Add Dynamo compile support for `truncate_long_and_double` compilation
argument by intercepting long/double type inputs and casting them to
their 32-bit counterparts prior to usage in TRT-accelerated subgraphs,
then casting back if necessary
- Add robust logic to handle 64-bit inputs and outputs
- Add test cases for long and double scenarios
@gs-olive gs-olive added component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths Story: Dynamo Compile Improvements Issues relating to improvement of the Dynamo compile path labels Jun 1, 2023
@gs-olive gs-olive self-assigned this Jun 1, 2023
@github-actions github-actions bot added the component: api [Python] Issues re: Python API label Jun 1, 2023
@github-actions github-actions bot requested a review from narendasan June 1, 2023 16:32
@gs-olive
Copy link
Collaborator Author

gs-olive commented Jun 5, 2023

Closed in place of #1983

@gs-olive gs-olive closed this Jun 5, 2023
@gs-olive gs-olive deleted the dynamo_compile_int64_fix branch June 5, 2023 17:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla signed component: api [Python] Issues re: Python API component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths Story: Dynamo Compile Improvements Issues relating to improvement of the Dynamo compile path
Projects
None yet
Development

Successfully merging this pull request may close these issues.

✨[Feature] Add support for truncate_long_and_double in Dynamo compile
3 participants