Skip to content

[ET-VK][7/n] Slice, with lots of codegen improvements #3171

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from

Conversation

yipjustin
Copy link
Contributor

@yipjustin yipjustin commented Apr 19, 2024

Stack from ghstack (oldest at bottom):

  1. Add slice operation. Instead of using copy in LI, we implement a simple shader with offsets.

  2. Improvement in codegen.

  • add support of optional variables
  • improve indent of the code, for better readability
  • allow user to specify tensor value generation, possible to generate sequential values for easier debugging for index operations
  • sample code improve test-case specification, particularly with long and optional values.

Differential Revision: D56295985

1. Add slice operation. Instead of using copy in LI, we implement a simple shader with offsets.

2. Improvement in codegen.
- add support of optional variables
- improve indent of the code, for better readability
- allow user to specify tensor value generation, possible to generate sequential values for easier debugging for index operations
- sample code improve test-case specification, particularly with long and optional values.

Differential Revision: [D56295985](https://our.internmc.facebook.com/intern/diff/D56295985/)

[ghstack-poisoned]
Copy link

pytorch-bot bot commented Apr 19, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/3171

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 73379b8 with merge base fa433cb (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 19, 2024
yipjustin added a commit that referenced this pull request Apr 19, 2024
1. Add slice operation. Instead of using copy in LI, we implement a simple shader with offsets.

2. Improvement in codegen.
- add support of optional variables
- improve indent of the code, for better readability
- allow user to specify tensor value generation, possible to generate sequential values for easier debugging for index operations
- sample code improve test-case specification, particularly with long and optional values.

Differential Revision: [D56295985](https://our.internmc.facebook.com/intern/diff/D56295985/)

ghstack-source-id: 223242316
Pull Request resolved: #3171
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D56295985

@yipjustin yipjustin requested review from SS-JIA and jorgep31415 and removed request for SS-JIA April 19, 2024 19:56
1. Add slice operation. Instead of using copy in LI, we implement a simple shader with offsets.

2. Improvement in codegen.
- add support of optional variables
- improve indent of the code, for better readability
- allow user to specify tensor value generation, possible to generate sequential values for easier debugging for index operations
- sample code improve test-case specification, particularly with long and optional values.

Differential Revision: [D56295985](https://our.internmc.facebook.com/intern/diff/D56295985/)

[ghstack-poisoned]
yipjustin added a commit that referenced this pull request Apr 19, 2024
Pull Request resolved: #3171

1. Add slice operation. Instead of using copy in LI, we implement a simple shader with offsets.

2. Improvement in codegen.
- add support of optional variables
- improve indent of the code, for better readability
- allow user to specify tensor value generation, possible to generate sequential values for easier debugging for index operations
- sample code improve test-case specification, particularly with long and optional values.
ghstack-source-id: 223247365

Differential Revision: [D56295985](https://our.internmc.facebook.com/intern/diff/D56295985/)
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D56295985

1. Add slice operation. Instead of using copy in LI, we implement a simple shader with offsets.

2. Improvement in codegen.
- add support of optional variables
- improve indent of the code, for better readability
- allow user to specify tensor value generation, possible to generate sequential values for easier debugging for index operations
- sample code improve test-case specification, particularly with long and optional values.

Differential Revision: [D56295985](https://our.internmc.facebook.com/intern/diff/D56295985/)

[ghstack-poisoned]
yipjustin added a commit that referenced this pull request Apr 19, 2024
Pull Request resolved: #3171

1. Add slice operation. Instead of using copy in LI, we implement a simple shader with offsets.

2. Improvement in codegen.
- add support of optional variables
- improve indent of the code, for better readability
- allow user to specify tensor value generation, possible to generate sequential values for easier debugging for index operations
- sample code improve test-case specification, particularly with long and optional values.
ghstack-source-id: 223254861

Differential Revision: [D56295985](https://our.internmc.facebook.com/intern/diff/D56295985/)
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D56295985

@facebook-github-bot
Copy link
Contributor

This pull request has been merged in 7469a28.

@mergennachin mergennachin mentioned this pull request Apr 26, 2024
jorgep31415 added a commit that referenced this pull request May 24, 2024
## The Operator
`nn.Module` invocations of [`torch.index_select`](https://pytorch.org/docs/stable/generated/torch.index_select.html) get compiled to `aten.index_select.default` in the Edge Dialect, which carries the following signature.
```
- func: index_select(Tensor self, int dim, Tensor index) -> Tensor
```

## Implementation
This is a C-packing-only implementation. It is very similar to `aten.slice`: #3171
```
- func: slice.Tensor(Tensor(a) self, int dim=0, SymInt? start=None, SymInt? end=None, SymInt step=1) -> Tensor(a)
```

It features a similar split between a shader for N,H,W and a shader for C, because copying from the C-dimension is more difficult due to C-packing.

Both `index_select` and `slice` copy specific indices across 1 dimension. The difference is in the way these indices are specified.
- `slice` uses `start=1`/`end=5`/`step=2` as three scalars for indices `1,3`.
- `index_select` lists the exact indices inside a tensor e.g. `index=torch.tensor([1,3])`.

Hence, `slice` uses a `offset=1` and `step=2` to compute input position. In `index_select`, we read the index tensor to compute input position.

Differential Revision: [D57745489](https://our.internmc.facebook.com/intern/diff/D57745489/)

[ghstack-poisoned]
jorgep31415 added a commit that referenced this pull request May 24, 2024
## The Operator
`nn.Module` invocations of [`torch.index_select`](https://pytorch.org/docs/stable/generated/torch.index_select.html) get compiled to `aten.index_select.default` in the Edge Dialect, which carries the following signature.
```
- func: index_select(Tensor self, int dim, Tensor index) -> Tensor
```

## Implementation
This is a C-packing-only implementation. It is very similar to `aten.slice`: #3171
```
- func: slice.Tensor(Tensor(a) self, int dim=0, SymInt? start=None, SymInt? end=None, SymInt step=1) -> Tensor(a)
```

It features a similar split between a shader for N,H,W and a shader for C, because copying from the C-dimension is more difficult due to C-packing.

Both `index_select` and `slice` copy specific indices across 1 dimension. The difference is in the way these indices are specified.
- `slice` uses `start=1`/`end=5`/`step=2` as three scalars for indices `1,3`.
- `index_select` lists the exact indices inside a tensor e.g. `index=torch.tensor([1,3])`.

Hence, `slice` uses a `offset=1` and `step=2` to compute input position. In `index_select`, we read the index tensor to compute input position.

Differential Revision: [D57745489](https://our.internmc.facebook.com/intern/diff/D57745489/)

ghstack-source-id: 227736336
Pull Request resolved: #3744
facebook-github-bot pushed a commit that referenced this pull request May 28, 2024
Summary:
Pull Request resolved: #3744

## The Operator
`nn.Module` invocations of [`torch.index_select`](https://pytorch.org/docs/stable/generated/torch.index_select.html) get compiled to `aten.index_select.default` in the Edge Dialect, which carries the following signature.
```
- func: index_select(Tensor self, int dim, Tensor index) -> Tensor
```

## Implementation
This is a C-packing-only implementation. It is very similar to `aten.slice`: #3171
```
- func: slice.Tensor(Tensor(a) self, int dim=0, SymInt? start=None, SymInt? end=None, SymInt step=1) -> Tensor(a)
```

It features a similar split between a shader for N,H,W and a shader for C, because copying from the C-dimension is more difficult due to C-packing.

Both `index_select` and `slice` copy specific indices across 1 dimension. The difference is in the way these indices are specified.
- `slice` uses `start=1`/`end=5`/`step=2` as three scalars for indices `1,3`.
- `index_select` lists the exact indices inside a tensor e.g. `index=torch.tensor([1,3])`.

Hence, `slice` uses a `offset=1` and `step=2` to compute input position. In `index_select`, we read the index tensor to compute input position.

Reviewed By: copyrightly

Differential Revision: D57745489

fbshipit-source-id: 4ef7f1a13d4bf74af20fe61149dbd5d461aaab0c
jorgep31415 added a commit that referenced this pull request May 29, 2024
## The Operator
`nn.Module` invocations of [`torch.index_select`](https://pytorch.org/docs/stable/generated/torch.index_select.html) get compiled to `aten.index_select.default` in the Edge Dialect, which carries the following signature.
```
- func: index_select(Tensor self, int dim, Tensor index) -> Tensor
```

## Implementation
This is a C-packing-only implementation. It is very similar to `aten.slice`: #3171
```
- func: slice.Tensor(Tensor(a) self, int dim=0, SymInt? start=None, SymInt? end=None, SymInt step=1) -> Tensor(a)
```

It features a similar split between a shader for N,H,W and a shader for C, because copying from the C-dimension is more difficult due to C-packing.

Both `index_select` and `slice` copy specific indices across 1 dimension. The difference is in the way these indices are specified.
- `slice` uses `start=1`/`end=5`/`step=2` as three scalars for indices `1,3`.
- `index_select` lists the exact indices inside a tensor e.g. `index=torch.tensor([1,3])`.

Hence, `slice` uses a `offset=1` and `step=2` to compute input position. In `index_select`, we read the index tensor to compute input position.

Differential Revision: [D57745489](https://our.internmc.facebook.com/intern/diff/D57745489/)

[ghstack-poisoned]
jorgep31415 added a commit that referenced this pull request May 29, 2024
## The Operator
`nn.Module` invocations of [`torch.index_select`](https://pytorch.org/docs/stable/generated/torch.index_select.html) get compiled to `aten.index_select.default` in the Edge Dialect, which carries the following signature.
```
- func: index_select(Tensor self, int dim, Tensor index) -> Tensor
```

## Implementation
This is a C-packing-only implementation. It is very similar to `aten.slice`: #3171
```
- func: slice.Tensor(Tensor(a) self, int dim=0, SymInt? start=None, SymInt? end=None, SymInt step=1) -> Tensor(a)
```

It features a similar split between a shader for N,H,W and a shader for C, because copying from the C-dimension is more difficult due to C-packing.

Both `index_select` and `slice` copy specific indices across 1 dimension. The difference is in the way these indices are specified.
- `slice` uses `start=1`/`end=5`/`step=2` as three scalars for indices `1,3`.
- `index_select` lists the exact indices inside a tensor e.g. `index=torch.tensor([1,3])`.

Hence, `slice` uses a `offset=1` and `step=2` to compute input position. In `index_select`, we read the index tensor to compute input position.

Differential Revision: [D57745489](https://our.internmc.facebook.com/intern/diff/D57745489/)

[ghstack-poisoned]
kedarnath03 pushed a commit to kedarnath03/executorch that referenced this pull request Jun 25, 2025
Pull Request resolved: pytorch/executorch#3744

## The Operator
`nn.Module` invocations of [`torch.index_select`](https://pytorch.org/docs/stable/generated/torch.index_select.html) get compiled to `aten.index_select.default` in the Edge Dialect, which carries the following signature.
```
- func: index_select(Tensor self, int dim, Tensor index) -> Tensor
```

## Implementation
This is a C-packing-only implementation. It is very similar to `aten.slice`: pytorch/executorch#3171
```
- func: slice.Tensor(Tensor(a) self, int dim=0, SymInt? start=None, SymInt? end=None, SymInt step=1) -> Tensor(a)
```

It features a similar split between a shader for N,H,W and a shader for C, because copying from the C-dimension is more difficult due to C-packing.

Both `index_select` and `slice` copy specific indices across 1 dimension. The difference is in the way these indices are specified.
- `slice` uses `start=1`/`end=5`/`step=2` as three scalars for indices `1,3`.
- `index_select` lists the exact indices inside a tensor e.g. `index=torch.tensor([1,3])`.

Hence, `slice` uses a `offset=1` and `step=2` to compute input position. In `index_select`, we read the index tensor to compute input position.

Differential Revision: [D57745489](https://our.internmc.facebook.com/intern/diff/D57745489/)
ghstack-source-id: 227954599
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. fb-exported Merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants