|
1 | 1 | # Recent Changes |
2 | 2 |
|
| 3 | +### June 8, 2021 |
| 4 | +* Add first ResMLP weights, trained in PyTorch XLA on TPU-VM w/ my XLA branch. 24 block variant, 79.2 top-1. |
| 5 | +* Add ResNet51-Q model w/ pretrained weights at 82.36 top-1. |
| 6 | + * NFNet inspired block layout with quad layer stem and no maxpool |
| 7 | + * Same param count (35.7M) and throughput as ResNetRS-50 but +1.5 top-1 @ 224x224 and +2.5 top-1 at 288x288 |
| 8 | + |
3 | 9 | ### May 25, 2021 |
4 | 10 | * Add LeViT, Visformer, Convit (PR by Aman Arora), Twins (PR by paper authors) transformer models |
5 | 11 | * Cleanup input_size/img_size override handling and testing for all vision transformer models |
|
122 | 128 | * 256x256 val, 0.94 crop top-1 - 83.75 |
123 | 129 | * 320x320 val, 1.0 crop - 84.36 |
124 | 130 | * Update results files |
125 | | - |
126 | | -### Dec 18, 2020 |
127 | | -* Add ResNet-101D, ResNet-152D, and ResNet-200D weights trained @ 256x256 |
128 | | - * 256x256 val, 0.94 crop (top-1) - 101D (82.33), 152D (83.08), 200D (83.25) |
129 | | - * 288x288 val, 1.0 crop - 101D (82.64), 152D (83.48), 200D (83.76) |
130 | | - * 320x320 val, 1.0 crop - 101D (83.00), 152D (83.66), 200D (84.01) |
131 | | - |
132 | | -### Dec 7, 2020 |
133 | | -* Simplify EMA module (ModelEmaV2), compatible with fully torchscripted models |
134 | | -* Misc fixes for SiLU ONNX export, default_cfg missing from Feature extraction models, Linear layer w/ AMP + torchscript |
135 | | -* PyPi release @ 0.3.2 (needed by EfficientDet) |
136 | | - |
137 | | -### Oct 30, 2020 |
138 | | -* Test with PyTorch 1.7 and fix a small top-n metric view vs reshape issue. |
139 | | -* Convert newly added 224x224 Vision Transformer weights from official JAX repo. 81.8 top-1 for B/16, 83.1 L/16. |
140 | | -* Support PyTorch 1.7 optimized, native SiLU (aka Swish) activation. Add mapping to 'silu' name, custom swish will eventually be deprecated. |
141 | | -* Fix regression for loading pretrained classifier via direct model entrypoint functions. Didn't impact create_model() factory usage. |
142 | | -* PyPi release @ 0.3.0 version! |
143 | | - |
144 | | -### Oct 26, 2020 |
145 | | -* Update Vision Transformer models to be compatible with official code release at https://github.com/google-research/vision_transformer |
146 | | -* Add Vision Transformer weights (ImageNet-21k pretrain) for 384x384 base and large models converted from official jax impl |
147 | | - * ViT-B/16 - 84.2 |
148 | | - * ViT-B/32 - 81.7 |
149 | | - * ViT-L/16 - 85.2 |
150 | | - * ViT-L/32 - 81.5 |
151 | | - |
152 | | -### Oct 21, 2020 |
153 | | -* Weights added for Vision Transformer (ViT) models. 77.86 top-1 for 'small' and 79.35 for 'base'. Thanks to [Christof](https://www.kaggle.com/christofhenkel) for training the base model w/ lots of GPUs. |
154 | | - |
155 | | -### Oct 13, 2020 |
156 | | -* Initial impl of Vision Transformer models. Both patch and hybrid (CNN backbone) variants. Currently trying to train... |
157 | | -* Adafactor and AdaHessian (FP32 only, no AMP) optimizers |
158 | | -* EdgeTPU-M (`efficientnet_em`) model trained in PyTorch, 79.3 top-1 |
159 | | -* Pip release, doc updates pending a few more changes... |
160 | | - |
161 | | -### Sept 18, 2020 |
162 | | -* New ResNet 'D' weights. 72.7 (top-1) ResNet-18-D, 77.1 ResNet-34-D, 80.5 ResNet-50-D |
163 | | -* Added a few untrained defs for other ResNet models (66D, 101D, 152D, 200/200D) |
164 | | - |
165 | | -### Sept 3, 2020 |
166 | | -* New weights |
167 | | - * Wide-ResNet50 - 81.5 top-1 (vs 78.5 torchvision) |
168 | | - * SEResNeXt50-32x4d - 81.3 top-1 (vs 79.1 cadene) |
169 | | -* Support for native Torch AMP and channels_last memory format added to train/validate scripts (`--channels-last`, `--native-amp` vs `--apex-amp`) |
170 | | -* Models tested with channels_last on latest NGC 20.08 container. AdaptiveAvgPool in attn layers changed to mean((2,3)) to work around bug with NHWC kernel. |
171 | | - |
172 | | -### Aug 12, 2020 |
173 | | -* New/updated weights from training experiments |
174 | | - * EfficientNet-B3 - 82.1 top-1 (vs 81.6 for official with AA and 81.9 for AdvProp) |
175 | | - * RegNetY-3.2GF - 82.0 top-1 (78.9 from official ver) |
176 | | - * CSPResNet50 - 79.6 top-1 (76.6 from official ver) |
177 | | -* Add CutMix integrated w/ Mixup. See [pull request](https://github.com/rwightman/pytorch-image-models/pull/218) for some usage examples |
178 | | -* Some fixes for using pretrained weights with `in_chans` != 3 on several models. |
179 | | - |
180 | | -### Aug 5, 2020 |
181 | | -Universal feature extraction, new models, new weights, new test sets. |
182 | | - |
183 | | -* All models support the `features_only=True` argument for `create_model` call to return a network that extracts features from the deepest layer at each stride. |
184 | | -* New models |
185 | | - * CSPResNet, CSPResNeXt, CSPDarkNet, DarkNet |
186 | | - * ReXNet |
187 | | - * (Modified Aligned) Xception41/65/71 (a proper port of TF models) |
188 | | -* New trained weights |
189 | | - * SEResNet50 - 80.3 top-1 |
190 | | - * CSPDarkNet53 - 80.1 top-1 |
191 | | - * CSPResNeXt50 - 80.0 top-1 |
192 | | - * DPN68b - 79.2 top-1 |
193 | | - * EfficientNet-Lite0 (non-TF ver) - 75.5 (submitted by [@hal-314](https://github.com/hal-314)) |
194 | | -* Add 'real' labels for ImageNet and ImageNet-Renditions test set, see [`results/README.md`](results/README.md) |
195 | | -* Test set ranking/top-n diff script by [@KushajveerSingh](https://github.com/KushajveerSingh) |
196 | | -* Train script and loader/transform tweaks to punch through more aug arguments |
197 | | -* README and documentation overhaul. See initial (WIP) documentation at https://rwightman.github.io/pytorch-image-models/ |
198 | | -* adamp and sgdp optimizers added by [@hellbell](https://github.com/hellbell) |
199 | | - |
200 | | -### June 11, 2020 |
201 | | -Bunch of changes: |
202 | | - |
203 | | -* DenseNet models updated with memory efficient addition from torchvision (fixed a bug), blur pooling and deep stem additions |
204 | | -* VoVNet V1 and V2 models added, 39 V2 variant (ese_vovnet_39b) trained to 79.3 top-1 |
205 | | -* Activation factory added along with new activations: |
206 | | - * select act at model creation time for more flexibility in using activations compatible with scripting or tracing (ONNX export) |
207 | | - * hard_mish (experimental) added with memory-efficient grad, along with ME hard_swish |
208 | | - * context mgr for setting exportable/scriptable/no_jit states |
209 | | -* Norm + Activation combo layers added with initial trial support in DenseNet and VoVNet along with impl of EvoNorm and InplaceAbn wrapper that fit the interface |
210 | | -* Torchscript works for all but two of the model types as long as using Pytorch 1.5+, tests added for this |
211 | | -* Some import cleanup and classifier reset changes, all models will have classifier reset to nn.Identity on reset_classifer(0) call |
212 | | -* Prep for 0.1.28 pip release |
213 | | - |
214 | | -### May 12, 2020 |
215 | | -* Add ResNeSt models (code adapted from https://github.com/zhanghang1989/ResNeSt, paper https://arxiv.org/abs/2004.08955)) |
216 | | - |
217 | | -### May 3, 2020 |
218 | | -* Pruned EfficientNet B1, B2, and B3 (https://arxiv.org/abs/2002.08258) contributed by [Yonathan Aflalo](https://github.com/yoniaflalo) |
219 | | - |
220 | | -### May 1, 2020 |
221 | | -* Merged a number of execellent contributions in the ResNet model family over the past month |
222 | | - * BlurPool2D and resnetblur models initiated by [Chris Ha](https://github.com/VRandme), I trained resnetblur50 to 79.3. |
223 | | - * TResNet models and SpaceToDepth, AntiAliasDownsampleLayer layers by [mrT23](https://github.com/mrT23) |
224 | | - * ecaresnet (50d, 101d, light) models and two pruned variants using pruning as per (https://arxiv.org/abs/2002.08258) by [Yonathan Aflalo](https://github.com/yoniaflalo) |
225 | | -* 200 pretrained models in total now with updated results csv in results folder |
0 commit comments