Skip to content

Commit 1c13edf

Browse files
author
Github Actions
committed
Difan Deng: [FIX] Numerical stability scaling for timeseries forecasting tasks (#467)
1 parent a5861b2 commit 1c13edf

35 files changed

+211
-234
lines changed

‎development/_downloads/bc82bea3a5dd7bdba60b65220891d9e5/examples_python.zip

0 Bytes
Binary file not shown.

‎development/_downloads/fb625db3c50d423b1b7881136ffdeec8/examples_jupyter.zip

0 Bytes
Binary file not shown.

‎development/_images/sphx_glr_example_plot_over_time_001.png

-1.25 KB
Loading

‎development/_images/sphx_glr_example_plot_over_time_thumb.png

302 Bytes
Loading

‎development/_images/sphx_glr_example_visualization_001.png

-876 Bytes
Loading

‎development/_images/sphx_glr_example_visualization_thumb.png

-780 Bytes
Loading

‎development/_sources/examples/20_basics/example_image_classification.rst.txt

Lines changed: 16 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -33,22 +33,22 @@ Image Classification
3333
3434
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-images-idx3-ubyte.gz
3535
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-images-idx3-ubyte.gz to ../datasets/FashionMNIST/raw/train-images-idx3-ubyte.gz
36-
0%| | 0/26421880 [00:00<?, ?it/s] 0%| | 32768/26421880 [00:00<01:53, 231928.47it/s] 0%| | 65536/26421880 [00:00<01:54, 230908.71it/s] 0%| | 131072/26421880 [00:00<01:18, 335717.81it/s] 1%| | 229376/26421880 [00:00<00:55, 475759.79it/s] 2%|1 | 425984/26421880 [00:00<00:32, 802339.36it/s] 3%|3 | 884736/26421880 [00:00<00:15, 1625120.17it/s] 7%|6 | 1736704/26421880 [00:00<00:08, 3049193.29it/s] 13%|#3 | 3473408/26421880 [00:01<00:03, 5962888.68it/s] 25%|##4 | 6488064/26421880 [00:01<00:01, 10712916.76it/s] 36%|###5 | 9469952/26421880 [00:01<00:01, 13863205.21it/s] 47%|####6 | 12320768/26421880 [00:01<00:00, 15731289.51it/s] 58%|#####8 | 15368192/26421880 [00:01<00:00, 17458237.61it/s] 69%|######9 | 18317312/26421880 [00:01<00:00, 18442026.10it/s] 80%|######## | 21200896/26421880 [00:01<00:00, 18980240.42it/s] 91%|######### | 24018944/26421880 [00:02<00:00, 19215170.92it/s] 100%|##########| 26421880/26421880 [00:02<00:00, 12287565.88it/s]
36+
0%| | 0/26421880 [00:00<?, ?it/s] 0%| | 32768/26421880 [00:00<01:55, 228713.09it/s] 0%| | 65536/26421880 [00:00<01:56, 227081.28it/s] 0%| | 131072/26421880 [00:00<01:19, 330056.26it/s] 1%| | 196608/26421880 [00:00<01:09, 378551.45it/s] 1%|1 | 393216/26421880 [00:00<00:35, 731618.16it/s] 3%|2 | 786432/26421880 [00:00<00:18, 1407417.98it/s] 6%|6 | 1605632/26421880 [00:01<00:08, 2797236.93it/s] 12%|#2 | 3211264/26421880 [00:01<00:04, 5436945.91it/s] 24%|##3 | 6324224/26421880 [00:01<00:01, 10460303.77it/s] 36%|###5 | 9404416/26421880 [00:01<00:01, 13800083.86it/s] 47%|####6 | 12320768/26421880 [00:01<00:00, 15735845.44it/s] 58%|#####8 | 15433728/26421880 [00:01<00:00, 17486398.69it/s] 70%|######9 | 18481152/26421880 [00:01<00:00, 18557130.94it/s] 82%|########1 | 21594112/26421880 [00:02<00:00, 19438388.55it/s] 94%|#########3| 24707072/26421880 [00:02<00:00, 20016132.86it/s] 100%|##########| 26421880/26421880 [00:02<00:00, 12140078.32it/s]
3737
Extracting ../datasets/FashionMNIST/raw/train-images-idx3-ubyte.gz to ../datasets/FashionMNIST/raw
3838
3939
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-labels-idx1-ubyte.gz
4040
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-labels-idx1-ubyte.gz to ../datasets/FashionMNIST/raw/train-labels-idx1-ubyte.gz
41-
0%| | 0/29515 [00:00<?, ?it/s] 100%|##########| 29515/29515 [00:00<00:00, 211498.05it/s] 100%|##########| 29515/29515 [00:00<00:00, 211025.39it/s]
41+
0%| | 0/29515 [00:00<?, ?it/s] 100%|##########| 29515/29515 [00:00<00:00, 217304.48it/s] 100%|##########| 29515/29515 [00:00<00:00, 216810.10it/s]
4242
Extracting ../datasets/FashionMNIST/raw/train-labels-idx1-ubyte.gz to ../datasets/FashionMNIST/raw
4343

4444
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-images-idx3-ubyte.gz
4545
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-images-idx3-ubyte.gz to ../datasets/FashionMNIST/raw/t10k-images-idx3-ubyte.gz
46-
0%| | 0/4422102 [00:00<?, ?it/s] 1%| | 32768/4422102 [00:00<00:18, 231252.57it/s] 1%|1 | 65536/4422102 [00:00<00:18, 230353.14it/s] 3%|2 | 131072/4422102 [00:00<00:12, 335518.37it/s] 5%|5 | 229376/4422102 [00:00<00:08, 475818.21it/s] 10%|9 | 425984/4422102 [00:00<00:04, 802625.41it/s] 20%|## | 884736/4422102 [00:00<00:02, 1625946.95it/s] 39%|###9 | 1736704/4422102 [00:00<00:00, 3050846.50it/s] 79%|#######8 | 3473408/4422102 [00:01<00:00, 5961671.44it/s] 100%|##########| 4422102/4422102 [00:01<00:00, 3878389.21it/s]
46+
0%| | 0/4422102 [00:00<?, ?it/s] 1%| | 32768/4422102 [00:00<00:18, 242036.89it/s] 1%|1 | 65536/4422102 [00:00<00:18, 241466.08it/s] 3%|2 | 131072/4422102 [00:00<00:12, 351200.05it/s] 5%|5 | 229376/4422102 [00:00<00:08, 497890.59it/s] 11%|#1 | 491520/4422102 [00:00<00:03, 1013070.38it/s] 21%|##1 | 950272/4422102 [00:00<00:01, 1815514.97it/s] 43%|####2 | 1900544/4422102 [00:00<00:00, 3507016.73it/s] 87%|########6 | 3833856/4422102 [00:01<00:00, 6913829.45it/s] 100%|##########| 4422102/4422102 [00:01<00:00, 4060498.86it/s]
4747
Extracting ../datasets/FashionMNIST/raw/t10k-images-idx3-ubyte.gz to ../datasets/FashionMNIST/raw
4848

4949
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-labels-idx1-ubyte.gz
5050
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-labels-idx1-ubyte.gz to ../datasets/FashionMNIST/raw/t10k-labels-idx1-ubyte.gz
51-
0%| | 0/5148 [00:00<?, ?it/s] 100%|##########| 5148/5148 [00:00<00:00, 40359396.25it/s]
51+
0%| | 0/5148 [00:00<?, ?it/s] 100%|##########| 5148/5148 [00:00<00:00, 40586986.83it/s]
5252
Extracting ../datasets/FashionMNIST/raw/t10k-labels-idx1-ubyte.gz to ../datasets/FashionMNIST/raw
5353

5454
Pipeline CS:
@@ -83,25 +83,25 @@ Image Classification
8383
Pipeline Random Config:
8484
________________________________________
8585
Configuration(values={
86-
'image_augmenter:GaussianBlur:use_augmenter': False,
87-
'image_augmenter:GaussianNoise:use_augmenter': False,
88-
'image_augmenter:RandomAffine:rotate': 0,
89-
'image_augmenter:RandomAffine:scale_offset': 0.09817883435719255,
90-
'image_augmenter:RandomAffine:shear': 3,
91-
'image_augmenter:RandomAffine:translate_percent_offset': 0.092968162759446,
92-
'image_augmenter:RandomAffine:use_augmenter': True,
93-
'image_augmenter:RandomCutout:use_augmenter': False,
86+
'image_augmenter:GaussianBlur:sigma_min': 0.40736851695519793,
87+
'image_augmenter:GaussianBlur:sigma_offset': 1.9154521101106374,
88+
'image_augmenter:GaussianBlur:use_augmenter': True,
89+
'image_augmenter:GaussianNoise:sigma_offset': 2.1494393981863014,
90+
'image_augmenter:GaussianNoise:use_augmenter': True,
91+
'image_augmenter:RandomAffine:use_augmenter': False,
92+
'image_augmenter:RandomCutout:p': 0.7558153204326064,
93+
'image_augmenter:RandomCutout:use_augmenter': True,
9494
'image_augmenter:Resize:use_augmenter': True,
95-
'image_augmenter:ZeroPadAndCrop:percent': 0.011876312992094795,
96-
'normalizer:__choice__': 'NoNormalizer',
95+
'image_augmenter:ZeroPadAndCrop:percent': 0.08168973511042621,
96+
'normalizer:__choice__': 'ImageNormalizer',
9797
})
9898

9999
Fitting the pipeline...
100100
________________________________________
101101
ImageClassificationPipeline
102102
________________________________________
103103
0-) normalizer:
104-
NoNormalizer
104+
ImageNormalizer
105105

106106
1-) preprocessing:
107107
EarlyPreprocessing
@@ -173,7 +173,7 @@ Image Classification
173173
174174
.. rst-class:: sphx-glr-timing
175175

176-
**Total running time of the script:** ( 0 minutes 6.988 seconds)
176+
**Total running time of the script:** ( 0 minutes 7.417 seconds)
177177

178178

179179
.. _sphx_glr_download_examples_20_basics_example_image_classification.py:

‎development/_sources/examples/20_basics/example_tabular_classification.rst.txt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -132,7 +132,7 @@ Search for an ensemble of machine learning algorithms
132132
.. code-block:: none
133133
134134
135-
<autoPyTorch.api.tabular_classification.TabularClassificationTask object at 0x7f494b86b6d0>
135+
<autoPyTorch.api.tabular_classification.TabularClassificationTask object at 0x7f7f3976b6a0>
136136
137137
138138
@@ -186,7 +186,7 @@ Print the final ensemble performance
186186
187187
.. rst-class:: sphx-glr-timing
188188

189-
**Total running time of the script:** ( 5 minutes 23.914 seconds)
189+
**Total running time of the script:** ( 5 minutes 20.861 seconds)
190190

191191

192192
.. _sphx_glr_download_examples_20_basics_example_tabular_classification.py:

‎development/_sources/examples/20_basics/example_tabular_regression.rst.txt

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -123,7 +123,7 @@ Search for an ensemble of machine learning algorithms
123123
.. code-block:: none
124124
125125
126-
<autoPyTorch.api.tabular_regression.TabularRegressionTask object at 0x7f48bb5eeca0>
126+
<autoPyTorch.api.tabular_regression.TabularRegressionTask object at 0x7f7ea94eb340>
127127
128128
129129
@@ -163,7 +163,7 @@ Print the final ensemble performance
163163
| 2 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.1 |
164164
| 3 | None | LGBMLearner | 0.04 |
165165
autoPyTorch results:
166-
Dataset name: 67f4ff2e-17dc-11ed-88b6-f144ec8da2bd
166+
Dataset name: a65f386c-17ee-11ed-88a4-a98c1c8ad0eb
167167
Optimisation Metric: r2
168168
Best validation score: 0.8670098636440993
169169
Number of target algorithm runs: 23
@@ -179,7 +179,7 @@ Print the final ensemble performance
179179
180180
.. rst-class:: sphx-glr-timing
181181

182-
**Total running time of the script:** ( 5 minutes 35.994 seconds)
182+
**Total running time of the script:** ( 5 minutes 34.705 seconds)
183183

184184

185185
.. _sphx_glr_download_examples_20_basics_example_tabular_regression.py:

‎development/_sources/examples/20_basics/example_time_series_forecasting.rst.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -150,7 +150,7 @@ Search for an ensemble of machine learning algorithms
150150
151151
.. rst-class:: sphx-glr-timing
152152

153-
**Total running time of the script:** ( 0 minutes 58.007 seconds)
153+
**Total running time of the script:** ( 0 minutes 57.742 seconds)
154154

155155

156156
.. _sphx_glr_download_examples_20_basics_example_time_series_forecasting.py:

‎development/_sources/examples/20_basics/sg_execution_times.rst.txt

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,14 +5,14 @@
55

66
Computation times
77
=================
8-
**12:04.902** total execution time for **examples_20_basics** files:
8+
**12:00.725** total execution time for **examples_20_basics** files:
99

1010
+----------------------------------------------------------------------------------------------------------------+-----------+--------+
11-
| :ref:`sphx_glr_examples_20_basics_example_tabular_regression.py` (``example_tabular_regression.py``) | 05:35.994 | 0.0 MB |
11+
| :ref:`sphx_glr_examples_20_basics_example_tabular_regression.py` (``example_tabular_regression.py``) | 05:34.705 | 0.0 MB |
1212
+----------------------------------------------------------------------------------------------------------------+-----------+--------+
13-
| :ref:`sphx_glr_examples_20_basics_example_tabular_classification.py` (``example_tabular_classification.py``) | 05:23.914 | 0.0 MB |
13+
| :ref:`sphx_glr_examples_20_basics_example_tabular_classification.py` (``example_tabular_classification.py``) | 05:20.861 | 0.0 MB |
1414
+----------------------------------------------------------------------------------------------------------------+-----------+--------+
15-
| :ref:`sphx_glr_examples_20_basics_example_time_series_forecasting.py` (``example_time_series_forecasting.py``) | 00:58.007 | 0.0 MB |
15+
| :ref:`sphx_glr_examples_20_basics_example_time_series_forecasting.py` (``example_time_series_forecasting.py``) | 00:57.742 | 0.0 MB |
1616
+----------------------------------------------------------------------------------------------------------------+-----------+--------+
17-
| :ref:`sphx_glr_examples_20_basics_example_image_classification.py` (``example_image_classification.py``) | 00:06.988 | 0.0 MB |
17+
| :ref:`sphx_glr_examples_20_basics_example_image_classification.py` (``example_image_classification.py``) | 00:07.417 | 0.0 MB |
1818
+----------------------------------------------------------------------------------------------------------------+-----------+--------+

‎development/_sources/examples/40_advanced/example_custom_configuration_space.rst.txt

Lines changed: 17 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -161,7 +161,7 @@ Search for an ensemble of machine learning algorithms
161161
.. code-block:: none
162162
163163
164-
<autoPyTorch.api.tabular_classification.TabularClassificationTask object at 0x7f48b3f26a00>
164+
<autoPyTorch.api.tabular_classification.TabularClassificationTask object at 0x7f7ea180a400>
165165
166166
167167
@@ -205,12 +205,12 @@ Print the final ensemble performance
205205
| 9 | None | SVMLearner | 0.02 |
206206
| 10 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
207207
autoPyTorch results:
208-
Dataset name: 6eb06a8c-17e0-11ed-88b6-f144ec8da2bd
208+
Dataset name: 7b335b0e-17f2-11ed-88a4-a98c1c8ad0eb
209209
Optimisation Metric: accuracy
210210
Best validation score: 0.8596491228070176
211211
Number of target algorithm runs: 18
212-
Number of successful target algorithm runs: 14
213-
Number of crashed target algorithm runs: 3
212+
Number of successful target algorithm runs: 15
213+
Number of crashed target algorithm runs: 2
214214
Number of target algorithms that exceeded the time limit: 1
215215
Number of target algorithms that exceeded the memory limit: 0
216216
@@ -268,7 +268,7 @@ Search for an ensemble of machine learning algorithms
268268
.. code-block:: none
269269
270270
271-
<autoPyTorch.api.tabular_classification.TabularClassificationTask object at 0x7f48b819aa60>
271+
<autoPyTorch.api.tabular_classification.TabularClassificationTask object at 0x7f7ea19b12b0>
272272
273273
274274
@@ -296,24 +296,22 @@ Print the final ensemble performance
296296

297297
.. code-block:: none
298298
299-
{'accuracy': 0.8728323699421965}
299+
{'accuracy': 0.8670520231213873}
300300
| | Preprocessing | Estimator | Weight |
301301
|---:|:---------------------------------------------------------------------------------------------|:----------------------------------------------------------------|---------:|
302-
| 0 | None | LGBMLearner | 0.36 |
303-
| 1 | None | RFLearner | 0.26 |
304-
| 2 | None | ETLearner | 0.14 |
305-
| 3 | SimpleImputer,Variance Threshold,NoCoalescer,NoEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.1 |
306-
| 4 | None | SVMLearner | 0.08 |
307-
| 5 | SimpleImputer,Variance Threshold,NoCoalescer,NoEncoder,Normalizer,KernelPCA | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
308-
| 6 | None | KNNLearner | 0.02 |
309-
| 7 | SimpleImputer,Variance Threshold,NoCoalescer,NoEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
302+
| 0 | None | LGBMLearner | 0.32 |
303+
| 1 | None | SVMLearner | 0.28 |
304+
| 2 | None | RFLearner | 0.26 |
305+
| 3 | None | ETLearner | 0.1 |
306+
| 4 | SimpleImputer,Variance Threshold,NoCoalescer,NoEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
307+
| 5 | SimpleImputer,Variance Threshold,NoCoalescer,NoEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
310308
autoPyTorch results:
311-
Dataset name: d4976415-17e0-11ed-88b6-f144ec8da2bd
309+
Dataset name: e4d3e27f-17f2-11ed-88a4-a98c1c8ad0eb
312310
Optimisation Metric: accuracy
313311
Best validation score: 0.8596491228070176
314-
Number of target algorithm runs: 21
315-
Number of successful target algorithm runs: 15
316-
Number of crashed target algorithm runs: 5
312+
Number of target algorithm runs: 17
313+
Number of successful target algorithm runs: 13
314+
Number of crashed target algorithm runs: 3
317315
Number of target algorithms that exceeded the time limit: 1
318316
Number of target algorithms that exceeded the memory limit: 0
319317
@@ -324,7 +322,7 @@ Print the final ensemble performance
324322
325323
.. rst-class:: sphx-glr-timing
326324

327-
**Total running time of the script:** ( 5 minutes 40.853 seconds)
325+
**Total running time of the script:** ( 6 minutes 1.438 seconds)
328326

329327

330328
.. _sphx_glr_download_examples_40_advanced_example_custom_configuration_space.py:

‎development/_sources/examples/40_advanced/example_parallel_n_jobs.rst.txt

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ with AutoPyTorch
3434

3535
.. code-block:: none
3636
37-
[ERROR] [2022-08-09 12:38:17,702:asyncio.events]
37+
[ERROR] [2022-08-09 14:48:40,164:asyncio.events]
3838
Traceback (most recent call last):
3939
File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/distributed/utils.py", line 799, in wrapper
4040
return await func(*args, **kwargs)
@@ -47,7 +47,7 @@ with AutoPyTorch
4747
File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/asyncio/tasks.py", line 659, in sleep
4848
return await future
4949
asyncio.exceptions.CancelledError
50-
[ERROR] [2022-08-09 12:38:17,706:asyncio.events]
50+
[ERROR] [2022-08-09 14:48:40,166:asyncio.events]
5151
Traceback (most recent call last):
5252
File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/distributed/utils.py", line 799, in wrapper
5353
return await func(*args, **kwargs)
@@ -64,15 +64,15 @@ with AutoPyTorch
6464
File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/asyncio/tasks.py", line 659, in sleep
6565
return await future
6666
asyncio.exceptions.CancelledError
67-
{'accuracy': 0.8554913294797688}
67+
{'accuracy': 0.8497109826589595}
6868
autoPyTorch results:
69-
Dataset name: 642e09dc-17df-11ed-88b6-f144ec8da2bd
69+
Dataset name: 9e3e4823-17f1-11ed-88a4-a98c1c8ad0eb
7070
Optimisation Metric: accuracy
7171
Best validation score: 0.8713450292397661
7272
Number of target algorithm runs: 46
7373
Number of successful target algorithm runs: 37
74-
Number of crashed target algorithm runs: 7
75-
Number of target algorithms that exceeded the time limit: 2
74+
Number of crashed target algorithm runs: 6
75+
Number of target algorithms that exceeded the time limit: 3
7676
Number of target algorithms that exceeded the memory limit: 0
7777
7878
@@ -149,7 +149,7 @@ with AutoPyTorch
149149
150150
.. rst-class:: sphx-glr-timing
151151

152-
**Total running time of the script:** ( 5 minutes 31.380 seconds)
152+
**Total running time of the script:** ( 5 minutes 27.912 seconds)
153153

154154

155155
.. _sphx_glr_download_examples_40_advanced_example_parallel_n_jobs.py:

0 commit comments

Comments
 (0)