Skip to content

Commit 0f18b59

Browse files
author
Github Actions
committed
Arlind Kadra: Update initialization to the normal xavier distribution (#221)
1 parent 2cd9448 commit 0f18b59

19 files changed

+173
-165
lines changed

‎refactor_development_regularization_cocktails/_downloads/2f0f54a9831653ac5c12ee0e06324a62/basics_tabular_jupyter.zip

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/8ef6602cf8ed40221edd89244fb031af/basics_tabular_python.zip

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/b38295cf56f02b1c4547385a8f389f90/advanced_tabular_python.zip

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/df038041811db9bd567de83692a8f994/advanced_tabular_jupyter.zip

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_sources/advanced_tabular/example_custom_configuration_space.rst.txt

Lines changed: 16 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ the search. Currently, there are two changes that can be made to the space:-
4646

4747
.. code-block:: none
4848
49-
<smac.runhistory.runhistory.RunHistory object at 0x7fb41833b2e0> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
49+
<smac.runhistory.runhistory.RunHistory object at 0x7f0415544fa0> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
5050
data_loader:batch_size, Value: 32
5151
encoder:__choice__, Value: 'OneHotEncoder'
5252
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -87,7 +87,7 @@ the search. Currently, there are two changes that can be made to the space:-
8787
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
8888
trainer:StandardTrainer:weighted_loss, Value: True
8989
trainer:__choice__, Value: 'StandardTrainer'
90-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0021953582763671875, budget=0), TrajEntry(train_perf=0.21052631578947367, incumbent_id=1, incumbent=Configuration:
90+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0018546581268310547, budget=0), TrajEntry(train_perf=0.16959064327485385, incumbent_id=1, incumbent=Configuration:
9191
data_loader:batch_size, Value: 32
9292
encoder:__choice__, Value: 'OneHotEncoder'
9393
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -128,14 +128,18 @@ the search. Currently, there are two changes that can be made to the space:-
128128
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
129129
trainer:StandardTrainer:weighted_loss, Value: True
130130
trainer:__choice__, Value: 'StandardTrainer'
131-
, ta_runs=1, ta_time_used=14.904691934585571, wallclock_time=16.4477219581604, budget=5.555555555555555)]
131+
, ta_runs=1, ta_time_used=15.174211978912354, wallclock_time=16.73828387260437, budget=5.555555555555555)]
132132
{'accuracy': 0.8670520231213873}
133-
| | Preprocessing | Estimator | Weight |
134-
|---:|:----------------------------------------------|:-------------------------------------------|---------:|
135-
| 0 | SimpleImputer,OneHotEncoder,NoScaler,Nystroem | embedding,MLPBackbone,NoHead,nn.Sequential | 0.94 |
136-
| 1 | None | ExtraTreesClassifier | 0.04 |
137-
| 2 | None | LGBMClassifier | 0.02 |
138-
<smac.runhistory.runhistory.RunHistory object at 0x7fb418322370> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
133+
| | Preprocessing | Estimator | Weight |
134+
|---:|:------------------------------------------------------------------|:----------------------------------------------------------|---------:|
135+
| 0 | SimpleImputer,OneHotEncoder,NoScaler,Nystroem | embedding,MLPBackbone,NoHead,nn.Sequential | 0.46 |
136+
| 1 | SimpleImputer,OneHotEncoder,NoScaler,Nystroem | embedding,MLPBackbone,NoHead,nn.Sequential | 0.44 |
137+
| 2 | None | LGBMClassifier | 0.02 |
138+
| 3 | None | CatBoostClassifier | 0.02 |
139+
| 4 | None | RFClassifier | 0.02 |
140+
| 5 | None | SVC | 0.02 |
141+
| 6 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
142+
<smac.runhistory.runhistory.RunHistory object at 0x7f0415790d00> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
139143
data_loader:batch_size, Value: 32
140144
encoder:__choice__, Value: 'NoEncoder'
141145
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -174,7 +178,7 @@ the search. Currently, there are two changes that can be made to the space:-
174178
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
175179
trainer:StandardTrainer:weighted_loss, Value: True
176180
trainer:__choice__, Value: 'StandardTrainer'
177-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0016777515411376953, budget=0), TrajEntry(train_perf=0.1871345029239766, incumbent_id=1, incumbent=Configuration:
181+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.001986980438232422, budget=0), TrajEntry(train_perf=0.19298245614035092, incumbent_id=1, incumbent=Configuration:
178182
data_loader:batch_size, Value: 32
179183
encoder:__choice__, Value: 'NoEncoder'
180184
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -213,7 +217,7 @@ the search. Currently, there are two changes that can be made to the space:-
213217
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
214218
trainer:StandardTrainer:weighted_loss, Value: True
215219
trainer:__choice__, Value: 'StandardTrainer'
216-
, ta_runs=1, ta_time_used=9.924669981002808, wallclock_time=11.428627252578735, budget=5.555555555555555)]
220+
, ta_runs=1, ta_time_used=10.018270254135132, wallclock_time=11.532697677612305, budget=5.555555555555555)]
217221
{'accuracy': 0.8728323699421965}
218222
| | Preprocessing | Estimator | Weight |
219223
|---:|:----------------|:---------------------|---------:|
@@ -350,7 +354,7 @@ the search. Currently, there are two changes that can be made to the space:-
350354
351355
.. rst-class:: sphx-glr-timing
352356

353-
**Total running time of the script:** ( 11 minutes 42.049 seconds)
357+
**Total running time of the script:** ( 11 minutes 51.197 seconds)
354358

355359

356360
.. _sphx_glr_download_advanced_tabular_example_custom_configuration_space.py:

‎refactor_development_regularization_cocktails/_sources/advanced_tabular/example_resampling_strategy.rst.txt

Lines changed: 27 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ a 67% train size split.
3838

3939
.. code-block:: none
4040
41-
<smac.runhistory.runhistory.RunHistory object at 0x7fb41aa982e0> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
41+
<smac.runhistory.runhistory.RunHistory object at 0x7f0424f56970> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
4242
data_loader:batch_size, Value: 64
4343
encoder:__choice__, Value: 'OneHotEncoder'
4444
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -77,7 +77,7 @@ a 67% train size split.
7777
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
7878
trainer:StandardTrainer:weighted_loss, Value: True
7979
trainer:__choice__, Value: 'StandardTrainer'
80-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.001760721206665039, budget=0), TrajEntry(train_perf=0.16959064327485385, incumbent_id=1, incumbent=Configuration:
80+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0020804405212402344, budget=0), TrajEntry(train_perf=0.19298245614035092, incumbent_id=1, incumbent=Configuration:
8181
data_loader:batch_size, Value: 64
8282
encoder:__choice__, Value: 'OneHotEncoder'
8383
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -116,19 +116,18 @@ a 67% train size split.
116116
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
117117
trainer:StandardTrainer:weighted_loss, Value: True
118118
trainer:__choice__, Value: 'StandardTrainer'
119-
, ta_runs=1, ta_time_used=14.044004440307617, wallclock_time=15.560070037841797, budget=5.555555555555555)]
120-
{'accuracy': 0.8728323699421965}
119+
, ta_runs=1, ta_time_used=14.983531713485718, wallclock_time=16.507847785949707, budget=5.555555555555555)]
120+
{'accuracy': 0.8670520231213873}
121121
| | Preprocessing | Estimator | Weight |
122122
|---:|:------------------------------------------------------------------|:----------------------------------------------------------------|---------:|
123-
| 0 | None | CatBoostClassifier | 0.2 |
124-
| 1 | None | RFClassifier | 0.18 |
125-
| 2 | None | LGBMClassifier | 0.16 |
126-
| 3 | SimpleImputer,OneHotEncoder,Normalizer,KitchenSink | no embedding,ShapedResNetBackbone,NoHead,nn.Sequential | 0.14 |
127-
| 4 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.12 |
128-
| 5 | None | ExtraTreesClassifier | 0.1 |
129-
| 6 | None | KNNClassifier | 0.08 |
130-
| 7 | None | SVC | 0.02 |
131-
<smac.runhistory.runhistory.RunHistory object at 0x7fb407c61c40> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
123+
| 0 | None | CatBoostClassifier | 0.28 |
124+
| 1 | None | LGBMClassifier | 0.22 |
125+
| 2 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.16 |
126+
| 3 | None | RFClassifier | 0.12 |
127+
| 4 | None | ExtraTreesClassifier | 0.12 |
128+
| 5 | None | SVC | 0.08 |
129+
| 6 | SimpleImputer,OneHotEncoder,Normalizer,KitchenSink | no embedding,ShapedResNetBackbone,NoHead,nn.Sequential | 0.02 |
130+
<smac.runhistory.runhistory.RunHistory object at 0x7f0424cca160> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
132131
data_loader:batch_size, Value: 64
133132
encoder:__choice__, Value: 'OneHotEncoder'
134133
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -167,7 +166,7 @@ a 67% train size split.
167166
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
168167
trainer:StandardTrainer:weighted_loss, Value: True
169168
trainer:__choice__, Value: 'StandardTrainer'
170-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0023446083068847656, budget=0), TrajEntry(train_perf=1.0, incumbent_id=1, incumbent=Configuration:
169+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.002246856689453125, budget=0), TrajEntry(train_perf=1.0, incumbent_id=1, incumbent=Configuration:
171170
data_loader:batch_size, Value: 64
172171
encoder:__choice__, Value: 'OneHotEncoder'
173172
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -206,7 +205,7 @@ a 67% train size split.
206205
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
207206
trainer:StandardTrainer:weighted_loss, Value: True
208207
trainer:__choice__, Value: 'StandardTrainer'
209-
, ta_runs=1, ta_time_used=32.852065563201904, wallclock_time=34.38716793060303, budget=5.555555555555555)]
208+
, ta_runs=1, ta_time_used=32.71918249130249, wallclock_time=34.36233854293823, budget=5.555555555555555)]
210209
{'accuracy': 0.8728323699421965}
211210
| | Preprocessing | Estimator | Weight |
212211
|---:|:----------------|:------------------|---------:|
@@ -216,7 +215,7 @@ a 67% train size split.
216215
| 3 | None | TabularClassifier | 0.16 |
217216
| 4 | None | TabularClassifier | 0.12 |
218217
| 5 | None | TabularClassifier | 0.06 |
219-
<smac.runhistory.runhistory.RunHistory object at 0x7fb41a4039d0> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
218+
<smac.runhistory.runhistory.RunHistory object at 0x7f0414074880> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
220219
data_loader:batch_size, Value: 64
221220
encoder:__choice__, Value: 'OneHotEncoder'
222221
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -255,7 +254,7 @@ a 67% train size split.
255254
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
256255
trainer:StandardTrainer:weighted_loss, Value: True
257256
trainer:__choice__, Value: 'StandardTrainer'
258-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.002129077911376953, budget=0), TrajEntry(train_perf=0.16374269005847952, incumbent_id=1, incumbent=Configuration:
257+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.002149343490600586, budget=0), TrajEntry(train_perf=0.16959064327485385, incumbent_id=1, incumbent=Configuration:
259258
data_loader:batch_size, Value: 64
260259
encoder:__choice__, Value: 'OneHotEncoder'
261260
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -294,18 +293,16 @@ a 67% train size split.
294293
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
295294
trainer:StandardTrainer:weighted_loss, Value: True
296295
trainer:__choice__, Value: 'StandardTrainer'
297-
, ta_runs=1, ta_time_used=15.055562496185303, wallclock_time=16.592270374298096, budget=5.555555555555555)]
298-
{'accuracy': 0.8670520231213873}
299-
| | Preprocessing | Estimator | Weight |
300-
|---:|:------------------------------------------------------------------|:----------------------------------------------------------------|---------:|
301-
| 0 | None | RFClassifier | 0.28 |
302-
| 1 | SimpleImputer,OneHotEncoder,Normalizer,KitchenSink | no embedding,ShapedResNetBackbone,NoHead,nn.Sequential | 0.22 |
303-
| 2 | None | CatBoostClassifier | 0.16 |
304-
| 3 | None | KNNClassifier | 0.14 |
305-
| 4 | None | LGBMClassifier | 0.1 |
306-
| 5 | None | ExtraTreesClassifier | 0.06 |
307-
| 6 | None | SVC | 0.02 |
308-
| 7 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
296+
, ta_runs=1, ta_time_used=15.943702936172485, wallclock_time=17.47943615913391, budget=5.555555555555555)]
297+
{'accuracy': 0.884393063583815}
298+
| | Preprocessing | Estimator | Weight |
299+
|---:|:---------------------------------------------------|:-------------------------------------------------------|---------:|
300+
| 0 | None | CatBoostClassifier | 0.66 |
301+
| 1 | None | ExtraTreesClassifier | 0.14 |
302+
| 2 | None | LGBMClassifier | 0.08 |
303+
| 3 | None | RFClassifier | 0.08 |
304+
| 4 | SimpleImputer,OneHotEncoder,Normalizer,KitchenSink | no embedding,ShapedResNetBackbone,NoHead,nn.Sequential | 0.02 |
305+
| 5 | None | SVC | 0.02 |
309306
310307
311308
@@ -469,7 +466,7 @@ a 67% train size split.
469466
470467
.. rst-class:: sphx-glr-timing
471468

472-
**Total running time of the script:** ( 10 minutes 20.331 seconds)
469+
**Total running time of the script:** ( 10 minutes 27.607 seconds)
473470

474471

475472
.. _sphx_glr_download_advanced_tabular_example_resampling_strategy.py:

0 commit comments

Comments
 (0)