Skip to content

Commit d1cbdf1

Browse files
author
Github Actions
committed
Ravin Kohli: Change weighted loss to categorical and fix for test adversarial trainer (#214)
1 parent 82ebc3b commit d1cbdf1

19 files changed

+333
-249
lines changed

‎refactor_development_regularization_cocktails/_downloads/2f0f54a9831653ac5c12ee0e06324a62/basics_tabular_jupyter.zip

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/8ef6602cf8ed40221edd89244fb031af/basics_tabular_python.zip

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/b38295cf56f02b1c4547385a8f389f90/advanced_tabular_python.zip

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/df038041811db9bd567de83692a8f994/advanced_tabular_jupyter.zip

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_sources/advanced_tabular/example_custom_configuration_space.rst.txt

Lines changed: 71 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ the search. Currently, there are two changes that can be made to the space:-
4646

4747
.. code-block:: none
4848
49-
<smac.runhistory.runhistory.RunHistory object at 0x7ff942707d30> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
49+
<smac.runhistory.runhistory.RunHistory object at 0x7fab88397f40> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
5050
data_loader:batch_size, Value: 32
5151
encoder:__choice__, Value: 'OneHotEncoder'
5252
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -85,9 +85,9 @@ the search. Currently, there are two changes that can be made to the space:-
8585
trainer:StandardTrainer:use_lookahead_optimizer, Value: True
8686
trainer:StandardTrainer:use_snapshot_ensemble, Value: True
8787
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
88-
trainer:StandardTrainer:weighted_loss, Constant: 1
88+
trainer:StandardTrainer:weighted_loss, Value: True
8989
trainer:__choice__, Value: 'StandardTrainer'
90-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0023491382598876953, budget=0), TrajEntry(train_perf=0.21052631578947367, incumbent_id=1, incumbent=Configuration:
90+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0017666816711425781, budget=0), TrajEntry(train_perf=0.21052631578947367, incumbent_id=1, incumbent=Configuration:
9191
data_loader:batch_size, Value: 32
9292
encoder:__choice__, Value: 'OneHotEncoder'
9393
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -126,22 +126,70 @@ the search. Currently, there are two changes that can be made to the space:-
126126
trainer:StandardTrainer:use_lookahead_optimizer, Value: True
127127
trainer:StandardTrainer:use_snapshot_ensemble, Value: True
128128
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
129-
trainer:StandardTrainer:weighted_loss, Constant: 1
129+
trainer:StandardTrainer:weighted_loss, Value: True
130130
trainer:__choice__, Value: 'StandardTrainer'
131-
, ta_runs=1, ta_time_used=13.149612665176392, wallclock_time=14.657279014587402, budget=5.555555555555555)]
132-
{'accuracy': 0.884393063583815}
133-
| | Preprocessing | Estimator | Weight |
134-
|---:|:------------------------------------------------------------------|:-------------------------------------------------------------|---------:|
135-
| 0 | None | CatBoostClassifier | 0.18 |
136-
| 1 | SimpleImputer,OneHotEncoder,NoScaler,TruncSVD | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.16 |
137-
| 2 | None | LGBMClassifier | 0.14 |
138-
| 3 | None | ExtraTreesClassifier | 0.14 |
139-
| 4 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.14 |
140-
| 5 | SimpleImputer,OneHotEncoder,NoScaler,TruncSVD | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.12 |
141-
| 6 | None | RFClassifier | 0.06 |
142-
| 7 | None | SVC | 0.04 |
143-
| 8 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
144-
<smac.runhistory.runhistory.RunHistory object at 0x7ff93092de50> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
131+
, ta_runs=1, ta_time_used=14.215990781784058, wallclock_time=15.694700717926025, budget=5.555555555555555), TrajEntry(train_perf=0.17543859649122806, incumbent_id=2, incumbent=Configuration:
132+
data_loader:batch_size, Value: 111
133+
encoder:__choice__, Value: 'OneHotEncoder'
134+
feature_preprocessor:TruncatedSVD:target_dim, Value: 5
135+
feature_preprocessor:__choice__, Value: 'TruncatedSVD'
136+
imputer:categorical_strategy, Value: 'constant_!missing!'
137+
imputer:numerical_strategy, Value: 'mean'
138+
lr_scheduler:StepLR:gamma, Value: 0.3916380858102349
139+
lr_scheduler:StepLR:step_size, Value: 5
140+
lr_scheduler:__choice__, Value: 'StepLR'
141+
network_backbone:MLPBackbone:activation, Value: 'relu'
142+
network_backbone:MLPBackbone:dropout_1, Value: 0.1523423145853786
143+
network_backbone:MLPBackbone:num_groups, Value: 1
144+
network_backbone:MLPBackbone:num_units_1, Value: 339
145+
network_backbone:MLPBackbone:use_dropout, Value: True
146+
network_backbone:__choice__, Value: 'MLPBackbone'
147+
network_embedding:LearnedEntityEmbedding:dimension_reduction_0, Value: 0.40351439845395365
148+
network_embedding:LearnedEntityEmbedding:dimension_reduction_1, Value: 0.2782048949477085
149+
network_embedding:LearnedEntityEmbedding:dimension_reduction_2, Value: 0.3389277893759589
150+
network_embedding:LearnedEntityEmbedding:dimension_reduction_3, Value: 0.27434990235055623
151+
network_embedding:LearnedEntityEmbedding:dimension_reduction_4, Value: 0.8301775423300523
152+
network_embedding:LearnedEntityEmbedding:dimension_reduction_5, Value: 0.013609031443506336
153+
network_embedding:LearnedEntityEmbedding:dimension_reduction_6, Value: 0.3503750454780089
154+
network_embedding:LearnedEntityEmbedding:dimension_reduction_7, Value: 0.8336578175236892
155+
network_embedding:LearnedEntityEmbedding:min_unique_values_for_embedding, Value: 6
156+
network_embedding:__choice__, Value: 'LearnedEntityEmbedding'
157+
network_head:__choice__, Value: 'fully_connected'
158+
network_head:fully_connected:activation, Value: 'tanh'
159+
network_head:fully_connected:num_layers, Value: 3
160+
network_head:fully_connected:units_layer_1, Value: 267
161+
network_head:fully_connected:units_layer_2, Value: 100
162+
network_init:XavierInit:bias_strategy, Value: 'Zero'
163+
network_init:__choice__, Value: 'XavierInit'
164+
optimizer:RMSpropOptimizer:alpha, Value: 0.902456140339138
165+
optimizer:RMSpropOptimizer:lr, Value: 0.0010342617079041403
166+
optimizer:RMSpropOptimizer:momentum, Value: 0.3964739559326338
167+
optimizer:RMSpropOptimizer:use_weight_decay, Value: True
168+
optimizer:RMSpropOptimizer:weight_decay, Value: 0.03884467847997589
169+
optimizer:__choice__, Value: 'RMSpropOptimizer'
170+
scaler:__choice__, Value: 'MinMaxScaler'
171+
trainer:RowCutOutTrainer:cutout_prob, Value: 0.2919538652350733
172+
trainer:RowCutOutTrainer:patch_ratio, Value: 0.07508439713858062
173+
trainer:RowCutOutTrainer:use_lookahead_optimizer, Value: False
174+
trainer:RowCutOutTrainer:use_snapshot_ensemble, Value: False
175+
trainer:RowCutOutTrainer:use_stochastic_weight_averaging, Value: True
176+
trainer:RowCutOutTrainer:weighted_loss, Value: False
177+
trainer:__choice__, Value: 'RowCutOutTrainer'
178+
, ta_runs=5, ta_time_used=78.13991332054138, wallclock_time=89.18238615989685, budget=5.555555555555555)]
179+
{'accuracy': 0.8728323699421965}
180+
| | Preprocessing | Estimator | Weight |
181+
|---:|:------------------------------------------------------------------|:----------------------------------------------------------|---------:|
182+
| 0 | SimpleImputer,OneHotEncoder,MinMaxScaler,TruncSVD | embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.22 |
183+
| 1 | None | RFClassifier | 0.14 |
184+
| 2 | None | CatBoostClassifier | 0.12 |
185+
| 3 | None | KNNClassifier | 0.12 |
186+
| 4 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.12 |
187+
| 5 | None | LGBMClassifier | 0.1 |
188+
| 6 | None | ExtraTreesClassifier | 0.08 |
189+
| 7 | SimpleImputer,OneHotEncoder,MinMaxScaler,TruncSVD | embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.04 |
190+
| 8 | None | SVC | 0.04 |
191+
| 9 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
192+
<smac.runhistory.runhistory.RunHistory object at 0x7fab883a9700> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
145193
data_loader:batch_size, Value: 32
146194
encoder:__choice__, Value: 'NoEncoder'
147195
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -178,9 +226,9 @@ the search. Currently, there are two changes that can be made to the space:-
178226
trainer:StandardTrainer:use_lookahead_optimizer, Value: True
179227
trainer:StandardTrainer:use_snapshot_ensemble, Value: True
180228
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
181-
trainer:StandardTrainer:weighted_loss, Constant: 1
229+
trainer:StandardTrainer:weighted_loss, Value: True
182230
trainer:__choice__, Value: 'StandardTrainer'
183-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0016913414001464844, budget=0), TrajEntry(train_perf=0.1871345029239766, incumbent_id=1, incumbent=Configuration:
231+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.001605987548828125, budget=0), TrajEntry(train_perf=0.1871345029239766, incumbent_id=1, incumbent=Configuration:
184232
data_loader:batch_size, Value: 32
185233
encoder:__choice__, Value: 'NoEncoder'
186234
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -217,9 +265,9 @@ the search. Currently, there are two changes that can be made to the space:-
217265
trainer:StandardTrainer:use_lookahead_optimizer, Value: True
218266
trainer:StandardTrainer:use_snapshot_ensemble, Value: True
219267
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
220-
trainer:StandardTrainer:weighted_loss, Constant: 1
268+
trainer:StandardTrainer:weighted_loss, Value: True
221269
trainer:__choice__, Value: 'StandardTrainer'
222-
, ta_runs=1, ta_time_used=8.595646619796753, wallclock_time=10.101649761199951, budget=5.555555555555555)]
270+
, ta_runs=1, ta_time_used=8.764430522918701, wallclock_time=10.22434663772583, budget=5.555555555555555)]
223271
{'accuracy': 0.8728323699421965}
224272
| | Preprocessing | Estimator | Weight |
225273
|---:|:----------------|:---------------------|---------:|
@@ -356,7 +404,7 @@ the search. Currently, there are two changes that can be made to the space:-
356404
357405
.. rst-class:: sphx-glr-timing
358406

359-
**Total running time of the script:** ( 11 minutes 46.494 seconds)
407+
**Total running time of the script:** ( 11 minutes 43.956 seconds)
360408

361409

362410
.. _sphx_glr_download_advanced_tabular_example_custom_configuration_space.py:

0 commit comments

Comments
 (0)