Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add module for calibrating impact functions #692

Merged
merged 87 commits into from
Jul 12, 2024
Merged
Changes from 1 commit
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
5d8278e
Initial draft for calibration from scipy.optimize
peanutfun Mar 31, 2023
443545e
Draft for impact function calibration
peanutfun May 4, 2023
819aab5
Add first unit tests of calibration module
peanutfun May 5, 2023
96e3cb3
ci: Add bayesian-optimization during Jenkins build
peanutfun May 5, 2023
123c632
Add __init__.py for util/calibarte/test module
peanutfun May 8, 2023
107a836
Add climada.util.calibrate.test module to test discovery
peanutfun May 8, 2023
2af6f09
Add unit and integration tests, update code base
peanutfun May 8, 2023
0d6e80b
Start documenting new calibrate module
peanutfun May 8, 2023
23cae6c
Actually add the intregration test
peanutfun May 8, 2023
50f3fd9
Add some documentation
peanutfun May 9, 2023
d321832
commit PLEASE CLEAN UP
peanutfun May 17, 2023
24c0fbc
Add more docstrings and simplify imports through __init__
peanutfun May 24, 2023
096a8d4
Add separate Output classes for each optimizer
peanutfun Jun 9, 2023
37c65d9
Merge branch 'develop' into calibrate-impact-functions
peanutfun Jun 9, 2023
e8abb1a
Restructure calibration module
peanutfun Jun 12, 2023
3d94151
Add tutorial on impact function calibration
peanutfun Jun 12, 2023
ea0eb47
Update tutorial
peanutfun Jun 13, 2023
0e5a557
Remove hazard event selection from calibrate.Input
peanutfun Jun 13, 2023
e1fe68a
Update calibration tutorial
peanutfun Jun 13, 2023
68c421b
Merge branch 'develop' into calibrate-impact-functions
emanuel-schmid Jun 26, 2023
df03b0d
Update climada/util/calibrate/bayesian_optimizer.py
peanutfun Jun 28, 2023
5ef4a01
Separate computing cost from transforming impact objects
peanutfun Jun 29, 2023
91cfd83
Merge branch 'calibrate-impact-functions' of https://github.com/CLIMA…
peanutfun Jul 10, 2023
4e1f104
Add evaluator for calibration output
peanutfun Jul 10, 2023
43f40b3
Add TestBayesianOptimizer test to test loader
peanutfun Jul 13, 2023
97d763a
Update code, docs, and tutorial
peanutfun Aug 3, 2023
d43eb8a
Update tutorial
peanutfun Aug 3, 2023
dda079d
Add option to adjust data frame alignment
peanutfun Aug 18, 2023
185866f
Merge branch 'develop' into calibrate-impact-functions
peanutfun Aug 21, 2023
5fdbf4e
Merge branch 'develop' into calibrate-impact-functions
peanutfun Sep 6, 2023
c2ede47
Merge branch 'develop' into calibrate-impact-functions
peanutfun Sep 20, 2023
c40b85a
Merge branch 'develop' into calibrate-impact-functions
peanutfun Oct 2, 2023
645862a
Merge branch 'develop' into calibrate-impact-functions
peanutfun Oct 26, 2023
357541a
Merge branch 'develop' into calibrate-impact-functions
emanuel-schmid Nov 9, 2023
2e536ef
add seaborn
emanuel-schmid Nov 9, 2023
2ace55f
Add function to plot Impf variability of calibration (#791)
timschmi95 Nov 14, 2023
423b5b8
Improve alignment and handling of NaNs
peanutfun Nov 15, 2023
8349016
Add seaborn to dependencies
peanutfun Nov 15, 2023
67ef797
Merge branch 'calibrate-impact-functions' of https://github.com/CLIMA…
peanutfun Nov 20, 2023
24c1fc3
Split tests into multiple files, finish up
peanutfun Nov 23, 2023
1538e78
Move impact transform and align to Input
peanutfun Nov 23, 2023
832da6a
Use MultiIndex in parameter space dataframe
peanutfun Nov 23, 2023
af959a7
Update tutorial
peanutfun Nov 23, 2023
042e6c9
Fix requirements for calibration module
peanutfun Nov 24, 2023
677fba9
Remove plot_impf_set function and improve exception type
peanutfun Nov 24, 2023
3a046c7
Add tests for OutputEvaluator
peanutfun Nov 24, 2023
066afe5
Fix name of bayes_opt package on PyPI
peanutfun Nov 24, 2023
fbc1701
Make sure latest seaborn is installed on Jenkins
peanutfun Nov 24, 2023
2986b51
Remove unused function definition
peanutfun Nov 24, 2023
8ab8600
Fix linter issues and remove unused code
peanutfun Nov 24, 2023
85a5826
Add BayesianOptimizerOutputEvaluator
peanutfun Nov 24, 2023
472d0c5
Fix typo in tutorial
peanutfun Nov 24, 2023
6ef09e5
Add GNU license header to new files
peanutfun Nov 24, 2023
661f991
Update CHANGELOG.md
peanutfun Nov 24, 2023
90f2749
edit authors.md
Nov 27, 2023
53feef6
Fix a bug in parameter space plot
peanutfun Nov 28, 2023
6fa9315
Merge branch 'develop' into calibrate-impact-functions
peanutfun Jan 30, 2024
d4d6777
Add suggestions from code review
peanutfun Jan 30, 2024
a0bada5
Add iteration controller for BayesianOptimizer
peanutfun Feb 20, 2024
cab68c4
Fix calibrate module init and add first controller tests
peanutfun Feb 20, 2024
a9c45d7
Merge branch 'develop' into calibrate-impact-functions
peanutfun Feb 20, 2024
9bf050c
Fix handling of instance maximum for constrained optimization
peanutfun Feb 21, 2024
17046db
Merge branch 'develop' into calibrate-impact-functions
peanutfun Feb 23, 2024
72f6263
Add tests for BayesianOptimizerController and fix verbosity
peanutfun Feb 23, 2024
5a8ac9f
Add test for plotting parameter space
peanutfun Feb 23, 2024
01f0f1c
Update integration tests for BayesianOptimizer
peanutfun Feb 23, 2024
b8f1cac
Add explanation of BayesianOptimizerController to tutorial
peanutfun Feb 23, 2024
10e9679
Merge branch 'develop' into calibrate-impact-functions
emanuel-schmid Mar 14, 2024
2326db2
Add option to store and load calibration results
peanutfun Apr 26, 2024
e270e1a
Merge branch 'develop' into calibrate-impact-functions
peanutfun Apr 29, 2024
5e2dd75
Update plots and tutorial
peanutfun Apr 29, 2024
10e0014
Add JOSS paper and associated GitHub workflow (#876)
peanutfun May 2, 2024
1c412da
JOSS Paper: Fix typo
peanutfun May 2, 2024
2b1ecda
JOSS Paper: Fix DOIs
peanutfun May 3, 2024
e7e441a
JOSS Paper: Do not call it 'natural disaster'.
peanutfun May 3, 2024
60b3a6e
JOSS: Add missing URL for Rougier et al.
peanutfun May 21, 2024
ba9bb7f
JOSS: Remove unused reference
peanutfun May 21, 2024
1d3d014
Add overview section to tutorial and include review suggestions
peanutfun Jul 8, 2024
9949f3e
Add quickstart section to tutorial
peanutfun Jul 8, 2024
a446edd
Shorten quickstart
peanutfun Jul 8, 2024
07adc6f
Guide readers through quickstart section
peanutfun Jul 8, 2024
1f14859
Replace Riedel et al. 2024 preprint with publication
peanutfun Jul 11, 2024
25fb7c3
Merge branch 'develop' into calibrate-impact-functions
peanutfun Jul 11, 2024
0183307
ci: Remove JOSS paper build job
peanutfun Jul 11, 2024
1511cc1
Fix CHANGELOG.md and add entry in Citation guide
peanutfun Jul 12, 2024
102a335
Merge branch 'develop' into calibrate-impact-functions
peanutfun Jul 12, 2024
2232c8c
Revert changes to Jenkinsfile
peanutfun Jul 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix handling of instance maximum for constrained optimization
  • Loading branch information
peanutfun committed Feb 21, 2024
commit 9bf050c776b8e25005be5f06118bdac903550fe4
6 changes: 4 additions & 2 deletions climada/util/calibrate/bayesian_optimizer.py
Original file line number Diff line number Diff line change
Expand Up @@ -190,11 +190,11 @@
class StopEarly(Exception):
"""An exception for stopping an optimization iteration early"""

pass

Check warning on line 193 in climada/util/calibrate/bayesian_optimizer.py

View check run for this annotation

Jenkins - WCR / Pylint

unnecessary-pass

NORMAL: Unnecessary pass statement
Raw output
Used when a "pass" statement that can be avoided is encountered.


@dataclass(eq=False)
class BayesianOptimizerController(object):

Check warning on line 197 in climada/util/calibrate/bayesian_optimizer.py

View check run for this annotation

Jenkins - WCR / Pylint

useless-object-inheritance

LOW: Class 'BayesianOptimizerController' inherits from object, can be safely removed from bases in python3
Raw output
no description found
"""A class for controlling the iterations of a :py:class:`BayesianOptimizer`.

Each iteration in the optimizer consists of a random sampling of the parameter space
Expand Down Expand Up @@ -291,7 +291,7 @@
return -np.inf
return self._improvements[-1].target

def is_converged(self) -> bool:

Check warning on line 294 in climada/util/calibrate/bayesian_optimizer.py

View check run for this annotation

Jenkins - WCR / Pylint

no-self-use

LOW: Method could be a function
Raw output
Used when a method doesn't use its bound instance, and so could be written asa function.
"""Check if convergence criteria are met"""
return True

Expand Down Expand Up @@ -329,14 +329,16 @@

def _is_new_max(self, instance):
"""Determine if a guessed value is the new maximum"""
if instance.max is None:
instance_max = instance.max
if not instance_max or instance_max.get("target") is None:
# During constrained optimization, there might not be a maximum
# value since the optimizer might've not encountered any points
# that fulfill the constraints.
return False

if instance.max["target"] > self._previous_max:
if instance_max["target"] > self._previous_max:
return True

return False

def _maybe_stop_early(self, instance):
Expand Down Expand Up @@ -499,7 +501,7 @@
"""Invert the cost function because BayesianOptimization maximizes the target"""
return -self.input.cost_func(data, predicted)

def run(

Check warning on line 504 in climada/util/calibrate/bayesian_optimizer.py

View check run for this annotation

Jenkins - WCR / Pylint

arguments-differ

NORMAL: Parameters differ from overridden 'run' method
Raw output
Used when a method has a different number of arguments than in the implementedinterface or in an overridden method.
self,
controller: BayesianOptimizerController,
) -> BayesianOptimizerOutput:
Expand Down Expand Up @@ -527,7 +529,7 @@

while controller.iterations < controller.max_iterations:
try:
LOGGER.info(f"Optimization iteration: {controller.iterations}")

Check warning on line 532 in climada/util/calibrate/bayesian_optimizer.py

View check run for this annotation

Jenkins - WCR / Pylint

logging-fstring-interpolation

NORMAL: Use lazy % formatting in logging functions
Raw output
no description found
self.optimizer.maximize(**controller.optimizer_params())
except StopEarly:
continue
Expand Down Expand Up @@ -570,7 +572,7 @@

super().__post_init__()

def plot_impf_variability(

Check warning on line 575 in climada/util/calibrate/bayesian_optimizer.py

View check run for this annotation

Jenkins - WCR / Pylint

too-many-locals

LOW: Too many local variables (29/15)
Raw output
Used when a function or method has too many local variables.
self,
cost_func_diff: float = 0.1,
p_space_df: Optional[pd.DataFrame] = None,
Expand Down Expand Up @@ -621,7 +623,7 @@

# Set plot defaults
color = plot_impf_kws.pop("color", "tab:blue")
lw = plot_impf_kws.pop("lw", 2)

Check warning on line 626 in climada/util/calibrate/bayesian_optimizer.py

View check run for this annotation

Jenkins - WCR / Pylint

invalid-name

LOW: Variable name "lw" doesn't conform to '(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$' pattern
Raw output
Used when the name doesn't match the regular expression associated to its type(constant, variable, class...).
zorder = plot_impf_kws.pop("zorder", 3)
label = plot_impf_kws.pop("label", "best fit")

Expand All @@ -630,7 +632,7 @@
axes = []

for impf_idx in range(n_impf):
_, ax = plt.subplots()

Check warning on line 635 in climada/util/calibrate/bayesian_optimizer.py

View check run for this annotation

Jenkins - WCR / Pylint

invalid-name

LOW: Variable name "ax" doesn't conform to '(([a-z][a-z0-9_]{2,30})|(_[a-z0-9_]*))$' pattern
Raw output
Used when the name doesn't match the regular expression associated to its type(constant, variable, class...).

# Plot best-fit impact function
best_impf = self.impf_set.get_func(haz_type=haz_type)[impf_idx]
Expand Down
Loading