Repository navigation
BFGS user should be able to pass parameters to the line search #1043
Description
Activity
Clarification: The purpose is for the user to be able to control the parameters of the actual line search algorithm, of which only
linesearch.hager_zhangis currently implemented.Hello @axch, is this issue fixed? If not, I would like to work on it as my first open source contribution. It would be very helpful if you could guide me through the process.
Thanks for your interest @NeelGhoshal. No, as far as I know the issue is not currently fixed, and we would welcome a contribution. As far as process goes, check out our contributing guidelines. Basically, you grab the development sources, run unit tests, make your change (adding relevant unit tests), make a PR when tests pass, and sign Google's Contributor License Agreement if you haven't already. Then someone on the team reviews your PR (for something like this I would expect it would mostly be for code style), you update the PR in response, and when that's done we merge it.
I checked out the Hager Zhang line search algorithm function, so it currently contains these parameters:
initial_step_size,
value_at_initial_step,value_at_zero,threshold_use_approximate_wolfe_condition=1e-06,shrinkage_param=0.66,expansion_param=5.0,sufficient_decrease_param=0.1,curvature_param=0.9,max_iterations=50
What params would you want added/changed w.r.t. these?Hi @axch, could you please point out what I need to change in the line search code? Which parameters should I focus on adding to the current list of parameters
PR #1070 was a good start (and see the comments thereupon). I don't think it's a good idea to list individual parameters of
hager_zhanganywhere else, because if we implement another line search algorithm at some point, it will presumably have different parameters. I would suggest taking a dictionary argument at the toplevelminimize, passing it through toline_search_step, and then**-expanding it at the call tohager_zhang.Hi! I checked the latest stable versions locally and
tfp.optimizer.bfgs_minimizestill does not have aline_search_kwargsparameter.Environment:
- Python 3.10.0
- TensorFlow 2.20.0
- TensorFlow Probability 0.25.0
- tf_keras 2.20.1
- Windows
The current signature of the function is:
(value_and_gradients_function, initial_position, tolerance=1e-08, x_tolerance=0, f_relative_tolerance=0, initial_inverse_hessian_estimate=None, scale_initial_inverse_hessian=True, max_iterations=50, parallel_iterations=1, stopping_condition=None, validate_args=True, max_line_search_iterations=50, f_absolute_tolerance=0, name=None)Is this feature still desired? Should I revive PR #1070 or open a new PR?
This should just be a matter of accepting a dict named
line_search_kwargsor such and passing it through to the actual line search.This was a feature request that came out of discussing #1034.