Optimisers progress

[amiszuda]

Is there a way to track the progress of the optimisers in the context of the proposed values of the fitted parameters to be optimised along with the goodness of the fit? In the *_sol.progress.ps file, I see only the number of completed iterations, the initial values of the parameters and their returned values, which at this point are equal to the initial ones, since the optimisation has not yet been completed.

[kecnry]

If progress_every_niters is set (to be non-zero), then the entire "solution" should be exported to the progress file, including the current positions of each of the parameters. You could set progress_every_niters=1, but note that there is some overhead with writing to this file which will slow down the run. If you either run the job in an external thread or on an external server, you should be able to access those intermediate results at any time and terminate the optimizer before it reaches its maximum iterations or target tolerances.

[kecnry]

ok, so -infinity even at the starting position. Is the forward model at the same parameter successful? And if so, check whether the b.calculate_lnlikelihood(model) on the resulting model and/or b.calculate_lnp('rv_priors') is the cause.

[amiszuda]

Yes, the forward model is successful. The other commands give

b.calculate_lnlikelihood() = -93.75078705526397 
b.calculate_lnp('rv_priors') = -10.596634733096074

[kecnry]

I am getting an atmospheres out-of-bounds error when trying to reproduce, which would result in a negative infinity at the starting position:

import phoebe
import numpy as np

b = phoebe.default_binary(starA='secondary', starB='primary')

data_rv = np.loadtxt('V1216Sco_RV.txt',unpack=True)

b.add_dataset('rv',times=data_rv[0,:],rvs={'primary': data_rv[2,:], 'secondary': data_rv[4,:]},
              sigmas={'primary': data_rv[3,:], 'secondary': data_rv[5,:]},dataset='rv01',
              intens_weighting='photon', ld_mode='manual', ld_func='logarithmic', overwrite=True)

b.add_compute('phoebe',compute='nm_solver')

fit_parameters = ['q', 'sma@binary', 'ecc', 'per0', 'vgamma', 't0_supconj']

b.add_distribution({'q':phoebe.uniform(2.0, 3.0),
                    'sma@binary':phoebe.uniform(20, 40),
                    'ecc':phoebe.uniform(0.0, 0.1),
                    'per0':phoebe.uniform(100, 200),
                    'vgamma':phoebe.uniform(-50, -30),
                    't0_supconj':phoebe.uniform(2360, 2370)},
                    distribution='rv_priors')

initial_values = {
    'q': 2.7,
    'sma@binary': 25.8,
    'ecc': 0.,
    'per0': 150,
    'vgamma': -40,
    't0_supconj': 2366.
}

for k,v in initial_values.items():
    b.set_value(k, context=['component', 'dataset', 'system'], value=v)

b.run_compute('nm_solver')

print(b.calculate_lnlikelihood())
print(b.calculate_lnp('rv_priors'))

I'm guessing you were comparing against the current face-values in the bundle, not the initial_values where you were telling the optimizer to start instead.

[amiszuda]

Sorry, I'm not sure I follow what you mean by "comparing against the current face-values in the bundle, not the initial_values ". Nevertheless, setting up
b['atm@primary@nm_solver'] = 'blackbody'
b['atm@secondary@nm_solver'] = 'blackbody'
did the trick.

[kecnry]

When you call run_compute that uses the current values of all the parameters. But by passing initial_values to the optimizer, you are overriding those current values with a different set of values to start the optimization. At these values, the forward model was failing (as can be seen by looping over the dictionary, setting those values, then calling run_compute()). Just be aware that blackbody atmospheres can easily go into unphysical territory!