Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Super duper thompson sampler #475

Merged

Conversation

henrymoss
Copy link
Collaborator

@henrymoss henrymoss commented Jan 25, 2022

We can now do continuous Thompson sampling with Trieste!

By continuous, I mean optimizing sample draws (trajectories) using L-BFG-B rather than just choosing the best points over a discrete set (as currently performed by our DiscreteThompsonSamplingRule).

In order to avoid retracing I added some update/resample logic to our RFF trajectory sampler, inspired by our AcquisitionFunctionBuilders. I also changed the expected input shape of trajectories so that they behave more like acquisition functions and so can be optimized using our optimizers.

In this PR, we sequentially maximize trajectory samples using our GreedyAcquisitionFuncton logic.

Future PRs will add (in order)

  1. A Matheron's rule trajectory sampler for our VGP and SVGP models to allow continuous Thompson sampling with these models.
  2. Added support for parallel continuous Thompson sampling, i.e maximising all the trajectories in parallel using our newish parallel acquisition optimiser.

@henrymoss henrymoss requested a review from uri-granta January 26, 2022 09:27
@henrymoss henrymoss requested a review from hstojic January 26, 2022 09:32
Copy link
Collaborator

@uri-granta uri-granta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Just some minor comments.

@henrymoss henrymoss merged commit 04d603c into secondmind-labs:develop Jan 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants