How to choose the final policy in the search phase?

[ee-hash]

Hi!
I have set learning rate=0.001. But when searching, the valid metric curve is not unstable.
Now I choose the top-25 sub_policy(epoch=36, 1 epoch=500 iter)，because the valid metric curve looks good. Then the second train stage is converged. I don't know why?
Additionally, I choose the sub_policy in the case of curve oscillation, the second train stage is not stable and model is not converged.
During the entire search phase, the valid metric are oscillating. Is this normal?

I don't know how to choose the final policy, looks like I select sub_policy randomly. Thanks!

The search stage figure:
X axis-epoch, Y axis-valid metric.

[CharalambosIoannou]

Hello, could you share how you adapted this code for CT images. I am trying to do something similar but i am having some troubles.
Maybe more specifically how did you add your own dataset to the train_paper.sh file?
Thanks

[latstars]

It is very hard to determine which DA policy is better.
It is better to evaluate the DA policy for training from scratch with a small-model or dataset.

[latstars]

Furthermore. We should stop the search when the valid loss is small enough. Then, we can choose the policy in the last epoch.
It is not good to search the policy too long.

[ee-hash]

Furthermore. We should stop the search when the valid loss is small enough. Then, we can choose the policy in the last epoch.
It is not good to search the policy too long.

Thanks. It helps.