Refine3D starting from 1/2 maps doesn't appear to initialise references from 1/2 map 2.

[huwjenkins]

The RELION 5 (and 4) tutorial states:

"By giving one of the half-maps as reference, both half-maps will actually be read in for gold-standard refinement. This prevents overfitting by reading in a joint reconstruction, and therefore one can start the refinement from higher initial resolutions"

However it is not clear how this works. In the relion50_tutorial_precalculated_results.tar.gz file the last Refine3D (job029) is run with:

--ref Refine3D/job025/run_half1_class001_unfil.mrc

running relion_image_handler on the half maps from job025 show they have different average pixel values:

Refine3D/job025/run_half1_class001_unfil.mrc : (x,y,z,n)= 256 x 256 x 256 x 1 ; avg= 0.000337476 stddev= 0.00821005 minval= -0.0463963 maxval= 0.110569; angpix = 1.24484
Refine3D/job025/run_half2_class001_unfil.mrc : (x,y,z,n)= 256 x 256 x 256 x 1 ; avg= 0.000336427 stddev= 0.00824318 minval= -0.0563466 maxval= 0.110049; angpix = 1.24484

however, the initial 1/2 maps in job029 do not:

Refine3D/job029/run_it000_half1_class001.mrc : (x,y,z,n)= 256 x 256 x 256 x 1 ; avg= 0.000337476 stddev= 0.00403042 minval= -0.0181095 maxval= 0.055298; angpix = 1.24484
Refine3D/job029/run_it000_half2_class001.mrc : (x,y,z,n)= 256 x 256 x 256 x 1 ; avg= 0.000337476 stddev= 0.00403042 minval= -0.0181095 maxval= 0.055298; angpix = 1.24484

in fact these files are identical:

$ diff Refine3D/job029/run_it000_half1_class001.mrc Refine3D/job029/run_it000_half2_class001.mrc

most concerningly if I replace Refine3D/job025/run_half2_class001_unfil.mrc with a map with all 0 pixels then the refinement works fine. However if I keep Refine3D/job025/run_half2_class001_unfil.mrc intact but instead replace Refine3D/job025/run_half1_class001_unfil.mrc with all zeros then the refinement starts off very badly:

 Auto-refine: Iteration= 1
 Auto-refine: Resolution= 7.96694 (no gain for 0 iter)
 Auto-refine: Changes in angles= 999 degrees; and in offsets= 999 Angstroms (no gain for 0 iter)
 Estimating accuracies in the orientational assignment ...
   1/   1 sec ............................................................~~(,_,">
 Auto-refine: Estimated accuracy angles= 30.1 degrees; offsets= 12.5728 Angstroms
 Auto-refine: WARNING: Iter = 1 The angular accuracy is worse than 10 degrees, so basically you cannot align your particles (yet)!
 Auto-refine: WARNING: You probably need not worry if the accuracy improves during the next few iterations.
 Auto-refine: WARNING: However, if the problem persists it may lead to spurious FSC curves, so be wary of inflated resolution estimates...
 Auto-refine: WARNING: Sometimes it is better to tune resolution yourself by adjusting T in a 3D-classification with a single class.
 CurrentResolution= 7.96694 Angstroms, which requires orientationSampling of at least 4.55696 degrees for a particle of diameter 200 Angstroms

in this case both initial half maps are all zeros:

Refine3D/test/run_it000_half1_class001.mrc : (x,y,z,n)= 256 x 256 x 256 x 1 ; avg= 0 stddev= 0 minval= 0 maxval= 0; angpix = 1.24484
Refine3D/test/run_it000_half2_class001.mrc : (x,y,z,n)= 256 x 256 x 256 x 1 ; avg= 0 stddev= 0 minval= 0 maxval= 0; angpix = 1.24484

and in the first few iterations both the 1/2 maps look very poor. Does this suggests that the initial references are only made from the first 1/2 map?

[biochem-fan]

I think this works only when you specify model.star as a reference. In other words, the documentation is wrong.

@scheres, what is the intended behaviour?

[scheres]

Hmm, this has been wrong for a long time. Thanks for reporting, I hope the commit above fixes this. Please re-open if it doesn't.

[huwjenkins]

Thanks for the quick fix! I can confirm that with 8514526 the initial references for half set 1 and half set 2 are different. Does this fix need back-porting to RELION-4 as the same code is present there?

[biochem-fan]

@scheres I am looking at the codes and your changes but where does "half1" in the fn_ref string actually get replaced into "half2"? I cannot find. MlOptimiser::initialiseReferences and MlModel::initialiseFromImages aren't responsible for it.

@huwjenkins I can git cherry-pick the commit back into 4.0.x once I understand the patch.

[scheres]

That was already done ml_optimiser_mpi.cpp line 730:
fn_ref.getHalf(fn_ref, my_halfset);

[huwjenkins]

deleted as answered by @scheres whilst I was writing the comment.

[biochem-fan]

@scheres @huwjenkins Thanks!

I applied the same patch to the ver4.0 branch.