New constructor for MPMD::Copier

[siddanib]

Summary

A new constructor, Copier::Copier(), is created in MPMD::Copier. This constructor is ONLY for a particular setup where the utilized BoxArray is exactly the same in both MPMD codes. The empty constructor builds its BoxArray based on the BoxArray supplied by the other code. Hence, the empty constructor MUST be used in conjunction with the other code's constructor being Copier::Copier(ba, dm, true).

[WeiqunZhang]

that's right

…

On Sat, Mar 16, 2024, 4:50 PM siddanib ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In Src/Base/AMReX_MPMD.cpp <#3806 (comment)>: > + } + + ParallelDescriptor::Bcast(oprocs.data(), oprocs.size()); + + // procs holds mpi ranks of our boxes, and oprocs holds + // mpi ranks of the other program's boxes. All mpi ranks are in + // MPI_COMM_WORLD. + + // Build communication meta-data + m_is_thread_safe = ba.ixType().cellCentered(); + + std::vector<std::pair<int,Box> > isects; + + for (int i = 0; i < this_nboxes; ++i) { + if (procs[i] == myproc) { + ba.intersections(m_bv[i], isects); Thanks for helping me with this, @WeiqunZhang <https://github.com/WeiqunZhang> ! For this special case if we ONLY do a copy of a (NODAL) box i to box i do you think we should also make m_is_thread_safe = true irrespective of the BoxArray being cell centered or not? — Reply to this email directly, view it on GitHub <#3806 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB37TYPWRNQQRCUBFNFUNVLYYTLDJAVCNFSM6AAAAABEXDIYN6VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTSNBRGM2DQMRSGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[siddanib]

Thanks for helping me with this, @WeiqunZhang! I wanted to mention that I included a minor logic for m_is_thread_safe in the general Copier(ba,dm,false).

https://github.com/siddanib/amrex/blob/6d644be78f76d2aa930a45ffc256364deacf18b6/Src/Base/AMReX_MPMD.cpp#L251

My intention was to check if there is any overlap among cell-centered Boxes in the oba. m_is_thread_safe = false if such an overlap is detected. I hope it is consistent with the objective of MPMD::Copier.

Thanks!

[WeiqunZhang]

I don't think the check is correct, because there are no race conditions for reading. I also don't think we need to check because under normal circumstances this is not any issue.

I just made some small changes. Could you please check if it works?

[siddanib]

Thanks for looking into this, @WeiqunZhang! I was able to successfully perform my local MPMD tests based on the latest changes. I removed the unused isects_o variable that I ended up including in my previous commits.