osc/sm: Fix a bus error on MPI_WIN_{POST,START}.

[kawashima-fj]

A bus error occurs in sm OSC under the following conditions.

• sparc64 or any other architectures which need strict alignment.
• MPI_WIN_POST or MPI_WIN_START is called for a window created by sm OSC.
• The communicator size is odd and greater than 3.

The lines 283-285 in current ompi/mca/osc/sm/osc_sm_component.c has the following code.

module->global_state = (ompi_osc_sm_global_state_t *) (module->segment_base);
module->node_states = (ompi_osc_sm_node_state_t *) (module->global_state + 1);
module->posts[0] = (uint64_t *) (module->node_states + comm_size);

The size of ompi_osc_sm_node_state_t is multiples of 4 but not multiples of 8. So if comm_size is odd, module->posts[0] does not aligned to 8. This causes a bus error when accessing module->posts[i][j].

This patch fixes the alignment of module->posts[0] by inserting a padding between node_states and module->posts[0].

@hjelmn Please review.
I don't know whether this is the best fix method.
If you have a better method, tell me or create a new PR please.

v2.x and v1.10 branches have the same problem. I'll create PRs for them if this PR is accepted.

[hppritcha]

@miked-mellanox could you check why the make dist failed for the mlnx jenkins?

[ggouaillardet]

@kawashima-fj @hjelmn wouldn't it be easier to have the posts at the beginning ?
here is an alternate patch

diff --git a/ompi/mca/osc/sm/osc_sm_component.c b/ompi/mca/osc/sm/osc_sm_component.c
index 2e2948e..91da943 100644
--- a/ompi/mca/osc/sm/osc_sm_component.c
+++ b/ompi/mca/osc/sm/osc_sm_component.c
@@ -280,9 +280,9 @@ component_select(struct ompi_win_t *win, void **base, size_t size, int disp_unit
         module->posts = calloc (comm_size, sizeof (module->posts[0]));
         if (NULL == module->posts) return OMPI_ERR_TEMP_OUT_OF_RESOURCE;

-        module->global_state = (ompi_osc_sm_global_state_t *) (module->segment_base);
+        module->posts[0] = (uint64_t *) (module->segment_base);
+        module->global_state = (ompi_osc_sm_global_state_t *) (module->posts + comm_size * post_size);
         module->node_states = (ompi_osc_sm_node_state_t *) (module->global_state + 1);
-        module->posts[0] = (uint64_t *) (module->node_states + comm_size);

         for (i = 0, total = state_size + posts_size ; i < comm_size ; ++i) {
             if (i > 0) {

[kawashima-fj]

Yes, it will be good.
The component owner (Nathan) will decide 😄

[kawashima-fj]

In Gilles' patch, module->posts should be module->posts[0].
This is the updated patch and I confirmed it works fine.

diff --git a/ompi/mca/osc/sm/osc_sm_component.c b/ompi/mca/osc/sm/osc_sm_component.c
index 2e2948e..91da943 100644
--- a/ompi/mca/osc/sm/osc_sm_component.c
+++ b/ompi/mca/osc/sm/osc_sm_component.c
@@ -280,9 +280,9 @@ component_select(struct ompi_win_t *win, void **base, size_t size, int disp_unit
         module->posts = calloc (comm_size, sizeof (module->posts[0]));
         if (NULL == module->posts) return OMPI_ERR_TEMP_OUT_OF_RESOURCE;

-        module->global_state = (ompi_osc_sm_global_state_t *) (module->segment_base);
+        module->posts[0] = (uint64_t *) (module->segment_base);
+        module->global_state = (ompi_osc_sm_global_state_t *) (module->posts[0] + comm_size * post_size);
         module->node_states = (ompi_osc_sm_node_state_t *) (module->global_state + 1);
-        module->posts[0] = (uint64_t *) (module->node_states + comm_size);

         for (i = 0, total = state_size + posts_size ; i < comm_size ; ++i) {
             if (i > 0) {

[jsquyres]

@hjelmn Can you have a look at this?

[hjelmn]

:bot:retest:

[hjelmn]

Hmm, either I can commit the corrected patch or you can update this PR with that patch.

[kawashima-fj]

@hjelmn Please review again. I pushed a new commit based on Gilles' patch.

[kawashima-fj]

@hjelmn Can you have a look at this?

[ggouaillardet]

@hjelmn what is the status of this PR ?
did you commit the fix already ?

[hjelmn]

Need to retest. If it comes back clean will merge and PR to 2.x.

:bot:retest: