@@ -90,7 +90,7 @@ process grid
9090 with ranks ``0, 1, ..., (N*M)-1 ``, process grid coordinate ``(i,j) ``
9191 corresponds to the process with rank ``i*M + j ``.
9292
93- (Note that the protocol's * process grid * is compatible with MPI's
93+ (Note that the protocol's process grid is compatible with MPI's
9494 ``MPI_Cart_create() `` command, and the MPI standard guarantees that
9595 Cartesian process coordinates are always assigned to ranks in the same way
9696 and are "C-order" by default [#mpivirtualtopologies ]_. The protocol makes
@@ -156,11 +156,11 @@ section of a distributed array.
156156
157157The value for the ``'dim_data' `` key shall be a tuple of dictionaries, called
158158"dimension dictionaries", containing one dictionary for each dimension of the
159- distributed array, with the zeroth dictionary associated with the zeroth
160- dimension of the array, and so on for each dimension in succession. There is
161- one dimension dictionary per dimension. These dictionaries are intended to
162- include all metadata required to fully specify a distributed array's dimension
163- information. This tuple may be empty, indicating a zero-dimensional array.
159+ distributed array. The zeroth dictionary in `` 'dim_data' `` shall describe the
160+ zeroth dimension of the array, the first dictionary shall describe the first
161+ dimension, and so on for each dimension in succession . These dictionaries
162+ include all metadata required to specify a distributed array's distribution.
163+ The `` 'dim_data' `` tuple may be empty, indicating a zero-dimensional array.
164164The number of elements in the ``'dim_data' `` tuple must match the number of
165165dimensions of the associated buffer object.
166166
@@ -193,23 +193,23 @@ Required key-value pairs
193193All dimension dictionaries (regardless of distribution type) must define the
194194following key-value pairs:
195195
196- * ``'dist_type' `` : ``{'b', 'c', 'u'} ``
196+ * ``'dist_type' `` : ``{'b', 'c', 'u'} ``.
197197
198198 The distribution type; the primary way to determine the kind of distribution
199199 for this dimension.
200200
201- * ``'size' `` : ``int ``, greater than or equal to 0
201+ * ``'size' `` : ``int ``, greater than or equal to 0.
202202
203203 Total number of global array elements along this dimension.
204204
205205 Indices considered "communication padding" *are not * counted towards this
206206 value; indices considered "boundary padding" *are * counted towards this
207207 value. More explicitly, to calculate the ``size `` along a particular
208208 dimension, one can sum the result of the function ``num_owned_indices `` (in
209- the provided ``utils.py ``) run on the appropriate dimension dictionary on
210- every process.
209+ the provided ``utils.py `` or in this document's appendix) run on the
210+ appropriate dimension dictionary on every process.
211211
212- * ``'proc_grid_size' `` : ``int ``, greater than or equal to 1
212+ * ``'proc_grid_size' `` : ``int ``, greater than or equal to 1.
213213
214214 The total number of processes in the process grid in this dimension.
215215 Necessary for computing the global / local index mapping, etc.
@@ -218,7 +218,7 @@ following key-value pairs:
218218 shall equal the total number of processes.
219219
220220* ``'proc_grid_rank' `` : ``int ``, greater than or equal to 0, less than
221- ``'proc_grid_size' ``
221+ ``'proc_grid_size' ``.
222222
223223 The rank of the process for this dimension in the process grid. This
224224 information allows the consumer to determine where the neighbor sections of
@@ -253,7 +253,7 @@ block (``dist_type`` is ``'b'``)
253253
254254 For a block-distributed dimension without communication padding, adjacent
255255 processes as determined by the dimension dictionary's ``proc_grid_rank ``
256- field shall have adjacent global index ranges. More explicitly, for two
256+ field shall have adjacent global index ranges. More explicitly, for two
257257 processes ``a `` and ``b `` with grid ranks ``i `` and ``i+1 `` respectively, the
258258 ``stop `` of ``a `` shall be equal the ``start `` of ``b ``. With communication
259259 padding present, the stop of ``a `` may be greater than the ``start `` of
@@ -268,8 +268,8 @@ block (``dist_type`` is ``'b'``)
268268* ``padding `` : 2-tuple of ``int ``, each greater than or equal to zero.
269269 Optional.
270270
271- If communication padding, must be less than or equal to the number of indices
272- owned by the neighboring process.
271+ If a value represents communication padding width, it must be less than or
272+ equal to the number of indices owned by the neighboring process.
273273
274274 The padding tuple describes the width of the padding region at the beginning
275275 and end of a buffer in a particular dimension. Padding represents extra
@@ -280,17 +280,17 @@ block (``dist_type`` is ``'b'``)
280280 in ``padding `` is the width of the boundary padding; this is extra allocation
281281 reserved for boundary logic in applications that need it. For the dimension
282282 dictionary with ``proc_grid_rank == proc_grid_size-1 ``, the second element in
283- ``padding `` is the width of the boundary padding. All other ``padding `` tuple
284- values are for communication padding and represent extra allocation reserved
285- for communication between processes. Every communication padding width must
286- equal its counterpart on its neighboring process; more specifically, the
287- "right" communication padding on rank ``i `` in a 1D grid must equal the
288- "left" communication padding on rank ``i+1 ``.
283+ ``padding `` is the width of the boundary padding. All other ``padding ``
284+ tuple values are for communication padding and represent extra allocation
285+ reserved for communication between processes. Every communication padding
286+ width must equal its counterpart on its neighboring process; more
287+ specifically, the "right" communication padding on rank ``i `` in a 1D grid
288+ must equal the "left" communication padding on rank ``i+1 ``.
289289
290290 For example, consider a one-dimensional block-distributed array distributed
291291 over four processes. Let its left boundary padding width be 4, its right
292292 boundary padding width be 0 and its communication padding widths be (1,) (1,
293- 2), (2, 3), and (3,). The padding tuple for the local array on each rank
293+ 2), (2, 3), and (3,). The padding tuple for the local array on each rank
294294 would be:
295295
296296 ============== ====== ====== ====== ======
@@ -302,7 +302,7 @@ block (``dist_type`` is ``'b'``)
302302 If the value associated with ``padding `` is the tuple ``(0,0) `` (the
303303 default), this indicates the local array is not padded in this dimension.
304304
305- * ``periodic `` : ``bool ``, optional
305+ * ``periodic `` : ``bool ``, optional.
306306
307307 Indicates whether this dimension is periodic. When not present, indicates
308308 this dimension is not periodic, equivalent to a value of ``False ``.
@@ -315,13 +315,13 @@ cyclic (``dist_type`` is ``'c'``)
315315 The start index (inclusive, 0-based) of the global index space available on
316316 this process.
317317
318- The cyclic distribution is what results from assigning global indices-- or
319- contiguous blocks of indices, in the case when ``block_size `` is greater than
320- one--to processes in round-robin fashion. When ``block_size `` equals one, a
321- Python slice formed from the ``start ``, ``size ``, and ``proc_grid_size ``
322- values selects the global indices that are owned by this local array.
318+ The cyclic distribution is what results from assigning global indices ( or
319+ contiguous blocks of indices when ``block_size `` is greater than one) to
320+ processes in round-robin fashion. When ``block_size `` equals one, a Python
321+ slice formed from the ``start ``, ``size ``, and ``proc_grid_size `` values
322+ selects the global indices that are owned by this local array.
323323
324- * ``block_size `` : ``int ``, greater than or equal to one. Optional.
324+ * ``block_size `` : ``int ``, greater than or equal to one. Optional.
325325
326326 Indicates the size of contiguous blocks of indices for this dimension. If
327327 absent, equivalent to the case when ``block_size `` is present and equal to
@@ -342,20 +342,20 @@ cyclic (``dist_type`` is ``'c'``)
342342 Note that since this protocol allows for block-distributed dimensions with
343343 irregular numbers of indices on each process, not all 'block'-distributed
344344 dimensions describable by this protocol can be represented as 'cyclic' with
345- the 'block-size' key.
345+ the `` block_size `` key.
346346
347347unstructured (``dist_type `` is ``'u' ``)
348348~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
349349
350- * ``indices ``: buffer (or buffer-compatible) of ``int ``
350+ * ``indices `` : buffer (or buffer-compatible) of ``int ``.
351351
352352 Global indices available on this process.
353353
354354 The only constraint that applies to the ``indices `` buffer is that the values
355355 are locally unique. The indices values are otherwise unconstrained: they can
356356 be negative, unordered, and non-contiguous.
357357
358- * ``one_to_one `` : bool, optional.
358+ * ``one_to_one `` : `` bool `` , optional.
359359
360360 If not present, shall be equivalent to being present with a ``False `` value.
361361
@@ -375,8 +375,8 @@ the current version of this protocol, but more may be added in future versions.
375375Empty dimension dictionary
376376~~~~~~~~~~~~~~~~~~~~~~~~~~
377377
378- An empty dimension dictionary in dimension ``i ``, will be interpreted as the
379- following:
378+ An empty dimension dictionary in dimension ``i `` of `` 'dim_data' `` , will be
379+ interpreted as the following:
380380
381381.. code :: python
382382
@@ -387,7 +387,7 @@ following:
387387 ' stop' : buf.shape[i],
388388 ' size' : buf.shape[i]}
389389
390- Where ``buf `` is the associated buffer object.
390+ where ``buf `` is the associated buffer object.
391391
392392This is intended to be a shortcut for defining undistributed dimensions.
393393
@@ -416,22 +416,23 @@ Undistributed dimensions
416416A dimension with ``proc_grid_size == 1 `` is essentially undistributed; it is
417417"distributed" over a single process. Block-distributed dimensions with
418418``proc_grid_size == 1 `` and with the ``periodic `` and ``padding `` keys present
419- are valid. The ``periodic == True `` and ``padding `` values indicate this array
420- is periodic on one processor, with associated padding regions.
419+ are valid. ``periodic == True `` and nonzero ``padding `` values indicate this
420+ array is periodic on one processor and has associated padding regions.
421421
422422Global array size
423423~~~~~~~~~~~~~~~~~
424424
425- The global number of elements in an array is the product of the ``size ``\s of
426- the dimension dictionaries, or 1 if the ``dim_data `` sequence is empty. In
427- Python syntax, this would be ``reduce(operator.mul, global_shape, 1) `` where
428- ``global_shape `` is a Python sequence of integers such that ``global_shape[i] ``
429- is the ``size `` of the dimension dictionary for dimension ``i ``. If
430- ``global_shape `` is an empty sequence, the result of the reduction above is
431- ``1 ``, indicating the distributed array is a zero-dimensional scalar.
425+ The global number of elements in an array is the product of the values of
426+ ``'size' `` in the dimension dictionaries, or ``1 `` if the ``'dim_data' ``
427+ sequence is empty. In Python syntax, this would be ``reduce(operator.mul,
428+ global_shape, 1) `` where ``global_shape `` is a Python sequence of integers such
429+ that ``global_shape[i] `` is the value of ``'size' `` in the dimension dictionary
430+ for dimension ``i ``. If ``global_shape `` is an empty sequence, the result of
431+ the reduction above is ``1 ``, indicating the distributed array is a
432+ zero-dimensional scalar.
432433
433- Identical ``dim_data `` along an axis
434- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
434+ Identical ``' dim_data' `` along an axis
435+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
435436
436437If ``dim_data `` is the tuple of dimension dictionaries for a process and ``rank
437438= dim_data[i]['proc_grid_rank'] `` for some dimension ``i ``, then all processes
@@ -455,17 +456,17 @@ References
455456 .. [#semver ] Semantic Versioning 2.0.0. http://semver.org/
456457 .. [#pep440 ] PEP 440: Version Identification and Dependency
457458 Specification. http://www.python.org/dev/peps/pep-0440/
458- .. [#trilinos ] Trilinos. http://trilinos.sandia.gov/
459+ .. [#trilinos ] Trilinos. http://trilinos.sandia.gov/
459460 .. [#pytrilinos ] PyTrilinos.
460461 http://trilinos.sandia.gov/packages/pytrilinos/
461- .. [#globalarrays ] Global Arrays. http://hpc.pnl.gov/globalarrays/
462+ .. [#globalarrays ] Global Arrays. http://hpc.pnl.gov/globalarrays/
462463 .. [#gain ] Global Arrays in NumPy.
463464 http://www.pnnl.gov/science/highlights/highlight.asp?id=1043
464- .. [#chapel ] Chapel. http://chapel.cray.com/
465- .. [#x10 ] X10. http://x10-lang.org/
466- .. [#hpfortran ] High Perfomance Fortran. http://dacnet.rice.edu/
467- .. [#julia ] Julia. http://docs.julialang.org
468- .. [#numpy ] NumPy. http://www.numpy.org/
465+ .. [#chapel ] Chapel. http://chapel.cray.com/
466+ .. [#x10 ] X10. http://x10-lang.org/
467+ .. [#hpfortran ] High Perfomance Fortran. http://dacnet.rice.edu/
468+ .. [#julia ] Julia. http://docs.julialang.org
469+ .. [#numpy ] NumPy. http://www.numpy.org/
469470 .. [#bcnetlib ] ScaLAPACK Users' Guide: The Two-dimensional Block-Cyclic Distribution.
470471 http://netlib.org/scalapack/slug/node75.html
471472 .. [#bcibm ] Parallel ESSL Guide and Reference: Block-Cyclic Distribution over Two-Dimensional Process Grids.
0 commit comments