Skip to content

Commit 727e44b

Browse files
committed
Merge pull request #107 from enthought/feature/edit-protocol-rst
LGTM. Merging.
2 parents 362143e + 0bfb6f5 commit 727e44b

File tree

1 file changed

+54
-53
lines changed

1 file changed

+54
-53
lines changed

protocol.rst

Lines changed: 54 additions & 53 deletions
Original file line numberDiff line numberDiff line change
@@ -90,7 +90,7 @@ process grid
9090
with ranks ``0, 1, ..., (N*M)-1``, process grid coordinate ``(i,j)``
9191
corresponds to the process with rank ``i*M + j``.
9292

93-
(Note that the protocol's *process grid* is compatible with MPI's
93+
(Note that the protocol's process grid is compatible with MPI's
9494
``MPI_Cart_create()`` command, and the MPI standard guarantees that
9595
Cartesian process coordinates are always assigned to ranks in the same way
9696
and are "C-order" by default [#mpivirtualtopologies]_. The protocol makes
@@ -156,11 +156,11 @@ section of a distributed array.
156156

157157
The value for the ``'dim_data'`` key shall be a tuple of dictionaries, called
158158
"dimension dictionaries", containing one dictionary for each dimension of the
159-
distributed array, with the zeroth dictionary associated with the zeroth
160-
dimension of the array, and so on for each dimension in succession. There is
161-
one dimension dictionary per dimension. These dictionaries are intended to
162-
include all metadata required to fully specify a distributed array's dimension
163-
information. This tuple may be empty, indicating a zero-dimensional array.
159+
distributed array. The zeroth dictionary in ``'dim_data'`` shall describe the
160+
zeroth dimension of the array, the first dictionary shall describe the first
161+
dimension, and so on for each dimension in succession. These dictionaries
162+
include all metadata required to specify a distributed array's distribution.
163+
The ``'dim_data'`` tuple may be empty, indicating a zero-dimensional array.
164164
The number of elements in the ``'dim_data'`` tuple must match the number of
165165
dimensions of the associated buffer object.
166166

@@ -193,23 +193,23 @@ Required key-value pairs
193193
All dimension dictionaries (regardless of distribution type) must define the
194194
following key-value pairs:
195195

196-
* ``'dist_type'`` : ``{'b', 'c', 'u'}``
196+
* ``'dist_type'`` : ``{'b', 'c', 'u'}``.
197197

198198
The distribution type; the primary way to determine the kind of distribution
199199
for this dimension.
200200

201-
* ``'size'`` : ``int``, greater than or equal to 0
201+
* ``'size'`` : ``int``, greater than or equal to 0.
202202

203203
Total number of global array elements along this dimension.
204204

205205
Indices considered "communication padding" *are not* counted towards this
206206
value; indices considered "boundary padding" *are* counted towards this
207207
value. More explicitly, to calculate the ``size`` along a particular
208208
dimension, one can sum the result of the function ``num_owned_indices`` (in
209-
the provided ``utils.py``) run on the appropriate dimension dictionary on
210-
every process.
209+
the provided ``utils.py`` or in this document's appendix) run on the
210+
appropriate dimension dictionary on every process.
211211

212-
* ``'proc_grid_size'`` : ``int``, greater than or equal to 1
212+
* ``'proc_grid_size'`` : ``int``, greater than or equal to 1.
213213

214214
The total number of processes in the process grid in this dimension.
215215
Necessary for computing the global / local index mapping, etc.
@@ -218,7 +218,7 @@ following key-value pairs:
218218
shall equal the total number of processes.
219219

220220
* ``'proc_grid_rank'`` : ``int``, greater than or equal to 0, less than
221-
``'proc_grid_size'``
221+
``'proc_grid_size'``.
222222

223223
The rank of the process for this dimension in the process grid. This
224224
information allows the consumer to determine where the neighbor sections of
@@ -253,7 +253,7 @@ block (``dist_type`` is ``'b'``)
253253

254254
For a block-distributed dimension without communication padding, adjacent
255255
processes as determined by the dimension dictionary's ``proc_grid_rank``
256-
field shall have adjacent global index ranges. More explicitly, for two
256+
field shall have adjacent global index ranges. More explicitly, for two
257257
processes ``a`` and ``b`` with grid ranks ``i`` and ``i+1`` respectively, the
258258
``stop`` of ``a`` shall be equal the ``start`` of ``b``. With communication
259259
padding present, the stop of ``a`` may be greater than the ``start`` of
@@ -268,8 +268,8 @@ block (``dist_type`` is ``'b'``)
268268
* ``padding`` : 2-tuple of ``int``, each greater than or equal to zero.
269269
Optional.
270270

271-
If communication padding, must be less than or equal to the number of indices
272-
owned by the neighboring process.
271+
If a value represents communication padding width, it must be less than or
272+
equal to the number of indices owned by the neighboring process.
273273

274274
The padding tuple describes the width of the padding region at the beginning
275275
and end of a buffer in a particular dimension. Padding represents extra
@@ -280,17 +280,17 @@ block (``dist_type`` is ``'b'``)
280280
in ``padding`` is the width of the boundary padding; this is extra allocation
281281
reserved for boundary logic in applications that need it. For the dimension
282282
dictionary with ``proc_grid_rank == proc_grid_size-1``, the second element in
283-
``padding`` is the width of the boundary padding. All other ``padding`` tuple
284-
values are for communication padding and represent extra allocation reserved
285-
for communication between processes. Every communication padding width must
286-
equal its counterpart on its neighboring process; more specifically, the
287-
"right" communication padding on rank ``i`` in a 1D grid must equal the
288-
"left" communication padding on rank ``i+1``.
283+
``padding`` is the width of the boundary padding. All other ``padding``
284+
tuple values are for communication padding and represent extra allocation
285+
reserved for communication between processes. Every communication padding
286+
width must equal its counterpart on its neighboring process; more
287+
specifically, the "right" communication padding on rank ``i`` in a 1D grid
288+
must equal the "left" communication padding on rank ``i+1``.
289289

290290
For example, consider a one-dimensional block-distributed array distributed
291291
over four processes. Let its left boundary padding width be 4, its right
292292
boundary padding width be 0 and its communication padding widths be (1,) (1,
293-
2), (2, 3), and (3,). The padding tuple for the local array on each rank
293+
2), (2, 3), and (3,). The padding tuple for the local array on each rank
294294
would be:
295295

296296
============== ====== ====== ====== ======
@@ -302,7 +302,7 @@ block (``dist_type`` is ``'b'``)
302302
If the value associated with ``padding`` is the tuple ``(0,0)`` (the
303303
default), this indicates the local array is not padded in this dimension.
304304

305-
* ``periodic`` : ``bool``, optional
305+
* ``periodic`` : ``bool``, optional.
306306

307307
Indicates whether this dimension is periodic. When not present, indicates
308308
this dimension is not periodic, equivalent to a value of ``False``.
@@ -315,13 +315,13 @@ cyclic (``dist_type`` is ``'c'``)
315315
The start index (inclusive, 0-based) of the global index space available on
316316
this process.
317317

318-
The cyclic distribution is what results from assigning global indices--or
319-
contiguous blocks of indices, in the case when ``block_size`` is greater than
320-
one--to processes in round-robin fashion. When ``block_size`` equals one, a
321-
Python slice formed from the ``start``, ``size``, and ``proc_grid_size``
322-
values selects the global indices that are owned by this local array.
318+
The cyclic distribution is what results from assigning global indices (or
319+
contiguous blocks of indices when ``block_size`` is greater than one) to
320+
processes in round-robin fashion. When ``block_size`` equals one, a Python
321+
slice formed from the ``start``, ``size``, and ``proc_grid_size`` values
322+
selects the global indices that are owned by this local array.
323323

324-
* ``block_size`` : ``int``, greater than or equal to one. Optional.
324+
* ``block_size`` : ``int``, greater than or equal to one. Optional.
325325

326326
Indicates the size of contiguous blocks of indices for this dimension. If
327327
absent, equivalent to the case when ``block_size`` is present and equal to
@@ -342,20 +342,20 @@ cyclic (``dist_type`` is ``'c'``)
342342
Note that since this protocol allows for block-distributed dimensions with
343343
irregular numbers of indices on each process, not all 'block'-distributed
344344
dimensions describable by this protocol can be represented as 'cyclic' with
345-
the 'block-size' key.
345+
the ``block_size`` key.
346346

347347
unstructured (``dist_type`` is ``'u'``)
348348
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
349349

350-
* ``indices``: buffer (or buffer-compatible) of ``int``
350+
* ``indices`` : buffer (or buffer-compatible) of ``int``.
351351

352352
Global indices available on this process.
353353

354354
The only constraint that applies to the ``indices`` buffer is that the values
355355
are locally unique. The indices values are otherwise unconstrained: they can
356356
be negative, unordered, and non-contiguous.
357357

358-
* ``one_to_one`` : bool, optional.
358+
* ``one_to_one`` : ``bool``, optional.
359359

360360
If not present, shall be equivalent to being present with a ``False`` value.
361361

@@ -375,8 +375,8 @@ the current version of this protocol, but more may be added in future versions.
375375
Empty dimension dictionary
376376
~~~~~~~~~~~~~~~~~~~~~~~~~~
377377

378-
An empty dimension dictionary in dimension ``i``, will be interpreted as the
379-
following:
378+
An empty dimension dictionary in dimension ``i`` of ``'dim_data'``, will be
379+
interpreted as the following:
380380

381381
.. code:: python
382382
@@ -387,7 +387,7 @@ following:
387387
'stop': buf.shape[i],
388388
'size': buf.shape[i]}
389389
390-
Where ``buf`` is the associated buffer object.
390+
where ``buf`` is the associated buffer object.
391391

392392
This is intended to be a shortcut for defining undistributed dimensions.
393393

@@ -416,22 +416,23 @@ Undistributed dimensions
416416
A dimension with ``proc_grid_size == 1`` is essentially undistributed; it is
417417
"distributed" over a single process. Block-distributed dimensions with
418418
``proc_grid_size == 1`` and with the ``periodic`` and ``padding`` keys present
419-
are valid. The ``periodic == True`` and ``padding`` values indicate this array
420-
is periodic on one processor, with associated padding regions.
419+
are valid. ``periodic == True`` and nonzero ``padding`` values indicate this
420+
array is periodic on one processor and has associated padding regions.
421421

422422
Global array size
423423
~~~~~~~~~~~~~~~~~
424424

425-
The global number of elements in an array is the product of the ``size``\s of
426-
the dimension dictionaries, or 1 if the ``dim_data`` sequence is empty. In
427-
Python syntax, this would be ``reduce(operator.mul, global_shape, 1)`` where
428-
``global_shape`` is a Python sequence of integers such that ``global_shape[i]``
429-
is the ``size`` of the dimension dictionary for dimension ``i``. If
430-
``global_shape`` is an empty sequence, the result of the reduction above is
431-
``1``, indicating the distributed array is a zero-dimensional scalar.
425+
The global number of elements in an array is the product of the values of
426+
``'size'`` in the dimension dictionaries, or ``1`` if the ``'dim_data'``
427+
sequence is empty. In Python syntax, this would be ``reduce(operator.mul,
428+
global_shape, 1)`` where ``global_shape`` is a Python sequence of integers such
429+
that ``global_shape[i]`` is the value of ``'size'`` in the dimension dictionary
430+
for dimension ``i``. If ``global_shape`` is an empty sequence, the result of
431+
the reduction above is ``1``, indicating the distributed array is a
432+
zero-dimensional scalar.
432433

433-
Identical ``dim_data`` along an axis
434-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
434+
Identical ``'dim_data'`` along an axis
435+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
435436

436437
If ``dim_data`` is the tuple of dimension dictionaries for a process and ``rank
437438
= dim_data[i]['proc_grid_rank']`` for some dimension ``i``, then all processes
@@ -455,17 +456,17 @@ References
455456
.. [#semver] Semantic Versioning 2.0.0. http://semver.org/
456457
.. [#pep440] PEP 440: Version Identification and Dependency
457458
Specification. http://www.python.org/dev/peps/pep-0440/
458-
.. [#trilinos] Trilinos. http://trilinos.sandia.gov/
459+
.. [#trilinos] Trilinos. http://trilinos.sandia.gov/
459460
.. [#pytrilinos] PyTrilinos.
460461
http://trilinos.sandia.gov/packages/pytrilinos/
461-
.. [#globalarrays] Global Arrays. http://hpc.pnl.gov/globalarrays/
462+
.. [#globalarrays] Global Arrays. http://hpc.pnl.gov/globalarrays/
462463
.. [#gain] Global Arrays in NumPy.
463464
http://www.pnnl.gov/science/highlights/highlight.asp?id=1043
464-
.. [#chapel] Chapel. http://chapel.cray.com/
465-
.. [#x10] X10. http://x10-lang.org/
466-
.. [#hpfortran] High Perfomance Fortran. http://dacnet.rice.edu/
467-
.. [#julia] Julia. http://docs.julialang.org
468-
.. [#numpy] NumPy. http://www.numpy.org/
465+
.. [#chapel] Chapel. http://chapel.cray.com/
466+
.. [#x10] X10. http://x10-lang.org/
467+
.. [#hpfortran] High Perfomance Fortran. http://dacnet.rice.edu/
468+
.. [#julia] Julia. http://docs.julialang.org
469+
.. [#numpy] NumPy. http://www.numpy.org/
469470
.. [#bcnetlib] ScaLAPACK Users' Guide: The Two-dimensional Block-Cyclic Distribution.
470471
http://netlib.org/scalapack/slug/node75.html
471472
.. [#bcibm] Parallel ESSL Guide and Reference: Block-Cyclic Distribution over Two-Dimensional Process Grids.

0 commit comments

Comments
 (0)