Update 2020-09-09-sparse-attention.md (deepspeedai#3432)

Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com>

Explicitly check for OPT activation function (deepspeedai#3278)

Co-authored-by: Michael Wyatt <michaelwyatt@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

[CPU support] Optionally bind each rank to different cores on host (d…

…eepspeedai#2881)

* add fallback path for kernels used in megatron

* temporary numactl WA for SPR 56core

* adapt core allocation according to number of ranks

* add switch to turn on numactl

* detect number of cores on the system

* allow select a subset of the cores on the system to bind

* remove unneeded changes

* use current_env to set OMP_NUM_THREADS in subprocess

* add test for ds_arguments

* change --bind_cores_to_rank option to store_true

* add test for parse_range_list

* add comment for parse range list

* add test for parse range list, rewrite parse_range_list

* fix format error

* fix format

* add -m parameter to numactl when necessary

* Check KMP_AFFINITY to avoid conflict with numactl

* fix format

* negative case for parse_range_list

* detect whether numactl is installed before use numactl to bind cores

* check numactl with package manager of distro

---------

Co-authored-by: sdp <sdp@aia-sdp-spr-108864.jf.intel.com>
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>

Fix Broken Links (deepspeedai#3048)

deepspeed.init_distributed() support for TCP protocols (deepspeedai#2905

)

Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

fix for cpu-only tests (deepspeedai#2849)

ZeRO3 handling frozen weights] (deepspeedai#2653)

get mask token from tokenizer (deepspeedai#2592)

Fix invalid check of recorded parameter orders in zero stage3. (deeps…

…peedai#2550)

Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>

Fix typos: deepseed -> deepspeed (deepspeedai#2499)