Skip to content

Commit

Permalink
Add Design section to README.
Browse files Browse the repository at this point in the history
  • Loading branch information
okeuday committed Jun 16, 2016
1 parent cd311e5 commit eed8928
Showing 1 changed file with 43 additions and 4 deletions.
47 changes: 43 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,8 @@ Purpose
-------

CPG provides a process group interface that is similar to the pg2 module
within Erlang OTP. The pg Erlang OTP module is experimental and people
have avoided using it. However, the pg2 module is used internally by
Erlang OTP, and is currently the most common approach to the combination of
within Erlang OTP. The pg2 module is used internally by
Erlang/OTP, and is currently the most common approach to the combination of
availability and partition tolerance in Erlang (as they relate to the
CAP theorem). When comparing these goals with gproc (and its usage of
`gen_leader`), gproc is focused on availability and consistency (as it relates
Expand All @@ -18,7 +17,8 @@ to the CAP theorem), which makes its goals similar to mnesia.
The cpg interface was created to avoid some problems with pg2 while pursuing
better availability and partition tolerance. pg2 utilizes ets (global
key/value storage in Erlang which requires internal memory locking,
which limits scalability) but cpg uses internal process memory. By default,
which limits scalability) but cpg uses internal process memory
(see the **Design** section for more information). By default,
cpg utilizes Erlang strings for group names (list of integers) and provides
the ability to set a pattern string as a group name. A pattern string
is a string that includes the`"*"`wildcard character (equivalent to ".+"
Expand Down Expand Up @@ -47,6 +47,45 @@ means these process group solutions are only targeting a cluster of Erlang
nodes, given the constraints of distributed Erlang and a fully-connected
network topology.

Design
------

cpg is a Commutative/Convergent Replicated Data-Type (CRDT) that uses
node ownership of Erlang processes to ensure a set of keys has
add and remove operations that commute with an internal map data structure.
The cpg module provides add and remove operations with the function names
join and leave, that may only be called on the node that owns the
Erlang process which is the value for the join or leave operation.
The key is the process group name which represents a list of Erlang processes
(with an single Erlang process being able to be added or removed any
number of times).

All cpg join and leave operations change global state as a
Commutative Replicated Data-Type (CmRDT) by sending the operation to the
associated cpg Erlang process as a Distributed Erlang message to all remote
nodes after the operation successfully completes on the local node.

cpg also uses Distributed Erlang node monitoring to handle netsplits as a
Convergent Replicated Data-Type (CvRDT) by sending all of the internal
cpg state to remote nodes that have recently connected. The associated
cpg Erlang process on the remote node then performs a merge operation to
make sure the count of each Erlang pid is consistent with the internal
cpg state it received.

The CRDT functionality in cpg may look similar to a PN-Set due to tracking
all the Erlang pids and the count of how many times they have been added.
However, the consistency of the internal cpg state relies on serialized
mutability on the local node (naturally, due to a single Erlang process
owning the internal cpg data) before operation is sent to the remote nodes
(for join or leave function calls that operate as a CmRDT).

The design description above assumes `GROUP_NAME_WITH_LOCAL_PIDS_ONLY` is
defined within `cpg_constants.hrl` when cpg is compiled, which is always
the default. If `GROUP_NAME_WITH_LOCAL_PIDS_ONLY` is not defined, then
cpg would use the global transaction locking that pg2 uses, which should
cause partition tolerance problems. The macro is present in case it is
necessary to replicate pg2 semantics with cpg.

Build
-----

Expand Down

0 comments on commit eed8928

Please sign in to comment.