Skip to content

Commit

Permalink
Add a README for @post-me/mpi
Browse files Browse the repository at this point in the history
  • Loading branch information
alesgenova committed Feb 10, 2021
1 parent d813105 commit 2a8a9e9
Show file tree
Hide file tree
Showing 7 changed files with 107 additions and 6 deletions.
12 changes: 9 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,9 @@ In this [live demo](https://alesgenova.github.io/post-me-demo) the main window c
5. [Callbacks as parameters](#callbacks)
6. [Transfer vs Clone](#transfer)
7. [Debugging](#debugging)
8. [API Documentation](#api)
9. [References](#references)
8. [Parallel Programming](#parallel)
9. [API Documentation](#api)
10. [References](#references)

<a id="install"></a>

Expand Down Expand Up @@ -387,11 +388,16 @@ ParentHandshake(messenger).then((connection) => {
Output:
![debug output](debug.png)
<a id="parallel"></a>
## Parallel Programming
[__@post-me/mpi__](https://github.com/alesgenova/post-me/tree/main/packages/mpi) is an experimental library to write parallel algorithms that run on a pool of workers using a MPI-like syntax. See the dedicated [README](https://github.com/alesgenova/post-me/tree/main/packages/mpi) for more information.
<a id="api"></a>
## API Documentation
The full [__API reference__](https://alesgenova.github.io/post-me/post-me.html) cab be found [here](https://alesgenova.github.io/post-me/post-me.html).
The full [__API reference__](https://alesgenova.github.io/post-me/post-me.html) can be found [here](https://alesgenova.github.io/post-me/post-me.html).
<a id="references"></a>
Expand Down
2 changes: 1 addition & 1 deletion packages/core/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -92,4 +92,4 @@
"typescript",
"postmate"
]
}
}
92 changes: 91 additions & 1 deletion packages/mpi/README.md
Original file line number Diff line number Diff line change
@@ -1 +1,91 @@
# @post-me/mpi
<h1 align="center">@post-me/mpi</h1>

<p align="center">Write parallel algorithms in JavaScript using a MPI-like API.</p>

![diagram](./diagram.png)

__@post-me/mpi__ is a library that allows programmers to write parallel algorithms that run on a pool of web workers, using an API similar to [MPI](https://en.wikipedia.org/wiki/Message_Passing_Interface).

A worker pool is a set of workers that are mutually interconnected. Each worker can communicate directly with any other worker in the pool.

The low level communication between workers and the parent application is managed by __post-me__.

## Usage

Below is a small example of using __@post-me/mpi__ in practice. In this example we will be sorting an array in parallel.

Worker code:
```javascript
import { joinPool } from '@post-me/mpi';

const connection = await joinPool(self);

// The parallel sort method
const sort = (communicator) => async (array) => {
const root = 0;
let subArray = await communicator.scatter(array, root);
subArray.sort((a, b) => a - b);
const sorted = await communicator.reduce(subArray, merge, root);

return sorted;
}

// Expose parallel methods to the application
connection.registerMethods({ sort });

// Merge two sorted arrays into a single sorted array
function merge(a0, a1) {/* ... */}
```

Parent code:
```javascript
import { createPool } from '@post-me/mpi';

const array = new Float32Array(1024);
const N_WORKERS = 4;

// Create the workers
const workers: Worker[] = [];
for (let i = 0; i < N_WORKERS; ++i) {
workers.push(new Worker('./worker.js'));
}

// Create a pool of mutually interconnected workers
const workerPool = await createPool(workers);

// Pass different parameter to the parallel method based on the rank of the worker
const root = 0;
const args = (rank) => rank === root ? array : null;
const transfer = (rank, [arr]) => rank === root ? [arr.buffer] : [];

// Call the parallel method 'sort'
const result = await workerPool.call('sort', args, transfer);

// The sorted array is returned by the root worker
const sortedArray = result[root];
```

## MPI Operations

The following MPI operations are already implemented in __@post-me/mpi__:
- `send`
- `recv`
- `bcast`
- `scatter`
- `gather`
- `reduce`
- `barrier`
- `allGather`
- `allReduce`

## Typescript

The library has extensive typescript support, all arguments, methods, return types, etc., are all type checked so that most coding mistakes can be caught at compile tim.

## Benchmark
Below is a quick non-scientific benchmark that shows that indeed running a parallel algorithm is faster than serial. In the plot I'm showing the speedup obtained when sorting arrays of various length as a function of the number of workers.

![benchmark](./benchmark.png)

## Demo
I created a small [demo page](https://alesgenova.github.io/post-me-demo/mpi/) where you can run a couple of test algorithms yourself ([source](https://github.com/alesgenova/post-me-demo/tree/main/examples/mpi)).
Binary file added packages/mpi/benchmark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added packages/mpi/diagram.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
5 changes: 5 additions & 0 deletions packages/mpi/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion packages/mpi/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -93,4 +93,4 @@
"back-end",
"typescript"
]
}
}

0 comments on commit 2a8a9e9

Please sign in to comment.