Skip to content

Commit 424787a

Browse files
committed
Merge branch 'master' of github.com:searchivarius/PyFastPFor
2 parents 6471674 + 3ad5f97 commit 424787a

File tree

1 file changed

+18
-1
lines changed

1 file changed

+18
-1
lines changed

README.md

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,25 @@
11
[![PyPI version](https://img.shields.io/pypi/v/pyfastpfor.svg)](https://pypi.python.org/pypi/pyfastpfor/)
22
[![Build Status](https://travis-ci.org/searchivarius/PyFastPFor.svg?branch=master)](https://travis-ci.org/searchivarius/PyFastPFor)
33
# PyFastPFor
4-
Python bindings for the fast integer compression library [FastPFor](https://github.com/lemire/FastPFor). Can be installed locally and vi pip.
4+
Python bindings for the fast integer compression library [FastPFor](https://github.com/lemire/FastPFor): A research library with integer compression schemes. FastPFor is broadly applicable to the compression of arrays of 32-bit integers where most integers are small. The library seeks to exploit SIMD instructions (SSE) whenever possible. This library can decode at least 4 billions of compressed integers per second on most desktop or laptop processors. That is, it can decompress data at a rate of 15 GB/s. This is significantly faster than generic codecs like gzip, LZO, Snappy or LZ4.
5+
6+
# Authors
7+
8+
Daniel Lemire, Leonid Boytsov, Owen Kaser, Maxime Caron, Louis Dionne, Michel Lemay, Erik Kruus, Andrea Bedini, Matthias Petri, Robson Braga Araujo, Patrick Damme. Bindings are created by Leonid Boytsov.
9+
10+
# Installation
11+
12+
Bindings can be installed locally:
13+
```
14+
cd python_bindings
15+
sudo setup.py build install
16+
```
17+
or via pip:
518
```
619
pip install pyfastpfor
720
```
821
Due to some compilation quirks this currently seem to work with GCC only. I will fix it in some not so distant future.
22+
23+
# Documentation
24+
25+
The library supports all the codecs implemented in the original [FastPFor](https://github.com/lemire/FastPFor) library by Feb 2018) as well as two types of data differencing approaches. The library compresses well only small integers. A common trick to deal with large numbers is to sort them and subsequently to encode the differences. Examples of three common use scenarios (no differencing, coarse and fine deltas) are outlined in [this Python notebook](python_bindings/examples.ipynb). To get a list of codecs, use the function ``getCodecList``.

0 commit comments

Comments
 (0)