adapter over various popular vector embeddings interfaces: AWS BedRock, OpenAI, word2vec
| sub-module | doc | about |
|---|---|---|
|
|
AWS Bedrock embeddings models |
|
|
OpenAI embeddings models |
|
|
Word2Vec embeddings model |
The library implements generic trait to transform text into vector embeddings.
type Embedder interface {
Embedding(ctx context.Context, text string) (Embedding, error)
}The library defines common embedding I/O utlities throught this generic trait:
- Caching of embeddings
- Embeddings I/O Rate Limiter
- Semantic Chunking (Sanning)
The library also defines adapter for common text Embeddings api, each define as own submodule:
The latest version of the library is available at main branch of this repository. All development, including new features and bug fixes, take place on the main branch using forking and pull requests as described in contribution guidelines. The stable version is available via Golang modules.
import embeddings "github.com/kshard/embeddings/{your-model}"
text, err := embeddings.New(/* config options */)
// Calculate embeddings
vector, err := text.Embedding(context.Background(), "text embeddings")The library is MIT licensed and accepts contributions via GitHub pull requests:
- Fork it
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Added some feature') - Push to the branch (
git push origin my-new-feature) - Create new Pull Request
The build and testing process requires Go version 1.13 or later.
build and test library.
git clone https://github.com/kshard/embeddings
cd embeddings
go test ./...The commit message helps us to write a good release note, speed-up review process. The message should address two question what changed and why. The project follows the template defined by chapter Contributing to a Project of Git book.
If you experience any issues with the library, please let us know via GitHub issues. We appreciate detailed and accurate reports that help us to identity and replicate the issue.
