Entity resolution (also known as data matching, data linkage, record linkage, and many other terms) is the task of finding entities in a dataset that refer to the same entity across different data sources (e.g., data files, books, websites, and databases). Entity resolution is necessary when joining different data sets based on entities that may or may not share a common identifier (e.g., database key, URI, National identification number), which may be due to differences in record shape, storage location, or curator style or preference.
A C library for parsing/normalizing street addresses around the world. Powered by statistical NLP and open geo data.
🆔 A python library for accurate and scalable fuzzy matching, record deduplication and entity-resolution.
Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends
A powerful and modular toolkit for record linkage and duplicate detection in Python
Straightforward fuzzy matching, information retrieval and NLP building blocks for JavaScript.
🆔 Command line tool for deduplicating CSV files
🆔 Examples for using the dedupe library
A list of free data matching and record linkage software.
Super Fast String Matching in Python
🔎 Finds fuzzy matches between CSV files
PyTorch library for transforming entities like companies, products, etc. into vectors to support scalable Record Linkage / Entity Resolution using Approximate Nearest Neighbors.
A convenient way to link, deduplicate, aggregate and cluster data(frames) in Python using deep learning
Link Discovery Framework for Metric Spaces.
List of entity resolution software and resources.
Spark RDD with Lucene's query and entity linkage capabilities
Resources for tackling record linkage / deduplication / data matching problems
Zero-config entity resolution & record linkage. The zero-tuning Fellegi-Sunter path beats hand-tuned Splink head-to-head and scales from a CSV to a verified 100M-row dedupe in 9.2 min. Fuzzy/exact/probabilistic + PPRL + LLM + identity graph. Python + edge-safe TypeScript (WASM), SQL-native in Postgres & DuckDB, MCP/REST + dbt/Airflow.
Record Linkage ToolKit (Find and link entities)
Link Wikidata items to large catalogs
Python package for deduplication/entity resolution using active learning
Created by Halbert L. Dunn
Released 1946