An awesome list of dataframe (and dataframe-like) libraries. This list focuses on libraries and tools intended for local (on your personal computer) manipulation of tabular data.
Python
- pandas - Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more.
- Polars - Fast multi-threaded DataFrame library in Rust and Python.
- Modin - Speed up your Pandas workflows by changing a single line of code.
- Ibis - A pandas-like deferred expression system, with first-class SQL support.
- Narwhals - Lightweight and extensible compatibility layer between dataframe libraries!
- agate - agate is a Python data analysis library that is optimized for humans instead of machines. It is an alternative to numpy and pandas that solves real-world problems with readable code.
- Lemuras - A small pure Python library to deal with big tables.
- datatable - A Python package for manipulating 2-dimensional tabular data structures.
- Prosto - A Python data processing toolkit to programmatically author and execute complex data processing workflows. Conceptually, it is an alternative to purely set-oriented approaches to data processing like map-reduce, relational algebra, SQL or data-frame-based tools like pandas.
- siuba - Python library for using dplyr like syntax with pandas and SQL.
- Vaex - A high performance Python library for lazy Out-of-Core DataFrames (similar to Pandas), to visualize and explore big tabular datasets.
- dfply - dplyr-style piping operations for pandas dataframes.
- kadro - A friendly pandas wrapper with a more composable grammar support.
- dexplo - Data exploration library with a pandas-like API.
- pands_cub - A detailed project that teaches you how to build your own Python data analysis library, pandas_cub, from scratch.
- fletcher - Pandas ExtensionDType/Array backed by Apache Arrow.
- tidypandas - A grammar of data manipulation for pandas inspired by tidyverse.
- redframes - [re]ctangular[d]ata[frames]
- static-frame - Immutable and grow-only Pandas-like DataFrames with a more explicit and consistent interface
R
- dplyr - A grammar of data manipulation, providing a consistent set of verbs that help you solve the most common data manipulation challenges.
- data.table - Provides a high-performance version of base R's
data.framewith syntax and feature enhancements for ease of use, convenience and programming speed. - dance - Dancing 💃 with the stats, aka
tibble()dancing 🕺. dance is a sort of reinvention of dplyr classic verbs, with a more modern stack underneath, i.e. it leverages a lot from vctrs and rlang.
JavaScript
- Arquero - A JavaScript library for query processing and transformation of array-backed data tables. Following the relational algebra and inspired by the design of dplyr, Arquero provides a fluent API for manipulating column-oriented data frames.
- dataflow-api - JavaScript API for dataflow processing using the vega-dataflow reactive engine. Perform common database operations (sorting, filtering, aggregation, window calculations) over JavaScript objects.
- datalib - A JavaScript data utility library. It provides facilities for data loading, type inference, common statistics, and string templates.
- Tidy.js - Tidy up your data with JavaScript, inspired by dplyr and the tidyverse.
- Data-Forge - The JavaScript data transformation and analysis toolkit inspired by Pandas and LINQ.
- zebras - A data manipulation and analysis library written in JavaScript offering the convenience of pandas or R.
- dataframe-js - A javascript library providing a new data structure for datascientists and developers.
- Simple Data Analysis (SDA) - Easy-to-use JavaScript library for most common data analysis tasks.
Julia
- DataFrames.jl - Tools for working with tabular data in Julia.
- DataKnots.jl - A Julia library for querying data with an extensible, practical and coherent algebra of query combinators.
- Volcanito.jl - Backend agnostic for tabular data operations in Julia.
- Query.jl - A package for querying julia data sources. It can filter, project, join and group data from any iterable data source, including all the sources supported in IterableTables.jl.
- TidierData.jl - 100% Julia implementation of the dplyr and tidyr R packages.
Clojure
- tech.ml.dataset - A Clojure high performance data processing system.
- tablecloth - Dataset manipulation library build on the top of tech.ml.dataset.
Common Lisp
- Data Frame - Data frames for Common Lisp
C++
- DataFrame - A C++ statistical library that provides an interface similar to Pandas package in Python.
Elixir
- explorer - Explorer brings series (one-dimensional) and dataframes (two-dimensional) for fast data exploration to Elixir.
Elm
- tidy - Leaning heavily on the principles of the tidyverse, and especially tidy data, this package makes it easy to reshape and tidy tabular data for easier data analysis and visualization.
Go
- column - High-performance, columnar, in-memory store with bitmap indexing in Go.
- gambas - Data analysis tool for Go. Similar to the famous Python library pandas.
Java
- Tablesaw - Java dataframe and visualization library.
Kotlin
- Kotlin Dataframe - Structured data processing in Kotlin.
- krangl - A {K}otlin library for data w{rangl}ing.
Lua
- Assistant - A data science library providing flexible dataframes for Lua 5.1+
Q
- qSQL - Query language embedded into Q.
Raku
- Data::Reshapers - Raku package with data reshaping functions for different data structures.
Ruby
- rover - Simple, powerful data frames for Ruby.
- daru - daru (Data Analysis in RUby) is a library for storage, analysis, manipulation and visualization of data in Ruby.
- polars-ruby - Blazingly fast DataFrames for Ruby.
Rust
- polars - A blazingly fast DataFrames library implemented in Rust.
- datafusion - DataFrame API in Apache Arrow DataFusion
Wolfram
- Dataset - Represents a structured dataset based on a hierarchy of lists and associations.
Database
- SQLite - A C-language library that implements a small, fast, self-contained, high-reliability, full-featured, SQL database engine
- DuckDB - An embeddable SQL OLAP Database Management System.
CLI
- VisiData - A terminal spreadsheet multitool for discovering and arranging data.
GUI
- Power Query - A core capability of Power Query is to filter and combine, that is, to mash-up data from one or more of a rich collection of supported data sources.
- prql - A modern language for transforming data — a simple, powerful, pipelined SQL replacement.
- Malloy - An experimental language for describing data relationships and transformations.
- Arrow - A cross-language development platform for in-memory data.
- Substrait - A cross platform way to express data transformation, relational algebra, standardized record expression and plans.
- Consortium for Python Data APIs
