Apache Celeborn is an elastic and high-performance service for shuffle and spilled data.

Mirror of Apache Spark

Apache Celeborn Site

Uniffle is a high performance, general purpose Remote Shuffle Service.

Apache Iceberg

A new C++ vectorized database acceleration library aimed to optimizing query engines and data processing systems.

Expressive types for Spark.

Native SQL Engine plugin for Spark SQL with vectorized SIMD optimizations.

A cluster computing framework for processing large-scale geospatial data

Terraform module to provision a fully managed AWS EKS Node Group

Apache Arrow DataFusion and Ballista query engines

An open-source storage layer that brings scalable, ACID transactions to Apache Spark™ and big data workloads.

This is the companion repository for the book How Query Engines Work.

Spark SQL Macros provides a mechanism similar to Spark User-Defined function registration; with the key enhancement being that custom code gets compiled to equivalent Catalyst Expressions at macro …

Spark SQL listener to record lineage information

Large-scale and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, on single node, hadoop yarn and more.

CockroachDB To-Do Apps

Self-Driving Database Management System from Carnegie Mellon University

Reliable Allreduce and Broadcast Interface for distributed machine learning

Morpheus brings the leading graph query language, Cypher, onto the leading distributed processing platform, Spark.

BigDL: Distributed Deep Learning Library for Apache Spark

Distributed Tensorflow, Keras and BigDL on Apache Spark

Easy TOC creation for GitHub README.md

A common bricks library for building scalable and portable distributed machine learning.