DBMS extension for multimodal query processing and optimization.
Explore the docs »

Landing Page | Report Bug | Request Feature

Table of Contents

1. About The Project
2. Features
3. Getting Started
   • Prerequisites
   • Installation
4. Usage
5. Roadmap
6. Feedback and Issues
7. License
8. Acknowledgments

📜 About The Project

Flock is an advanced DuckDB extension that seamlessly integrates analytics with semantic analysis through declarative SQL queries. Designed for modern data analysis needs, Flock empowers users to work with structured and unstructured data, combining OLAP workflows with the capabilities of LLMs (Large Language Models) and RAG (Retrieval-Augmented Generation) pipelines.

To cite the project:

@article{10.14778/3750601.3750685,
  author  = {Dorbani, Anas and Yasser, Sunny and Lin, Jimmy and Mhedhbi, Amine},
  title   = {Beyond Quacking: Deep Integration of Language Models and RAG into DuckDB},
  journal = {Proc. VLDB Endow.},
  year    = {2025},
  volume  = {18},
  number  = {12},
  doi     = {10.14778/3750601.3750685},
  url     = {https://doi.org/10.14778/3750601.3750685}
}

🔝 back to top

🔥 Features

• Declarative SQL Interface: Perform text generation, classification, summarization, filtering, and embedding generation using SQL queries.
• Multi-Provider Support: Easily integrate with OpenAI, Azure, and Ollama for your AI needs.
• End-to-End RAG Pipelines: Enable retrieval and augmentation workflows for enhanced analytics.
• Map and Reduce Functions: Intuitive APIs for combining semantic tasks and data analytics directly in DuckDB.

🔝 back to top

🚀 Getting Started

📝 Prerequisites

1. DuckDB: Version 1.1.1 or later. Install it from the official DuckDB installation guide.
2. Supported Providers: Ensure you have credentials or API keys for at least one of the supported providers:
   • OpenAI
   • Azure
   • Ollama
3. Supported OS:
   • Linux
   • macOS
   • Windows

🔝 back to top

⚙️ Installation

Flock can be installed in two ways:

Option 1: Install from Community Extension (Recommended)

Flock is a Community Extension available directly from DuckDB's community catalog.

1. Install the extension:

   INSTALL flock FROM community;

2. Load the extension:

   LOAD flock;

Option 2: Build from Source

If you want to build Flock from source or contribute to the project, you can use our automated build script:

1. Clone the repository with submodules:

   git clone --recursive https://github.com/dais-polymtl/flock.git
   cd flock

   Or if you've already cloned without submodules:

   git submodule update --init --recursive

2. Run the build and run script:

   ./scripts/build_and_run.sh

   This interactive script will guide you through:

   • Checking prerequisites (CMake, build tools, compilers)
   • Setting up vcpkg (dependency manager)
   • Building the project (Debug or Release mode)
   • Running DuckDB with the Flock extension

   The script will automatically detect your system configuration and use the appropriate build tools (Ninja or Make).

3. The script will launch DuckDB with Flock extension ready to use. Make sure to check the documentation for usage examples.

Requirements for building from source:

• CMake (3.5 or later)
• C++ compiler (GCC, Clang, or MSVC)
• Build system (Ninja or Make)
• Git
• Python 3 (optional, for integration tests)

🔝 back to top

💻 Usage

🔧 Example Query

Using Flock, you can run semantic analysis tasks directly in DuckDB. For example:

SELECT llm_complete(
            { 'model_name': 'summarizer'},
            { 'prompt_name': 'description-generation', 'context_columns': [{ 'data': product_name }]}
       ) AS product_description
  FROM UNNEST(['Wireless Headphones', 'Gaming Laptop', 'Smart Watch']) AS t(product_name);

Explore more usage examples in the documentation.

🔝 back to top

🛣️ Roadmap

Our roadmap outlines upcoming features and improvements. Stay updated by checking out our detailed plan.

🔝 back to top

🛠️ Feedback and Issues

We value your feedback! If you’d like to report an issue or suggest a new feature, please use the links below:

• Report a Bug
• Request a Feature

For contributing code or other contributions, please refer to our dedicated Contribution Guidelines.

🔝 back to top

📝 License

This project is licensed under the MIT License. See the LICENSE file for details.

🔝 back to top

✨ Team

This project is under active development by the Data & AI Systems Laboratory (DAIS Lab) at Polytechnique Montréal.

🔝 back to top