End-to-End Example of Splitgraph, Seafowl and Madatdata

This is a full, end-to-end example built with Next.js that uses madatdata, Splitgraph and Seafowl. It renders a website where users can import GitHub repository data into Splitgraph, and then export that data to Seafowl, and query it directly from the client. In doing so, it demonstrates:

• Importing data to Splitgraph (using the airbyte-github madatdata plugin)
• Exporting it to Seafowl (using the export-to-seafowl madatdata plugin)
• And then querying it, at both Splitgraph DDN and demo.seafowl.cloud, sending queries directly from the client with @madatdata/react and useSql

The example uses Next.js with backend routes that call the Splitgraph API to trigger data import (from GitHub) and data export (to Seafowl). The client side pages query Seafowl and Splitgraph DDN directly by sending raw SQL queries in HTTP requests, which is what both Splitgraph and Seafowl are ultimately designed for.

Try Now

Preview Immediately

No signup required, just click the button!

Or, deploy to Vercel (signup required)

Signup, fork the repo, and import it

Local Development

Make sure you're using Yarn Berry

Make sure your local Yarn executable is set to Yarn Berry, so that it will read from .yarnrc.yml which will direct it to use the executable in .yarn/releases, which at the moment of writing is version 3.2.1

yarn set version berry

Sanity check, yarn --version should be 3.2.1:

yarn --version
# 3.2.1

Also make sure you're using Node v18+:

node --version
# v18.10.0

Install latest `node` with `nvm`, and then install `yarn` as global package

You can use nvm to install the latest Node version (assuming you've setup nvm):

nvm install
nvm use
npm install -g yarn

Note: When calling nvm install, you can tell it to migrate global packages from your current version to the new version, which could save you from running npm install -g yarn, if for some reason this is easier for you:

nvm install --reinstall-packages-from=current
nvm use

Install

To install using the same versions of all packages as listed in the lockfile:

yarn install --immutable

Configure

You will need:

• Splitgraph API key and secret (manage credentials)
• Splitgraph namespace to use (it should be your username)
• GitHub Personal Access Token (get one)

The specific environment variable keys are documented via comments in the .env.test.local file, which you should copy to the (ignored) path of .env.local:

cp .env.test.local .env.local

Then, edit the .env.local file to insert the appropriate values for the required variables.

Run

To run in development mode:

yarn dev

---