Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
159 lines (113 loc) · 5.36 KB

CONTRIBUTING.md

File metadata and controls

159 lines (113 loc) · 5.36 KB

Contributing to Flowfile

Thanks for your interest in contributing! This guide covers how to set up a dev environment, the conventions we follow, and how to get a change merged.

Flowfile is MIT-licensed. By contributing, you agree that your contribution will be released under the same license.

Before you start

  • Small fixes (typos, clear bugs, small doc improvements) — just open a PR.
  • Anything bigger — open a GitHub Discussion or an issue first so we can agree on the shape before you write code. This saves everyone time.
  • Questions, not contributions — head to Discussions. See the Community page.

Repository layout

This is a monorepo managed by Poetry (Python) and npm (frontend):

  • flowfile_core/ — FastAPI backend (ETL engine, flow execution, auth, catalog) — port 63578
  • flowfile_worker/ — FastAPI compute worker for heavy data processing — port 63579
  • flowfile_frame/ — Python API library (Polars-like interface for programmatic flows)
  • flowfile_frontend/ — Tauri 2 (Rust shell) + Vue 3 UI (VueFlow graph editor)
  • flowfile_wasm/ — Browser-only WASM version (Pyodide)
  • flowfile/ — CLI entry point
  • kernel_runtime/ — Docker-based sandbox for user Python code
  • docs/ — MkDocs site

Dev setup

Prerequisites

  • Python >=3.10, <3.14
  • Node.js 20+
  • Poetry
  • Rust stable toolchain (required only for the Tauri desktop shell — npm run dev:web works without it)
  • Docker (only required for the kernel test marker)
  • Linux only: libwebkit2gtk-4.1-dev, libgtk-3-dev, libayatana-appindicator3-dev, librsvg2-dev

Backend

poetry install

# Run the services in two terminals
poetry run flowfile_worker   # :63579
poetry run flowfile_core     # :63578

Frontend

cd flowfile_frontend
npm install
npm run dev:web              # web dev server, :8080 → backend on :63578 (no Rust needed)
# or, for the full Tauri desktop shell (requires Rust + staged sidecars):
#   1. From repo root: make build_python_services && make rename_sidecars
#   2. cd flowfile_frontend && npm run dev

Full stack via Docker

cp .env.example .env
docker compose up -d
# Frontend: http://localhost:8080  Core: :63578  Worker: :63579

Code style

Python — Ruff

  • Line length 120, target Python 3.10
  • Rules: Pyflakes (F), pycodestyle (E/W), isort (I), pyupgrade (UP), flake8-bugbear (B)
  • Double quotes, space indentation
  • Use Polars, not pandas
  • Import order: stdlib, third-party, then first-party (flowfile, flowfile_core, flowfile_worker, flowfile_frame, shared, test_utils, tools, build_backends)
poetry run ruff check .
poetry run ruff check --fix .
poetry run ruff format .

Frontend — ESLint + Prettier

  • Semicolons, 2-space tabs, double quotes, 100-char width, trailing commas, LF line endings
  • Vue 3 Composition API + TypeScript, Pinia for state, Element Plus for UI
  • Path aliases: @ → src/renderer/app/
cd flowfile_frontend
npm run lint

Tests

Any code change that can be tested should come with a test.

Python (pytest)

poetry run pytest flowfile_core/tests
poetry run pytest flowfile_worker/tests
poetry run pytest flowfile_frame/tests

# With coverage (core + worker)
make test_coverage

# Kernel integration (Docker required)
poetry run pytest -m kernel

Markers: worker, core, kernel.

Frontend E2E (Playwright)

cd flowfile_frontend
npx playwright install --with-deps chromium
npm run test:web         # needs backend + preview server running

Or via Make:

make test_e2e            # web E2E, builds frontend and starts servers

Tauri-shell E2E (via tauri-driver) is a follow-up. The web Playwright suite exercises the same renderer code path.

WASM (Vitest)

cd flowfile_wasm
npm run test

Commits and PRs

  • Branch naming: fix/..., feat/..., docs/... is fine — nothing strict.
  • Commit messages: short imperative subject (e.g. fix flow parameter serialization on reload). Explain the why in the body if it isn't obvious from the diff.
  • One logical change per PR. Smaller PRs get reviewed faster.
  • Fill in the PR description: what changed, why, and how you tested it. Screenshots or short clips help for UI changes.
  • CI must be green before merge. If a check is flaky, say so in the PR — don't just re-run silently.
  • Don't force-push to main. Releases build from it.

Things to avoid

  • Don't commit master_key.txt, .env, or any credentials.
  • Don't introduce pandas — the project uses Polars throughout.
  • On Windows, Polars is pinned to <=1.25.2 (build constraint) — don't bump it unilaterally.
  • Don't add features or abstractions "for later." Keep changes scoped to the task.

Reporting security issues

Please do not open a public issue for security vulnerabilities. Instead, report them privately via GitHub's "Report a vulnerability" form.

Code of conduct

Be kind and assume good intent. Flowfile follows the Contributor Covenant v2.1.