Formcraft AI

AI‑powered form builder and response collection app built with Next.js, Clerk, Drizzle ORM, Neon Postgres, Tailwind CSS, and OpenAI.

Live Preview

Demo Video

LLM as a Judge Tests

Features

• AI form builder: Chat to describe your form; the AI returns a strict JSON schema.
• Save & manage forms: Persist schemas, name forms, and manage publish/unpublish state.
• Public forms: Share a link (read-only, no auth) that renders inputs from the saved schema and validates required fields.
• Responses & analytics:
  • View recent submissions for any form you own.
  • See a realtime total submissions counter.
  • Export responses as CSV.
• Authentication: Clerk-secured dashboard; user records are provisioned on first visit.
• Postgres schema (Drizzle): users, forms, form_responses.

Tech Stack

• Next.js 15 (App Router), React 19
• Clerk for authentication
• Drizzle ORM with Neon Postgres
• Tailwind CSS v4
• Vercel AI SDK and OpenAI for the builder

---

Quickstart

Prerequisites

• Node 20+ and pnpm
• Neon (or any Postgres) connection string
• Clerk application (keys + redirect URLs)
• OpenAI API key

1) Install dependencies

pnpm install

2) Configure environment

Create .env.local and set the following variables:

NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=
CLERK_SECRET_KEY=
NEXT_PUBLIC_CLERK_SIGN_IN_URL=/auth/sign-in
NEXT_PUBLIC_CLERK_SIGN_UP_URL=/auth/sign-up
NEXT_PUBLIC_CLERK_AFTER_SIGN_IN_URL=/dashboard
NEXT_PUBLIC_CLERK_AFTER_SIGN_UP_URL=/dashboard

DATABASE_URL=
OPENAI_API_KEY=

# (optional) Used to build share links in the dashboard; defaults to request host
# NEXT_PUBLIC_APP_URL=https://your-domain.com

Notes:

• DATABASE_URL should be a full Postgres connection string (e.g., Neon serverless).
• OPENAI_API_KEY is required for the AI form builder.
• The Clerk URLs above match the app routes already in this repo.

3) Create the database schema

pnpm db:push

This applies the Drizzle schema in db/schema.ts to your Postgres database.

4) Run the app

pnpm dev

Open http://localhost:3000. Sign up/sign in with Clerk.

---

Using the App

Dashboard

• Go to /dashboard to see your forms.
• Click Create New Form to start a draft (a UUID form id is generated).
• The builder is at /dashboard/form/:id.

AI Form Builder

• Describe the form you want in natural language.
• The AI returns strict JSON with:
  • schema: { name?, description?, fields: [{ name, label, type, required?, ... }] }
  • followUps: questions if constraints are ambiguous
  • humanisedDescription: short natural-language summary
• You’ll see a live preview. Name the form and Save.

Publish & Share

• In /dashboard, toggle Publish/Unpublish.
• Copy the share link to /form/:id and open it in a new tab.

Public Form

• Renders inputs from the saved schema.
• Validates required fields client-side.
• On submit, persists to form_responses and redirects to /form/:id/thank-you.

Responses & Analytics

• Visit /dashboard/form/:id/responses:
  • Table of recent responses (last 200).
  • Realtime total submissions.
  • Export CSV via the page button (downloads from /dashboard/form/:id/responses/export?format=csv).

---

Important Endpoints & Files

• AI builder API (Edge): src/app/api/ai/form-builder/route.ts
• Responses API: src/app/api/forms/[id]/responses/route.ts
• Analytics API: src/app/api/forms/[id]/analytics/route.ts
• CSV export: src/app/dashboard/form/[id]/responses/export/route.ts
• Public form page: src/app/form/[id]/page.tsx
• Builder UI: src/app/dashboard/form/[id]/Builder.tsx
• Response submit action: src/app/form/[id]/actions.ts
• Auth utilities: src/lib/auth.ts, user provisioning: src/lib/users.ts
• DB client: src/lib/db.ts, schema: db/schema.ts

---

Scripts

pnpm dev           # Start Next.js in development
pnpm build         # Build for production
pnpm start         # Start production server
pnpm db:generate   # Generate Drizzle SQL snapshots
pnpm db:push       # Apply current schema to the database
pnpm eval:dev      # Run evalite watcher (optional/dev-only)

---

Evalite: LLM‑as‑a‑Judge Evals

This repo includes Evalite tests that exercise the AI form builder end‑to‑end and score outputs.

• Location: src/tests/form-builder.eval.ts
• Model: uses the Vercel AI SDK with OpenAI (gpt-4)
• Scoring: a structural scorer checks that outputs are strict JSON with a valid schema.fields array; you can add additional scorers (including LLM‑as‑a‑judge) easily.

Run the evals

pnpm eval:dev   # watches and runs evals on change
# or, once-off:
npx evalite run src/tests/form-builder.eval.ts

Requires OPENAI_API_KEY in your environment.

Add tracing (optional)

Evalite can trace Vercel AI SDK calls for better visibility. Wrap your model with traceAISDKModel:

// inside an eval task using streamText
import { traceAISDKModel } from "evalite/ai-sdk";
const result = streamText({
  model: traceAISDKModel(openai("gpt-4o-mini")),
  messages,
});

See the Evalite AI SDK examples for details: Evalite × Vercel AI SDK.

LLM‑as‑a‑Judge example

You can score generations with an LLM judge via createScorer by prompting a model to grade the output:

import { createScorer } from "evalite";
import { streamText } from "ai";
import { openai } from "@ai-sdk/openai";

export const JudgeScorer = createScorer({
  name: "LLM Judge",
  description: "Grades schema quality (0..1)",
  scorer: async ({ output }) => {
    const r = await streamText({
      model: openai("gpt-4o-mini"),
      system: "Return ONLY a number 0..1 for quality of the provided schema.",
      prompt: `Schema JSON:\n${output}`,
    });
    const text = await r.text;
    const score = Number.parseFloat(text.trim());
    return Number.isFinite(score) ? Math.max(0, Math.min(1, score)) : 0;
  },
});

Add JudgeScorer to the scorers: [] array in your eval to record the judgment alongside structural checks.

---

Deployment

• Set the same environment variables in your hosting provider.
• Ensure DATABASE_URL, CLERK_*, and OPENAI_API_KEY are configured.
• The AI builder uses gpt-4 via the Vercel AI SDK.

---

Troubleshooting

• 401 from analytics/responses APIs: ensure you’re signed in and own the form.
• AI responses empty or errors: check OPENAI_API_KEY and usage limits.
• DB errors: verify DATABASE_URL and that pnpm db:push succeeded.
• Share links wrong domain: set NEXT_PUBLIC_APP_URL in production.

---

License

Proprietary – for internal/demo use unless a license is added.