Feature: Auto-generate llms.txt for church websites

[mbyrdLCS]

Platform

Web

Application

B1.church

Description

Feature Request: Auto-Generated llms.txt for B1 Websites

Summary

Automatically generate llms.txt files for all B1-hosted church websites to improve AI/LLM discoverability. This makes church websites visible to ChatGPT, Claude, Perplexity, and other AI assistants when users ask questions like "find me a church near me" or "what churches have youth programs."

Background

• The llms.txt standard is emerging as a way for websites to communicate with AI crawlers
• Similar to robots.txt but optimized for LLM consumption
• Churches using B1 would automatically benefit without any manual configuration
• Could be a competitive advantage for B1 over other church website builders

Implementation Plan

Phase 1: Simple Auto-Generation (MVP)

Goal: Extract existing content and serve as llms.txt - no AI needed

New Files:

B1App/src/app/[sdSlug]/llms.txt/route.ts
B1App/src/helpers/LlmsTxtGenerator.ts

Route Handler (route.ts):

import { NextResponse } from "next/server";
import { ApiHelper } from "@churchapps/apphelper";
import { LlmsTxtGenerator } from "@/helpers/LlmsTxtGenerator";

export const revalidate = 3600; // Cache for 1 hour

export async function GET(
  request: Request,
  { params }: { params: Promise<{ sdSlug: string }> }
) {
  const { sdSlug } = await params;

  try {
    // Load church data
    const church = await ApiHelper.getAnonymous(
      `/churches/lookup/?subDomain=${sdSlug}`,
      "MembershipApi"
    );

    if (!church?.id) {
      return new NextResponse("Not found", { status: 404 });
    }

    // Generate llms.txt content
    const content = await LlmsTxtGenerator.generate(church);

    return new NextResponse(content, {
      headers: {
        "Content-Type": "text/plain; charset=utf-8",
        "Cache-Control": "public, max-age=3600",
      },
    });
  } catch (error) {
    console.error("Error generating llms.txt:", error);
    return new NextResponse("Error generating llms.txt", { status: 500 });
  }
}

Generator Helper (LlmsTxtGenerator.ts):

import { ApiHelper } from "@churchapps/apphelper";
import { ChurchInterface, LinkInterface, PageInterface } from "./interfaces";

export class LlmsTxtGenerator {

  static async generate(church: ChurchInterface): Promise<string> {
    const baseUrl = `https://${church.subDomain}.b1.church`;

    // Load navigation links to get all page URLs
    const links: LinkInterface[] = await ApiHelper.getAnonymous(
      `/links/church/${church.id}?category=b1Tab`,
      "ContentApi"
    );

    // Load content from each page
    const pages = await this.loadAllPages(church.id, links);

    // Build llms.txt
    return this.buildLlmsTxt(church, baseUrl, pages);
  }

  private static async loadAllPages(
    churchId: string,
    links: LinkInterface[]
  ): Promise<PageContent[]> {
    const pages: PageContent[] = [];

    for (const link of links) {
      if (link.url && link.url.startsWith("/")) {
        try {
          const page: PageInterface = await ApiHelper.getAnonymous(
            `/pages/${churchId}/tree?url=${link.url}`,
            "ContentApi"
          );

          if (page?.title) {
            const text = this.extractTextFromPage(page);
            pages.push({
              title: page.title,
              url: link.url,
              content: text,
            });
          }
        } catch (e) {
          // Skip pages that fail to load
        }
      }
    }

    return pages;
  }

  private static extractTextFromPage(page: PageInterface): string {
    const textParts: string[] = [];

    for (const section of page.sections || []) {
      for (const element of section.elements || []) {
        if (element.answers?.text) {
          // Strip markdown formatting for cleaner text
          const cleanText = this.stripMarkdown(element.answers.text);
          if (cleanText.trim()) {
            textParts.push(cleanText.trim());
          }
        }
      }
    }

    return textParts.join(" ");
  }

  private static stripMarkdown(text: string): string {
    return text
      .replace(/#{1,6}\s?/g, "")           // Headers
      .replace(/\*\*(.+?)\*\*/g, "$1")     // Bold
      .replace(/\*(.+?)\*/g, "$1")         // Italic
      .replace(/\[(.+?)\]\(.+?\)/g, "$1")  // Links
      .replace(/!\[.*?\]\(.+?\)/g, "")     // Images
      .replace(/`(.+?)`/g, "$1")           // Inline code
      .replace(/\n{3,}/g, "\n\n")          // Multiple newlines
      .trim();
  }

  private static buildLlmsTxt(
    church: ChurchInterface,
    baseUrl: string,
    pages: PageContent[]
  ): string {
    const lines: string[] = [];

    // Header
    lines.push(`# ${church.name}`);
    lines.push("");
    lines.push(`> ${church.name} is a church community.`);
    lines.push("");
    lines.push(`Website: ${baseUrl}`);
    lines.push("");

    // Pages section
    lines.push("## Pages");
    lines.push("");

    for (const page of pages) {
      lines.push(`- [${page.title}](${baseUrl}${page.url})`);

      // Add brief content summary (first 200 chars)
      if (page.content) {
        const summary = page.content.substring(0, 200);
        if (summary) {
          lines.push(`  ${summary}${page.content.length > 200 ? "..." : ""}`);
        }
      }
      lines.push("");
    }

    // Footer
    lines.push("---");
    lines.push("This website is powered by B1.church - Free church websites and apps.");

    return lines.join("\n");
  }
}

interface PageContent {
  title: string;
  url: string;
  content: string;
}

Effort: ~2-4 hours
Impact: Immediate - all B1 sites become AI-discoverable

---

Phase 2: AI-Enhanced Generation (Future)

Goal: Use AI to create smarter, more structured summaries

Enhancements:

1. Intelligent Summarization

   • Call OpenAI/Anthropic API to summarize page content
   • Extract key services, programs, and contact info
   • Generate FAQ section from content

2. Structured Data Extraction

   interface ChurchAISummary {
     denomination?: string;
     services: { day: string; time: string; name: string }[];
     programs: string[];  // Youth, Children, Music, etc.
     location: { address: string; city: string; state: string };
     contactInfo: { phone?: string; email?: string };
     keyMessages: string[];  // Core beliefs, mission, vision
   }

3. Enhanced llms.txt Output

   # First Baptist Church
   
   > A welcoming Baptist church in Austin, TX serving families since 1985.
   
   ## Quick Facts
   - Denomination: Baptist
   - Location: Austin, TX
   - Sunday Services: 9:00 AM, 11:00 AM
   
   ## Programs
   - Youth Ministry (grades 6-12)
   - Children's Ministry (ages 0-5)
   - Small Groups
   - Music Ministry
   
   ## Contact
   - Phone: (512) 555-1234
   - Email: info@firstbaptist.org
   
   ## Pages
   [structured page list with AI summaries]
   

4. Caching Strategy

   • Store generated AI summary in database
   • Regenerate when pages are saved (webhook/event)
   • Invalidate cache on content changes

New API Endpoint (optional):

POST /content/llms-txt/regenerate/:churchId

• Called when pages are saved
• Regenerates and caches the AI summary
• Could be triggered by admin action or automatically

Effort: ~1-2 days
Dependencies: AI API key (OpenAI/Anthropic), caching infrastructure

---

Technical Details

Data Flow

Request → /llms.txt route
           ↓
    Load church by subdomain
           ↓
    Load navigation links (b1Tab category)
           ↓
    For each link URL:
      Load page tree → Extract text from elements
           ↓
    [Phase 2: AI summarization]
           ↓
    Generate llms.txt format
           ↓
    Return with caching headers

Existing APIs Used

• GET /churches/lookup/?subDomain=xxx (MembershipApi)
• GET /links/church/:churchId?category=b1Tab (ContentApi)
• GET /pages/:churchId/tree?url=/about (ContentApi)

Content Extraction Path

Page → sections[] → elements[] → answers.text (markdown)

---

Success Metrics

• All B1 websites automatically have /llms.txt endpoint
• Pages load in < 500ms (with caching)
• AI tools can discover and recommend B1-hosted churches
• Zero configuration required from church admins

Future Considerations

• Add to sitemap.xml
• Create /llms-full.txt for complete content dump
• Support for multi-language churches
• Analytics on AI crawler visits

---

References

• llms.txt Specification
• Generative Engine Optimization (GEO)
• Related: HubSpot AEO Grader showed ChurchApps at 50-54/100, with Brand Recognition at 3/20

Additional Context

No response