Remove clean markdown generation and artifacts.#833
Conversation
Stop generating and committing clean markdown files in CI, simplify markdown serving/indexing to use source content directly, and remove .md URL suffixes from llms output and markdown alternates for the current main behavior. Made-with: Cursor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| # Agentic Development | ||
|
|
||
| Every page on the Arcade docs site renders as clean markdown. When an AI agent or coding assistant visits any docs URL, the site automatically returns `Content-Type: text/markdown` instead of HTML if: | ||
| Every page on the Arcade docs site can be served as markdown. When an AI agent or coding assistant visits any docs URL, the site automatically returns `Content-Type: text/markdown` instead of HTML if: |
There was a problem hiding this comment.
Google.Passive: Changed 'can be served' to 'serves' to use active voice
| Every page on the Arcade docs site can be served as markdown. When an AI agent or coding assistant visits any docs URL, the site automatically returns `Content-Type: text/markdown` instead of HTML if: | |
| Every page on the Arcade docs site serves as markdown. When an AI agent or coding assistant visits any docs URL, the site automatically returns `Content-Type: text/markdown` instead of HTML if: |
jottakka
requested review from
evantahler,
sdserranog and
teallarson
and removed request for
evantahler
| - name: Check for changes | ||
| id: check-changes | ||
| run: | | ||
| if [ -n "$(git status --porcelain)" ]; then | ||
| echo "has_changes=true" >> $GITHUB_OUTPUT | ||
| else | ||
| echo "has_changes=false" >> $GITHUB_OUTPUT | ||
| fi |
There was a problem hiding this comment.
I'm pretty sure we want to keep most of this - it only regenerates the summary for changed pages. It's very slow/expensive to do them all
| setLoading(true); | ||
| try { | ||
| const response = await fetch(`/api/markdown${pathname}.md`); | ||
| const response = await fetch(`/api/markdown${pathname}`); |
There was a problem hiding this comment.
won't we have to fetch with the new application/markdown header?
| alternates: ENABLE_MARKDOWN_ALTERNATE | ||
| ? { | ||
| types: { | ||
| "text/markdown": getMarkdownAlternatePath(pathname), | ||
| }, | ||
| } | ||
| : undefined, |
There was a problem hiding this comment.
I thought we were removing all of this from our code and having cloudflare do it?
There was a problem hiding this comment.
Yeh, tomorrow I have to take a better look on this, there are more things then I expected related to the MD files.
| @@ -101,15 +101,15 @@ function getLocaleFromPathname(pathname: string, request: NextRequest): string { | |||
|
|
|||
| function buildMarkdownPath(pathname: string, locale: string): string { | |||
There was a problem hiding this comment.
I think this function is no longer needed
| @@ -19,10 +18,10 @@ const LOCALE_REGEX = /^app\/([a-z]{2}(?:-[A-Z]{2})?)\//; | |||
| */ | |||
| function getMarkdownAlternatePath(pathname: string): string { | |||
There was a problem hiding this comment.
most of this logic is also not necessary
| /** | ||
| * Converts clean markdown to HTML for Pagefind indexing. | ||
| * This function expects pre-cleaned markdown (no MDX syntax). | ||
| * Converts markdown to HTML for Pagefind indexing. | ||
| */ | ||
| async function markdownToHtml(markdownContent: string): Promise<string> { |
There was a problem hiding this comment.
Maybe I am missing something but pagefind didn't need to convert to html since it's a postbuild step.
| const fetchAndCopyMarkdown = useCallback(async (): Promise<boolean> => { | ||
| try { | ||
| const markdownUrl = `/api/markdown${pathname}.md`; | ||
| const markdownUrl = `/api/markdown${pathname}`; |
There was a problem hiding this comment.
we need to add the header application/markdown here.
3 participants
Stop generating and committing clean markdown files in CI, simplify markdown serving/indexing to use source content directly, and remove .md URL suffixes from llms output and markdown alternates for the current main behavior.
Note
Medium Risk
Removes CI generation/committing of prebuilt markdown artifacts and changes markdown serving/copy behavior to no longer rely on
public/_markdownor.mdsuffixes; risk is mainly broken markdown links/clients expecting.mdor missing alternate metadata.Overview
Stops generating/committing “clean markdown” artifacts. Deletes the
generate-markdownGitHub workflow, removes thegenerate:markdownscript and itspostbuildhook, and drops thepublic/_markdown/generated files from the repo.Simplifies markdown delivery to use source content and no
.mdsuffix. The markdown API no longer attempts to serve pre-generated clean markdown and instead serves raw MDX (or toolkit markdown when available), while UI copy actions now fetch/api/markdown${pathname}(no.md). Markdown content negotiation/rewrites are updated to build non-.mdpaths, and thetext/markdownalternate metadata is gated behind a disabled flag.Written by Cursor Bugbot for commit 577bbec. This will update automatically on new commits. Configure here.