Fetch versioned references from repos #12987
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
| const SDK_REFERENCES_FOLDER_PATHS = [ | ||
| { | ||
| id: 'posthog-python', | ||
| repo: 'gewenyu99/posthog-python', |
There was a problem hiding this comment.
I will update these to point to real SDK repo folders once I merge changes in the SDK repos
| SDK Version: {fullReference.info.version} | ||
| SDK Version: | ||
| </p> | ||
| <Popover |
There was a problem hiding this comment.
Always appreciate help from design and website with UI :)
gatsby/createPages.ts
Outdated
| description: node.info.description, | ||
| fullReference: node, | ||
| regex: `/docs/references/${node.referenceId}`, | ||
| types: sdkTypesByReference[node.referenceId][node.version], |
There was a problem hiding this comment.
null check on sdkTypesByReference[node.referenceId]
gatsby/sourceNodes.ts
Outdated
| // Fetch all folder contents in parallel | ||
| const folderPromises = SDK_REFERENCES_FOLDER_PATHS.map(async (folderPath) => { | ||
| const url = `https://api.github.com/repos/${folderPath.repo}/contents/${folderPath.folder_path}?ref=${folderPath.repo_branch}` | ||
| const response = await fetch(url) | ||
| const data = await response.json() | ||
| return { folderPath, data } | ||
| }) |
There was a problem hiding this comment.
add error handling if the api.github.com fetch fails?
There was a problem hiding this comment.
Added. Maybe we should block the build though, when the requests fail, we'll end up with silent 404 on some pages 🤔 These will be hard to catch quickly.
Maybe we do save a version of these locally that we fall back to if GitHub has an outage or something. Like: attempt to fetch files -> attempt to save as filename+commit hash.json -> read from local files.
@smallbrownbike thoughts on what makes sense?
| .filter((item) => item.type === 'file') | ||
| .map(async (item) => { | ||
| const fileUrl = `https://api.github.com/repos/${folderPath.repo}/contents/${folderPath.folder_path}/${item.name}?ref=${folderPath.repo_branch}` | ||
| const fileResponse = await fetch(fileUrl) |
There was a problem hiding this comment.
add error handling if the api.github.com fetch fails?
| { | ||
| resolve: `gatsby-source-filesystem`, | ||
| options: { | ||
| name: `sdkReferences`, | ||
| path: `${__dirname}/src/data/sdkReferences`, | ||
| ignore: [`**/*.{png,jpg,jpeg,gif,svg,webp,mp4,avi,mov}`], | ||
| }, | ||
| }, |
There was a problem hiding this comment.
so this is replaced by sourceNodes yea?
Changes
We want automatically updated, versioned, and persisted references for all SDKs.⚠️ We won't merge this until we land the PRs on the SDK repos
See related RFC
This PR does the following:
<sdk-id>-<sdk-version>whereposthog-pythonwill be the latest version,posthog-python-6.0.0will be a specific versiongatsby/sourceNodes.tsduring build.Separately in other PRs:
Screen.Recording.2025-09-30.at.4.52.44.PM.mov