feat: implement cloud sync for vibe catalog system #296
Conversation
✅ Deploy Preview for fireproof-ai-builder ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
![charliecreates[bot]](https://avatars.githubusercontent.com/in/1140780?s=60&v=4){kind=small}
There was a problem hiding this comment.
- cidUtils:
atob-based base64 decoding is memory-heavy and not SSR-safe; prefer fetching data URLs to ArrayBuffer and hashing that, and clarify that outputs are not IPFS CIDs. - publishUtils: Assumes
result.rows[0]is the newest screenshot without guaranteeing order; risk of selecting older screenshots. Also, converting Files to base64 data URLs is inefficient. - Tests:
global.fetchis mocked without teardown inuseCatalog.test.tsx, risking cross-test pollution. - Tests: Redundant per-file
use-fireproofmocks (includingtoCloud) despite a shared manual mock—consolidation would reduce drift and maintenance cost.
Additional notes (2)
-
Maintainability |
vibes.diy/pkg/app/utils/cidUtils.ts:5-9
This function returns asha256-<hex>label, which is not a true CID (e.g., multihash + multibase). Given the PR mentions "CID-based deduplication", the name/docstring can mislead future maintainers into assuming IPFS-compatible CIDs. Consider renaming or clarifying the comment to reflect that this is a simple SHA-256 content hash string. -
Maintainability |
vibes.diy/tests/__mocks__/use-fireproof.ts:51-51
You’ve added a centralizedtoCloudmock here, but many tests still redefineuse-fireproof(includingtoCloud) inline. This duplication invites drift and inconsistent behavior across tests. Prefer relying on this module mock viavi.mock('use-fireproof')and customizing per-test behavior by mutating the exported mock instead of re-declaring it in each test.
Summary of changes
- Added cid utilities (cidUtils.ts) with functions to hash data URLs and Files using SHA-256.
- Enhanced publish workflow (publishUtils.ts) to store the latest session screenshot and transformed source in the catalog via
addCatalogScreenshotStandalone. - Introduced sync preference utilities (syncPreference.ts) for SSR-safe localStorage access.
- Expanded tests to mock
toCloudand added comprehensiveuseCatalogtests, plus updates to existing tests and shareduse-fireproofmock to includetoCloud.
See: utils added for CID generation, catalog update hook during publish, and broad test updates with new mocks and assertions.
| const binaryString = atob(base64Content); | ||
| const bytes = new Uint8Array(binaryString.length); | ||
| for (let i = 0; i < binaryString.length; i++) { | ||
| bytes[i] = binaryString.charCodeAt(i); | ||
| } |
There was a problem hiding this comment.
The atob + binary-string loop is both memory-intensive for large screenshots and not SSR-safe (atob is browser-only). It also assumes base64-encoded data URLs; non-base64 data: URLs would be mishandled. Prefer getting an ArrayBuffer via fetch(dataUrl) (works for base64 and URL-encoded data URLs) and hashing that. This avoids large intermediate strings and improves compatibility across environments.
Suggestion
Consider replacing the base64 decode with a data URL fetch to ArrayBuffer to avoid atob and binary string allocations, and add a minimal SSR-safe fallback:
export async function generateCid(dataUrl: string): Promise<string> {
if (!dataUrl.startsWith("data:")) throw new Error("Expected data URL");
let buffer: ArrayBuffer;
if (typeof fetch === "function") {
const res = await fetch(dataUrl);
buffer = await res.arrayBuffer();
} else {
// Fallback for SSR/Node: decode base64 payload
const base64 = dataUrl.split(",")[1] ?? "";
if (!base64) throw new Error("Invalid data URL format");
// Use Node Buffer when available to avoid atob
const B: any = (globalThis as any).Buffer;
if (B) {
buffer = B.from(base64, "base64");
} else {
const bin = typeof atob === "function" ? atob(base64) : "";
const bytes = new Uint8Array(bin.length);
for (let i = 0; i < bin.length; i++) bytes[i] = bin.charCodeAt(i);
buffer = bytes.buffer;
}
}
const hashBuffer = await crypto.subtle.digest("SHA-256", buffer);
const hashHex = Array.from(new Uint8Array(hashBuffer))
.map((b) => b.toString(16).padStart(2, "0"))
.join("");
return `sha256-${hashHex}`;
}Reply with "@CharlieHelps yes please" if you'd like me to add a commit with this change.
| // Store screenshot in catalog database for this published vibe | ||
| if (sessionId && userId) { | ||
| try { | ||
| // Get the most recent screenshot from session database | ||
| if (result.rows.length > 0) { | ||
| const screenshotDoc = result.rows[0].doc as any; | ||
| if (screenshotDoc._files && screenshotDoc._files.screenshot) { | ||
| try { | ||
| // Get the File and convert to data URL | ||
| const screenshotFile = | ||
| await screenshotDoc._files.screenshot.file(); | ||
| const screenshotDataUrl = await new Promise<string>( | ||
| (resolve, reject) => { | ||
| const reader = new FileReader(); | ||
| reader.onload = () => resolve(reader.result as string); | ||
| reader.onerror = reject; | ||
| reader.readAsDataURL(screenshotFile); | ||
| }, | ||
| ); | ||
|
|
||
| // Use shared catalog utility function | ||
| await addCatalogScreenshotStandalone( | ||
| userId, | ||
| sessionId, | ||
| screenshotDataUrl, | ||
| transformedCode, | ||
| ); | ||
| } catch (err) { | ||
| console.error("Failed to store catalog screenshot:", err); | ||
| } | ||
| } | ||
| } | ||
| } catch (error) { |
There was a problem hiding this comment.
"Get the most recent screenshot" relies on result.rows[0], but no ordering is ensured here. If the upstream query doesn’t guarantee sort order, you may pick an older screenshot. This is a latent correctness bug that will surface with multiple screenshots.
Suggestion
Ensure you select the newest screenshot deterministically. If the docs have a timestamp (e.g., created_at), sort before selecting, or update the query to return newest-first:
if (result.rows.length > 0) {
const rows = [...result.rows].sort(
(a, b) => (b.doc?.created_at ?? 0) - (a.doc?.created_at ?? 0)
);
const screenshotDoc = rows[0].doc as any;
// continue as before
}Alternatively, fix the query to order by created_at DESC and add a code comment asserting that contract. Reply with "@CharlieHelps yes please" if you'd like me to add the sort (and tests) here.
| const screenshotFile = | ||
| await screenshotDoc._files.screenshot.file(); | ||
| const screenshotDataUrl = await new Promise<string>( | ||
| (resolve, reject) => { | ||
| const reader = new FileReader(); | ||
| reader.onload = () => resolve(reader.result as string); | ||
| reader.onerror = reject; | ||
| reader.readAsDataURL(screenshotFile); | ||
| }, | ||
| ); | ||
|
|
||
| // Use shared catalog utility function | ||
| await addCatalogScreenshotStandalone( | ||
| userId, | ||
| sessionId, | ||
| screenshotDataUrl, | ||
| transformedCode, | ||
| ); |
There was a problem hiding this comment.
Converting the File to a base64 data URL via FileReader is unnecessary and amplifies memory usage (~33% overhead). If addCatalogScreenshotStandalone can accept a File/Blob, passing it directly avoids redundant encoding/decoding and is faster.
Suggestion
Refactor addCatalogScreenshotStandalone to accept a File (and compute any CID internally). Then pass the file directly:
// Change util signature to e.g. addCatalogScreenshotStandalone(userId, sessionId, { screenshot: File, sourceCode: string })
await addCatalogScreenshotStandalone(
userId,
sessionId,
{ screenshot: screenshotFile, sourceCode: transformedCode },
);If changing the util is undesirable, at least compute the hash on the ArrayBuffer and avoid making a data URL. Reply with "@CharlieHelps yes please" if you want me to update the util and callers accordingly (including tests).
| // Mock fetch for screenshot data | ||
| global.fetch = vi.fn(() => | ||
| Promise.resolve({ | ||
| blob: () => | ||
| Promise.resolve(new Blob(["screenshot"], { type: "image/png" })), | ||
| }), | ||
| ) as any; |
There was a problem hiding this comment.
This test stubs global.fetch but never restores it, which can leak into subsequent tests and cause order-dependent failures. Use vi.spyOn(global, 'fetch') or vi.stubGlobal('fetch', ...) with cleanup in afterEach.
Suggestion
Add setup/teardown to isolate the fetch mock per test file:
let fetchSpy: any;
beforeEach(() => {
fetchSpy = vi.spyOn(global as any, "fetch").mockResolvedValue({
blob: async () => new Blob(["screenshot"], { type: "image/png" }),
} as any);
});
afterEach(() => {
fetchSpy?.mockRestore();
});Alternatively, use vi.stubGlobal('fetch', ...) and call vi.unstubAllGlobals() in afterEach. Reply with "@CharlieHelps yes please" if you'd like me to add this isolation to the test file.
…lling - Fix 'Illegal invocation' error when using fetch in pollForAuthToken - Bind fetch to window context to preserve proper 'this' binding - Affects both pollForAuthToken and extendToken functions 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Summary
Implements comprehensive cloud synchronization for the vibe catalog system, extending the local-first architecture to support seamless cloud sync capabilities.
This PR supersedes #256 with a complete cloud-enabled implementation that maintains backward compatibility while adding robust sync functionality.
Key Features
• Cloud Sync Integration: Full
toCloud()integration for authenticated users• Dual-Mode Operation: Works locally for anonymous users, syncs for authenticated users
• Enhanced Catalog System: Extends #256's catalog with screenshot storage and source code preservation
• User Settings Sync: Synchronized user preferences across devices
• Catalog Screenshot Storage: Persists app screenshots with Uint8Array optimization
• Content ID Generation: CID-based deduplication system for efficient storage
Architecture Changes
• useUserSettings hook: Manages sync preferences and user configuration
• Enhanced useCatalog: Cloud-aware catalog with screenshot/source storage
• Auth Integration: useAuth integration throughout component tree
• Sync Preference: localStorage-based sync toggle with real-time updates
Cloud Sync Components
• VibeCatalog: Enhanced to show sync status and cloud connectivity
• Settings Route: Cloud sync toggle and user preference management
• Mine Route: Cloud-aware vibe listing with sync indicators
• Publishing Utils: Screenshot and source code sync on app publish
Technical Implementation
• Fireproof toCloud(): Seamless database sync when authenticated
• Graceful Degradation: Full offline functionality for anonymous users
• Type Safety: Complete TypeScript integration with proper error handling
• Test Coverage: Updated test suite with cloud sync mocking
Migration from #256
This implementation includes all features from #256 plus:
Closes #256 in favor of this more complete implementation.
🤖 Generated with Claude Code