Document how to use Wasp with ChatGPT embeddings (pgvector)

[Martinsos]

We should document this! Probably best to make a guide of it in our docs.
@maksim36ua managed to get this working, so we should probably start with extracting as much info from him as we can :D and we should replicate our own project that uses it (template or just example wasp app) and write guide based on that also and then we have a nice thing that people can look at and to their own projects.

We might also want to look into recommending users how to set up such postgres database in production, since finding a hosting service where you can set up pgvector is also a bit of a challenge potentially.

@infomiho and @vincanger I believe you have the most experience here.

[Martinsos]

Here is what @maksim36ua said:

I'm using https://github.com/pgvector/pgvector to add vector embeddings to my Postgres database .
You'll need to install is as an extension to Postgres and create a vector field. Wasp's Prisma won't be able to interact with that field so you'll need to use Prisma's queryRaw for all your CRUDs for that field.
Or install pg package to interact with your vector embeddings, either way is ok .

[Lezzz]

This is what @maksim36ua offered me as aditional help while we discussed this use case in private, should be quite useful:

Here's the function that can get you an embedding from OpenAI API:
https://github.com/maksim36ua/feedback_hub/blob/main/src/server/services/apiService.ts#LL34C23-L34C23

Here's how you can save your vector into the database:
https://github.com/maksim36ua/feedback_hub/blob/main/src/server/actions/saveReducedFeedback.ts#L9

This line is how you can perform SQL queries with the vectors in the database (eg find the distance between points):
https://github.com/maksim36ua/feedback_hub/blob/main/src/server/actions/getFeedbackSimilarities.ts#L17

[Martinsos]

Awesome, thanks @Lezzz !

Here is also corresponding Discord convo: https://discord.com/channels/686873244791210014/1109254973549727784/1110233646968872990

[vincanger]

Ok. After two days of messing around with embeddings, Prisma, and Supabase, this is what I've found:

1. vector data types are not currently supported by Prisma, so the Unsupported("vector (1536)") type for OpenAI embeddings must be used, as openAI always returns a vector with 1536 dimensions. This is fine, but results in us having to import the prismaClient and use raw Prisma Queries with prismaClient.$queryRaw instead of prisma's built-in methods

2. Providers such as Supabase support the pgvector extension for vector similarity search in Postgres. I was able to initialize the DB with Unsupported("vector (1536)") type, and perform actions on it. It wasn't until I made changes to an entity and needed to migrate that I ran into issues. The solution I found in order to keep migration history consistent, was to make the prisma schema aware of the extension:

datasource db {
  provider = "postgresql"
  url      = env("DATABASE_URL")
  shadowDatabaseUrl = env("SHADOW_DATABASE_URL")
  extensions = [pgvector(map: "vector", schema: "extensions")]
}

generator client {
  provider = "prisma-client-js"
  output = env("PRISMA_CLIENT_OUTPUT_DIR")
  previewFeatures = ["postgresqlExtensions"]
}

Side note: if developing with a Supabase DB in the cloud, we need to create a second "shadow" database which Prisma creates and drops each time we migrate dev in order to compare changes and detect drift.

3. With this in mind, and until we find a way to allow the user to configure extensions, as @shayneczyzewski mentioned, it may be best to perform all embeddings operations on a dedicated vector database such as Pinecone using their SDK

[infomiho]

We have an example app that shows how to do this https://github.com/wasp-lang/ask-the-documents