Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize schema check storage of composite sdl and supergraph #2603

Closed
n1ru4l opened this issue Jul 14, 2023 · 4 comments · Fixed by #2992
Closed

Optimize schema check storage of composite sdl and supergraph #2603

n1ru4l opened this issue Jul 14, 2023 · 4 comments · Fixed by #2992
Labels
good first issue Good for newcomers help wanted Extra attention is needed

Comments

@n1ru4l
Copy link
Collaborator

n1ru4l commented Jul 14, 2023

Right now we store the composite SDL and supergraph on every record within the schema_checks table.

When a schema check runs that has no active changes within the schema, we store a lot of redundant data within our database. E.g. bots like renovate cause a lot of these runs. Some GraphQL schemas are quite huge.

A lot of schema_checks are linked to the schema_versions table, which also stores the composite SDL and supergraph.
In case a schema check has no active changes (read as the same checksum), we could omit to store the composite SDL and supergraph within the schema_checks table and instead retrieve them from the schema_versions table when consuming the field via the GraphQL API.

A further optimization could be to even store schemas and their checksums in a completely new table, e.g. checksum_schemas, that is a checksum to SDL mapping. That way we could furthermore reduce the redundancy of large SDL strings.

Note: If we do the latter, we also need to make sure the cleanup happens when the checksum entry is no longer referenced by any schema_checks entry, which would furthermore complicate the database schema.

@n1ru4l n1ru4l added help wanted Extra attention is needed good first issue Good for newcomers labels Jul 17, 2023
@andriihrachov
Copy link
Contributor

would like to take it

@n1ru4l
Copy link
Collaborator Author

n1ru4l commented Jul 21, 2023

Hey @andriihrachov, feel free to send a pull request! I would be happy to review it ☺️

@kamilkisiela
Copy link
Collaborator

@andriihrachov before touching code, do you mind sharing with us when do you plan to work on it and how you want to implement it?

@andriihrachov
Copy link
Contributor

andriihrachov commented Aug 5, 2023

@kamilkisiela sorry I missed your message, but opened a PR with first steps (migration, recording schema SDL and checksum in separate table, etc,) - it's small enough to kill if I took wrong way, pls feedback :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants