-
Notifications
You must be signed in to change notification settings - Fork 15
Introduce database import and export protocol messages #224
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
message Res { | ||
DatabaseReplicas database = 1; | ||
} | ||
} | ||
|
||
message Import { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've placed all migration-related messages in a single file, so it's "encapsulated". However, this introduces an additional layer of optional client
and server
in Rust while unpacking this message, so it's a little irritating. Not sure if it's worth it or not. I like this design more, I guess.
|
||
// This is an emulation of the google ErrorDetails message. Generally, ErrorDetails are submitted via the GRPC error | ||
// mechanism, but a manual error sending is required in streams | ||
message Error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was planning to reuse this error, but I understood that I don't need non-terminal errors in my protocol. However, I think that it's reasonable to generalize error messages for the whole TypeDB protocol.
Release notes: usage and product changes
Add
migration
protocol messages for usage in database import and database export operations:database_export
stream from a TypeDB server to export a specific database, similar to TypeDB 2.x;databases_import
stream between a client and a server to import an exported 2.x/3.x TypeDB database into a TypeDB 3.x server from a client.The format of migration items used for these operations is an extended version of TypeDB 2.x's migration items, so it is backward compatible with 2.x database files. Important: it's not intended to import 3.x databases into 2.x servers.
Implementation
Add
Migration { Item }
message. The format is an extended version of the 2.x protocol, so it contains "outdated" fields for its compatibility with old databases.Add
Migration { Export }
message. This operation consists of a single clientReq { database }
and multiple streamed server responses:Done
message to signal that the server is ready to close the stream without errors. It can be substituted by a silent stream closure, but I preferred explicitness here.Add
Migration { Import }
message. This operation consists of a stream of client requests:Done
message to signal that the client is finished without errors, and the server can perform the final validation. ThisDone
message is required and cannot be removed because the client has to check whether there were finalization errors or not.and a stream of server responses (actually, there is either a single
Done
or a single error, but the stream is needed to return errors at any stage of the communication).