Skip to content

Conversation

dirkmc
Copy link
Contributor

@dirkmc dirkmc commented Nov 5, 2021

Note: This doesn't compile, as some of the changes are made in the other open PRs: #20 #21 #22 #23
I separated out the PRs to try to make it easier to review them instead of having a single giant PR.

@dirkmc dirkmc mentioned this pull request Nov 5, 2021
Copy link
Collaborator

@aarshkshah1992 aarshkshah1992 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finished reviewing the db part. Moving on to the storage provider.

Offset INT,
Length INT,
Checkpoint TEXT,
Error TEXT,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dirk, my experience has been that persisting errors to DB never works out well, especially when there's resumption involved. Can we get rid of this and add it back only if we absolutely need it ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a checkpoint value called "Complete" but we don't have a way to distinguish between "Completed successfully" and "Completed with error". I'm open to suggestions here.

}

// TODO: use a generic data transfer descriptor instead of a URL
u := url.URL{}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this should change to the opaque transfer descriptor byte token.

@dirkmc dirkmc merged commit 8db044a into main Nov 10, 2021
@dirkmc dirkmc deleted the feat/db-obj-translation branch November 10, 2021 07:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants