Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(datastore): Ignore field mismatch errors #8694

Merged
merged 9 commits into from
Jul 30, 2024

Conversation

bhshkh
Copy link
Contributor

@bhshkh bhshkh commented Oct 12, 2023

ErrFieldMismatch is returned when a field is to be loaded into a different type than the one it was stored from, or when a field is missing or unexported in the destination struct.

This PR allows user to suppress such errors. Some scenarios in which this would be useful are listed in #913

Points to note

  1. Suppressing the error does not mean that Datastore will not fetch missing field. It just means that the field was still fetched from Datastore service but not loaded in user's struct.
  2. When writing the data back to Datastore, it will be overwritten. E.g.
    If entity written to Datastore is {X: 1, Y:2} and it is loaded into type NewStruct struct{X int}, then {X:1} is returned to the user. Now, if user modifies this X = 5 and writes it back to Datastore, there will be no Y field left for this entity in Datastore.

Fixes: #913

@bhshkh bhshkh requested review from a team as code owners October 12, 2023 08:53
@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: datastore Issues related to the Datastore API. labels Oct 12, 2023
@bhshkh bhshkh requested a review from enocom October 12, 2023 08:53
// IgnoreFieldMismatch allows ignoring ErrFieldMismatch error while
// reading or querying data.
// WARNING: Ignoring ErrFieldMismatch can cause data loss
func (c *Client) IgnoreFieldMismatch() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider expressing this as a client specific ClientIOption. Here is an example from storage:

func WithXMLReads() option.ClientOption {
return &withReadAPI{useJSON: false}
}
type withReadAPI struct {
internaloption.EmbeddableAdapter
useJSON bool
}
func (w *withReadAPI) ApplyStorageOpt(c *storageConfig) {
c.useJSONforReads = w.useJSON
c.readAPIWasSet = true
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the PR

@product-auto-label product-auto-label bot added the stale: old Pull request is old and needs attention. label Nov 11, 2023
@product-auto-label product-auto-label bot added stale: extraold Pull request is critically old and needs prioritization. and removed stale: old Pull request is old and needs attention. labels Dec 11, 2023
@bhshkh bhshkh requested a review from a team as a code owner April 16, 2024 19:04
@bhshkh bhshkh requested a review from gkevinzheng July 22, 2024 23:04
@bhshkh bhshkh enabled auto-merge (squash) July 23, 2024 19:38
// WithIgnoreFieldMismatch allows ignoring ErrFieldMismatch error while
// reading or querying data.
// WARNING: Ignoring ErrFieldMismatch can cause data loss
func WithIgnoreFieldMismatch() option.ClientOption {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Is it better to call this type/field IgnoreFieldMismatch without the "with"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@enocom enocom removed their request for review July 24, 2024 18:36
@codyoss
Copy link
Member

codyoss commented Jul 24, 2024

@bhshkh do we still want this PR?

@bhshkh
Copy link
Contributor Author

bhshkh commented Jul 26, 2024

@bhshkh do we still want this PR?

Yes

@bhshkh bhshkh merged commit 6625d12 into googleapis:main Jul 30, 2024
8 checks passed
@bhshkh bhshkh deleted the fix/issue-913 branch July 30, 2024 20:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: datastore Issues related to the Datastore API. size: m Pull request size is medium. stale: extraold Pull request is critically old and needs prioritization.
Projects
None yet
3 participants