added a strategy to manage unknown fields during deserialization #660

mcerina · 2015-06-19T09:50:04Z

See issue #188
While I understand the design choice of ignoring unknown fields during deserialization by default, I definitely think that there are use cases in which throwing an exception, or at least being able to log something is preferred.

googlebot · 2015-06-19T09:50:05Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project, in which case you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed, please reply here (e.g. I signed it!) and we'll verify. Thanks.

If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
If you signed the CLA as a corporation, please let us know the company's name.

mcerina · 2015-06-19T09:52:49Z

I signed it!

googlebot · 2015-06-19T09:52:51Z

CLAs look good, thanks!

inder123 · 2015-06-20T07:30:54Z

Appreciate you taking the time to write this. However, I think the usecase for this is too infrequent to warrant inclusion in standard Gson. Also, the approach is too narrow (for just unknown properties). We have had many requests to provide ability to intercept and validate a Gson generated instance. We would like to see a simple elegant solution to the broader issue.

itERRatOR · 2016-02-15T03:35:09Z

It is quite frequent. And I would reject solutions as not good enough only when there are other better alternatives presented. But it doesn't look like @inder123 is going to come up with any idea regarding this at all.

ghuh · 2016-03-14T06:46:12Z

I would agree that this is a common problem. For instance, a client sends a JSON object with a field name misspelled. If it is valid for that field to be omitted, you would never know that there was an issue. It would simply get silently ignored.

horalstvo · 2016-09-22T14:13:52Z

I would like to have a choice to decide whether GSON should ignore unknown fields or should reject such requests. Sadly, I'm switching now to Jackson to get that.

awbgithub · 2016-10-26T15:49:38Z

@inder123 I see a comment from you from over a year ago where you mention many requests like this one to support validation ... can you point me to a discussion where you are working on a more general and elegant solution? If not, please reconsider this PR as it is a simple and elegant approach to the specific problem of receiving unexpected fields in the JSON.

mrkb80 · 2016-11-08T23:08:56Z

I agree that the user should have the choice.

vertex-github · 2016-11-15T13:48:38Z

"I think the usecase for this is too infrequent to warrant inclusion in standard Gson"

We would like this use case - and from the looks of the comments so do many others. Jackson has this feature and it seems fairly frequently used (do a search on tutorials and examples).

"Also, the approach is too narrow (for just unknown properties). We have had many requests to provide ability to intercept and validate a Gson generated instance. We would like to see a simple elegant solution to the broader issue."

That maybe the case, but we came here looking for this exact feature, and only this feature. We want a way to be be able to detect/intercept additional fields in the JSON so that we can report/handle fields that we were not expecting. We spent quite a while trying to shoe-horn this feature in, and there are no hooks available to us to provide this functionality without duplicating all of the core type adapter factory and its supporting classes - that seems a tad wasteful.

We took a quick review of the PR code, and it seems very simple, logical and well thought out. Keep the default behavior as is, and existing users wont have to change any code. Provide the builder option and everyone is happy/happier. :-)

+1 for taking this PR.

inder123 · 2016-11-27T23:22:37Z

Throwing Exception on encountering an unnknown fields seems weird. JSON is an extensible and flexible format: a client shouldn't be broken by adding a new property in JSON.

What about missing fields? Wouldn't you want a similar action?

A better way is to use https://github.com/google/gson/blob/master/extras/src/main/java/com/google/gson/typeadapters/PostConstructAdapterFactory.java

awbgithub · 2016-11-28T03:26:14Z

@inder123 As @mcerina mentions in the first comment above, the UnknownFieldHandlingStrategy is extensible and could be used to just log unknown fields if throwing an exception doesn't suit your needs.

I looked at PostConstructAdapterFactoryTest and see that your test throws an exception when the sandwich is too cheesy ... and I can see how that could also be used to take some action if a "required" field was missing (i.e. still null during post construct validation) ... but I can't figure out how to use the PostConstructAdapterFactory to catch when an unknown field was present in the original JSON. Can you provide a code example of how the PostConstructAdapterFactory could be used to solve the OP's issue about handling unknown fields?

romankulyk1 · 2016-12-16T10:33:02Z

Can you deserialize the JSON object to get your Java object, then serialize your Java object back to JSON and see if you get the same result (compare the original JSON object to the converted Java JSON)! if not...extra fields! Haven't tried, yet, but this is just a thought.

adam42a · 2019-03-19T10:59:07Z

I am looking for this feature right now. Is the reason for its absence just adherence to dogma?

rubearen · 2020-06-02T09:29:46Z

Still no solution for this problem.

Marcono1234

I am not a maintainer of this project and they will sadly probably not change their mind.
Though maybe this review is helpful nonetheless in case you or someone else is using this code in a fork.
The review comments are mostly documentation related.

Marcono1234 · 2020-08-22T15:39:14Z

gson/src/main/java/com/google/gson/Gson.java

   *   consideration for serialization and deserialization. You can change this behavior through
   *   {@link GsonBuilder#excludeFieldsWithModifiers(int...)}.</li>
+   *   <li>By default, during deserialization Gson silently ignores fields that are present in the
+   *   incoming Json but not in the Java classes. You can change this policy through


Suggested change

* incoming Json but not in the Java classes. You can change this policy through

* incoming JSON but not in the Java classes. You can change this policy through

Marcono1234 · 2020-08-22T15:40:47Z

gson/src/main/java/com/google/gson/GsonBuilder.java

+  /**
+   * Configures Gson to apply a specific unknown field strategy during deserialization.
+   *
+   * @param unknownFieldHandlingStrategy the actual naming strategy to apply to the fields


actual naming strategy

This appears to be wrong

Marcono1234 · 2020-08-22T15:41:21Z

gson/src/main/java/com/google/gson/UnknownFieldHandlingPolicy.java

+
+/**
+ * An enumeration that defines a few standard handling strategies for fields
+ * present in the incoming Json but not in the Java classes.


Suggested change

* present in the incoming Json but not in the Java classes.

* present in the incoming JSON but not in the Java classes.

Marcono1234 · 2020-08-22T15:42:16Z

gson/src/main/java/com/google/gson/UnknownFieldHandlingPolicy.java

+ * An enumeration that defines a few standard handling strategies for fields
+ * present in the incoming Json but not in the Java classes.
+ * This enumeration should be used in conjunction with {@link com.google.gson.GsonBuilder}
+ * to configure a {@link com.google.gson.Gson} instance


Suggested change

* to configure a {@link com.google.gson.Gson} instance

* to configure a {@link com.google.gson.Gson} instance.

Marcono1234 · 2020-08-22T15:44:55Z

gson/src/main/java/com/google/gson/UnknownFieldHandlingPolicy.java

+   */
+  THROW_EXCEPTION() {
+    public void handleUnknownField(JsonReader in, Object instance, String name) throws IOException {
+        String msg = String.format("Unrecognized field \"%s\" (Class %s)", name, instance.getClass().getName());


Maybe use String.format(Locale, String, Object...), e.g. String.format(Locale.ENGLISH, ...) to make the message independent of the system locale.

Marcono1234 · 2020-08-22T15:46:58Z

gson/src/main/java/com/google/gson/UnknownFieldHandlingStrategy.java

+
+/**
+ * A mechanism for providing custom unknown property handling in Gson during deserialization.
+ * This allows to provide custom behavior to handle fields present in the incoming Json which


Suggested change

* This allows to provide custom behavior to handle fields present in the incoming Json which

* This allows to provide custom behavior to handle fields present in the incoming JSON which

Marcono1234 · 2020-08-22T15:47:21Z

gson/src/main/java/com/google/gson/UnknownFieldHandlingStrategy.java

+public interface UnknownFieldHandlingStrategy {
+
+  /**
+   * Handles a field present in the incoming Json but not in the Java class


Suggested change

* Handles a field present in the incoming Json but not in the Java class

* Handles a field present in the incoming JSON but not in the Java class

Marcono1234 · 2020-08-22T15:48:29Z

gson/src/main/java/com/google/gson/UnknownFieldHandlingStrategy.java

+  /**
+   * Handles a field present in the incoming Json but not in the Java class
+   *
+   * @param in the {@link JsonReader} reading the incoming Json


Links in javadoc should be used sparingly and here the generated documentation already links to the type so there is no need to have an additional link here as well.

Suggested change

* @param in the {@link JsonReader} reading the incoming Json

* @param in the {@code JsonReader} reading the incoming JSON

Marcono1234 · 2020-08-22T15:49:23Z

gson/src/main/java/com/google/gson/UnknownFieldHandlingStrategy.java

+   *
+   * @param in the {@link JsonReader} reading the incoming Json
+   * @param instance the instance of the Java class we are trying to deserialize to
+   * @param name the name of the field in the incoming Json


Suggested change

* @param name the name of the field in the incoming Json

* @param name the name of the unknown field in the incoming JSON

Marcono1234 · 2020-08-22T16:02:27Z

gson/src/test/java/com/google/gson/functional/UnknownFieldHandlingTest.java

+    Gson gson = new GsonBuilder().setUnknownFieldHandlingStrategy(THROW_EXCEPTION).create();
+    try {
+      gson.fromJson(JSON, ClassWithTwoFields.class);
+      Assert.fail("A JsonParseException was expected");


You should be able to use TestCase's fail:

Suggested change

Assert.fail("A JsonParseException was expected");

fail("A JsonParseException was expected");

(and then also remove the import)

added a strategy to manage unknown fields during deserialization

471a022

mcerina mentioned this pull request Jun 19, 2015

Support handlers for unknown properties, useful for error handling #188

Open

fixed javadoc

7ae7fbe

inder123 closed this Jun 20, 2015

Marcono1234 reviewed Aug 22, 2020

View reviewed changes

Marcono1234 mentioned this pull request Mar 26, 2023

Add strategies for unknown and missing fields #2358

Draft

9 tasks

	* incoming Json but not in the Java classes. You can change this policy through
	* incoming JSON but not in the Java classes. You can change this policy through

	* present in the incoming Json but not in the Java classes.
	* present in the incoming JSON but not in the Java classes.

	* to configure a {@link com.google.gson.Gson} instance
	* to configure a {@link com.google.gson.Gson} instance.

	* This allows to provide custom behavior to handle fields present in the incoming Json which
	* This allows to provide custom behavior to handle fields present in the incoming JSON which

	* Handles a field present in the incoming Json but not in the Java class
	* Handles a field present in the incoming JSON but not in the Java class

	* @param in the {@link JsonReader} reading the incoming Json
	* @param in the {@code JsonReader} reading the incoming JSON

	* @param name the name of the field in the incoming Json
	* @param name the name of the unknown field in the incoming JSON

	Assert.fail("A JsonParseException was expected");
	fail("A JsonParseException was expected");

added a strategy to manage unknown fields during deserialization #660

added a strategy to manage unknown fields during deserialization #660

Uh oh!

Conversation

mcerina commented Jun 19, 2015

Uh oh!

googlebot commented Jun 19, 2015

Uh oh!

mcerina commented Jun 19, 2015

Uh oh!

googlebot commented Jun 19, 2015

Uh oh!

inder123 commented Jun 20, 2015

Uh oh!

itERRatOR commented Feb 15, 2016

Uh oh!

ghuh commented Mar 14, 2016

Uh oh!

horalstvo commented Sep 22, 2016

Uh oh!

awbgithub commented Oct 26, 2016

Uh oh!

mrkb80 commented Nov 8, 2016

Uh oh!

vertex-github commented Nov 15, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

inder123 commented Nov 27, 2016

Uh oh!

awbgithub commented Nov 28, 2016

Uh oh!

romankulyk1 commented Dec 16, 2016

Uh oh!

adam42a commented Mar 19, 2019

Uh oh!

rubearen commented Jun 2, 2020

Uh oh!

Marcono1234 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

13 participants

vertex-github commented Nov 15, 2016 •

edited

Loading