strictness2 fixes & improvements

Troubleshooting.md

@@ -147,9 +147,12 @@ To spot syntax errors in the JSON data easily you can open it in an editor with
 
 **Reason:** Due to legacy reasons Gson performs parsing by default in lenient mode
 
-**Solution:** See [`Gson` class documentation](https://www.javadoc.io/doc/com.google.code.gson/gson/latest/com.google.gson/com/google/gson/Gson.html#default-lenient) section "Lenient JSON handling"
-
-Note: Even in non-lenient mode Gson deviates slightly from the JSON specification, see [`JsonReader.setStrictness`](https://www.javadoc.io/doc/com.google.code.gson/gson/latest/com.google.gson/com/google/gson/stream/JsonReader.html#setStrictness(Strictness)) for more details.
+**Solution:** If you are using Gson 2.11.0 or newer, call [`GsonBuilder.setStrictness`](https://www.javadoc.io/doc/com.google.code.gson/gson/latest/com.google.gson/com/google/gson/GsonBuilder.html#setStrictness(com.google.gson.Strictness)),
+[`JsonReader.setStrictness`](https://www.javadoc.io/doc/com.google.code.gson/gson/latest/com.google.gson/com/google/gson/stream/JsonReader.html#setStrictness(com.google.gson.Strictness))
+and [`JsonWriter.setStrictness`](https://www.javadoc.io/doc/com.google.code.gson/gson/latest/com.google.gson/com/google/gson/stream/JsonWriter.html#setStrictness(com.google.gson.Strictness))
+with `Strictness.STRICT` to overwrite the default lenient behavior of `Gson` and make these classes strictly adhere to the JSON specification.
+Otherwise if you are using an older Gson version, see the [`Gson` class documentation](https://www.javadoc.io/doc/com.google.code.gson/gson/latest/com.google.gson/com/google/gson/Gson.html#default-lenient)
+section "JSON Strictness handling" for alternative solutions.
 
 ## <a id="unexpected-json-structure"></a> `IllegalStateException`: "Expected ... but was ..."
 
@@ -313,6 +316,8 @@ Note: For newer Gson versions these rules might be applied automatically; make s
 
 **Symptom:** A `JsonIOException` with the message 'Abstract classes can't be instantiated!' is thrown; the class mentioned in the exception message is not actually `abstract` in your source code, and you are using the code shrinking tool R8 (Android app builds normally have this configured by default).
 
+Note: If the class which you are trying to deserialize is actually abstract, then this exception is probably unrelated to R8 and you will have to implement a custom [`InstanceCreator`](https://javadoc.io/doc/com.google.code.gson/gson/latest/com.google.gson/com/google/gson/InstanceCreator.html) or [`TypeAdapter`](https://javadoc.io/doc/com.google.code.gson/gson/latest/com.google.gson/com/google/gson/TypeAdapter.html) which creates an instance of a non-abstract subclass of the class.
+
 **Reason:** The code shrinking tool R8 performs optimizations where it removes the no-args constructor from a class and makes the class `abstract`. Due to this Gson cannot create an instance of the class.
 
 **Solution:** Make sure the class has a no-args constructor, then adjust your R8 configuration file to keep the constructor of the class. For example:
@@ -324,6 +329,10 @@ Note: For newer Gson versions these rules might be applied automatically; make s
 }
 ```
 
+You can also use `<init>(...);` to keep all constructors of that class, but then you might actually rely on `sun.misc.Unsafe` on both JDK and Android to create classes without no-args constructor, see [`GsonBuilder.disableJdkUnsafe()`](https://javadoc.io/doc/com.google.code.gson/gson/latest/com.google.gson/com/google/gson/GsonBuilder.html#disableJdkUnsafe()) for more information.
+
 For Android you can add this rule to the `proguard-rules.pro` file, see also the [Android documentation](https://developer.android.com/build/shrink-code#keep-code). In case the class name in the exception message is obfuscated, see the Android documentation about [retracing](https://developer.android.com/build/shrink-code#retracing).
 
-Note: If the class which you are trying to deserialize is actually abstract, then this exception is probably unrelated to R8 and you will have to implement a custom [`InstanceCreator`](https://javadoc.io/doc/com.google.code.gson/gson/latest/com.google.gson/com/google/gson/InstanceCreator.html) or [`TypeAdapter`](https://javadoc.io/doc/com.google.code.gson/gson/latest/com.google.gson/com/google/gson/TypeAdapter.html) which creates an instance of a non-abstract subclass of the class.
+For Android you can alternatively use the [`@Keep` annotation](https://developer.android.com/studio/write/annotations#keep) on the class or constructor you want to keep. That might be easier than having to maintain a custom R8 configuration.
+
+Note that the latest Gson versions (> 2.10.1) specify a default R8 configuration. If your class is a top-level class or is `static`, has a no-args constructor and its fields are annotated with Gson's [`@SerializedName`](https://www.javadoc.io/doc/com.google.code.gson/gson/latest/com.google.gson/com/google/gson/annotations/SerializedName.html), you might not have to perform any additional R8 configuration.

examples/android-proguard-example/README.md

@@ -12,6 +12,9 @@ details on how ProGuard can be configured.
 The R8 code shrinker uses the same rule format as ProGuard, but there are differences between these two
 tools. Have a look at R8's Compatibility FAQ, and especially at the [Gson section](https://r8.googlesource.com/r8/+/refs/heads/main/compatibility-faq.md#gson).
 
-Note that newer Gson versions apply some of the rules shown in `proguard.cfg` automatically by default,
+Note that the latest Gson versions (> 2.10.1) apply some of the rules shown in `proguard.cfg` automatically by default,
 see the file [`gson/META-INF/proguard/gson.pro`](/gson/src/main/resources/META-INF/proguard/gson.pro) for
-the Gson version you are using.
+the Gson version you are using. In general if your classes are top-level classes or are `static`, have a no-args constructor and their fields are annotated with Gson's [`@SerializedName`](https://www.javadoc.io/doc/com.google.code.gson/gson/latest/com.google.gson/com/google/gson/annotations/SerializedName.html), you might not have to perform any additional ProGuard or R8 configuration.
+
+An alternative to writing custom keep rules for your classes in the ProGuard configuration can be to use
+Android's [`@Keep` annotation](https://developer.android.com/studio/write/annotations#keep).

gson/pom.xml

@@ -62,7 +62,7 @@
     <dependency>
       <groupId>com.google.guava</groupId>
       <artifactId>guava-testlib</artifactId>
-      <version>32.0.1-jre</version>
+      <version>32.1.1-jre</version>
       <scope>test</scope>
     </dependency>
   </dependencies>

gson/src/main/java/com/google/gson/Gson.java

@@ -136,6 +136,10 @@
  *       to make sure there is no trailing data
  * </ol>
  *
+ * Note that the {@code JsonReader} created this way is only 'legacy strict', it mostly adheres
+ * to the JSON specification but allows small deviations. See {@link JsonReader#setStrictness(Strictness)}
+ * for details.
+ *
  * @see TypeToken
  *
  * @author Inderjeet Singh
@@ -915,7 +919,7 @@ public void toJson(JsonElement jsonElement, Appendable writer) throws JsonIOExce
    *   <li>{@link GsonBuilder#serializeNulls()}</li>
    *   <li>{@link GsonBuilder#setStrictness(Strictness)}. If no
    *   {@linkplain GsonBuilder#setStrictness(Strictness) explicit strictness has been set} the created
-   *   writer will have a strictness of {@link Strictness#LEGACY_STRICT}. Otherwise, this strictness of
+   *   writer will have a strictness of {@link Strictness#LEGACY_STRICT}. Otherwise, the strictness of
    *   the {@code Gson} instance will be used for the created writer.</li>
    *   <li>{@link GsonBuilder#setPrettyPrinting()}</li>
    *   <li>{@link GsonBuilder#setFormattingStyle(FormattingStyle)}</li>
@@ -940,7 +944,7 @@ public JsonWriter newJsonWriter(Writer writer) throws IOException {
    * <ul>
    *   <li>{@link GsonBuilder#setStrictness(Strictness)}. If no
    *   {@linkplain GsonBuilder#setStrictness(Strictness) explicit strictness has been set} the created
-   *   reader will have a strictness of {@link Strictness#LEGACY_STRICT}. Otherwise, this strictness of
+   *   reader will have a strictness of {@link Strictness#LEGACY_STRICT}. Otherwise, the strictness of
    *   the {@code Gson} instance will be used for the created reader.</li>
    * </ul>
    */

gson/src/main/java/com/google/gson/GsonBuilder.java

@@ -58,8 +58,7 @@
  * use {@code new Gson()}. {@code GsonBuilder} is best used by creating it, and then invoking its
  * various configuration methods, and finally calling create.</p>
  *
- * <p>The following is an example shows how to use the {@code GsonBuilder} to construct a Gson
- * instance:
+ * <p>The following example shows how to use the {@code GsonBuilder} to construct a Gson instance:
  *
  * <pre>
  * Gson gson = new GsonBuilder()
@@ -125,7 +124,7 @@ public GsonBuilder() {
    * Constructs a GsonBuilder instance from a Gson instance. The newly constructed GsonBuilder
    * has the same configuration as the previously built Gson instance.
    *
-   * @param gson the gson instance whose configuration should by applied to a new GsonBuilder.
+   * @param gson the gson instance whose configuration should be applied to a new GsonBuilder.
    */
   GsonBuilder(Gson gson) {
     this.excluder = gson.excluder;
@@ -279,7 +278,7 @@ public GsonBuilder serializeNulls() {
    * {"x":2,"y":3}}.
    *
    * <p>Given the assumption above, a {@code Map<Point, String>} will be
-   * serialize as an array of arrays (can be viewed as an entry set of pairs).
+   * serialized as an array of arrays (can be viewed as an entry set of pairs).
    *
    * <p>Below is an example of serializing complex types as JSON arrays:
    * <pre> {@code
@@ -601,7 +600,7 @@ public GsonBuilder setDateFormat(String pattern) {
   }
 
   /**
-   * Configures Gson to to serialize {@code Date} objects according to the style value provided.
+   * Configures Gson to serialize {@code Date} objects according to the style value provided.
    * You can call this method or {@link #setDateFormat(String)} multiple times, but only the last
    * invocation will be used to decide the serialization format.
    *
@@ -622,7 +621,7 @@ public GsonBuilder setDateFormat(int style) {
   }
 
   /**
-   * Configures Gson to to serialize {@code Date} objects according to the style value provided.
+   * Configures Gson to serialize {@code Date} objects according to the style value provided.
    * You can call this method or {@link #setDateFormat(String)} multiple times, but only the last
    * invocation will be used to decide the serialization format.
    *

gson/src/main/java/com/google/gson/internal/ConstructorConstructor.java

@@ -76,14 +76,15 @@ static String checkInstantiable(Class<?> c) {
     if (Modifier.isAbstract(modifiers)) {
       // R8 performs aggressive optimizations where it removes the default constructor of a class
       // and makes the class `abstract`; check for that here explicitly
-      if (c.getDeclaredConstructors().length == 0) {
-        return "Abstract classes can't be instantiated! Adjust the R8 configuration or register"
-            + " an InstanceCreator or a TypeAdapter for this type. Class name: " + c.getName()
-            + "\nSee " + TroubleshootingGuide.createUrl("r8-abstract-class");
-      }
-
-      return "Abstract classes can't be instantiated! Register an InstanceCreator"
-          + " or a TypeAdapter for this type. Class name: " + c.getName();
+      /*
+       * Note: Ideally should only show this R8-specific message when it is clear that R8 was
+       * used (e.g. when `c.getDeclaredConstructors().length == 0`), but on Android where this
+       * issue with R8 occurs most, R8 seems to keep some constructors for some reason while
+       * still making the class abstract
+       */
+      return "Abstract classes can't be instantiated! Adjust the R8 configuration or register"
+          + " an InstanceCreator or a TypeAdapter for this type. Class name: " + c.getName()
+          + "\nSee " + TroubleshootingGuide.createUrl("r8-abstract-class");
     }
     return null;
   }

gson/src/main/java/com/google/gson/reflect/TypeToken.java

@@ -337,9 +337,12 @@ public static <T> TypeToken<T> get(Class<T> type) {
    * As seen here the result is a {@code TypeToken<?>}; this method cannot provide any type safety,
    * and care must be taken to pass in the correct number of type arguments.
    *
+   * <p>If {@code rawType} is a non-generic class and no type arguments are provided, this method
+   * simply delegates to {@link #get(Class)} and creates a {@code TypeToken(Class)}.
+   *
    * @throws IllegalArgumentException
-   *   If {@code rawType} is not of type {@code Class}, if it is not a generic type, or if the
-   *   type arguments are invalid for the raw type
+   *   If {@code rawType} is not of type {@code Class}, or if the type arguments are invalid for
+   *   the raw type
    */
   public static TypeToken<?> getParameterized(Type rawType, Type... typeArguments) {
     Objects.requireNonNull(rawType);
@@ -354,10 +357,16 @@ public static TypeToken<?> getParameterized(Type rawType, Type... typeArguments)
     Class<?> rawClass = (Class<?>) rawType;
     TypeVariable<?>[] typeVariables = rawClass.getTypeParameters();
 
-    // Note: Does not check if owner type of rawType is generic because this factory method
-    // does not support specifying owner type
-    if (typeVariables.length == 0) {
-      throw new IllegalArgumentException(rawClass.getName() + " is not a generic type");
+    int expectedArgsCount = typeVariables.length;
+    int actualArgsCount = typeArguments.length;
+    if (actualArgsCount != expectedArgsCount) {
+      throw new IllegalArgumentException(rawClass.getName() + " requires " + expectedArgsCount +
+          " type arguments, but got " + actualArgsCount);
+    }
+
+    // For legacy reasons create a TypeToken(Class) if the type is not generic
+    if (typeArguments.length == 0) {
+      return get(rawClass);
     }
 
     // Check for this here to avoid misleading exception thrown by ParameterizedTypeImpl
@@ -366,13 +375,6 @@ public static TypeToken<?> getParameterized(Type rawType, Type... typeArguments)
           + " it requires specifying an owner type");
     }
 
-    int expectedArgsCount = typeVariables.length;
-    int actualArgsCount = typeArguments.length;
-    if (actualArgsCount != expectedArgsCount) {
-      throw new IllegalArgumentException(rawClass.getName() + " requires " + expectedArgsCount +
-          " type arguments, but got " + actualArgsCount);
-    }
-
     for (int i = 0; i < expectedArgsCount; i++) {
       Type typeArgument = Objects.requireNonNull(typeArguments[i], "Type argument must not be null");
       Class<?> rawTypeArgument = $Gson$Types.getRawType(typeArgument);

gson/src/main/java/com/google/gson/stream/JsonReader.java

@@ -182,7 +182,7 @@
  * <p>Prefixing JSON files with <code>")]}'\n"</code> makes them non-executable
  * by {@code <script>} tags, disarming the attack. Since the prefix is malformed
  * JSON, strict parsing fails when it is encountered. This class permits the
- * non-execute prefix when {@link #setStrictness(Strictness) lenient parsing} is
+ * non-execute prefix when {@linkplain #setStrictness(Strictness) lenient parsing} is
  * enabled.
  *
  * <p>Each {@code JsonReader} may be used to read a single JSON stream. Instances
@@ -295,7 +295,7 @@ public JsonReader(Reader in) {
   /**
    * Sets the strictness of this reader.
    *
-   * @deprecated Please use {@link JsonReader#setStrictness(Strictness)} instead.
+   * @deprecated Please use {@link #setStrictness(Strictness)} instead.
    * {@code JsonReader.setLenient(true)} should be replaced by {@code JsonReader.setStrictness(Strictness.LENIENT)}
    * and {@code JsonReader.setLenient(false)} should be replaced by {@code JsonReader.setStrictness(Strictness.LEGACY_STRICT)}.<br>
    * However, if you used {@code setLenient(false)} before, you might prefer {@link Strictness#STRICT} now instead.

gson/src/main/java/com/google/gson/stream/JsonWriter.java

@@ -271,7 +271,7 @@ public final FormattingStyle getFormattingStyle() {
   /**
    * Sets the strictness of this writer.
    *
-   * @deprecated Please use {@link JsonWriter#setStrictness(Strictness)} instead.
+   * @deprecated Please use {@link #setStrictness(Strictness)} instead.
    * {@code JsonWriter.setLenient(true)} should be replaced by {@code JsonWriter.setStrictness(Strictness.LENIENT)}
    * and {@code JsonWriter.setLenient(false)} should be replaced by {@code JsonWriter.setStrictness(Strictness.LEGACY_STRICT)}.<br>
    * However, if you used {@code setLenient(false)} before, you might prefer {@link Strictness#STRICT} now instead.
@@ -320,7 +320,7 @@ public final void setStrictness(Strictness strictness) {
   }
 
   /**
-   * Returns how strict this writer is.
+   * Returns the {@linkplain Strictness strictness} of this writer.
    *
    * @see #setStrictness(Strictness)
    * @since $next-version$

gson/src/main/resources/META-INF/proguard/gson.pro

@@ -13,7 +13,7 @@
 
 # Keep Gson annotations
 # Note: Cannot perform finer selection here to only cover Gson annotations, see also https://stackoverflow.com/q/47515093
--keepattributes *Annotation*
+-keepattributes RuntimeVisibleAnnotations,AnnotationDefault
 
 
 ### The following rules are needed for R8 in "full mode" which only adheres to `-keepattribtues` if
@@ -24,18 +24,20 @@
 -keep class com.google.gson.reflect.TypeToken { *; }
 
 # Keep any (anonymous) classes extending TypeToken
--keep class * extends com.google.gson.reflect.TypeToken
+-keep,allowobfuscation class * extends com.google.gson.reflect.TypeToken
 
 # Keep classes with @JsonAdapter annotation
--keep @com.google.gson.annotations.JsonAdapter class *
+-keep,allowobfuscation,allowoptimization @com.google.gson.annotations.JsonAdapter class *
 
 # Keep fields with @SerializedName annotation, but allow obfuscation of their names
 -keepclassmembers,allowobfuscation class * {
   @com.google.gson.annotations.SerializedName <fields>;
 }
 
 # Keep fields with any other Gson annotation
--keepclassmembers class * {
+# Also allow obfuscation, assuming that users will additionally use @SerializedName or
+# other means to preserve the field names
+-keepclassmembers,allowobfuscation class * {
   @com.google.gson.annotations.Expose <fields>;
   @com.google.gson.annotations.JsonAdapter <fields>;
   @com.google.gson.annotations.Since <fields>;
@@ -44,15 +46,25 @@
 
 # Keep no-args constructor of classes which can be used with @JsonAdapter
 # By default their no-args constructor is invoked to create an adapter instance
--keep class * extends com.google.gson.TypeAdapter {
+-keepclassmembers class * extends com.google.gson.TypeAdapter {
   <init>();
 }
--keep class * implements com.google.gson.TypeAdapterFactory {
+-keepclassmembers class * implements com.google.gson.TypeAdapterFactory {
   <init>();
 }
--keep class * implements com.google.gson.JsonSerializer {
+-keepclassmembers class * implements com.google.gson.JsonSerializer {
   <init>();
 }
--keep class * implements com.google.gson.JsonDeserializer {
+-keepclassmembers class * implements com.google.gson.JsonDeserializer {
   <init>();
 }
+
+# If a class is used in some way by the application, and has fields annotated with @SerializedName
+# and a no-args constructor, keep those fields and the constructor
+# Based on https://issuetracker.google.com/issues/150189783#comment11
+# See also https://github.com/google/gson/pull/2420#discussion_r1241813541 for a more detailed explanation
+-if class *
+-keepclasseswithmembers,allowobfuscation,allowoptimization class <1> {
+  <init>();
+  @com.google.gson.annotations.SerializedName <fields>;
+}