Android: Add host_jar_path vs device_jar_path to build logic

Fixes desugared .jar files incorrectly being used in java_binary() /
junit_binary().

Bug: 1078484
Change-Id: Ia49a10d3e907844ed75ee658604e8eeaa2993fb4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2241865
Commit-Queue: Andrew Grieve <agrieve@chromium.org>
Commit-Queue: Peter Wen <wnwen@chromium.org>
Reviewed-by: Peter Wen <wnwen@chromium.org>
Cr-Commit-Position: refs/heads/master@{#778433}

BUILD.gn

@@ -356,7 +356,6 @@ group("gn_all") {
       "//tools/android:push_apps_to_background",
       "//tools/android/audio_focus_grabber:audio_focus_grabber_apk",
       "//tools/android/customtabs_benchmark:customtabs_benchmark_apk",
-      "//tools/android/errorprone_plugin:errorprone_plugin_java",
       "//tools/android/kerberos/SpnegoAuthenticator:spnego_authenticator_apk",
       "//ui/android:ui_junit_tests",
       "//weblayer/public/java:client_aar",

build/android/bytecode/BUILD.gn

@@ -17,5 +17,4 @@ java_binary("bytecode_processor") {
   ]
   wrapper_script_name = "helper/bytecode_processor"
   enable_bytecode_checks = false
-  enable_desugar = false
 }

build/android/docs/java_toolchain.md

@@ -20,6 +20,12 @@ also have a default `jar_excluded_patterns` set (more on that later):
 All target names must end with "_java" so that the build system can distinguish
 them from non-java targets (or [other variations](https://cs.chromium.org/chromium/src/build/config/android/internal_rules.gni?rcl=ec2c17d7b4e424e060c3c7972842af87343526a1&l=20)).
 
+Most targets produce two separate `.jar` files:
+* Device `.jar`: Used to produce `.dex.jar`, which is used on-device.
+* Host `.jar`: For use on the host machine (`junit_binary` / `java_binary`).
+  * Host `.jar` files live in `lib.java/` so that they are archived in
+    builder/tester bots (which do not archive `obj/`).
+
 ## From Source to Final Dex
 
 ### Step 1: Create interface .jar with turbine or ijar
@@ -72,15 +78,24 @@ This step can be disabled via GN arg: `use_errorprone_java_compiler = false`
 [ErrorProne]: https://errorprone.info/
 [ep_plugins]: /tools/android/errorprone_plugin/
 
-### Step 3: Desugaring
+### Step 3: Desugaring (Device .jar Only)
 
-This step happens only when targets have `supports_android = true`.
+This step happens only when targets have `supports_android = true`. It is not
+applied to `.jar` files used by `junit_binary`.
 
 * `//third_party/bazel/desugar` converts certain Java 8 constructs, such as
   lambdas and default interface methods, into constructs that are compatible
   with Java 7.
 
-### Step 4: Filtering
+### Step 4: Instrumenting (Device .jar Only)
+
+This step happens only when this GN arg is set: `use_jacoco_coverage = true`
+
+* [Jacoco] adds instrumentation hooks to methods.
+
+[Jacoco]: https://www.eclemma.org/jacoco/
+
+### Step 5: Filtering
 
 This step happens only when targets that have `jar_excluded_patterns` or
 `jar_included_patterns` set (e.g. all `android_` targets).
@@ -97,27 +112,12 @@ This step happens only when targets that have `jar_excluded_patterns` or
 [Android Resources]: life_of_a_resource.md
 [apphooks]: /chrome/android/java/src/org/chromium/chrome/browser/AppHooksImpl.java
 
-### Step 5: Instrumentation
-
-This step happens only when this GN arg is set: `use_jacoco_coverage = true`
-
-* [Jacoco] adds instrumentation hooks to methods.
-
-[Jacoco]: https://www.eclemma.org/jacoco/
-
-### Step 6: Copy to lib.java
-
-* The `.jar` is copied into `$root_build_dir/lib.java` (under target-specific
-  subdirectories) so that it will be included by bot archive steps.
-  * These `.jar` files are the ones used when running `java_binary` and
-    `junit_binary` targets.
-
-### Step 7: Per-Library Dexing
+### Step 6: Per-Library Dexing
 
 This step happens only when targets have `supports_android = true`.
 
 * [d8] converts `.jar` files containing `.class` files into `.dex.jar` files
-  containing `.dex` files.
+  containing `classes.dex` files.
 * Dexing is incremental - it will reuse dex'ed classes from a previous build if
   the corresponding `.class` file is unchanged.
 * These per-library `.dex.jar` files are used directly by [incremental install],
@@ -128,7 +128,7 @@ This step happens only when targets have `supports_android = true`.
 [d8]: https://developer.android.com/studio/command-line/d8
 [incremental install]: /build/android/incremental_install/README.md
 
-### Step 8: Apk / Bundle Module Compile
+### Step 7: Apk / Bundle Module Compile
 
 * Each `android_apk` and `android_bundle_module` template has a nested
   `java_library` target. The nested library includes final copies of files
@@ -139,7 +139,7 @@ This step happens only when targets have `supports_android = true`.
 
 [JNI glue]: /base/android/jni_generator/README.md
 
-### Step 9: Final Dexing
+### Step 8: Final Dexing
 
 This step is skipped when building using [Incremental Install].
 
@@ -149,19 +149,11 @@ When `is_java_debug = true`:
 When `is_java_debug = false`:
 * [R8] performs whole-program optimization on all library `lib.java` `.jar`
   files and outputs a final `.r8dex.jar`.
-  * For App Bundles, R8 creates a single `.r8dex.jar` with the code from all
-    modules.
+  * For App Bundles, R8 creates a `.r8dex.jar` for each module.
 
 [Incremental Install]: /build/android/incremental_install/README.md
 [R8]: https://r8.googlesource.com/r8
 
-### Step 10: Bundle Module Dex Splitting
-
-This step happens only when `is_java_debug = false`.
-
-* [dexsplitter.py] splits the single `*dex.jar` into per-module `*dex.jar`
-  files.
-
 ## Test APKs with apk_under_test
 
 Test APKs are normal APKs that contain an `<instrumentation>` tag within their

build/android/gyp/dist_aar.py

@@ -14,7 +14,7 @@
 import tempfile
 import zipfile
 
-from filter_zip import CreatePathTransform
+import filter_zip
 from util import build_utils
 
 
@@ -117,8 +117,8 @@ def main(args):
         build_utils.AddToZipHermetic(
             z, 'AndroidManifest.xml', src_path=options.android_manifest)
 
-        path_transform = CreatePathTransform(options.jar_excluded_globs,
-                                             options.jar_included_globs, [])
+        path_transform = filter_zip.CreatePathTransform(
+            options.jar_excluded_globs, options.jar_included_globs, [])
         with tempfile.NamedTemporaryFile() as jar_file:
           build_utils.MergeZips(
               jar_file.name, options.jars, path_transform=path_transform)

build/android/gyp/filter_zip.py

@@ -5,6 +5,7 @@
 # found in the LICENSE file.
 
 import argparse
+import shutil
 import sys
 
 from util import build_utils
@@ -20,6 +21,21 @@
 
 def CreatePathTransform(exclude_globs, include_globs,
                         strip_resource_classes_for):
+  """Returns a function to strip paths for the given patterns.
+
+  Args:
+    exclude_globs: List of globs that if matched should be excluded.
+    include_globs: List of globs that if not matched should be excluded.
+    strip_resource_classes_for: List of Java packages for which to strip
+       R.java classes from.
+
+  Returns:
+    * None if no filters are needed.
+    * A function "(path) -> path" that returns None when |path| should be
+          stripped, or |path| otherwise.
+  """
+  if not (exclude_globs or include_globs or strip_resource_classes_for):
+    return None
   exclude_globs = list(exclude_globs or [])
   if strip_resource_classes_for:
     exclude_globs.extend(p.replace('.', '/') + '/' + f
@@ -52,19 +68,18 @@ def main():
   argv = build_utils.ExpandFileArgs(sys.argv[1:])
   args = parser.parse_args(argv)
 
-  if args.exclude_globs:
-    args.exclude_globs = build_utils.ParseGnList(args.exclude_globs)
-  if args.include_globs:
-    args.include_globs= build_utils.ParseGnList(args.include_globs)
-  if args.strip_resource_classes_for:
-    args.strip_resource_classes_for = build_utils.ParseGnList(
-        args.strip_resource_classes_for)
+  args.exclude_globs = build_utils.ParseGnList(args.exclude_globs)
+  args.include_globs = build_utils.ParseGnList(args.include_globs)
+  args.strip_resource_classes_for = build_utils.ParseGnList(
+      args.strip_resource_classes_for)
 
   path_transform = CreatePathTransform(args.exclude_globs, args.include_globs,
                                        args.strip_resource_classes_for)
   with build_utils.AtomicOutput(args.output) as f:
-    build_utils.MergeZips(
-        f.name, [args.input], path_transform=path_transform)
+    if path_transform:
+      build_utils.MergeZips(f.name, [args.input], path_transform=path_transform)
+    else:
+      shutil.copy(args.input, f.name)
 
 
 if __name__ == '__main__':