Description
openedon Jul 11, 2020
Is your feature request related to a problem? Please describe.
For accurate incremental transpiling, external tooling (Maven, Gradle, etc.) need to determine which files to re-transpile when a given file changes.
The mapping from source file to generated files is needed too (naming conventions are not enough, because it's not unusual for a Java file to contain multiple types, and can even contain several top-level types; and source files aren't even required to follow the common naming convention based on the package and type names) to accurately cleanup the output directory when a source file is deleted.
J2CL also copies source files (.java
and .js
) to the output, possibly relocating them, so the mapping from source path to output path is needed as well.
Even better if it could also map to JARs or directories in the -classpath
for external dependencies, such that if the JAR (or class in the directory) changes the build tool could re-process all files that depend on it (and possibly on all following path entries in the classpath, in case class is now shadowed, or a class that was shadowed no longer is; if it could map to the actual class file, this could be made even more precise, by detecting those shadowing cases). Otherwise, each change to the classpath would mean reprocessing all files. Gradle, for instance, does such processing for its incremental javac
.
For example, given those two source files:
package p;
class A {
B a() {
return new B(1);
}
}
and
package p;
class B {
B(Number n) {}
}
If src/p/B.java
is modified, then we'll want to reprocess both src/p/B.java
and src/p/A.java
because p.B
would have had its API modified in a way that would change p.A
's output (e.g. here adding a B(int)
constructor overload).
Processing those 2 files will result in 8 files being generated or copied:
out/p/A.java
(copied from the sources)out/p/A.java.js
out/p/A.impl.java.js
out/p/A.js.map
out/p/B.java
(copied from the sources)out/p/B.java.js
out/p/B.impl.java.js
out/p/B.js.map
If src/p/A.java
is deleted, we'll want to delete all the out/p/A.*
files.
Now if a src/p/A.native.js
is created, then src/p/A.java
would have to be reprocessed. And then again if src/p/A.native.js
is deleted. But because J2CL uses a naming convention here, this does not need to appear in the mapping (correct me if I'm wrong).
If src/p/B.java
is modified to add an inner class, it would generate 3 new files (out/p/B$Inner.java.js
, out/p/B$Inner.impl.java.js
, and out/p/B$Inner.js.map
). If src/p/B.java
is then deleted, then all 7 files need to be deleted, and src/p/A.java
be reprocessed (which would fail, unless it was also modified to remove its dependency on p.B
).
Describe the solution you'd like
Define a stable file format that J2clCommandLineRunner
could output for use by external (non-Bazel) tooling.
It needs to contain 2 things:
- the list of dependencies between source files (
src/p/A.java
depends onsrc/p/B.java
; this could use types as an indirection, e.g. typep.A
comes fromsrc/p/A.java
and references typep.B
that comes fromsrc/p/B.java
, andfoo.Bar
that comes fromjar:foo.jar!/foo/Bar.class
) - the mapping back to the source file(s) (there could be a
.java
and a.native.js
) for each output file
In an incremental use of the transpiler, the build tool would have to merge the generated mapping file with the previously known mapping (and removing entries from that global mapping when a file is deleted).
Describe alternatives you've considered
J2CL outputs a source map file (.js.map
) for each generated file (pair of .java.js
and .impl.java.js
), containing the path of the source; but that references the source that J2CL copied to the output, not the actual path to the source (as can be seen in bazel-bin/third_party/jbox2d.js.zip
vs bazel-bin/third_party/libjbox2d-src.jar
after a bazel build //third_party:jbox2d.js.zip
: the .js.zip
contains a java/lang/StrictMath.js.map
referencing, using a relative path, java/lang/StrictMath.java
, but the actual source in the -src.jar
was external/org_jbox2d/src/main/java/org/jbox2d/gwtemul/java/lang/StrictMath.java
), because it's targeted at tools that need to access those .java
files.
Additional context
J2CL internally already has all the information; it populates a LibraryInfo
protobuf that it can serialize to a file (for later use by RTA) but the flag to do so is not exposed to J2clCommandLineRunner
(probably to keep the LibraryInfo
as an implementation detail of J2CL). Just like the source map though, the library_info does not contain the actual source path either (modifying third_party/BUILD
to pass readable_library_info = True
to //third_party:jbox2d
generates a library_info_debug.json
in the .js.zip
but it does not contain the full src/main/java/org/jbox2d/gwtemul/java/lang/StrictMath.java
path)
The Kythe indexing metadata includes the information though (I modified build_defs/internal_do_not_use/j2cl_common.bzl
to unconditionally pass -generatekytheindexingmetadata
), so J2CL has all the needed information to generate such a mapping file.