Javadoc JARs built with JDK 11

[lassana]

Android Gradle plugin 4.2.0 is now bundled with JDK 11 which seems to build javadoc JARs quite different from what JDK 8 was producing.

1. There is no longer package-list file in the JAR file; it seems to be changed to element-list: https://bugs.openjdk.java.net/browse/JDK-8211194

This causes a build error when using a JAR built with JDK 11 as JavaDocJar in a Xamarin.Android binding project:

/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Bindings.ClassParse.targets(5,5): Error XACLP7004: System.ArgumentException: Directory '//Users/user/Documents/cobrowseio-llc/feature-remote_control_consent/cobrowse-sdk-xamarin/Android/CobrowseIO.Android/obj/Debug/javadocs/cobrowse-sdk-android-LATEST-javadoc/' does not appear to be an android doc reference directory.
  at Xamarin.Android.Tools.Bytecode.AndroidDocScraper..ctor (System.String dir, System.String patternHead, System.String resetPatternHead, System.String parameterPairSplitter, System.Boolean continuousParamLines, System.String openMethod, System.String paramSep, System.String closeMethod, System.String postCloseMethodParens) [0x000ce] in /Users/builder/azdo/_work/278/s/xamarin-android/external/Java.Interop/src/Xamarin.Android.Tools.Bytecode/JavaDocumentScraper.cs:172 
  at Xamarin.Android.Tools.Bytecode.AndroidDocScraper..ctor (System.String dir, System.String patternHead, System.String resetPatternHead, System.String parameterPairSplitter, System.Boolean continuousParamLines) [0x00000] in /Users/builder/azdo/_work/278/s/xamarin-android/external/Java.Interop/src/Xamarin.Android.Tools.Bytecode/JavaDocumentScraper.cs:145 
  at Xamarin.Android.Tools.Bytecode.DroidDocScraper..ctor (System.String dir) [0x00000] in /Users/builder/azdo/_work/278/s/xamarin-android/external/Java.Interop/src/Xamarin.Android.Tools.Bytecode/JavaDocumentScraper.cs:41 
  at Xamarin.Android.Tools.Bytecode.ClassPath.CreateDocScraper (System.String src) [0x00030] in /Users/builder/azdo/_work/278/s/xamarin-android/external/Java.Interop/src/Xamarin.Android.Tools.Bytecode/ClassPath.cs:249 
  at Xamarin.Android.Tools.Bytecode.ClassPath.FixupParametersFromDocs (System.Xml.Linq.XElement api, System.String path) [0x00000] in /Users/builder/azdo/_work/278/s/xamarin-android/external/Java.Interop/src/Xamarin.Android.Tools.Bytecode/ClassPath.cs:260 
  at Xamarin.Android.Tools.Bytecode.ClassPath.FixupParametersFromDocs (System.Xml.Linq.XElement api) [0x0002e] in /Users/builder/azdo/_work/278/s/xamarin-android/external/Java.Interop/src/Xamarin.Android.Tools.Bytecode/ClassPath.cs:241 
  at Xamarin.Android.Tools.Bytecode.ClassPath.ToXElement () [0x0009b] in /Users/builder/azdo/_work/278/s/xamarin-android/external/Java.Interop/src/Xamarin.Android.Tools.Bytecode/ClassPath.cs:314 
  at Xamarin.Android.Tools.Bytecode.ClassPath.SaveXmlDescription (System.IO.TextWriter textWriter) [0x0001b] in /Users/builder/azdo/_work/278/s/xamarin-android/external/Java.Interop/src/Xamarin.Android.Tools.Bytecode/ClassPath.cs:333 
  at Xamarin.Android.Tasks.ClassParse.RunTask () [0x000aa] in <5a1df33220564ef787578ac3c7e67d03>:0 
  at Xamarin.Android.Tasks.AndroidTask.Execute () [0x00000] in <5a1df33220564ef787578ac3c7e67d03>:0  (XACLP7004) (CobrowseIO.Android)

Apparently Xamarin searches for package-list or packages.html file in the JAR but neither of them exist:

https://github.com/xamarin/java.interop/blob/a3de91efdaec04b3927b755a07018e8d26cfdc71/src/Xamarin.Android.Tools.Bytecode/JavaDocumentScraper.cs#L167-L168

I've managed to workaround this by creating a new package-list file and copying the content of element-list into it.

2. There is also a change in HTML tags in javadoc: URLs in href attribute no longer contain the package name.

Here is what we had with JDK 8:

...
<tr id="i0" class="altColor">
<td class="colFirst"><code>void</code></td>
<td class="colLast"><code><span class="memberNameLink"><a href="../../io/cobrowse/CobrowseIO.RemoteControlRequestDelegate.html#handleRemoteControlRequest-android.app.Activity-io.cobrowse.Session-">handleRemoteControlRequest</a></span>(android.app.Activity&nbsp;activity,
                          <a href="../../io/cobrowse/Session.html" title="class in io.cobrowse">Session</a>&nbsp;session)</code>
<div class="block">Called when an agent requests remote control.</div>
</td>
...

and now with JDK 11:

...
<tr id="i0" class="altColor">
<td class="colFirst"><code>void</code></td>
<th class="colSecond" scope="row"><code><span class="memberNameLink"><a href="#handleRemoteControlRequest(android.app.Activity,io.cobrowse.Session)">handleRemoteControlRequest</a></span>&#8203;(android.app.Activity&nbsp;activity,
                          <a href="Session.html" title="class in io.cobrowse">Session</a>&nbsp;session)</code></th>
<td class="colLast">
<div class="block">Called when an agent requests remote control.</div>
</td>
</tr>
...

This causes an issue when Xamarin fails to detect correct parameter names for methods:

/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Bindings.ClassParse.targets(27,5): warning : Warning : no match for CobrowseIO.RemoteControlRequestDelegate.handleRemoteControlRequest (rex: <span class="sympad"><a href=".*io/cobrowse/CobrowseIO.RemoteControlRequestDelegate.html#handleRemoteControlRequest\(android.app.Activity, io.cobrowse.Session\)".*\(([^\(\)]*)\))

Apparently Xamarin expects the package name to be in this URL but JDK 11 no longer includes it:

https://github.com/xamarin/java.interop/blob/a3de91efdaec04b3927b755a07018e8d26cfdc71/src/Xamarin.Android.Tools.Bytecode/JavaDocumentScraper.cs#L195-L207

---

Javadoc JAR examples (renamed from *.jar to *.zip):

• Built with JDK 8: cobrowse-sdk-android-2.11.0-javadoc.zip

• Built with JDK 11: cobrowse-sdk-android-2.11.2-javadoc.zip

---

About Visual Studio

=== Visual Studio Community 2019 for Mac ===

Version 8.9.10 (build 4)
Installation UUID: 3fa90cee-79c4-4b50-93fb-cfc2f09b209a
	GTK+ 2.24.23 (Raleigh theme)
	Xamarin.Mac 6.18.0.23 (d16-6 / 088c73638)

	Package version: 612000125

=== Mono Framework MDK ===

Runtime:
	Mono 6.12.0.125 (2020-02/8c552e98bd6) (64-bit)
	Package version: 612000125

=== Roslyn (Language Service) ===

3.9.0-6.21152.10+c10f884b30737542ddd84ca889a4aad9281ce210

=== NuGet ===

Version: 5.8.0.6860

=== .NET Core SDK ===

SDK: /usr/local/share/dotnet/sdk/5.0.203/Sdks
SDK Versions:
	5.0.203
	3.1.409
MSBuild SDKs: /Applications/Visual Studio.app/Contents/Resources/lib/monodevelop/bin/MSBuild/Current/bin/Sdks

=== .NET Core Runtime ===

Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
	5.0.6
	3.1.15

=== .NET Core 3.1 SDK ===

SDK: 3.1.409

=== Xamarin.Profiler ===

Version: 1.6.12.26
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

=== Updater ===

Version: 11

=== Xamarin.Android ===

Version: 11.2.2.1 (Visual Studio Community)
Commit: xamarin-android/d16-9/877f572
Android SDK: /usr/local/share/android-sdk/
	Supported Android versions:
		8.0 (API level 26)
		8.1 (API level 27)

SDK Tools Version: 26.1.1
SDK Platform Tools Version: 31.0.2
SDK Build Tools Version: 31.0.0 rc3

Build Information: 
Mono: 5e9cb6d
Java.Interop: xamarin/java.interop/d16-9@54f8c24
ProGuard: Guardsquare/proguard/v7.0.1@912d149
SQLite: xamarin/sqlite/3.34.1@daff8f4
Xamarin.Android Tools: xamarin/xamarin-android-tools/d16-9@d210f11

=== Microsoft OpenJDK for Mobile ===

Java SDK: /Users/user/Library/Developer/Xamarin/jdk/microsoft_dist_openjdk_1.8.0.25
1.8.0-25
Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

=== Android SDK Manager ===

Version: 16.9.0.22
Hash: a391de2
Branch: remotes/origin/dev/bretjohn/readme-tweak~3
Build date: 2021-05-13 17:03:56 UTC

=== Android Device Manager ===

Version: 16.9.0.17
Hash: fc2b3db
Branch: remotes/origin/dev/jmt/d16-9bump~1
Build date: 2021-05-13 17:04:12 UTC

=== Apple Developer Tools ===

Xcode 12.5 (18205)
Build 12E262

=== Xamarin.Mac ===

Xamarin.Mac not installed. Can't find /Library/Frameworks/Xamarin.Mac.framework/Versions/Current/Version.

=== Xamarin.iOS ===

Version: 14.16.0.5 (Visual Studio Community)
Hash: ab40b131d
Branch: xcode12.5
Build date: 2021-04-22 08:56:42-0400

=== Xamarin Designer ===

Version: 16.9.0.324
Hash: b1e216c75
Branch: remotes/origin/d16-9
Build date: 2021-04-16 00:02:50 UTC

=== Build Information ===

Release ID: 809100004
Git revision: 30612572b7defb0273729cb6b5ea5adc8bd9a147
Build date: 2021-05-13 13:06:24-04
Build branch: release-8.9
Xamarin extensions: 30612572b7defb0273729cb6b5ea5adc8bd9a147

=== Operating System ===

Mac OS X 10.16.0
Darwin 20.4.0 Darwin Kernel Version 20.4.0
    Thu Apr 22 21:46:47 PDT 2021
    root:xnu-7195.101.2~1/RELEASE_X86_64 x86_64

=== Enabled user installed extensions ===

DeepClean 1.2.5
NuGet Package Explorer 0.5
AddinMaker 1.6.0
Eto.Forms Support Addin 2.5.6

[jonpryor]

Javadoc support is a maze of twisty passages, all alike. We've been eaten by the implied Grue multiple times, as everything has their own idea of what HTML should be produced by Javadoc.

Xamarin.Android 11.2 deprecated the @(JavaDocJar) Build action. Instead, the @(JavaSourceJar) build action should be preferred, which works by using tools/java-source-utils; see also:

• Commit 69e1b80
• Commit 7574f16
• https://docs.microsoft.com/en-us/xamarin/android/deploy-test/building-apps/build-items#javasourcejar

This is still not as stable as I'd like -- see also PR #844 -- but parsing Javadoc comments is at least a (reasonably) stable format to parse, unlike the myriad HTML dialects that we otherwise need to deal with.

@(JavaDocJar) is dead. Long Live @(JavaSourceJar)!

[lassana]

@(JavaDocJar) is dead. Long Live @(JavaSourceJar)!

Basically this kills support for proprietary libraries.

[lassana]

@jonpryor is there a better alternative to JavaSourceJar? I maintain a Xamarin version of a proprietary Android AAR library that does not provide a sources jar but provides a Javadoc jar instead. Since JavaDocJar is broken in Xamarin.Android (if built using JDK 11) and not supported in .NET 6, what am I supposed to do? Is manually reflecting all parameter names the only option?