8334714: Implement JEP 484: Class-File API

[asotona]

Class-File API is leaving preview.
This is a removal of all @PreviewFeature annotations from Class-File API.
It also bumps all @since tags and removes jdk.internal.javac.PreviewFeature.Feature.CLASSFILE_API.

Please review.

Thanks,
Adam

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change requires a JEP request to be targeted
• Change requires CSR request JDK-8334720 to be approved

Issues

• JDK-8334714: Implement JEP 484: Class-File API (Sub-task - P4)
• JDK-8334712: JEP 484: Class-File API (JEP)
• JDK-8334720: Implement JEP 484: Class-File API (CSR)

Reviewers

• Vicente Romero (@vicente-romero-oracle - Reviewer) 🔄 Re-review required (review applies to e5f96464)
• Chen Liang (@liach - Reviewer) 🔄 Re-review required (review applies to 4a51c456)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/19826/head:pull/19826
$ git checkout pull/19826

Update a local copy of the PR:
$ git checkout pull/19826
$ git pull https://git.openjdk.org/jdk.git pull/19826/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 19826

View PR using the GUI difftool:
$ git pr show -t 19826

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/19826.diff

Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back asotona! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

❗ This change is not yet ready to be integrated.
See the Progress checklist in the description for automated requirements.

[openjdk]

@asotona The following labels will be automatically applied to this pull request:

• compiler
• core-libs

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 06: Full (29936e33)
• 05: Full (4a51c456)
• 04: Full (bb857e58)
• 03: Full (9571ecb9)
• 02: Full (aeebc58c)
• 01: Full (e5f96464)
• 00: Full (56c8573a)

[liach]

Do we intend to do it after we finish our planned API changes or as soon as possible? Also we might need to look at the preview-enabled test treatments.

[AlanBateman]

Looks like this has been accidentally created as a sub-task of the JEP issue, I assume you'll fix that. Will there be another issue to update the tests and drop @enablePreview?

[asotona]

Any potential API changes should be merged first, to avoid impression of changing a non-preview API.
However any discussion and approval processes of API and documentation changes should not block the JEP process (and each other).

Cleaning up preview-enabled tests has a lower priority and should follow.

[asotona]

Looks like this has been accidentally created as a sub-task of the JEP issue, I assume you'll fix that. Will there be another issue to update the tests and drop @enablePreview?

I thought the implementation is usually created as a subtask of JEP, however I can make it a standalone bug if neede.

Yes, there will be follow up task to clean all the @enablePreview mess, I'll create it.

[ExE-Boss]

Note that Utf8Entry::equalsString has inconsistent behaviour when called with a null value, which I’ve reported as JI‑9077307, and should probably be fixed before this API leaves preview.

[liach]

@ExE-Boss Your report has been promoted to https://bugs.openjdk.org/browse/JDK-8336430 and will be addressed in a separate patch.

[vicente-romero-oracle]

lgtm

[bridgekeeper]

@asotona This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

[xxDark]

I've been trying to work with classfile api, and:

• AttributesProcessingOption is unused, does nothing? That seems to be just a weird way to write a check... Yikes.

  jdk/src/java.base/share/classes/jdk/internal/classfile/impl/Util.java

  Line 104 in 5671f83

  ? ATTRIBUTE_STABILITY_COUNT - attr.attributeMapper().stability().ordinal() > processingOption.ordinal()

• Classfiles with oak format seem to be unparsable. They also cannot be written. (maxStack/maxLocals in Code attribute depend on classfile version). Constants such as JAVA_1_VERSION exist in java.lang.classfile.ClassFile.

[liach]

Classfiles with oak format seem to be unparsable. They also cannot be written. (maxStack/maxLocals in Code attribute depend on classfile version). Constants such as JAVA_1_VERSION exist in java.lang.classfile.ClassFile.

Unfortunately, ClassFile API's scope is only to interpret correctly the Class Files that are accepted by the current VM and support writing such class files; for example, for release N, we will not support correct parsing or writing of preview class files from N-1, N-2, etc.

While your account of oak format seems interesting (from a search, it seems to use u1 for max stacks/locals, u2 for Code size), it is neither recognized by hotspot (the reference implementation for JVM):

jdk/src/hotspot/share/classfile/classFileParser.cpp

Lines 2332 to 2334 in 5671f83

	max_stack = cfs->get_u2_fast();
	max_locals = cfs->get_u2_fast();
	code_length = cfs->get_u4_fast();

Nor by the JVMS: https://docs.oracle.com/javase/specs/jvms/se22/html/jvms-4.html#jvms-4.7.3

And as such, it falls out of the goals of the API, and is likely not going to be added. You most likely will resort to a third party library to handle such frozen format, as the main issue ClassFile API is solving is that a library built against JDK 21 may not run on JDK 23 unless it bumps its ASM dependency to support reading JDK 23 Class File format, which doesn't exist for the said oak format.

[xxDark]

While your account of oak format seems interesting (from a search, it seems to use u1 for max stacks/locals, u2 for Code size), it is neither recognized by hotspot (the reference implementation for JVM):

jdk/src/hotspot/share/classfile/classFileParser.cpp

Lines 2332 to 2334 in 5671f83

	max_stack = cfs->get_u2_fast();
	max_locals = cfs->get_u2_fast();
	code_length = cfs->get_u4_fast();

Interesting. I guess handling of oak classfiles was removed a long time ago by this commit.
But, does this mean that now, Hotspot blindly accepts classfiles with version 45 with, technically, incorrect data?

java -cp . Test
Hello, World

jdk-1.8.0_402\bin\java -cp . Test
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.ClassFormatError: Invalid code attribute name index 10 in class file Test
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:756)

Also now, this library which took special care to parse oak files reports wrong instructions does not parse classfile at all.

		byte[] bytes = ClassFile.of(ClassFile.StackMapsOption.GENERATE_STACK_MAPS)
				.build(ClassDesc.ofInternalName("Test"), builder -> {
					builder.withVersion(ClassFile.JAVA_1_VERSION, 0);
					builder.withMethod(
							"main",
							MethodTypeDesc.ofDescriptor("([Ljava/lang/String;)V"),
							AccessFlags.ofMethod(AccessFlag.PUBLIC, AccessFlag.STATIC).flagsMask(),
							mb -> {
								mb.withCode(cb -> {
									ClassDesc printStream = PrintStream.class.describeConstable().orElseThrow();
									cb.getstatic(System.class.describeConstable().orElseThrow(), "out", printStream)
											.ldc("Hello, World")
											.invokevirtual(printStream, "println", MethodTypeDesc.ofDescriptor("(Ljava/lang/String;)V"))
											.return_();
								});
							});
					builder.with(SourceFileAttribute.of("Test Attr"));
				});
		Path path = Path.of("Test.class");
		Files.write(path, bytes);
		var file = new ClassFileReader().read(bytes);

As noted about Hotspot technically parsing invalid data now (if you could consider that "invalid") and we make a correct oak classfile:

jdk-1.8.0_402\bin\java -cp . Test
Hello World

JDK 17
java -cp . Test
Error: LinkageError occurred while loading main class Test
        java.lang.ClassFormatError: Invalid method Code length 2986347538 in class file Test

[liach]

I think so. See https://mail.openjdk.org/pipermail/hotspot-dev/2019-October/039795.html for some context. There are also historical evaluations available at https://bugs.openjdk.org/browse/JDK-8232890 and https://bugs.openjdk.org/browse/JDK-8232967, notably the judgement in Compatibility Risk Description:

Class files with versions < 45.3 predate Java.

But, does this mean that now, Hotspot blindly accepts classfiles with version 45 with, technically, incorrect data?

It appears so. See note in ClassFileFormatVersion:

jdk/src/java.base/share/classes/java/lang/reflect/ClassFileFormatVersion.java

Lines 413 to 414 in 5671f83

	// RELEASE_0 and RELEASE_1 both have a major version of 45;
	// return RELEASE_1 for an argument of 45.

It seems that now major version 45, regardless of the minor version, is simply seen as the Class File format for Java 1.1.

Also now, this library which took special care to parse oak files reports wrong instructions does not parse classfile at all.

Indeed, it would be a good RFE to allow users to override default attribute mappers to parse attributes; this would be extremely useful if users wish to support previous previews that only differed in the attribute formats.

jdk-1.8.0_402\bin\java -cp . Test
Hello World

For the behavioral inconsistencies, we can backport this special handling removal to active jdk update projects or even request specification changes on LTS versions (see https://github.com/openjdk/jdk8u-ri) if this is deemed important enough.

[xxDark]

Indeed, it would be a good RFE to allow users to override default attribute mappers to parse attributes; this would be extremely useful if users wish to support previous previews that only differed in the attribute formats.

I don't think at this point in time there would be any change in the format of any existing attributes, as that would be a binary incompatible change.
But yes, having a way to override parsing of builtin attributes would be nice.

For the behavioral inconsistencies, we can backport this special handling removal to active jdk update projects or even request specification changes on LTS versions (see https://github.com/openjdk/jdk8u-ri) if this is deemed important enough.

I don't think that many projects use pre-Java 1 format at this point. While I saw some libraries that kept their version set to versions like Java 4 for no apparent reason... Hey, at least it's not Java 1 version.
If Hotspot now parses these files incorrectly, it might be worth to just throw UnsupportedClassVersionError for oak class files?

[Col-E]

Unfortunately, ClassFile API's scope is only to interpret correctly the Class Files that are accepted by the current VM and support writing such class files; for example, for release N, we will not support correct parsing or writing of preview class files from N-1, N-2, etc.

I understand that there is no explicit goal for this API to be used for general purposes, but it seems really odd that I cannot safely assume that classes that are successfully loaded in the current runtime will be parsable. Oak classes are a notable exception, and nobody should realistically expect to see an oak class ever. However, if we only can guarantee that the current class file version will be supported and not even N-1 then no application/library expecting reliability (without constant recompilation to ensure the class versions are current) should use the API.

Again, its clearly not listed as a goal but it seems disappointing that this new API is unrealistic to use if this statement is to be taken at face value.

Edit: I missed the word preview in the quote. Does this only apply to preview class file features?

[liach]

If Hotspot now parses these files incorrectly, it might be worth to just throw UnsupportedClassVersionError for oak class files?

This would be an important decision; I would wait till Monday when more engineers are back to discuss about the histroy of dropping support for such files, why this format never made its way to the JVMS, and the correct way to handle so as to avoid inconsistency between different versions.

Edit: I missed the word preview in the quote. Does this only apply to preview class file features?

Yes, Class-File API is supposed to handle whatever the VM handles. It's actually considered a "JVM API" in Oracle's documentations: https://docs.oracle.com/en/java/javase/22/vm/class-file-api.html

OpenJDK's preview version policy allows shipping the preview features for testing to a wider audience without any implied burden of long-term maintenance. So for example, if the Nullable Types JEP preview adds ~ as a null marker in Signature attribute but changes it to ? in a subsequent preview, the JVM and core libraries will act as if a previous iteration with ~ marker never existed; the policy is similar to that for preview libraries.

Unfortunately, there seems to be some misunderstanding around the role of preview, so some are asking if preview features that is largely unchanged and finalized in a future release can become finalized in an LTS as well. Short answer, no: it's an experiment format enabled by the new 6-month release schedule.

And note that javap uses ClassFile API, and as a result, javap for release N cannot correctly decompile Class Files with version N-1.65535, etc. We are planning to decompile on a best-effort basis, that we try to interpret such as class as if it's N.65535 (maybe emit a warning), trying to salvage the most out of the Class File and printing errors (but continue to output) if we encounter anything we don't expect.

[liach]

If Hotspot now parses these files incorrectly, it might be worth to just throw UnsupportedClassVersionError for oak class files?

I was informed in offline discussion that this is unlikely to be changed and this change is unlikely to be backported unless for security purposes.

[openjdk]

@asotona this pull request can not be integrated into master due to one or more merge conflicts. To resolve these merge conflicts and update this pull request you can run the following commands in the local repository for your personal fork:

git checkout JDK-8334714-final
git fetch https://git.openjdk.org/jdk.git master
git merge FETCH_HEAD
# resolve conflicts and follow the instructions given by git merge
git commit -m "Merge master"
git push

[liach]

All @since tags in the java.lang.classfile directory are @since 24.

@nizarbenalla Is it possible to enable the since checker in java.lang.classfile and its nested packages.

[nizarbenalla]

I ran it and all the tags seem to be correct.

[asotona]

I ran it and all the tags seem to be correct.

Thank you!

[ExE-Boss]

I wish that the concrete PoolEntry subtypes had of(…) factory methods which would create instances using the TemporaryConstantPool, as I find it necessary to create PoolEntry instances in contexts where a ConstantPoolBuilder is not easily available (and ConstantPoolBuilder.of() is relatively expensive when creating a one‑off PoolEntry instance) and the current TemporaryConstantPool implementation doesn’t support all the creation methods.

[asotona]

I wish that the concrete PoolEntry subtypes had of(…) factory methods...

Please forward the proposal on clasfile-api-dev mailing list, where it can be discussed. Thank you.

[liach]

/jep JEP-484

[openjdk]

@liach
This pull request will not be integrated until the JEP-484 has been targeted.

[ExE-Boss]

It should probably be possible for ClassFile​::verify(…) to be able to verify the bytecode of java.lang.Object, java.lang.Class, java.lang.String, and java.lang.Throwable, as the main reason the HotSpot verifier skips those is that there’s circular verification bootstrap dependencies between those, but the Class‑File API verifier uses the offline ClassHierarchyResolver instead for determining assignability checks.

jdk/src/java.base/share/classes/jdk/internal/classfile/impl/verifier/VerifierImpl.java

Lines 144 to 150 in 18bcbf7

	public static boolean is_eligible_for_verification(VerificationWrapper klass) {
	String name = klass.thisClassName();
	return !java_lang_Object.equals(name) &&
	!java_lang_Class.equals(name) &&
	!java_lang_String.equals(name) &&
	!java_lang_Throwable.equals(name);
	}