JITServer roadmap and status

[mpirvu]

Currently, the JITServer code in the master branch is protected by #ifdef JITSERVER_SUPPORT and thus it is not present by default in the compiled openj9 package. In order to allow liberty (and others) to easily use JITServer technology we would like to have this code compiled-in and enabled at runtime by a command line option. Ideally, this would happen in time for the 0.18 release.

Status of porting JITServer code from jitaas branch into master branch:

• x86: porting is complete
• P: porting is complete
  Note: delivery on Z will happen at a later date.

Building and test support:

• Create a conditional build flag: --enable-jitserver
• Build JITServer code with the same openssl version as the JDK: Build JITaaS with the same OpenSSL as the rest of OpenJ9 #5720
• For PR/nightly testing create a different platform/spec: Implement PR testing for JITServer with the master branch #7782
• Enable nightly testing
• Create a launcher for JITServer: Create a launcher for the JITServer process #7800

Functional stability:

• Presence of JITServer code should cause absolutely no issue to non-jisever runs
• Pass all sanity tests. This is true for Java8. One intermittent issue seen on Java11: Intermittent JITServer crash while running sanity test in Java 11 #7567
• Pass all test from extended bucket.
• No memory leaks at the server.

Note: private builds from @dchopra001 on P were passing sanity.
Also, system tests performed by @harryyu1994 were passing on x86

Performance:

• Measure package (on disk) footprint. JITServer code (including protobufs) increase JIT dll size by 4MB (JRE increases by 3% from 123M to 127M)
• Make sure that compiling-in JITServer code does not regress any benchmarks in non-jitserver mode. Perf team has validated this aspect. Results sumarized here Performance comparison JITServer on/off #8062
• Measure Acme/Daytrader7 throughput/rampup/footprint/start-up with JITServer.
• Measure performance in openshift
• Study the effect of network latency on performance results

Current performance status:

• AcmeAir runs in 4P/1G config on x86: JITServes achieves 75% reduction in CPU for compilation, no change in steady-state footprint and 1-2% loss in throughput. About the same results were observed without/with SCC/AOT and in cold/warm runs. Similar results were seen on P.
• AcmeAir runs in 1P/150m config on x86: JITServer ramps up considerably faster.
• Spikes in CPU/footprint due to compilation are eliminated (as expected)
• Startup time is somewhat slower in 4P config and somewhat faster in 1P config.

Note: from what we've seen so far, JITServer technology can help applications that suffer from JIT compilation interference because they need to compile many methods in a relatively short interval of time and they don't have enough resources (CPU and memory) to do so. A hindrance to JITServer is network latency. We will make recommendations on when to use it.

Documentation:

• Check existing documentation at https://github.com/eclipse/openj9/tree/master/doc/compiler/jitserver for staleness
• Work with Sue on release notes
• Write a blog

Timeline:
As soon as possible:

• PR testing and nightly testing of JITServer code

Before code split (Dec 15):

• Finish porting on P/Z
• Clear all sanity/extended failures
• Performance sniff

Between Dec 15 and Jan 2 (Milestone 2)

• Performance tuning
• Documentation
• Blog

[mpirvu]

Attn: @DanHeidinga @pshipton @mstoodle

[mpirvu]

@vijaysun-omr is ok with the 1-2% footprint increase from compiling-in the JITServer related code (including protobufs).
Experimental results are captured in #8062

[SueChaplain]

@mpirvu - Can we discuss what support might be needed here from the docs team please?

[mpirvu]

@SueChaplain I am guessing there will be a "what's new" section in the 0.18 release. We would like to mention the JITServer capability as a preview and a few words about it: when to use it, how to use it, what to expect from it. At the moment there are still changes being done on how to compile and how to launch it and I was waiting for that to settle before writing a draft.

[SueChaplain]

@mpirvu - ok no problem. As this is quite a big project I was concerned that there might be a lot of work needed in the user docs to accommodate it. A preview sounds like an easier introduction and we can plan for anything more significant further out.

[pshipton]

@mpirvu will this be closed off, or moved to the next milestone? We expect to start creating the final release builds later tonight or tomorrow.

[mpirvu]

It can be closed off.
The only item remaining is collecting some performance numbers. I already have some results and I am working on a blog which can be published after the release

[mpirvu]

I have collected all the performance numbers and they will make the subject of an upcoming blog post. This issue can be closed now.