Skip to content
This repository was archived by the owner on Jan 23, 2023. It is now read-only.

Initial draft of the composite R2R format #25344

Closed
wants to merge 12 commits into from
Closed

Initial draft of the composite R2R format #25344

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
6918775
Update R2R format doc to 3.0
trylek Jun 20, 2019
231197c
Addressed [hopefully] most of JanK's initial PR feedback
trylek Jun 20, 2019
02b2e60
Address additional Jan's feedback
trylek Jun 21, 2019
7a44810
Remove VSD flag from the helper enumeration
trylek Jun 21, 2019
7dc137b
Address additional Jan's PR feedback
trylek Jun 23, 2019
0e5bf59
Proposed changes for composite R2R format
trylek Jun 24, 2019
4cd6e6c
Remove design bits into a new doc readytorun-composite-format-design
trylek Jun 24, 2019
e71c767
Moved design file under design-docs, rewording per Simon's feedback
trylek Jun 24, 2019
4564692
Clarify versioning considerations based on David Wrighton's feedback
trylek Jun 24, 2019
6c1c58b
Further clarify that ATTRIBUTEPRESENCE is per component assembly
trylek Jun 24, 2019
0ac4fe9
Slight facelift of the composite R2R design docs
trylek Oct 1, 2019
fca1274
Fix casing (nGen -> NGen) per JanK's PR feedback
trylek Oct 1, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 89 additions & 11 deletions Documentation/botr/readytorun-format.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,22 @@ ReadyToRun File Format
Revisions:
* 1.1 - [Jan Kotas](https://github.com/jkotas) - 2015
* 3.1 - [Tomas Rylek](https://github.com/trylek) - 2019
* 3.2 - [Tomas Rylek](https://github.com/trylek) - 2019

# Introduction

This document describes ReadyToRun format implemented in CoreCLR as of June 2019.
This document describes ReadyToRun format 3.1 implemented in CoreCLR as of June 2019 and not yet
implemented proposed extensions 3.2 for the support of composite R2R file format.
**Composite R2R file format** has basically the same structure as the traditional R2R file format
defined in earlier revisions except that the output file represents a larger number of input MSIL
assemblies compiled together as a logical unit.

**Note**: The addition of the new composite R2R file format flavor doesn't require bumping up the
major ReadyToRun format version number. This is because, according to the format definition, the
composite R2R file format doesn't technically conform to the single-input R2R supported by older
versions of CoreCLR so there's no risk of *"the old loader"* messing up by incorrectly trying to
run *"the new file"*. The only downside is that the *"new composite R2R file"* won't run with the
*"old loader"* and thus it will somewhat violate the design principle that R2R is a mere code cache.

# PE Headers and CLI Headers
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it is implicit but all PE sections will be merged. One consequence for debuggers and profilers will be searching through the Debug Directory. All such tools I'm familiar with read through the first two entries and rely on the order. I believe both PerfView and WPA expect max 2 entries (one for MSIL, one for precompiled code) so no action required other than maybe a mental note that if we decide to get fancy with how we merge pe sections.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the precompiled code, I believe that should get merged automatically by the compiler. For the MSIL it's somewhat more tricky - I think it should be no problem to put all MSIL metadata in the same section - in fact it's probably the logical thing to do - but it's not immediately obvious whether it will automatically fix the entire problem - e.g. if PerfView expects the entire MSIL section to be a single MSIL metadata block, it still wouldn't work.


Expand All @@ -20,6 +32,16 @@ customizations:

The image contains full copy of the IL and metadata that it was generated from.

For **single-file R2R PE files**, the COR header and ECMA 335 metadata pointed to by the COM
descriptor data directory item in the COFF header represents the actual input MSIL metadata of
the compiled module.

For **composite R2R files** (denoted by the `READYTORUN_FLAG_COMPOSITE` flag in the R2R header)
there is no global COR header (as there are potentially multiple metadata blocks in the file).
The ReadyToRun header structure is pointed to by the well-known export symbol `RTR_HEADER`.
The "actual" metadata for the individual component assemblies is accessed via the R2R section
`READYTORUN_SECTION_ASSEMBLIES`.

## Future Improvements

The limitations of the current format are:
Expand Down Expand Up @@ -80,6 +102,7 @@ native code from image of version 3.0.
| Flag | Value | Description
|:----------------------------------------|-----------:|:-----------
| READYTORUN_FLAG_PLATFORM_NEUTRAL_SOURCE | 0x00000001 | Set if the original IL image was platform neutral. The platform neutrality is part of assembly name. This flag can be used to reconstruct the full original assembly name.
| READYTORUN_FLAG_COMPOSITE | 0x00000002 | The image represents a composite R2R file resulting from a combined compilation of a larger number of input MSIL assemblies.

## READYTORUN_SECTION

Expand Down Expand Up @@ -119,6 +142,7 @@ enum ReadyToRunSectionType
READYTORUN_SECTION_PROFILEDATA_INFO = 111, // Added in V2.2
READYTORUN_SECTION_MANIFEST_METADATA = 112, // Added in V2.3
READYTORUN_SECTION_ATTRIBUTEPRESENCE = 113, // Added in V3.1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ATTRIBUTEPRESENCE section is questionable in composite mode as implemented now, but it could be tweaked to be viable by adjusting the hashing function to include which assembly was a token associated with.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's what we're also discussing here with @jkotas right now - basically some of the R2R sections need to be per-assembly in the composite case. One special case there is the input MSIL metadata that previously used to be the global singleton addressed by the COM descriptor directory; newly this also needs to be per component assembly. It would be ideal to end up with a generally usable and sufficiently performant design covering all the necessary tables.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now I marked the available types, method entrypoints and the MSIL metadata as eligible; and I added a TODO for some I was unsure about; ATTRIBUTEPRESENCE is a big todo right now by itself, thanks for pointing it out. According to our current discussion with JanK we should probably add a jagged array of per component assembly sections to the composite file. As a potential alternative approach I suggested dual encoding - switching over the encoding for a known subset of the R2R sections to adopt the per-assembly behavior, most likely by means of an additional indirection.

READYTORUN_SECTION_ASSEMBLIES = 114, // Added in V3.2
};
```

Expand Down Expand Up @@ -308,7 +332,7 @@ filled before method can be executed executing.
The index of the method is shift left by 1 bit, with the low bit indicating whether the list of slots to fixup
follows. The list of slots is encoded as follows (same encoding as used by NGen):

``
```
READYTORUN_IMPORT_SECTIONS absolute index
absolute slot index
slot index delta
Expand All @@ -328,7 +352,7 @@ READYTORUN_IMPORT_SECTIONS index delta
slot delta
0
0
``
```

The fixup list is a stream of integers encoded as nibbles (1 nibble = 4 bits). 3 bits of a nibble are used to
store 3 bits of the value, and the top bit indicates if the following nibble contains rest of the value. If the
Expand All @@ -339,6 +363,10 @@ means that the i-th value is the sum of values [1..i].

The list is terminated by a 0 (0 is not meaningful as valid delta).

**Note:** This section is only present in single-file R2R files. In composite R2R files created
by compiling multiple input MSIL assemblies, method entrypoints need to be split by assembly and
are addressed through `READYTORUN_SECTION_ASSEMBLIES` section instead.

## READYTORUN_SECTION_EXCEPTION_INFO

Exception handling information. This section contains array of
Expand Down Expand Up @@ -391,11 +419,26 @@ This section contains a native hashtable of all defined & export types within th
| 0 | defined type
| 1 | exported type

The version-resilient hashing algorithm used for hashing the type names is implemented in [vm/versionresilienthashcode.cpp](https://github.com/dotnet/coreclr/blob/ec2a74e7649f1c0ecff32ce86724bf3ca80bfd46/src/vm/versionresilienthashcode.cpp#L75).
The version-resilient hashing algorithm used for hashing the type names is implemented in
[vm/versionresilienthashcode.cpp](https://github.com/dotnet/coreclr/blob/ec2a74e7649f1c0ecff32ce86724bf3ca80bfd46/src/vm/versionresilienthashcode.cpp#L75).

**Note:** This section is only present in single-file R2R files. In composite R2R files created
by compiling multiple input MSIL assemblies, the available types need to be split by assembly
and are addressed through `READYTORUN_SECTION_ASSEMBLIES` section instead.

## READYTORUN_SECTION_INSTANCE_METHOD_ENTRYPOINTS

This section contains a native hashtable of all generic method instantiations compiled into the R2R executable. The key is the method instance signature; the appropriate version-resilient hash code calculation is implemented in [vm/versionresilienthashcode.cpp](https://github.com/dotnet/coreclr/blob/ec2a74e7649f1c0ecff32ce86724bf3ca80bfd46/src/vm/versionresilienthashcode.cpp#L128); the value, represented by the `EntryPointWithBlobVertex` class, stores the method index in the runtime function table, the fixups blob and a blob encoding the method signature.
This section contains a native hashtable of all generic method instantiations compiled into
the R2R executable. The key is the method instance signature; the appropriate version-resilient
hash code calculation is implemented in
[vm/versionresilienthashcode.cpp](https://github.com/dotnet/coreclr/blob/ec2a74e7649f1c0ecff32ce86724bf3ca80bfd46/src/vm/versionresilienthashcode.cpp#L128);
the value, represented by the `EntryPointWithBlobVertex` class, stores the method index in the
runtime function table, the fixups blob and a blob encoding the method signature.

**Note:** In contrast to non-generic method entrypoints, this section is executable-wide for
composite R2R images. It represents all generics needed by all assemblies within the composite
executable. As mentioned elsewhere in this document, CoreCLR runtime requires changes to
properly look up methods stored in this section in the composite R2R case.

## READYTORUN_SECTION_INLINING_INFO

Expand All @@ -407,11 +450,21 @@ This section contains a native hashtable of all generic method instantiations co

## READYTORUN_SECTION_MANIFEST_METADATA

Manifest metadata is an [ECMA-335] metadata blob containing extra reference assemblies within the version bubble introduced by inlining on top of assembly references stored in the input MSIL. As of R2R version 3.1, the metadata is only searched for the AssemblyRef table. This is used to translate module override indices in signatures to the actual reference modules (using either the `READYTORUN_FIXUP_ModuleOverride` bit flag on the signature fixup byte or the `ELEMENT_TYPE_MODULE_ZAPSIG` COR element type).
Manifest metadata is an [ECMA-335] metadata blob containing extra reference assemblies within
the version bubble introduced by inlining on top of assembly references stored in the input MSIL.
As of R2R version 3.1, the metadata is only searched for the AssemblyRef table. This is used to
translate module override indices in signatures to the actual reference modules (using either
the `READYTORUN_FIXUP_ModuleOverride` bit flag on the signature fixup byte or the
`ELEMENT_TYPE_MODULE_ZAPSIG` COR element type).

**Disclaimer:** The manifest metadata is a new feature that hasn't shipped yet; it involves straightforward adaptation of a fragile nGen technology to ReadyToRun images as an expedite means for enabling new functionality (larger version bubble support). The precise details of this encoding are still work in progress and likely to further evolve.
**Disclaimer:** The manifest metadata is a new feature that hasn't shipped yet; it involves
straightforward adaptation of a fragile NGen technology to ReadyToRun images as an expedite
means for enabling new functionality (larger version bubble support). The precise details of
this encoding are still work in progress and likely to further evolve.

**Note:** It doesn't make sense to store references to assemblies external to the version bubble in the manifest metadata as there's no guarantee that their metadata token values remain constant; thus we cannot encode signatures relative to them.
**Note:** It doesn't make sense to store references to assemblies external to the version bubble
in the manifest metadata as there's no guarantee that their metadata token values remain
constant; thus we cannot encode signatures relative to them.

The module override index translation algorithm is as follows (**ILAR** = *the number of `AssemblyRef` rows in the input MSIL*):

Expand All @@ -427,11 +480,36 @@ The module override index translation algorithm is as follows (**ILAR** = *the n

**TODO**: document attribute presence encoding

**Note**: We already know this table uses assembly-relative token encoding so it has similar
characteristics like `READYTORUN_SECTION_AVAILABLE_TYPES` or `READYTORUN_SECTION_METHOD_ENTRYPOINTS`.
No matter what component assembly-relative encoding we end up choosing for these tables, we
should use the same encoding for ATTRIBUTEPRESENCE.

## READYTORUN_SECTION_ASSEMBLIES (v3.2+)

This section is only present in composite R2R files. It is a straight binary array of the
entries `READYTORUN_SECTION_ASSEMBLIES_ENTRY` parallel to the indices in the manifest metadata
AssemblyRef table in the sense that it's a linear table where the row indices correspond to the
equivalent AssemblyRef indices. Just like in the AssemblyRef ECMA 335 table, the indexing is
1-based (the first entry in the table corresponds to index 1).

```C++
struct READYTORUN_SECTION_ASSEMBLIES_ENTRY
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant to make this look like this:

    DWORD                   NumberOfSections;

    // Array of sections follows. The array entries are sorted by Type
    // READYTORUN_SECTION   Sections[];

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I apologize, I'm still not fully getting what you're proposing.

a) We can make the sections a jagged array so that each component assembly can potentially have a different NumberOfSections. That seems to correspond to what you're suggesting - but then the table will either require an additional indirection level for the section array, or the individual per-assembly entries would have different lengths and would require linear scanning that is generally perf-concerning.

b) We can make the sections a two-dimensional array but that would correspond to a global value for NumberOfSections (that should, in fact, correspond to a global set of section type ID's). Say, if we put NumberOfSections = 3 (corresponding to method entrypoints, available types and a new previously nonexistent section for the MSIL metadata), READYTORUN_SECTION elements 0-2 would correspond to AssemblyRef #1, 3-5 to AssemblyRef #2, 6-8 to AssemblyRef #3 and so on.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It just occurred to me that yet another way of approaching this would be dual encoding - in the presence of the COMPOSITE flag, the appropriate sections like available types would automatically switch over to a different mode where there's the additional indirection to cater for multiple per component-assembly entries. The advantage of this would be not having the appropriate tables unused in composite mode and not having to introduce a new ASSEMBLIES table, the downside is naturally the dual encoding.

Copy link
Member

@jkotas jkotas Jun 24, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can make the sections a jagged array so that each component assembly can potentially have a different NumberOfSections

This is what I meant.

the individual per-assembly entries would have different lengths and would require linear scanning that is generally perf-concerning.

This is the same structure as what is used in R2R header today. The number of entries is small and this is a top level table that won't be scanned that often.

The advantage of it is that it is easy to version (add and remove from). I think it is pretty safe bet that there are going to be more than the 3 things that are here now.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, but if we put this in the READYTORUN_SECTION_ASSEMBLIES table, it will have the number of entries equal to the number of component assemblies in the composite R2R image i.e. 1500 in Mukul's case. I concur with your overall strategy that we should leave perf optimizations for the future, when we have solid framework for measuring the actual benefits of particular changes, but isn't linear scanning of 1500 assemblies too much of an initial perf risk w.r.t. things like locating a particular MSIL metadata block for a given module override?

But perhaps you're right that the runtime is likely to mitigate some of these by means of various caches and the like and we should initially strive to make the implementation simple so that it's primarily easy to reason about semantically. Perf is a bonus we can always add later assuming we keep it in mind so that some of our design decisions don't block it.

Copy link
Member

@jkotas jkotas Jun 24, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I expect that runtime will need to hash the 1500 entries during startup anyway to understand what is loaded. Hashing 1500 entries is no big deal. It will take like 10 miliseconds. Note that we will be replacing opening of 1500 files in the current state with hashing 1500 entries.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. One other interesting angle is @MichalStrehovsky's change where he's independently adding a precursor of what we're just discussing - a per component-assembly table.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, upon second thinking it might be best to leave that alone, after all that's mostly required for external files, not for the composite file, as discussed in other places on this PR. Originally, when I had a straight struct, I was imagining adding Michal's MVID as one of the fields, but that makes sense no longer.

IMAGE_DATA_DIRECTORY CorHeader; // Input MSIL metadata COR header
IMAGE_DATA_DIRECTORY AvailableTypes; // Available types table
IMAGE_DATA_DIRECTORY MethodEntrypoints; // Method entrypoint table
};
```

**TODO:** It remains to be seen whether `READYTORUN_SECTION_METHODCALL_THUNKS` and / or
`READYTORUN_SECTION_INLINING_INFO` also require changes specific to the composite R2R file format.

# Native Format

Native format is set of encoding patterns that allow persisting type system data in a binary format that is
efficient for runtime access - both in working set and CPU cycles. (Originally designed for and extensively
used by .NET Native.)
Native format is set of encoding patterns that allow persisting type system data in a binary
format that is efficient for runtime access - both in working set and CPU cycles. (Originally
designed for and extensively used by .NET Native.)

## Integer encoding

Expand Down
Loading