Implement GcInfo v2 #6522

swaroop-sridhar · 2016-07-29T03:07:38Z

Ref #4379

This change implements GcInfo version 2 for all platforms that use the
GcInfo library (all architectures other than X86).

Changes are:

Defines ReturnKind enumeration for all platforms
Change the GcInfo encoder library to encode the ReturnKind and ReversePInvokeFrame slot
Change the CM's GcInfo decoder to encode the ReturnKind and ReversePInvokeFrame slot for GCINFO_VERSION 2
Some corrections to GCINFO_MEASUREments
Changes to RYU Jit to provide the correct information to the encoder
Changes to the VM to use the ReturnKind information while hijacking a thread

If ReturnKind is available from GcInfo, new hijack routines are used
Otherwise, fall back to old method (for compatibility)
7) Rework and simplify the thread hijack routines by scanning HijackFrames directly for gcroots
8) Supporting code to implement the above features.

Returning Structs in multiple registers

Hijacking for StructInRegs is currently only implemented for
Unix SystemV ABI Multi-reg struct returns. However, the hijack-workers that use
ReturnKind are ready to handle other platforms (ex: ARM/ARM64 Windows)
once the corresponding HijackTripThread() assembly routines are defined.

The New feature flag: FEATURE_MULTIREG_RETURN is set for platforms where a struct value
can be returned in multiple registers [ex: Windows/Unix ARM/ARM64, Unix-AMD64]
FEATURE_UNIX_AMD64_STRUCT_PASSING is a specific kind of FEATURE_MULTIREG_RETURN
specified by SystemV ABI for AMD64

Compatibility with other JITs

All new GCInfo generated by RYU Jit is in GcInfo version 2
All Ngen images must be regenerated with the new GcInfo version.
Ready-to-run images with old GcInfo will continue to work.
Jit64/X64 uses the GcInfo library, so it generates GcInfo version 2.
However, it doesn't (yet) provide the data to encode the correct ReturnKind
Similar is the case for ARM32 code running on JIT32, and any other JITs
that may be using GcInfo library but not yet modified to use the new API.
So, compatibility is achived using RT_Unset flag.

When ReturnKind is RT_Unset, it means that the JIT did not set
the ReturnKind in the GCInfo, and therefore the VM cannot rely on it,
and must use other mechanisms (similar to GcInfo ver 1) to determine
the Return type's GC information.

Implement GC root scanning for Hijack-frames

This change implements GCScanRoots() method for Hijacke-frames
based on the ReturnKind information available from the GcInfo.
If the exact ReturnKind is not available in the GcInfo, the
thread-suspension logic will compute the ReturnKind based on
the method-signature.

As a result of this change, several hijack-helpers in the VM
are cleaned up. There's only one implementation of HijackWorker()
which handles all returnKinds, and one assembly helper for handling
all methods that return a GC value.

ARM64

Only GcIndfo v2 is reliably supported for ARM64 platform.
The changes to thread-hijack mechanism fixes #6494 for ARM64.

No measurable change in JIT throughput, performance or native-image
size from this change.

swaroop-sridhar · 2016-07-29T04:31:58Z

@dotnet-bot test Windows_NT amd64 checked gcstress0xc

swaroop-sridhar · 2016-07-29T19:23:40Z

@dotnet-bot test Windows_NT amd64 checked gcstress0xc

swaroop-sridhar · 2016-07-29T19:23:53Z

@dotnet-bot test Windows_NT arm64 checked

swaroop-sridhar · 2016-07-29T19:24:01Z

@dotnet-bot test Windows_NT arm64 checked gcstress0xc

swaroop-sridhar · 2016-07-30T02:47:26Z

@dotnet-bot test Windows_NT arm64 checked

swaroop-sridhar · 2016-07-30T02:47:36Z

@dotnet-bot test Windows_NT arm64 checked

swaroop-sridhar · 2016-07-30T02:48:11Z

@dotnet-bot test Windows_NT gcstress0xc

swaroop-sridhar · 2016-07-30T02:48:39Z

@dotnet-bot test Ubuntu gcstress0xc

swaroop-sridhar · 2016-07-30T02:50:03Z

@dotnet-bot test Windows_NT arm64 checked pri1r2r

swaroop-sridhar · 2016-07-30T02:50:11Z

@dotnet-bot test Windows_NT arm64 checked

swaroop-sridhar · 2016-08-01T20:51:16Z

@jkotas please review.
CC: @dotnet/jit-contrib

jkotas · 2016-08-01T20:58:18Z

src/debug/daccess/nidump.cpp

@@ -9440,7 +9440,7 @@ void NativeImageDumper::DumpReadyToRunMethod(PCODE pEntryPoint, PTR_RUNTIME_FUNC
        GCDump gcDump(GCINFO_VERSION);
        gcDump.gcPrintf = stringOutFn;
 #if !defined(_TARGET_X86_) && defined(USE_GC_INFO_DECODER)
-        GcInfoDecoder gcInfoDecoder({ curGCInfoPtr, GCINFO_VERSION }, DECODE_CODE_LENGTH, 0);
+        GcInfoDecoder gcInfoDecoder({ curGCInfoPtr, GCINFO_VERSION }, DECODE_CODE_LENGTH);


This is dumping method in R2R image. Should this take the version that matches the file format version?

(It is not crucial to fix this because of it is nidump ... if it is hard to do, comment would be nice at least.)

janvorli · 2016-08-04T21:02:00Z

LGTM

swaroop-sridhar · 2016-08-04T21:54:45Z

@dotnet-bot test Ubuntu gcstress0xc

swaroop-sridhar · 2016-08-04T21:59:20Z

@dotnet-bot test Windows_NT gcstress0xc
@dotnet-bot test Ubuntu gcstress0xc
@dotnet-bot test Windows_NT arm64 Checked
@dotnet-bot test Windows_NT arm64 checked pri1r2r
@dotnet-bot test Windows_NT arm64 Release
@dotnet-bot test Windows_NT arm64 checked gcstress0xc

swaroop-sridhar · 2016-08-04T22:00:01Z

@dotnet-bot test Windows_NT arm64 Checked
@dotnet-bot test Windows_NT arm64 checked gcstress0xc

swaroop-sridhar · 2016-08-04T22:46:29Z

@dotnet-bot test OSX x64 Checked Build and Test

swaroop-sridhar · 2016-08-04T22:46:39Z

@dotnet-bot test Ubuntu x64 Checked Build and Test

swaroop-sridhar · 2016-08-04T22:52:53Z

@dotnet-bot test Linux ARM Emulator Cross Debug Build

swaroop-sridhar · 2016-08-04T23:24:05Z

@dotnet-bot test Windows_NT arm64 Checked

briansull · 2016-08-04T23:51:06Z

src/vm/threadsuspend.cpp

+            }
+        }
+        ReturnKind structReturnKind = GetStructReturnKind(regKinds[0], regKinds[1]);
+        return structReturnKind;


You told me that this section is there for GC info v1]

Can you add a comment so that we can understand why there is no ARM64 section here.

This code only exists to support GcInfo v1 images, which are not supported for ARM64.
I'll add a comment about it.
ARM64 struct-in-regs scanning happens in HijackFrame::GcScanRoots()

briansull · 2016-08-04T23:54:01Z

LGTM

swaroop-sridhar · 2016-08-04T23:59:43Z

You mean the ARM64 work for struct-in-regs return, right?
There's and issue for it #6494. and it'll be fixed by this change.

swaroop-sridhar · 2016-08-05T00:02:58Z

Windows NT GCStress 0xC:
The two failures look like they are pre-existing:

Random failure writing to protected memory in Console.WriteLine()
Failure in GetGenerationWR2.exe
Here is the last week's run for comparison.

swaroop-sridhar · 2016-08-05T01:15:40Z

@dotnet-bot test Windows_NT arm64 Checked

swaroop-sridhar · 2016-08-05T03:04:07Z

@dotnet-bot test Windows_NT arm64 Checked

swaroop-sridhar · 2016-08-05T05:46:21Z

@dotnet-bot test Linux ARM Emulator Cross Debug Build

swaroop-sridhar · 2016-08-05T06:40:06Z

@dotnet-bot test Windows_NT arm64 Checked

swaroop-sridhar · 2016-08-05T06:40:50Z

@dotnet-bot test Windows_NT arm64 Checked

swaroop-sridhar · 2016-08-05T06:42:24Z

@dotnet-bot Details Linux ARM Emulator Cross Debug Build

swaroop-sridhar · 2016-08-05T18:56:33Z

@dotnet-bot test Windows_NT arm64 checked

Ref #4379 This change implements GcInfo version 2 for all platforms that use the GcInfo library (all architectures other than X86). Changes are: 1) Defines ReturnKind enumeration for all platforms 2) Change the GcInfo encoder library to encode the ReturnKind and ReversePInvokeFrame slot 3) Change the CM's GcInfo decoder to encode the ReturnKind and ReversePInvokeFrame slot for GCINFO_VERSION 2 4) Some corrections to GCINFO_MEASUREments 5) Changes to RYU Jit to provide the correct information to the encoder 6) Changes to the VM to use the ReturnKind information while hijacking a thread - If ReturnKind is available from GcInfo, new hijack routines are used - Otherwise, fall back to old method (for compatibility) 7) Rework and simplify the thread hijack routines by scanning HijackFrames directly for gcroots 8) Supporting code to implement the above features. Returning Structs in multiple registers Hijacking for StructInRegs is currently only implemented for Unix SystemV ABI Multi-reg struct returns. However, the hijack-workers that use ReturnKind are ready to handle other platforms (ex: ARM/ARM64 Windows) once the corresponding HijackTripThread() assembly routines are defined. The New feature flag: FEATURE_MULTIREG_RETURN is set for platforms where a struct value can be returned in multiple registers [ex: Windows/Unix ARM/ARM64, Unix-AMD64] FEATURE_UNIX_AMD64_STRUCT_PASSING is a specific kind of FEATURE_MULTIREG_RETURN specified by SystemV ABI for AMD64 Compatibility with other JITs - All new GCInfo generated by RYU Jit is in GcInfo version 2 - All Ngen images must be regenerated with the new GcInfo version. - Ready-to-run images with old GcInfo will continue to work. - Jit64/X64 uses the GcInfo library, so it generates GcInfo version 2. However, it doesn't (yet) provide the data to encode the correct ReturnKind Similar is the case for ARM32 code running on JIT32, and any other JITs that may be using GcInfo library but not yet modified to use the new API. So, compatibility is achived using RT_Unset flag. When ReturnKind is RT_Unset, it means that the JIT did not set the ReturnKind in the GCInfo, and therefore the VM cannot rely on it, and must use other mechanisms (similar to GcInfo ver 1) to determine the Return type's GC information. Implement GC root scanning for Hijack-frames This change implements GCScanRoots() method for Hijacke-frames based on the ReturnKind information available from the GcInfo. If the exact ReturnKind is not available in the GcInfo, the thread-suspension logic will compute the ReturnKind based on the method-signature. As a result of this change, several hijack-helpers in the VM are cleaned up. There's only one implementation of HijackWorker() to handle all returnKinds. This change also simplifies the thread-hijack logic by using a single assembly helper OnHijackTripThread() in most cases. The only other helper used is for X86 floating point return values for save/restoring the top of the FP stack. ARM64 Only GcIndfo v2 is reliably supported for ARM64 platform. The changes to thread-hijack mechanism fixes #6494 for ARM64. No measurable change in JIT throughput, performance or native-image size from this change.

PR dotnet#6522 changed from OnHijackScalarTripThread to OnHijackTripThread; however, missed changing this definition for arm windows.

Implement GcInfo v2 Commit migrated from dotnet/coreclr@6bd9557

PR dotnet/coreclr#6522 changed from OnHijackScalarTripThread to OnHijackTripThread; however, missed changing this definition for arm windows. Commit migrated from dotnet/coreclr@49b8766

dnfclas added the cla-already-signed label Jul 29, 2016

swaroop-sridhar added the * NO MERGE * The PR is not ready for merge yet (see discussion for detailed reasons) label Jul 29, 2016

swaroop-sridhar force-pushed the gcinfov2 branch 2 times, most recently from a5a1877 to 8f71f63 Compare July 29, 2016 04:30

swaroop-sridhar force-pushed the gcinfov2 branch from 8f71f63 to ea4efbb Compare July 29, 2016 19:23

swaroop-sridhar force-pushed the gcinfov2 branch 3 times, most recently from 1a6ca38 to 552217f Compare July 29, 2016 20:31

JosephTremoulet mentioned this pull request Jul 29, 2016

Optimize bounds checks with correct assertion set #6529

Merged

swaroop-sridhar force-pushed the gcinfov2 branch 4 times, most recently from 5091c8b to a605dc8 Compare July 30, 2016 02:41

swaroop-sridhar removed the * NO MERGE * The PR is not ready for merge yet (see discussion for detailed reasons) label Aug 1, 2016

jkotas reviewed Aug 1, 2016
View reviewed changes

swaroop-sridhar force-pushed the gcinfov2 branch from a605dc8 to 24135e1 Compare August 3, 2016 22:27

swaroop-sridhar force-pushed the gcinfov2 branch from f10be42 to 4a49994 Compare August 4, 2016 21:59

briansull reviewed Aug 4, 2016
View reviewed changes

swaroop-sridhar force-pushed the gcinfov2 branch from 4a49994 to bbaf762 Compare August 5, 2016 01:14

swaroop-sridhar force-pushed the gcinfov2 branch from bbaf762 to d302e64 Compare August 5, 2016 21:06

swaroop-sridhar merged commit 6bd9557 into dotnet:master Aug 5, 2016

jashook pushed a commit to jashook/coreclr that referenced this pull request Aug 8, 2016

Fix for missing symbol OnHijackTripThread

417f681

PR dotnet#6522 changed from OnHijackScalarTripThread to OnHijackTripThread; however, missed changing this definition for arm windows.

jashook mentioned this pull request Aug 8, 2016

Fix for missing symbol OnHijackTripThread #6656

Merged

jashook pushed a commit to jashook/coreclr that referenced this pull request Aug 8, 2016

Fix for missing symbol OnHijackTripThread

49b8766

PR dotnet#6522 changed from OnHijackScalarTripThread to OnHijackTripThread; however, missed changing this definition for arm windows.

picenka21 pushed a commit to picenka21/runtime that referenced this pull request Feb 18, 2022

Merge pull request dotnet/coreclr#6522 from swaroop-sridhar/gcinfov2

ab6f722

Implement GcInfo v2 Commit migrated from dotnet/coreclr@6bd9557

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement GcInfo v2 #6522

Implement GcInfo v2 #6522

swaroop-sridhar commented Jul 29, 2016 •

edited

Loading

swaroop-sridhar commented Jul 29, 2016

swaroop-sridhar commented Jul 29, 2016

swaroop-sridhar commented Jul 29, 2016

swaroop-sridhar commented Jul 29, 2016

swaroop-sridhar commented Jul 30, 2016

swaroop-sridhar commented Jul 30, 2016

swaroop-sridhar commented Jul 30, 2016

swaroop-sridhar commented Jul 30, 2016

swaroop-sridhar commented Jul 30, 2016

swaroop-sridhar commented Jul 30, 2016

swaroop-sridhar commented Aug 1, 2016

jkotas Aug 1, 2016

janvorli commented Aug 4, 2016

swaroop-sridhar commented Aug 4, 2016

swaroop-sridhar commented Aug 4, 2016

swaroop-sridhar commented Aug 4, 2016

swaroop-sridhar commented Aug 4, 2016

swaroop-sridhar commented Aug 4, 2016

swaroop-sridhar commented Aug 4, 2016

swaroop-sridhar commented Aug 4, 2016

briansull Aug 4, 2016 •

edited

Loading

swaroop-sridhar Aug 5, 2016

briansull commented Aug 4, 2016 •

edited

Loading

swaroop-sridhar commented Aug 4, 2016

swaroop-sridhar commented Aug 5, 2016

swaroop-sridhar commented Aug 5, 2016

swaroop-sridhar commented Aug 5, 2016

swaroop-sridhar commented Aug 5, 2016

swaroop-sridhar commented Aug 5, 2016

swaroop-sridhar commented Aug 5, 2016

swaroop-sridhar commented Aug 5, 2016

swaroop-sridhar commented Aug 5, 2016

Implement GcInfo v2 #6522

Implement GcInfo v2 #6522

Conversation

swaroop-sridhar commented Jul 29, 2016 • edited Loading

swaroop-sridhar commented Jul 29, 2016

swaroop-sridhar commented Jul 29, 2016

swaroop-sridhar commented Jul 29, 2016

swaroop-sridhar commented Jul 29, 2016

swaroop-sridhar commented Jul 30, 2016

swaroop-sridhar commented Jul 30, 2016

swaroop-sridhar commented Jul 30, 2016

swaroop-sridhar commented Jul 30, 2016

swaroop-sridhar commented Jul 30, 2016

swaroop-sridhar commented Jul 30, 2016

swaroop-sridhar commented Aug 1, 2016

jkotas Aug 1, 2016

Choose a reason for hiding this comment

janvorli commented Aug 4, 2016

swaroop-sridhar commented Aug 4, 2016

swaroop-sridhar commented Aug 4, 2016

swaroop-sridhar commented Aug 4, 2016

swaroop-sridhar commented Aug 4, 2016

swaroop-sridhar commented Aug 4, 2016

swaroop-sridhar commented Aug 4, 2016

swaroop-sridhar commented Aug 4, 2016

briansull Aug 4, 2016 • edited Loading

Choose a reason for hiding this comment

swaroop-sridhar Aug 5, 2016

Choose a reason for hiding this comment

briansull commented Aug 4, 2016 • edited Loading

swaroop-sridhar commented Aug 4, 2016

swaroop-sridhar commented Aug 5, 2016

swaroop-sridhar commented Aug 5, 2016

swaroop-sridhar commented Aug 5, 2016

swaroop-sridhar commented Aug 5, 2016

swaroop-sridhar commented Aug 5, 2016

swaroop-sridhar commented Aug 5, 2016

swaroop-sridhar commented Aug 5, 2016

swaroop-sridhar commented Aug 5, 2016

swaroop-sridhar commented Jul 29, 2016 •

edited

Loading

briansull Aug 4, 2016 •

edited

Loading

briansull commented Aug 4, 2016 •

edited

Loading