[cdac] ExecutionManager contract and lookup map tests

*  add RangeSectionMap docs

*  exhaustively test RangeSectionMap.EffectiveBitsForLevel

*  fix lookup in RangeSection.Find

    RangeSectionLookupAlgorithm.FindFragment finds the slot containing the range section fragment pointer.
    Have to dereference it once to get the actual RangeSectionFragment pointer from the slot.

    This "worked" before because RangeSectionFragment.Next is at offset 0, so the first lookup would have a garbage range, so we would follow the "next" field and get to the correct fragment

*   make a testable RangeSectionMap.FindFragmentInternal

*   brief nibble map summary

*   [cdac] Implement NibbleMap lookup and tests

    The execution manager uses a nibble map to quickly map program counter
    pointers to the beginnings of the native code for the managed method.

    Implement the lookup algorithm for a nibble map.

    Start adding unit tests for the nibble map

    Also for testing in MockMemorySpace simplify ReaderContext, there's nothing special about the descriptor HeapFragments anymore.  We can use a uniform reader.

*   NibbleMap: fix bug and add test

    Previously we incorrectly computed the prior map index when doing the backward linear search

*   [testing] display Target values in hex in debugger views

*   MockMemorySpace: simplify ReaderContext

    there's nothing special about the descriptor HeapFragments anymore.  We can use a uniform reader

*   refactor ExecutionManager

*   ExecutionManager contract

    the presence of RangeSection.R2RModule is a discriminator for whether we're looking at EE code or R2R code

*   WIP NibbleMap

*   WIP JitCodeToMethodInfo

*   WIP RangeSectionMap

docs/design/datacontracts/ExecutionManager.md

@@ -6,12 +6,80 @@ managed method corresponding to that address.
 
 ## APIs of contract
 
-**TODO**
+```csharp
+internal struct EECodeInfoHandle
+{
+    // no public constructor
+    public readonly TargetPointer Address;
+    internal EECodeInfoHandle(TargetPointer address) => Address = address;
+}
+```
+
+```csharp
+    // Collect execution engine info for a code block that includes the given instruction pointer.
+    // Return a handle for the information, or null if an owning code block cannot be found.
+    EECodeInfoHandle? GetEECodeInfoHandle(TargetCodePointer ip);
+    // Get the method descriptor corresponding to the given code block
+    TargetPointer GetMethodDesc(EECodeInfoHandle codeInfoHandle) => throw new NotImplementedException();
+    // Get the instruction pointer address of the start of the code block
+    TargetCodePointer GetStartAddress(EECodeInfoHandle codeInfoHandle) => throw new NotImplementedException();
+```
 
 ## Version 1
 
+The execution manager uses two data structures to map the entire target address space to native executable code.
+The range section map is used to partition the address space into large chunks which point to range section fragments.  Each chunk is relatively large.  If there is any executable code in the chunk, the chunk will contain one or more range section fragments that cover subsets of the chunk.  Conversely if a massive method is JITed a single range section fragment may span multiple adjacent chunks.
+
+Within a range section fragment, a nibble map structure is used to map arbitrary IP addresses back to the start of the method (and to the code header which immediately preceeeds the entrypoint to the code).
+
+Data descriptors used:
+| Data Descriptor Name | Field | Meaning |
+| --- | --- | --- |
+| RangeSectionMap | TopLevelData | pointer to the outermost RangeSection |
+| RangeSectionFragment| ? | ? |
+| RangeSection | ? | ? |
+| RealCodeHeader | ? | ? |
+| HeapList | ? | ? |
+
+
+
+Global variables used:
+| Global Name | Type | Purpose |
+| --- | --- | --- |
+| ExecutionManagerCodeRangeMapAddress | TargetPointer | Pointer to the global RangeSectionMap
+| StubCodeBlockLast | uint8 | Maximum sentinel code header value indentifying a stub code block
+
+Contracts used:
+| Contract Name |
+| --- |
+
+```csharp
+```
+
 **TODO** Methods
 
+### RangeSectionMap
+
+The range section map logically partitions the entire 32-bit or 64-bit addressable space into chunks.
+The map is implemented with multiple levels, where the bits of an address are used as indices into an array of pointers.  The upper levels of the map point to the next level down. At the lowest level of the map, the pointers point to the first range section fragment containing addresses in the chunk.
+
+On 32-bit targets a 2 level map is used
+
+| 31-24 | 23-16 | 15-0 |
+|:----:|:----:|:----:|
+| L2 | L1 | chunk |
+
+That is, level 2 in the map has 256 entries pointing to level 1 maps (or null if there's nothing allocated), each level 1 map has 256 entries pointing covering a 64 KiB chunk and pointing to a linked list of range section fragments that fall within that 64 KiB chunk.
+
+On 64-bit targets, we take advantage of the fact that the most architectures don't support a full 64-bit addressable space: arm64 supports 52 bits of addressable memoryy and x86-64 supports 57 bits.  The runtime ignores the top bits 63-57 and uses 5 levels of mapping
+
+| 63-57 | 56-49 | 48-41 | 40-33 | 32-25 | 24-17 | 16-0 |
+|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:----:|
+| unused | L5 | L4 | L3 | L2 | L1 | chunk |
+
+That is, level 5 has 256 entires pointing to level 4 maps (or nothing if there's no
+code allocated in that address range), level 4 entires point to level 3 maps and so on.  Each level 1 map has 256 entries cover a 128 KiB chunk and pointing to a linked list of range section fragments that fall within that 128 KiB chunk.
+
 ### NibbleMap
 
 Version 1 of this contract depends on a "nibble map" data structure

src/coreclr/debug/runtimeinfo/.editorconfig

@@ -0,0 +1,2 @@
+[contracts.jsonc]
+indent_size = 2

src/coreclr/debug/runtimeinfo/contracts.jsonc

@@ -12,6 +12,7 @@
   "DacStreams": 1,
   "EcmaMetadata" : 1,
   "Exception": 1,
+  "ExecutionManager": 1,
   "Loader": 1,
   "Object": 1,
   "RuntimeTypeSystem": 1,

src/coreclr/debug/runtimeinfo/datadescriptor.h

@@ -335,6 +335,47 @@ CDAC_TYPE_INDETERMINATE(DynamicMethodDesc)
 CDAC_TYPE_FIELD(DynamicMethodDesc, /*pointer*/, MethodName, cdac_data<DynamicMethodDesc>::MethodName)
 CDAC_TYPE_END(DynamicMethodDesc)
 
+CDAC_TYPE_BEGIN(CodePointer)
+CDAC_TYPE_SIZE(sizeof(PCODE))
+CDAC_TYPE_END(CodePointer)
+
+CDAC_TYPE_BEGIN(RangeSectionMap)
+CDAC_TYPE_INDETERMINATE(RangeSectionMap)
+CDAC_TYPE_FIELD(RangeSectionMap, /*pointer*/, TopLevelData, cdac_data<RangeSectionMap>::TopLevelData)
+CDAC_TYPE_END(RangeSectionMap)
+
+CDAC_TYPE_BEGIN(RangeSectionFragment)
+CDAC_TYPE_INDETERMINATE(RangeSectionFragment)
+CDAC_TYPE_FIELD(RangeSectionFragment, /*pointer*/, RangeBegin, cdac_data<RangeSectionMap>::RangeSectionFragment::RangeBegin)
+CDAC_TYPE_FIELD(RangeSectionFragment, /*pointer*/, RangeEndOpen, cdac_data<RangeSectionMap>::RangeSectionFragment::RangeEndOpen)
+CDAC_TYPE_FIELD(RangeSectionFragment, /*pointer*/, RangeSection, cdac_data<RangeSectionMap>::RangeSectionFragment::RangeSection)
+CDAC_TYPE_FIELD(RangeSectionFragment, /*pointer*/, Next, cdac_data<RangeSectionMap>::RangeSectionFragment::Next)
+CDAC_TYPE_END(RangeSectionFragment)
+
+CDAC_TYPE_BEGIN(RangeSection)
+CDAC_TYPE_INDETERMINATE(RangeSection)
+CDAC_TYPE_FIELD(RangeSection, /*pointer*/, RangeBegin, cdac_data<RangeSection>::RangeBegin)
+CDAC_TYPE_FIELD(RangeSection, /*pointer*/, RangeEndOpen, cdac_data<RangeSection>::RangeEndOpen)
+CDAC_TYPE_FIELD(RangeSection, /*pointer*/, NextForDelete, cdac_data<RangeSection>::NextForDelete)
+CDAC_TYPE_FIELD(RangeSection, /*pointer*/, JitManager, cdac_data<RangeSection>::JitManager)
+CDAC_TYPE_FIELD(RangeSection, /*int32_t*/, Flags, cdac_data<RangeSection>::Flags)
+CDAC_TYPE_FIELD(RangeSection, /*pointer*/, HeapList, cdac_data<RangeSection>::HeapList)
+CDAC_TYPE_FIELD(RangeSection, /*pointer*/, R2RModule, cdac_data<RangeSection>::R2RModule)
+CDAC_TYPE_END(RangeSection)
+
+CDAC_TYPE_BEGIN(RealCodeHeader)
+CDAC_TYPE_INDETERMINATE(RealCodeHeader)
+CDAC_TYPE_FIELD(RealCodeHeader, /*pointer*/, MethodDesc, offsetof(RealCodeHeader, phdrMDesc))
+CDAC_TYPE_END(RealCodeHeader)
+
+CDAC_TYPE_BEGIN(CodeHeapListNode)
+CDAC_TYPE_FIELD(CodeHeapListNode, /*pointer*/, Next, offsetof(HeapList, hpNext))
+CDAC_TYPE_FIELD(CodeHeapListNode, /*pointer*/, StartAddress, offsetof(HeapList, startAddress))
+CDAC_TYPE_FIELD(CodeHeapListNode, /*pointer*/, EndAddress, offsetof(HeapList, endAddress))
+CDAC_TYPE_FIELD(CodeHeapListNode, /*pointer*/, MapBase, offsetof(HeapList, mapBase))
+CDAC_TYPE_FIELD(CodeHeapListNode, /*pointer*/, HeaderMap, offsetof(HeapList, pHdrMap))
+CDAC_TYPE_END(CodeHeapListNode)
+
 CDAC_TYPES_END()
 
 CDAC_GLOBALS_BEGIN()
@@ -364,6 +405,7 @@ CDAC_GLOBAL(DirectorySeparator, uint8, (uint8_t)DIRECTORY_SEPARATOR_CHAR_A)
 CDAC_GLOBAL(MethodDescAlignment, uint64, MethodDesc::ALIGNMENT)
 CDAC_GLOBAL(ObjectHeaderSize, uint64, OBJHEADER_SIZE)
 CDAC_GLOBAL(SyncBlockValueToObjectOffset, uint16, OBJHEADER_SIZE - cdac_data<ObjHeader>::SyncBlockValue)
+CDAC_GLOBAL(StubCodeBlockLast, uint8, STUB_CODE_BLOCK_LAST)
 CDAC_GLOBAL_POINTER(ArrayBoundsZero, cdac_data<ArrayBase>::ArrayBoundsZero)
 CDAC_GLOBAL_POINTER(ExceptionMethodTable, &::g_pExceptionClass)
 CDAC_GLOBAL_POINTER(FreeObjectMethodTable, &::g_pFreeObjectMethodTable)
@@ -373,6 +415,7 @@ CDAC_GLOBAL_POINTER(StringMethodTable, &::g_pStringClass)
 CDAC_GLOBAL_POINTER(SyncTableEntries, &::g_pSyncTable)
 CDAC_GLOBAL_POINTER(MiniMetaDataBuffAddress, &::g_MiniMetaDataBuffAddress)
 CDAC_GLOBAL_POINTER(MiniMetaDataBuffMaxSize, &::g_MiniMetaDataBuffMaxSize)
+CDAC_GLOBAL_POINTER(ExecutionManagerCodeRangeMapAddress, cdac_data<ExecutionManager>::CodeRangeMapAddress)
 CDAC_GLOBALS_END()
 
 #undef CDAC_BASELINE

src/coreclr/vm/codeman.h

@@ -552,6 +552,8 @@ class Range
     {
         return end;
     }
+
+    template<typename T> friend struct ::cdac_data;
 };
 
 struct RangeSection
@@ -623,6 +625,19 @@ struct RangeSection
 
 
     RangeSection* _pRangeSectionNextForDelete = NULL; // Used for adding to the cleanup list
+
+    template<typename T> friend struct ::cdac_data;
+};
+
+template<> struct cdac_data<RangeSection>
+{
+    static constexpr size_t RangeBegin = offsetof(RangeSection, _range.begin);
+    static constexpr size_t RangeEndOpen = offsetof(RangeSection, _range.end);
+    static constexpr size_t NextForDelete = offsetof(RangeSection, _pRangeSectionNextForDelete);
+    static constexpr size_t JitManager = offsetof(RangeSection, _pjit);
+    static constexpr size_t Flags = offsetof(RangeSection, _flags);
+    static constexpr size_t HeapList = offsetof(RangeSection, _pHeapList);
+    static constexpr size_t R2RModule = offsetof(RangeSection, _pR2RModule);
 };
 
 enum class RangeSectionLockState
@@ -832,7 +847,7 @@ class RangeSectionMap
             {
                 // Upgrade to non-collectible
 #ifdef _DEBUG
-                TADDR initialValue = 
+                TADDR initialValue =
 #endif
                 InterlockedCompareExchangeT(&_ptr, ptr - 1, ptr);
                 assert(initialValue == ptr || initialValue == (ptr - 1));
@@ -948,7 +963,7 @@ class RangeSectionMap
             auto levelNew = static_cast<decltype(&(outerLevel->VolatileLoad(NULL))[0])>(AllocateLevel());
             if (levelNew == NULL)
                 return NULL;
-            
+
             if (!outerLevel->Install(levelNew, collectible))
             {
                 // Handle race where another thread grew the table
@@ -1014,7 +1029,7 @@ class RangeSectionMap
         auto rangeSectionL3 = rangeSectionL3Ptr->VolatileLoadWithoutBarrier(pLockState);
         if (rangeSectionL3 == NULL)
             return NULL;
-        
+
         auto rangeSectionL2Ptr = &((*rangeSectionL3)[EffectiveBitsForLevel(address, 3)]);
         if (level == 2)
             return rangeSectionL2Ptr;
@@ -1068,7 +1083,7 @@ class RangeSectionMap
 
         // Account for the range not starting at the beginning of a last level fragment
         rangeSize += pRangeSection->_range.RangeStart() & (bytesAtLastLevel - 1);
-        
+
         uintptr_t fragmentCount = ((rangeSize - 1) / bytesAtLastLevel) + 1;
         return fragmentCount;
     }
@@ -1311,7 +1326,7 @@ class RangeSectionMap
                     else
                     {
                         // Since the fragment linked lists are sorted such that the collectible ones are always after the non-collectible ones, this should never happen.
-                        assert(!seenCollectibleRangeList); 
+                        assert(!seenCollectibleRangeList);
                     }
 #endif
                     entryInMapToUpdate = &(entryInMapToUpdate->VolatileLoadWithoutBarrier(pLockState))->pRangeSectionFragmentNext;
@@ -1352,7 +1367,7 @@ class RangeSectionMap
 
                         if (foundMeaningfulValue)
                             break;
-                        
+
                         // This level is completely empty. Free it, and then null out the pointer to it.
                         pointerToLevelData->Uninstall();
 #if defined(__GNUC__)
@@ -1429,6 +1444,21 @@ class RangeSectionMap
     }
 #endif// DACCESS_COMPILE
 
+    template<typename T> friend struct ::cdac_data;
+};
+
+template<>
+struct cdac_data<RangeSectionMap>
+{
+    static constexpr size_t TopLevelData = offsetof(RangeSectionMap, _topLevelData);
+
+    struct RangeSectionFragment
+    {
+        static constexpr size_t RangeBegin = offsetof(RangeSectionMap::RangeSectionFragment, _range.begin);
+        static constexpr size_t RangeEndOpen = offsetof(RangeSectionMap::RangeSectionFragment, _range.end);
+        static constexpr size_t RangeSection = offsetof(RangeSectionMap::RangeSectionFragment, pRangeSection);
+        static constexpr size_t Next = offsetof(RangeSectionMap::RangeSectionFragment, pRangeSectionFragmentNext);
+    };
 };
 
 struct RangeSectionMapData
@@ -2243,8 +2273,18 @@ class ExecutionManager
         JumpStubBlockHeader * m_pBlocks;
         JumpStubTable m_Table;
     };
+
+    template<typename T> friend struct ::cdac_data;
 };
 
+#ifndef DACCESS_COMPILE
+template<>
+struct cdac_data<ExecutionManager>
+{
+    static constexpr void* const CodeRangeMapAddress = (void*)&ExecutionManager::g_codeRangeMap.Data[0];
+};
+#endif
+
 inline CodeHeader * EEJitManager::GetCodeHeader(const METHODTOKEN& MethodToken)
 {
     LIMITED_METHOD_DAC_CONTRACT;
@@ -2549,6 +2589,8 @@ class EECodeInfo
     // Simple helper to return a pointer to the UNWIND_INFO given the offset to the unwind info.
     UNWIND_INFO * GetUnwindInfoHelper(ULONG unwindInfoOffset);
 #endif // TARGET_AMD64
+
+    template<typename T> friend struct ::cdac_data;
 };
 
 #include "codeman.inl"

src/native/managed/cdacreader/Microsoft.Diagnostics.DataContractReader.Abstractions/ContractRegistry.cs

@@ -39,4 +39,8 @@ internal abstract class ContractRegistry
     /// Gets an instance of the DacStreams contract for the target.
     /// </summary>
     public abstract IDacStreams DacStreams { get; }
+    /// <summary>
+    /// Gets an instance of the ExecutionManager contract for the target.
+    /// </summary>
+    public abstract IExecutionManager ExecutionManager { get; }
 }

src/native/managed/cdacreader/Microsoft.Diagnostics.DataContractReader.Abstractions/Contracts/IExecutionManager.cs

@@ -0,0 +1,26 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+
+using System;
+
+namespace Microsoft.Diagnostics.DataContractReader.Contracts;
+
+internal struct EECodeInfoHandle
+{
+    public readonly TargetPointer Address;
+    internal EECodeInfoHandle(TargetPointer address) => Address = address;
+}
+
+internal interface IExecutionManager : IContract
+{
+    static string IContract.Name { get; } = nameof(ExecutionManager);
+    EECodeInfoHandle? GetEECodeInfoHandle(TargetCodePointer ip) => throw new NotImplementedException();
+    TargetPointer GetMethodDesc(EECodeInfoHandle codeInfoHandle) => throw new NotImplementedException();
+    TargetCodePointer GetStartAddress(EECodeInfoHandle codeInfoHandle) => throw new NotImplementedException();
+
+}
+
+internal readonly struct ExecutionManager : IExecutionManager
+{
+
+}

src/native/managed/cdacreader/Microsoft.Diagnostics.DataContractReader.Abstractions/DataType.cs

@@ -50,4 +50,10 @@ public enum DataType
     InstantiatedMethodDesc,
     DynamicMethodDesc,
     StoredSigMethodDesc,
+    RangeSectionMap,
+    RangeSectionFragment,
+    RangeSection,
+    RealCodeHeader,
+    CodeHeapListNode,
+
 }

src/native/managed/cdacreader/Microsoft.Diagnostics.DataContractReader.Contracts/Constants.cs

@@ -38,5 +38,8 @@ internal static class Globals
 
         internal const string MethodDescTokenRemainderBitCount = nameof(MethodDescTokenRemainderBitCount);
         internal const string DirectorySeparator = nameof(DirectorySeparator);
+
+        internal const string ExecutionManagerCodeRangeMapAddress = nameof(ExecutionManagerCodeRangeMapAddress);
+        internal const string StubCodeBlockLast = nameof(StubCodeBlockLast);
     }
 }

src/native/managed/cdacreader/Microsoft.Diagnostics.DataContractReader.Contracts/Contracts/ExecutionManagerFactory.cs

@@ -0,0 +1,20 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+
+using System;
+
+namespace Microsoft.Diagnostics.DataContractReader.Contracts;
+
+internal sealed class ExecutionManagerFactory : IContractFactory<IExecutionManager>
+{
+    IExecutionManager IContractFactory<IExecutionManager>.CreateContract(Target target, int version)
+    {
+        TargetPointer executionManagerCodeRangeMapAddress = target.ReadGlobalPointer(Constants.Globals.ExecutionManagerCodeRangeMapAddress);
+        Data.RangeSectionMap rangeSectionMap = target.ProcessedData.GetOrAdd<Data.RangeSectionMap>(executionManagerCodeRangeMapAddress);
+        return version switch
+        {
+            1 => new ExecutionManager_1(target, rangeSectionMap),
+            _ => default(ExecutionManager),
+        };
+    }
+}