Performance: Not possible to prevent inlining the IL by the F# compiler without the MethodImplOptions.NoInlining flags

[zpodlovics]

I am trying to do some performance optimalizations using the ThrowHelper like construct as of #5019 (comment) :

The main method does its checks (say argument validation) and then conditionally calls throw helper with any needed args. Type the helper so no implicit boxing or parms array allocation is needed.

The throw helper does all formatting and exception object creation, then unconditionally throws. It is deliberately not marked with [MethodImpl(MethodImplOptions.NoInlining)].

However it seems the F# compiler still inlines all the small functions IL without the MethodImplOptions.NoInlining flag, and also inlines the small static method calls IL without this attribute. It should be possible express non IL inlining for F# functions and static and non-static methods without marking it MethodImplOptions.NoInlining.

Please provide a succinct description of the issue.

Repro steps

open System
open System.Runtime.InteropServices
open System.Runtime.CompilerServices

type [<RequireQualifiedAccess>] EnumType =
  | F = 0uy
  | T = 1uy 

type ThrowHelper() =
  static member Call() =
    invalidArg "value" "invalid value"

module EnumType = begin
  [<MethodImpl(MethodImplOptions.NoInlining)>]
  let noInliningThrowHelper() =
    invalidArg "value" "invalid value"

  let throwHelper() =
    invalidArg "value" "invalid value"

  let ofValue1 (value: uint8) : EnumType = 
    match value with
    | v when v = 0uy -> EnumType.F
    | v when v = 1uy -> EnumType.T
    | _ -> throwHelper()

  let ofValue2 (value: uint8) : EnumType = 
    match value with
    | v when v = 0uy -> EnumType.F
    | v when v = 1uy -> EnumType.T
    | _ -> ThrowHelper.Call()

  let ofValue3 (value: uint8) : EnumType = 
    match value with
    | v when v = 0uy -> EnumType.F
    | v when v = 1uy -> EnumType.T
    | _ -> noInliningThrowHelper()

end

[<EntryPoint>]
let main argv =
    let v1 = EnumType.ofValue1 1uy
    printfn "Enum: %s" (v1.ToString())
    let v2 = EnumType.ofValue2 1uy
    printfn "Enum: %s" (v2.ToString())
    let v3 = EnumType.ofValue2 1uy
    printfn "Enum: %s" (v3.ToString())
    0 // return an integer exit code

Provide the steps required to reproduce the problem

The compiled IL code will looks like this:

  .class auto ansi serializable nested public ThrowHelper
         extends [System.Runtime]System.Object
  {
    .custom instance void [FSharp.Core]Microsoft.FSharp.Core.CompilationMappingAttribute::.ctor(valuetype [FSharp.Core]Microsoft.FSharp.Core.SourceConstructFlags) = ( 01 00 03 00 00 00 00 00 ) 
    .method public specialname rtspecialname 
            instance void  .ctor() cil managed
    {
      // Code size       9 (0x9)
      .maxstack  8
      IL_0000:  ldarg.0
      IL_0001:  callvirt   instance void [System.Runtime]System.Object::.ctor()
      IL_0006:  ldarg.0
      IL_0007:  pop
      IL_0008:  ret
    } // end of method ThrowHelper::.ctor

    .method public static !!a  Call<a>() cil managed
    {
      // Code size       16 (0x10)
      .maxstack  8
      IL_0000:  ldstr      "invalid value"
      IL_0005:  ldstr      "value"
      IL_000a:  newobj     instance void [System.Runtime]System.ArgumentException::.ctor(string,
                                                                                         string)
      IL_000f:  throw
    } // end of method ThrowHelper::Call

  } // end of class ThrowHelper

  .class abstract auto ansi sealed nested public EnumTypeModule
         extends [System.Runtime]System.Object
  {
    .custom instance void [FSharp.Core]Microsoft.FSharp.Core.CompilationMappingAttribute::.ctor(valuetype [FSharp.Core]Microsoft.FSharp.Core.SourceConstructFlags) = ( 01 00 07 00 00 00 00 00 ) 
    .method public static !!a  noInliningThrowHelper<a>() cil managed noinlining
    {
      // Code size       16 (0x10)
      .maxstack  8
      IL_0000:  ldstr      "invalid value"
      IL_0005:  ldstr      "value"
      IL_000a:  newobj     instance void [System.Runtime]System.ArgumentException::.ctor(string,
                                                                                         string)
      IL_000f:  throw
    } // end of method EnumTypeModule::noInliningThrowHelper

    .method public static !!a  throwHelper<a>() cil managed
    {
      // Code size       16 (0x10)
      .maxstack  8
      IL_0000:  ldstr      "invalid value"
      IL_0005:  ldstr      "value"
      IL_000a:  newobj     instance void [System.Runtime]System.ArgumentException::.ctor(string,
                                                                                         string)
      IL_000f:  throw
    } // end of method EnumTypeModule::throwHelper

    .method public static valuetype Program/EnumType 
            ofValue1(uint8 'value') cil managed
    {
      // Code size       27 (0x1b)
      .maxstack  8
      IL_0000:  ldarg.0
      IL_0001:  brtrue.s   IL_0005

      IL_0003:  ldc.i4.0
      IL_0004:  ret

      IL_0005:  ldarg.0
      IL_0006:  ldc.i4.1
      IL_0007:  bne.un.s   IL_000b

      IL_0009:  ldc.i4.1
      IL_000a:  ret

      IL_000b:  ldstr      "invalid value"
      IL_0010:  ldstr      "value"
      IL_0015:  newobj     instance void [System.Runtime]System.ArgumentException::.ctor(string,
                                                                                         string)
      IL_001a:  throw
    } // end of method EnumTypeModule::ofValue1

    .method public static valuetype Program/EnumType 
            ofValue2(uint8 'value') cil managed
    {
      // Code size       27 (0x1b)
      .maxstack  8
      IL_0000:  ldarg.0
      IL_0001:  brtrue.s   IL_0005

      IL_0003:  ldc.i4.0
      IL_0004:  ret

      IL_0005:  ldarg.0
      IL_0006:  ldc.i4.1
      IL_0007:  bne.un.s   IL_000b

      IL_0009:  ldc.i4.1
      IL_000a:  ret

      IL_000b:  ldstr      "invalid value"
      IL_0010:  ldstr      "value"
      IL_0015:  newobj     instance void [System.Runtime]System.ArgumentException::.ctor(string,
                                                                                         string)
      IL_001a:  throw
    } // end of method EnumTypeModule::ofValue2

    .method public static valuetype Program/EnumType 
            ofValue3(uint8 'value') cil managed
    {
      // Code size       17 (0x11)
      .maxstack  8
      IL_0000:  ldarg.0
      IL_0001:  brtrue.s   IL_0005

      IL_0003:  ldc.i4.0
      IL_0004:  ret

      IL_0005:  ldarg.0
      IL_0006:  ldc.i4.1
      IL_0007:  bne.un.s   IL_000b

      IL_0009:  ldc.i4.1
      IL_000a:  ret

      IL_000b:  call       !!0 Program/EnumTypeModule::noInliningThrowHelper<valuetype Program/EnumType>()
      IL_0010:  ret
    } // end of method EnumTypeModule::ofValue3

  } // end of class EnumTypeModule

Expected behavior

Should be possible to express non IL inlining small functions and static and non static method without marking it with MethodImplOptions.NoInlining.

Actual behavior

F# compiler IL inlining small functions and static and non static method without marking it with MethodImplOptions.NoInlining.

Possible/Known workarounds

Implement the helper in C# (not yet tested)

The best future solution probably would be using the noinline keyword as inline already used for functions and static methods/methods (but this will complicate the language a little bit more). However it would be a perfecly fine future solution to have a [<NoInline>] flag in F# for functions/static methods/methods.

Related information

• Operating system: Ubuntu 16.04 x86_64
• .NET Runtime, CoreCLR or Mono Version:

ii  dotnet-host                                                 2.1.0-1                                                     amd64        Microsoft .NET Core Host - 2.1.0
ii  dotnet-hostfxr-2.0.7                                        2.0.7-1                                                     amd64        Microsoft .NET Core Host FX Resolver - 2.0.7 2.0.7
ii  dotnet-hostfxr-2.1                                          2.1.0-1                                                     amd64        Microsoft .NET Core Host FX Resolver - 2.1.0 2.1.0
ii  dotnet-runtime-2.0.7                                        2.0.7-1                                                     amd64        Microsoft .NET Core Runtime - 2.0.7 Microsoft.NETCore.App 2.0.7
ii  dotnet-runtime-2.1                                          2.1.0-1                                                     amd64        Microsoft .NET Core Runtime - 2.1.0 Microsoft.NETCore.App 2.1.0
ii  dotnet-runtime-deps-2.1                                     2.1.0-1                                                     amd64        dotnet-runtime-deps-2.1 2.1.0
ii  dotnet-sdk-2.1                                              2.1.300-1                                                   amd64        Microsoft .NET Core SDK 2.1.300

[dsyme]

@zpodlovics This is a language suggestion, best to put it at http://github.com/fsharp/fslang-suggestions

[dsyme]

@zpodlovics Am I correct that the workaround is to use [<MethodImpl(MethodImplOptions.NoInlining)>]?

[zpodlovics]

@dsyme Thanks, I will move it to F# suggestion repo.

Unfortunately the noinlining marker is not a workaround, because of this comment (#5019 (comment)):

"The throw helper does all formatting and exception object creation, then unconditionally throws. It is deliberately not marked with [MethodImpl(MethodImplOptions.NoInlining)]."

[dsyme]

@zpodlovics I'm confused.

• inline causes the F# compiler to inline
• [<MethodImpl(MethodImplOptions.NoInlining)>] cause neither F# nor JIT to inline
• no attribute means both F# and the JIT can inline if they want

But you additionally want noinline, but what would that do? Cause F# to not inline but allow the JIT to inline?

It is deliberately not marked with [MethodImpl(MethodImplOptions.NoInlining)]."

Could you briefly explain why?

[zpodlovics]

@dsyme Right now the C# compiler will not inline the static methods IL automatically (at least not in the examples I tried), so it possible to create method with split-path execution. This is the model of the execution and optimalization (hopefully not far from the current .NET Core implementation):

Let assume I have a busy webserver with an existing standard API with millions of requests per second and I have to do some simple calculation C1 on an array (100 IL instructions). I also have some domain knownledge: for this usage the array one in million execution will be empty and require a bit more complex logic to handle this case. If I split the execution path in two part: one for the non-empty case (25 IL instruction) and one for the empty case (75 IL instruction), than the overall performance could be significantly higher:

C1 Original: 100 IL instruction * 1 million execution
C1 Split path execution: 25 IL instruction * 1 million execution + 75 IL instruction * 1 execution

However sometimes I have an another busy functionality in the other part of the software that use the same calculation but in this execution path the most common path is the empty case. The NoInlining marker in this case will prevent the JIT to inline to this execution path.

Right now with F# I have the following cases:

• use inline and both execution path will use 100 IL instructions
• use NoInlining the the first execution path will be inlined and fast (25IL) and the second will be prevented from inlining
• use no F# attribute means no optimalization even if I have domain knowledge

Using the not yet implemented noinline functions/methods could allow split path executions but without preventing the JIT from optimalization based on actual execution information.

.NET Core JIT is now tiered, and could make decisions based on execution profile and relatively safe to assume that this profile based optimalization path will be explored more deeply in the future:
http://mattwarren.org/2017/12/15/How-does-.NET-JIT-a-method-and-Tiered-Compilation/

I know it's a hard decision to introduce another concepts in the language (but at least it will be symmetric: inline + noinline) and make it more complex. So I am open for suggestion if there is an existing way or a simpler solution to express this split path execution in F# (other than write it in C#).

Unfortunately even a few additional instructions matters a lot when it executed million times+ a second.

[dsyme]

OK, thanks, that's helpful context.

So to be clear: you want something which tells F# not to inline, but allows the JIT to inline if it wants? If so that's a reasonable request.

But just to mention we couldn't call it noinline because to anyone reading the code that would clearly imply "neither F# nor JIT can inline this". I suppose we would do it as an F#-specific attribute of some kind.

[zpodlovics]

@dsyme Yes, exactly. I would like to tell the F# compiler not to inline, but allows the JIT to inline if it wants. You are right, naming thing correctly is critical. I would be happy to use any constructs (attribute, keyword, whatever) to control this.: an F# specific attribute would be a perfect choice.

[forki]

Isn't that the default? Isn't the jit always free to inline until told otherwise?

[cartermp]

@zpodlovics I'll close this out since this is a language suggestion and should be tracked at http://github.com/fsharp/fslang-suggestions as previously mentioned.

[zpodlovics]

If anyone interested there is a developer flag to change the default inlining behaviour, however it's hard to use correctly and the developer flag will be applied to the whole project:

  <PropertyGroup>
    <TargetFramework>netcoreapp2.1</TargetFramework>
    <OtherFlags>--inlinethreshold:0</OtherFlags>
  </PropertyGroup>

A more fine grained solution will be needed to control the compiler inlining behaviour.