Commit f44be79
Merge pull request #41931 from JuliaLang/avi/noinfer
* introduce `@nospecializeinfer` macro to tell the compiler to avoid excess inference
This commit introduces a new compiler annotation called `@nospecializeinfer`,
which allows us to request the compiler to avoid excessive inference.
\## `@nospecialize` mechanism
T discuss `@nospecializeinfer`, let's first understand the behavior of
`@nospecialize`.
Its docstring says that
> This is only a hint for the compiler to avoid excess code generation.
, and it works by suppressing dispatches with complex runtime
occurrences of the annotated arguments. This could be understood with
the example below:
```julia
julia> function call_func_itr(func, itr)
local r = 0
r += func(itr[1])
r += func(itr[2])
r += func(itr[3])
r
end;
julia> _isa = isa; # just for the sake of explanation, global variable to prevent inlining
julia> func_specialize(a) = _isa(a, Function);
julia> func_nospecialize(@nospecialize a) = _isa(a, Function);
julia> dispatchonly = Any[sin, muladd, nothing]; # untyped container can cause excessive runtime dispatch
julia> @code_typed call_func_itr(func_specialize, dispatchonly)
CodeInfo(
1 β %1 = Ο (0, Int64)
β %2 = Base.arrayref(true, itr, 1)::Any
β %3 = (func)(%2)::Any
β %4 = (%1 + %3)::Any
β %5 = Base.arrayref(true, itr, 2)::Any
β %6 = (func)(%5)::Any
β %7 = (%4 + %6)::Any
β %8 = Base.arrayref(true, itr, 3)::Any
β %9 = (func)(%8)::Any
β %10 = (%7 + %9)::Any
βββ return %10
) => Any
julia> @code_typed call_func_itr(func_nospecialize, dispatchonly)
CodeInfo(
1 β %1 = Ο (0, Int64)
β %2 = Base.arrayref(true, itr, 1)::Any
β %3 = invoke func(%2::Any)::Any
β %4 = (%1 + %3)::Any
β %5 = Base.arrayref(true, itr, 2)::Any
β %6 = invoke func(%5::Any)::Any
β %7 = (%4 + %6)::Any
β %8 = Base.arrayref(true, itr, 3)::Any
β %9 = invoke func(%8::Any)::Any
β %10 = (%7 + %9)::Any
βββ return %10
) => Any
```
The calls of `func_specialize` remain to be `:call` expression (so that
they are dispatched and compiled at runtime) while the calls of
`func_nospecialize` are resolved as `:invoke` expressions. This is
because `@nospecialize` requests the compiler to give up compiling
`func_nospecialize` with runtime argument types but with the declared
argument types, allowing `call_func_itr(func_nospecialize, dispatchonly)`
to avoid runtime dispatches and accompanying JIT compilations
(i.e. "excess code generation").
The difference is evident when checking `specializations`:
```julia
julia> call_func_itr(func_specialize, dispatchonly)
2
julia> length(Base.specializations(only(methods(func_specialize))))
3 # w/ runtime dispatch, multiple specializations
julia> call_func_itr(func_nospecialize, dispatchonly)
2
julia> length(Base.specializations(only(methods(func_nospecialize))))
1 # w/o runtime dispatch, the single specialization
```
The problem here is that it influences dispatch only, and does not
intervene into inference in anyway. So there is still a possibility of
"excess inference" when the compiler sees a considerable complexity of
argument types during inference:
```julia
julia> func_specialize(a) = _isa(a, Function); # redefine func to clear the specializations
julia> @Assert length(Base.specializations(only(methods(func_specialize)))) == 0;
julia> func_nospecialize(@nospecialize a) = _isa(a, Function); # redefine func to clear the specializations
julia> @Assert length(Base.specializations(only(methods(func_nospecialize)))) == 0;
julia> withinfernce = tuple(sin, muladd, "foo"); # typed container can cause excessive inference
julia> @time @code_typed call_func_itr(func_specialize, withinfernce);
0.000812 seconds (3.77 k allocations: 217.938 KiB, 94.34% compilation time)
julia> length(Base.specializations(only(methods(func_specialize))))
4 # multiple method instances inferred
julia> @time @code_typed call_func_itr(func_nospecialize, withinfernce);
0.000753 seconds (3.77 k allocations: 218.047 KiB, 92.42% compilation time)
julia> length(Base.specializations(only(methods(func_nospecialize))))
4 # multiple method instances inferred
```
The purpose of this PR is to implement a mechanism that allows us to
avoid excessive inference to reduce the compilation latency when
inference sees a considerable complexity of argument types.
\## Design
Here are some ideas to implement the functionality:
1. make `@nospecialize` block inference
2. add nospecializeinfer effect when `@nospecialize`d method is annotated as `@noinline`
3. implement as `@pure`-like boolean annotation to request nospecializeinfer effect on top of `@nospecialize`
4. implement as annotation that is orthogonal to `@nospecialize`
After trying 1 ~ 3., I decided to submit 3.
\### 1. make `@nospecialize` block inference
This is almost same as what Jameson has done at <vtjnash@8ab7b6b>.
It turned out that this approach performs very badly because some of
`@nospecialize`'d arguments still need inference to perform reasonably.
For example, it's obvious that the following definition of
`getindex(@nospecialize(t::Tuple), i::Int)` would perform very badly if
`@nospecialize` blocks inference, because of a lack of useful type
information for succeeding optimizations:
<https://github.com/JuliaLang/julia/blob/12d364e8249a07097a233ce7ea2886002459cc50/base/tuple.jl#L29-L30>
\### 2. add nospecializeinfer effect when `@nospecialize`d method is annotated as `@noinline`
The important observation is that we often use `@nospecialize` even when
we expect inference to forward type and constant information.
Adversely, we may be able to exploit the fact that we usually don't
expect inference to forward information to a callee when we annotate it
with `@noinline` (i.e. when adding `@noinline`, we're usually fine with
disabling inter-procedural optimizations other than resolving dispatch).
So the idea is to enable the inference suppression when `@nospecialize`'d
method is annotated as `@noinline` too.
It's a reasonable choice and can be efficiently implemented with #41922.
But it sounds a bit weird to me to associate no infer effect with
`@noinline`, and I also think there may be some cases we want to inline
a method while partly avoiding inference, e.g.:
```julia
\# the compiler will always infer with `f::Any`
@noinline function twof(@nospecialize(f), n) # this method body is very simple and should be eligible for inlining
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
\### 3. implement as `@pure`-like boolean annotation to request nospecializeinfer effect on top of `@nospecialize`
This is what this commit implements. It basically replaces the previous
`@noinline` flag with a newly-introduced annotation named `@nospecializeinfer`.
It is still associated with `@nospecialize` and it only has effect when
used together with `@nospecialize`, but now it is not associated to
`@noinline`, and it would help us reason about the behavior of `@nospecializeinfer`
and experiment its effect more safely:
```julia
\# the compiler will always infer with `f::Any`
Base.@nospecializeinfer function twof(@nospecialize(f), n) # the compiler may or not inline this method
if occursin('+', string(typeof(f).name.name::Symbol))
2 + n
elseif occursin('*', string(typeof(f).name.name::Symbol))
2n
else
zero(n)
end
end
```
\### 4. implement as annotation that is orthogonal to `@nospecialize`
Actually, we can have `@nospecialize` and `@nospecializeinfer` separately, and it
would allow us to configure compilation strategies in a more
fine-grained way.
```julia
function noinfspec(Base.@nospecializeinfer(f), @nospecialize(g))
...
end
```
I'm fine with this approach but at the same time I'm afraid to have too
many annotations that are related to some sort (I expect we will
annotate both `@nospecializeinfer` and `@nospecialize` in this scheme).
---
experiment `@nospecializeinfer` on `Core.Compiler`
This commit adds `@nospecializeinfer` macro on various `Core.Compiler`
functions and achieves the following sysimage size reduction:
| | this commit | master | % |
| --------------------------------- | ----------- | ----------- | ------- |
| `Core.Compiler` compilation (sec) | `66.4551` | `71.0846` | `0.935` |
| `corecompiler.jl` (KB) | `17638080` | `18407248` | `0.958` |
| `sys.jl` (KB) | `88736432` | `89361280` | `0.993` |
| `sys-o.a` (KB) | `189484400` | `189907096` | `0.998` |
---------
Co-authored-by: Mosè Giordano <giordano@users.noreply.github.com>
Co-authored-by: Tim Holy <tim.holy@gmail.com>File tree
18 files changed
+302
-118
lines changed- base
- compiler
- doc/src/base
- src
- stdlib/Serialization/src
- test/compiler
18 files changed
+302
-118
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
508 | 508 | | |
509 | 509 | | |
510 | 510 | | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
511 | 515 | | |
512 | 516 | | |
513 | 517 | | |
| |||
2645 | 2649 | | |
2646 | 2650 | | |
2647 | 2651 | | |
2648 | | - | |
| 2652 | + | |
2649 | 2653 | | |
2650 | 2654 | | |
2651 | 2655 | | |
2652 | | - | |
| 2656 | + | |
2653 | 2657 | | |
2654 | 2658 | | |
2655 | | - | |
| 2659 | + | |
2656 | 2660 | | |
2657 | 2661 | | |
2658 | 2662 | | |
2659 | | - | |
| 2663 | + | |
2660 | 2664 | | |
2661 | 2665 | | |
2662 | 2666 | | |
| |||
2668 | 2672 | | |
2669 | 2673 | | |
2670 | 2674 | | |
2671 | | - | |
| 2675 | + | |
2672 | 2676 | | |
2673 | 2677 | | |
2674 | 2678 | | |
| |||
2705 | 2709 | | |
2706 | 2710 | | |
2707 | 2711 | | |
2708 | | - | |
| 2712 | + | |
2709 | 2713 | | |
2710 | 2714 | | |
2711 | 2715 | | |
| |||
2723 | 2727 | | |
2724 | 2728 | | |
2725 | 2729 | | |
2726 | | - | |
| 2730 | + | |
2727 | 2731 | | |
2728 | 2732 | | |
2729 | 2733 | | |
| |||
2742 | 2746 | | |
2743 | 2747 | | |
2744 | 2748 | | |
2745 | | - | |
| 2749 | + | |
2746 | 2750 | | |
2747 | 2751 | | |
2748 | | - | |
| 2752 | + | |
2749 | 2753 | | |
2750 | 2754 | | |
2751 | | - | |
| 2755 | + | |
2752 | 2756 | | |
2753 | 2757 | | |
2754 | 2758 | | |
| |||
2771 | 2775 | | |
2772 | 2776 | | |
2773 | 2777 | | |
2774 | | - | |
| 2778 | + | |
2775 | 2779 | | |
2776 | 2780 | | |
2777 | | - | |
| 2781 | + | |
2778 | 2782 | | |
2779 | 2783 | | |
2780 | | - | |
| 2784 | + | |
2781 | 2785 | | |
2782 | 2786 | | |
2783 | 2787 | | |
2784 | 2788 | | |
2785 | | - | |
| 2789 | + | |
2786 | 2790 | | |
2787 | 2791 | | |
2788 | | - | |
| 2792 | + | |
2789 | 2793 | | |
2790 | 2794 | | |
2791 | 2795 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
161 | 161 | | |
162 | 162 | | |
163 | 163 | | |
164 | | - | |
| 164 | + | |
165 | 165 | | |
166 | 166 | | |
167 | 167 | | |
168 | 168 | | |
169 | 169 | | |
170 | 170 | | |
171 | 171 | | |
172 | | - | |
| 172 | + | |
173 | 173 | | |
174 | 174 | | |
175 | 175 | | |
176 | 176 | | |
177 | 177 | | |
178 | 178 | | |
179 | 179 | | |
180 | | - | |
| 180 | + | |
181 | 181 | | |
182 | 182 | | |
183 | 183 | | |
| |||
186 | 186 | | |
187 | 187 | | |
188 | 188 | | |
189 | | - | |
| 189 | + | |
190 | 190 | | |
191 | 191 | | |
192 | 192 | | |
| |||
197 | 197 | | |
198 | 198 | | |
199 | 199 | | |
200 | | - | |
| 200 | + | |
201 | 201 | | |
202 | | - | |
| 202 | + | |
203 | 203 | | |
204 | 204 | | |
205 | 205 | | |
206 | 206 | | |
207 | | - | |
| 207 | + | |
208 | 208 | | |
209 | 209 | | |
210 | 210 | | |
211 | 211 | | |
212 | 212 | | |
213 | 213 | | |
214 | 214 | | |
215 | | - | |
| 215 | + | |
216 | 216 | | |
217 | 217 | | |
218 | 218 | | |
219 | 219 | | |
220 | 220 | | |
221 | 221 | | |
222 | 222 | | |
223 | | - | |
| 223 | + | |
224 | 224 | | |
225 | | - | |
| 225 | + | |
226 | 226 | | |
227 | 227 | | |
228 | 228 | | |
| |||
236 | 236 | | |
237 | 237 | | |
238 | 238 | | |
239 | | - | |
| 239 | + | |
240 | 240 | | |
241 | 241 | | |
242 | 242 | | |
| |||
245 | 245 | | |
246 | 246 | | |
247 | 247 | | |
248 | | - | |
| 248 | + | |
249 | 249 | | |
250 | | - | |
| 250 | + | |
251 | 251 | | |
252 | | - | |
| 252 | + | |
253 | 253 | | |
254 | 254 | | |
255 | 255 | | |
256 | | - | |
| 256 | + | |
257 | 257 | | |
258 | 258 | | |
259 | 259 | | |
260 | 260 | | |
261 | | - | |
| 261 | + | |
262 | 262 | | |
263 | 263 | | |
264 | 264 | | |
265 | | - | |
| 265 | + | |
266 | 266 | | |
267 | 267 | | |
268 | 268 | | |
| |||
281 | 281 | | |
282 | 282 | | |
283 | 283 | | |
284 | | - | |
| 284 | + | |
285 | 285 | | |
286 | | - | |
| 286 | + | |
287 | 287 | | |
288 | 288 | | |
289 | 289 | | |
| |||
306 | 306 | | |
307 | 307 | | |
308 | 308 | | |
309 | | - | |
310 | | - | |
311 | | - | |
312 | | - | |
313 | | - | |
314 | | - | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
315 | 315 | | |
316 | 316 | | |
317 | 317 | | |
| |||
0 commit comments