Merge pull request #73398 from Snowy1803/6.0-lost-vars-stats-onone

[6.0] Lost variables statistics + Fix -Onone

Author: adrian-prantl

docs/DebuggingTheCompiler.md

@@ -260,7 +260,7 @@ If one builds swift using ninja and wants to dump the SIL of the
 stdlib using some of the SIL dumping options from the previous
 section, one can use the following one-liner:
 
-    ninja -t commands | grep swiftc | grep Swift.o | grep " -c "
+    ninja -t commands | grep swiftc | grep 'Swift\.o'
 
 This should give one a single command line that one can use for
 Swift.o, perfect for applying the previous sections options to.
@@ -302,6 +302,13 @@ with the proper attributes to ensure they'll be available in the debugger. In
 particular, if you see `SWIFT_DEBUG_DUMP` in a class declaration, that class
 has a `dump()` method you can call.
 
+### Pass statistics
+
+There are options to output a lot of different statistics, including about
+SIL passes. More information is available in
+[Compiler Performance](CompilerPerformance.md) for the unified statistics, and
+[Optimizer Counter Analysis](OptimizerCountersAnalysis.md) for pass counters.
+
 ## Debugging and Profiling on SIL level
 
 ### SIL source level profiling

docs/HowToUpdateDebugInfo.md

@@ -16,6 +16,11 @@ merge drop and copy locations, since all the same considerations apply. Helpers
 like `SILBuilderWithScope` make it easy to copy source locations when expanding
 SIL instructions.
 
+> [!Warning]
+> Don't use `SILBuilderWithScope` when replacing a single instruction of type
+> `AllocStackInst` or `DebugValueInst`. These meta instructions are skipped,
+> so the wrong scope will be inferred.
+
 ## Variables
 
 Each `debug_value` (and variable-carrying instruction) defines an update point
@@ -254,12 +259,37 @@ debug_value %1 : $Int, var, name "pair", type $Pair, expr op_fragment:#Pair.a //
 ```
 
 ## Rules of thumb
-- Optimization passes may never drop a variable entirely. If a variable is
-  entirely optimized away, an `undef` debug value should still be kept.
-- A `debug_value` must always describe a correct value for that source variable
-  at that source location. If a value is only correct on some paths through that
-  instruction, it must be replaced with `undef`. Debug info never speculates.
-- When a SIL instruction is deleted, call salvageDebugInfo(). It will try to
-  capture the effect of the deleted instruction in a debug expression, so the
-  location can be preserved. You can also use an `InstructionDeleter` which will
-  automatically call `salvageDebugInfo`.
+
+### Correctness
+A `debug_value` must always describe a correct value for that source variable
+at that source location. If a value is only correct on some paths through that
+instruction, it must be replaced with `undef`. Debug info never speculates.
+
+### Don't drop debug info
+
+Optimization passes may never drop a variable entirely. If a variable is
+entirely optimized away, an `undef` debug value should still be kept. The only
+exception is when the variable is in an unreachable function or scope, where it
+can be removed with the rest of the instructions.
+
+### Instruction Deletion
+
+When a SIL instruction is deleted, call `salvageDebugInfo`. It will try to
+capture the effect of the deleted instruction in a debug expression, so the
+location can be preserved.
+
+Alternatively, you can use an `InstructionDeleter`, which will automatically
+call `salvageDebugInfo`.
+
+If the debug info cannot be salvaged by `salvageDebugInfo`, and the pass has a
+special knowledge of the value, the pass can directly replace the debug value
+with the known value.
+
+If an instruction is being replaced by another, use `replaceAllUsesWith`. It
+will also update debug values to use the new instruction.
+
+> [!Tip]
+> To detect when a pass drops a variable, you can use the
+> `-Xllvm -sil-stats-lost-variables` to print when a variable is lost by a pass.
+> More information about this option is available in
+> [Optimizer Counter Analysis](OptimizerCountersAnalysis.md)

docs/OptimizerCountersAnalysis.md

@@ -54,7 +54,8 @@ The following statistics can be recorded:
 
   * For SILFunctions: the number of SIL basic blocks for each SILFunction, the
     number of SIL instructions, the number of SILInstructions of a specific
-    kind (e.g. a number of alloc_ref instructions)
+    kind (e.g. a number of alloc_ref instructions), the number of debug
+    variables
 
   * For SILModules: the number of SIL basic blocks in the SILModule, the number
     of SIL instructions, the number of SILFunctions, the number of
@@ -118,6 +119,16 @@ e.g. `-Xllvm -sil-stats-only-instructions=alloc_ref,alloc_stack`. If you need to
 collect stats about all kinds of SIL instructions, you can use this syntax: 
 `-Xllvm -sil-stats-only-instructions=all`.
 
+### Debug variable level counters
+A different type of counter is the lost debug variables counter. It is enabled
+by using the `-Xllvm -sil-stats-lost-variables` command-line option. It only
+tracks statistics about lost variables in SILFunctions. It is not enabled by
+any other command-line option, but can be combined with the others. It is not
+compatible with thresholds, it always counts lost variables. Note that it does
+not track the number of debug variables: it counts the number of debug variables
+that were present, but aren't anymore. If a variable changes location or scope,
+which is not allowed, it will be counted as lost.
+
 ## Configuring which counters changes should be recorded
 
 The user has a possibility to configure a number of thresholds, which control
@@ -181,9 +192,9 @@ And for counter stats it looks like this:
   * `function_history` corresponds to the verbose mode of function
     counters collection, when changes to the SILFunction counters are logged 
     unconditionally, without any on-line filtering.
-* `CounterName` is typically one of `block`, `inst`, `function`, `memory`, 
-   or `inst_instruction_name` if you collect counters for specific kinds of SIL
-   instructions.
+* `CounterName` is typically one of `block`, `inst`, `function`, `memory`,
+   `lostvars`, or `inst_instruction_name` if you collect counters for specific
+   kinds of SIL instructions.
 * `Symbol` is e.g. the name of a function
 * `StageName` is the name of the current optimizer pipeline stage
 * `TransformName` is the name of the current optimizer transformation/pass
@@ -192,6 +203,14 @@ And for counter stats it looks like this:
    want to reproduce the result later using 
    `-Xllvm -sil-opt-pass-count -Xllvm TransformPassNumber`
 
+## Extract Lost Variables per Pass
+
+For lost variables, there is a script to output a CSV with only the amount of
+lost variables per pass. You can then easily open the resulting CSV in Numbers
+to make graphs.
+
+`utils/process-stats-lost-variables csv_file_with_counters > csv_aggregate`
+
 ## Storing the produced statistics into a database
 
 To store the set of produced counters into a database, you can use the
@@ -345,4 +364,3 @@ from Counters C where C.counter = 'inst' and C.kind = 'module'
 group by Stage
 order by sum(C.Delta);
 ```
-

lib/IRGen/AllocStackHoisting.cpp

@@ -157,9 +157,8 @@ void moveAllocStackToBeginningOfBlock(
   // of the debug_value to the original position.
   if (haveMovedElt) {
     if (auto varInfo = AS->getVarInfo()) {
-      SILBuilderWithScope Builder(AS);
       // SILBuilderWithScope skips over meta instructions when picking a scope.
-      Builder.setCurrentDebugScope(AS->getDebugScope());
+      SILBuilder Builder(AS, AS->getDebugScope());
       auto *DVI = Builder.createDebugValue(AS->getLoc(), AS, *varInfo);
       DVI->setUsesMoveableValueDebugInfo();
       DebugValueToBreakBlocksAt.push_back(DVI);
@@ -198,14 +197,14 @@ void Partition::assignStackLocation(
     if (AssignedLoc == AllocStack) continue;
     eraseDeallocStacks(AllocStack);
     AllocStack->replaceAllUsesWith(AssignedLoc);
-    if (hasAtLeastOneMovedElt) {
-      if (auto VarInfo = AllocStack->getVarInfo()) {
-        SILBuilderWithScope Builder(AllocStack);
-        auto *DVI = Builder.createDebugValue(AllocStack->getLoc(), AssignedLoc,
-                                             *VarInfo);
+    if (auto VarInfo = AllocStack->getVarInfo()) {
+      SILBuilder Builder(AllocStack, AllocStack->getDebugScope());
+      auto *DVI = Builder.createDebugValueAddr(AllocStack->getLoc(),
+                                               AssignedLoc, *VarInfo);
+      if (hasAtLeastOneMovedElt) {
         DVI->setUsesMoveableValueDebugInfo();
-        DebugValueToBreakBlocksAt.push_back(DVI);
       }
+      DebugValueToBreakBlocksAt.push_back(DVI);
     }
     AllocStack->eraseFromParent();
   }

lib/IRGen/LoadableByAddress.cpp

@@ -33,6 +33,7 @@
 #include "swift/SILOptimizer/Analysis/DeadEndBlocksAnalysis.h"
 #include "swift/SILOptimizer/PassManager/Transforms.h"
 #include "swift/SILOptimizer/Utils/BasicBlockOptUtils.h"
+#include "swift/SILOptimizer/Utils/DebugOptUtils.h"
 #include "swift/SILOptimizer/Utils/InstOptUtils.h"
 #include "swift/SILOptimizer/Utils/StackNesting.h"
 #include "llvm/ADT/MapVector.h"
@@ -1777,7 +1778,7 @@ static void rewriteUsesOfSscalar(StructLoweringState &pass,
       createOutlinedCopyCall(copyBuilder, address, dest, pass);
       storeUser->eraseFromParent();
     } else if (auto *dbgInst = dyn_cast<DebugValueInst>(user)) {
-      SILBuilderWithScope dbgBuilder(dbgInst);
+      SILBuilder dbgBuilder(dbgInst, dbgInst->getDebugScope());
       // Rewrite the debug_value to point to the variable in the alloca.
       dbgBuilder.createDebugValueAddr(dbgInst->getLoc(), address,
                                       *dbgInst->getVarInfo());
@@ -2150,9 +2151,8 @@ static void rewriteFunction(StructLoweringState &pass,
       } else {
         assert(currOperand->getType().isAddress() &&
                "Expected an address type");
-        SILBuilderWithScope debugBuilder(instr);
         // SILBuilderWithScope skips over metainstructions.
-        debugBuilder.setCurrentDebugScope(instr->getDebugScope());
+        SILBuilder debugBuilder(instr, instr->getDebugScope());
         debugBuilder.createDebugValueAddr(instr->getLoc(), currOperand,
                                           *instr->getVarInfo());
         instr->getParent()->erase(instr);
@@ -3637,6 +3637,7 @@ class AssignAddressToDef : SILInstructionVisitor<AssignAddressToDef> {
 
     builder.createCopyAddr(load->getLoc(), load->getOperand(), addr, IsTake,
                            IsInitialization);
+    swift::salvageLoadDebugInfo(load);
     assignment.markForDeletion(load);
   }
 

lib/SIL/IR/SILInstruction.cpp

@@ -1475,9 +1475,8 @@ bool SILInstruction::mayTrap() const {
 }
 
 bool SILInstruction::isMetaInstruction() const {
-  // Every instruction that implements getVarInfo() should be in this list.
+  // Every instruction that doesn't generate code should be in this list.
   switch (getKind()) {
-  case SILInstructionKind::AllocBoxInst:
   case SILInstructionKind::AllocStackInst:
   case SILInstructionKind::DebugValueInst:
     return true;