Skip to content

arm64: Reuse SVE mask constants in LSRA - #131309

Open
jonathandavies-arm wants to merge 8 commits into
dotnet:mainfrom
jonathandavies-arm:upstream/sve/lsra-ptrue-reuse
Open

arm64: Reuse SVE mask constants in LSRA#131309
jonathandavies-arm wants to merge 8 commits into
dotnet:mainfrom
jonathandavies-arm:upstream/sve/lsra-ptrue-reuse

Conversation

@jonathandavies-arm

Copy link
Copy Markdown
Contributor

This is the ptrue reuse work done in LSRA instead of a new pass.

@github-actions github-actions Bot added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Jul 24, 2026
@dotnet-policy-service dotnet-policy-service Bot added the community-contribution Indicates that the PR has been added by a community member label Jul 24, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 7 pipeline(s).
9 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Comment thread src/coreclr/jit/lsra.h Outdated
Comment on lines +2148 to +2157
#if defined(TARGET_ARM64) && defined(FEATURE_MASKED_HW_INTRINSICS)
struct SveMaskIntervalEntry
{
GenTree* tree;
Interval* interval;
SveMaskIntervalEntry* next;
};

SveMaskIntervalEntry* reusableSveMaskIntervals = nullptr;
#endif

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was not expecting to see anything like this. What makes this needed over the existing constant reuse mechanism that LSRA already has? E.g. members like LinearScan::m_RegistersWithConstants and LinearScan::getMatchingConstants.

Did you figure out why the existing mechanism does not handle the cases this tries to handle? It would be good to understand that first.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the shared-interval mechanism. The existing LSRA reuse machinery already handles GT_CNS_MSK, but SVE ptrue values are often represented as GT_HWINTRINSIC and were not marked as constants. Therefore isMatchingConstant did not consider them reusable. The updated change now uses the existing mechanism.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, that seems cleaner now.
With #127520 merged do we need to recognize HWINTRINSIC at all? My understanding is that it converts these HW intrinsics recognized in this PR into GT_CNS_MASK (and if not, that it would be possible to). cc @a74nh

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We import TrueMask/FalseMask as GT_CNS_MASK where possible.

If the pattern is in a variable, then we have to import the HWINTRINSIC - but that's not something we can optimise in the PR as we don't know the pattern.

Then there are all the "all ptrue" we introduce during lowering. They are also added as GT_CNS_MASK nodes now too.

So, yes, this most likely doesn't need to check for HWINTRINSIC any more.

Also, this PR might need some JitUseScalableVectorT() checks now too

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've done that

Comment thread src/coreclr/jit/lsra.cpp Outdated
Comment on lines +2720 to +2726
#if defined(TARGET_ARM64) && defined(FEATURE_MASKED_HW_INTRINSICS)
if (areMatchingSveMaskConstants(refPosition->treeNode, otherTreeNode))
{
return true;
}
#endif

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should not be needed, can you please fix the code called in the GT_CNS_MSK case below if that isn't already sufficient?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

@jonathandavies-arm

Copy link
Copy Markdown
Contributor Author

An earlier implementation preserved SVE mask constants across certain basic-block boundaries:

BasicBlock* nextBlock             = getNextBlock();
bool        preserveMaskConstants = false;

preserveMaskConstants =
    m_compiler->opts.OptimizationEnabled() &&
    (nextBlock != nullptr) &&
    (nextBlock->GetUniquePred(m_compiler) == currentBlock) &&
    !blockInfo[nextBlock->bbNum].hasEHBoundaryIn &&
    !blockInfo[currentBlock->bbNum].hasEHBoundaryOut;

...

if (preserveMaskConstants && assignedInterval->isConstant &&
    varTypeIsMask(assignedInterval->registerType))
{
    setConstantReg(reg, assignedInterval->registerType);
    continue;
}

Should LSRA allow constant registers to survive a block boundary when the value is not live into the successor?

Is the current behavior (clearing constant-register state at every block boundary and rematerializing constants in each basic block) the intended approach, or should LSRA support preserving these constants across eligible block boundaries?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI community-contribution Indicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants