Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the execution performance of the azure ad provider #155

Conversation

tkol2022
Copy link
Collaborator

@tkol2022 tkol2022 commented Feb 16, 2023

🗣 Description

Made adjustments to ExportAADProvider functions Get-PrivilegedUser and Get-PrivilegedRole for performance improvements. The execution speed improvements made by these code changes will be more noticeable on tenants with a larger number of privileged users (e.g. over 25) and on client machines running ScubaGear that have less resources (CPU, memory, network bandwidth) such as smaller sized virtual machines.

Closes #35.

  • Reduced the number of calls to Graph cmdlets that have to traverse the network and hit the back-end cloud API. The number of calls to the following cmdlets were reduced to a minimum: Get-MgUser and Get-MgRoleManagementDirectoryRoleEligibilityScheduleInstance

  • Added and modified comments to better describe the logic performed by each block of code.

  • Modified a block of code that detects when an AAD Group is assigned to a role to fix a logic error that could arise under certain circumstances.

🧪 How to Test

  • Test these changes on Test Tenant 1 (G5), Test Tenant 3 (E5), Test Tenant 2 (G3).

  • The code changes affect AAD Rego policies 2.13, 2.14, 2.15, 2.16.

Regression tests (to ensure these code updates didn't break the evaluation of the relevant Rego policies)

  • Download the main branch and invoke ScubaGear against AAD to produce a TestResults.json file.
  • Download the fix branch and invoke ScubaGear against AAD to produce a TestResults.json file.
  • Compare the two TestResults.json files and ensure there are no unexpected differences. Check the specific policies referenced above. Ensure that the RequirementMet value is the same between the two files. Also examine the ReportDetails value - ensure that the number of roles identified by the policy is the same between the two files.

Expected changes between files. If you run a Diff tool between the files such as VSC code, you may notice that the main branch has some extra text "View all CA policies" in the ReportDetails field for some of the policies. This is expected because some code was added to the report generator after I rebased my branch from main. You can ignore that extra text since it is not related to the AAD functions that I modified.

Performance tests (to ensure the code enhancements produce a faster execution of the AAD provider)

  • When you take the timing measurements that you do not include the Login process (i.e. entering your credentials) because that code is unrelated to my code updates and can skew the measurements.
  • Follow the steps below

Time the main branch
Import the ScubaGear module
Run the AAD provider at least one time with the authentication prompts
Execute the Powershell commands below to compute an average execution time for the main branch

$TotalExecution = 0
1..10 | Foreach-Object {
    $ExecutionTime = Measure-Command { invoke-scuba -productnames aad -Login $false}
    $TotalExecution += $ExecutionTime.TotalSeconds
}
$AverageTime = $TotalExecution / 10
Write-Warning "InvokeScuba main branch average execution time:"
Write-Warning $AverageTime

Time the fix branch
Import the ScubaGear module
Run the AAD provider at least one time with the authentication prompts
Execute the Powershell commands below to compute an average execution time for the fix branch

$TotalExecution = 0
1..10 | Foreach-Object {
    $ExecutionTime = Measure-Command { invoke-scuba -productnames aad -Login $false}
    $TotalExecution += $ExecutionTime.TotalSeconds
}
$AverageTime = $TotalExecution / 10
Write-Warning "InvokeScuba fix branch average execution time:"
Write-Warning $AverageTime

Document the execution time results for the main and fix branches in the pull request

✅ Pre-approval checklist

  • This PR has an informative and human-readable title.
  • Changes are limited to a single goal - eschew scope creep!
  • All future TODOs are captured in issues, which are referenced
    in code comments.
  • All relevant type-of-change labels have been added.
  • I have read the CONTRIBUTING document.
  • These code changes follow cisagov code standards.
  • All relevant repo and/or project documentation has been updated
    to reflect the changes in this PR.
  • Tests have been added and/or modified to cover the changes in this PR.
  • All new and existing tests pass.

✅ Pre-merge checklist

  • Revert dependencies to default branches.
  • Finalize version.

✅ Post-merge checklist

  • Add a tag or create a release.

@tkol2022 tkol2022 self-assigned this Feb 16, 2023
@schrolla schrolla added the enhancement This issue or pull request will add new or improve existing functionality label Feb 16, 2023
@schrolla schrolla added this to the Dolphin milestone Feb 16, 2023
Copy link
Contributor

@crutchfield crutchfield left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested performance on cisaent (45.42 sec to 35.34) y2zji (30.27 to 17.40) and scubag3forthee (5.80 to 4.86).

Copy link
Contributor

@ssatyapal123 ssatyapal123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed per test instructions on three tenants: G3 (scuba3forthee), G5 (cisaent), E5 (y2zj1). Performance improvement was best for G5.
Results are below:
cisaent main: 33.9 seconds, cisaentfeature: 20.8 seconds
y2zj1main: 12.15 seconds, y2zj1 feature: 12.3 seconds
scuba3forthee main: 3.18 seconds , scuba3forthee feature: 3.21 seconds

@tkol2022
Copy link
Collaborator Author

@schrolla Ready for merge.

@schrolla schrolla merged commit e4557d9 into main Feb 23, 2023
@schrolla
Copy link
Collaborator

Merge testing shows that the only changes to outputs are those expected from a previous merge adding CAP policies to the HTML output. No regressions or errors found in comparison to current main branch. Merging changes.

@buidav buidav deleted the 35-explore-how-to-improve-the-execution-performance-of-the-azure-ad-provider branch July 6, 2023 15:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement This issue or pull request will add new or improve existing functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Explore how to improve the execution performance of the Azure AD provider
4 participants