[O11y][Azure] Migration of Azure AD Identity Protection dashboard to Lens

[rajvi-patel-22]

Urgency

• High

Activity Type

• Enhancement

What does this PR do?

• Manually migrate Azure AD Identity Protection visualizations to the lens in the current Kibana version 8.6.0 itself.

• Statistics for Azure AD Identity Protection Lens migration:

Migration stats

	Before Migration		After Migration
Dashboad Name	Lens	Visualization	Lens	Visualization
[Logs Azure] Azure AD Identity Protection	3	2	5	0

Checklist

• I have added an entry to my package's changelog.yml file.
• I have verified that panels are populated with data.
• I have verified that panels are not distorted after being migrated to the lens.
• I have updated screenshots of the dashboard.
• I have verified that the data counts are matching and panel aggregations are the same as before.

Author's Checklist

• Migrated panels should be removed from visualization folder.
• Migrated visualizations are populating in current Kibana version 8.6.0 itself.

Related issues

• Relates [O11y] [Azure Active Directory] Lens Migration #7230
• Closes [O11y][Azure] Incorrect filter in Azure AD Identity Protection dashboard #7583

Screenshot

Note: To get actual data is not straightforward in this case. User have to trigger it accessing from an unknown ip in order to generate these logs, and also it's a premium feature. Hence dashboard is not populated.

Dashboard is getting populated after lens migration and updating data-stream filter.

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-08-29T08:03:43.857+0000

• Duration: 15 min 58 sec

Test stats 🧪

Test	Results
Failed	0
Passed	83
Skipped	0
Total	83

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (10/10)	💚
Files	86.364% (19/22)	👎 -13.636
Classes	86.364% (19/22)	👎 -13.636
Methods	83.333% (155/186)	👍 10.0
Lines	85.003% (2817/3314)	👎 -14.997
Conditionals	100.0% (0/0)	💚

[kush-elastic]

LGTM!

[zmoog]

Hey @rajvi-elastic, I know setting up a real Identity Protection is not straightforward and involves enabling an Active Directory Premium P2 license.

However, for the purpose of this PR, we don't have to set up the whole thing; we can send sample event logs to an event hub using a CLI tool:

┌─────────────┐         ┌───────────┐        ┌───────────────┐
│  event hub  │────────▶│   agent   │───────▶│  data stream  │
└─────────────┘         └───────────┘        └───────────────┘
       ▲                                                      
                                                              
       │                                                      
     send                                                     
    sample                                                    
    events                                                    
       │                                                      
┌─────────────┐                                               
│  cli tool   │                                               
└─────────────┘                                               

[zmoog]

We have a few sample events in the repo, and I probably have more on my laptop; we can use these as a template.

To send data, we'll soon have support for the event hub in https://github.com/elastic/stream, but in the meantime, you can use https://github.com/zmoog/eventhubs to read sample event logs and publish them in the event hub.

Let me know if you need more information. I can put together a short guide if you need it.

[zmoog]

We should test the dashboard sending sample events using the CLI tool.

[zmoog]

The problem with this integration is that it's using azure.identityprotection for field names and azure.identity_protection as dataset. We usually use the same for both. This wasn't a wise choice, and it's on me.

We need to have just one of the two eventually, but the very first change we need to to is to make the dashboard work.

[zmoog]

@rajvi-elastic, I added a couple of suggestions to replac azure.identityprotection with azure.identity_protection as the dataset used for queries in the dashboard.

This makes the dashboard display data.

[zmoog]

The priority is to make the integration work; after it's merged, I'll resolve the naming inconsistencies.

[rajvi-patel-22]

@zmoog, I have updated the filters as per the suggestion and dashboards are also getting populated. I have added screenshot in description.

[zmoog]

LGTM, thanks!

[elasticmachine]

Package azure - 1.5.31 containing this change is available at https://epr.elastic.co/search?package=azure