[DISCUSS] Alerting + Security

[kobelb]

Stack Monitoring's Alerts

The stack monitoring team would like to create their own alerts which run "automatically" to do things like create cluster alerts.

Saved Object Security Model

The following largely assumes that alerts themselves will be "saved objects", and as such will have to abide by the "saved objects security model". If this is an invalid assumption, this section is largely invalid but we'll want to discuss alternate plans.

I'm assuming that they'd like for these alerts to be "space agnostic" so that they can be visible regardless of the space that the user is in. For an alert to be "space agnostic" this is specified per "saved object type" like the following:

kibana/x-pack/plugins/xpack_main/index.js

Line 98 in 03cef22

isNamespaceAgnostic: true,

Additionally, we'll likely only want users of monitoring to be able to see their own monitoring related alerts. To enable this, we'll likely want to implement a dedicate "monitoring alerts saved object type" and add this to the list of saved object types which users have when they have monitoring:

kibana/x-pack/plugins/monitoring/init.js

Lines 70 to 73 in 03cef22

	savedObject: {
	all: [],
	read: []
	},

Where do we store the results?

We'll only want the results of the alerts themselves to be visible by those with access to monitoring. We could potentially store the results in an index which only the monitoring_user has access to, or we could store these in a dedicated "saved object type" themselves. I don't know if there's any infrastructure we should put in place to make this easier for all of the various applications consuming alerting.

Which "identity" do we use for the background job?

A large amount of the discussion which I've heard regarding alerting assumes that we'd like the alerts to run in the context of the user which scheduled them. I don't know if this is true for monitoring's use of alerts, and it feels like we'd want this to run under the identity of the Kibana internal server user, or another dedicated service account.

Generalizing stack management's requirements

It sounds like each "consumer" of alerts will likely be creating their own "alerting type" which will be able to choose between the following:

1. global or space specific
2. run as the internal Kibana user or the user which created it
   and we'll have to create a dedicated "saved object type" for each. Thoughts?

[kobelb]

/cc @bmcconaghy @peterschretlen @mikecote

Apologies if these are already things that you all have thought through, I don't want to be stepping on anyones toes.

[bmcconaghy]

A few (random) thoughts:

• I think more often than not alerts would be visible by anyone with the proper role/privilege, not just by the user who created them. I think most use cases would involve an alert intended as a production check, and all users in the admin role would need to be able to see it, ack it, mute it, etc.
• I don't see alerting as tied to monitoring at all, as people could be using alerting without monitoring. All they need is an index with data that they want to get alerts about.
• Not sure that we want to store the events that alerting will create as saved objects. I guess I was thinking about storing them in an alerting events index. Maybe that doesn't make sense though.
• I haven't thought much about how spaces plays into all of this. I could see users wanting the alerts to be space specific in certain cases, not in others, so offering the user the option makes sense.

[mikecote]

Few notes from me as well:

• For alert saved objects, I haven't ruled out saving alerts as saved objects with also a task manager record. I could enhance the task manager APIs to avoid having saved objects alongside of it. But this may remove some RBAC functionality since the task manager is more generic? I would also have to think about import / export since task manager records aren't saved objects.
• For history purposes (storing results) I'm thinking saving in a single place (to remove work from developers), but need to make sure it's properly protected. Attribute based access control?
• For limiting alerts visibility by user, we could down the line use attribute based access control for this? I may have to ensure I capture the proper information (user id, etc)

[bmcconaghy]

@epixa @pmuellr

[bmcconaghy]

@tvernum

[mikecote]

After discussing with @kobelb I now understand where he's coming from for having multiple types per saved object. It comes down to feature controls and removing scheduled alerts from API responses the user shouldn't see. (ex: Disabling APM should hide APM scheduled alerts from the user). The same should happen to history, one type per application. We can automate this process within the alerting code so developers don't need to know about the multiple object types and this would also satisfy the feature controls requirements without having to change security code.

We also covered that ideally object level security would be a perfect example for this but the feature won't be ready on time for us to use.

[bmcconaghy]

@jkakavas interested in your input re: the "Which "identity" do we use for the background job?" question.

[tvernum]

Attribute based access control?

Assuming you mean Elasticsearch ABAC, I wouldn't encourage that path. It's a platinum feature and I don't think you want to leave this wide open users on Gold or Basic.

If you are storing things with mixed access policies into a single index, then I think the only feasible option is to use the kibana system user to read from the index and then implement the necesary checks in Kibana server.

[bmcconaghy]

Thanks for taking a look at this @tvernum . Do you have any thoughts on the subject of "Which "identity" do we use for the background job?" We want to make sure that a user cannot create an alert that can access data the user herself cannot, but also want to avoid the situation where a user creates a lot of mission critical alerts and then leaves the company and the alerts all break. I think we are leaning towards running the ES queries for an alert as a service account but with an API key that has been limited down to mirror the permissions the user who created the alert has.

[jkakavas]

Given that we both want to:

• Keep the long running job running even if the user that initiated it is not available any more
• Do not run all the jobs with the elevated privileges a service account might have

the solution of getting an API Key as the service account but passing the roles of the authenticated user seems like a good compromise and IIRC this is the best solution we came up with during the GAH session too.

This leaves a few outstanding items to consider:

• What happens when the user's that initiated the task roles change and they shouldn't have privileges to make that specific query ? These changes are not reflected in the API key, which will still carry the old roles.

• What happens if the use case expects the long running task to stop being executed when the user is not there. We could then get the api key as the authenticated user themselves. Could this be surfaced in the UI as a choice for the user? We don't necessarily give them all the internal details but point out that the task can be tied to the user or not and make the relevant call.

[bmcconaghy]

@jkakavas @tvernum @mikecote It sounds like we have a general consensus that it's an OK compromise to run the alerts as a system user but pass the roles of the logged in user in when we obtain the API key. I think it would be an OK user experience to ask the user when the alert is created how they want the alert to run (as the logged in user or as a system user with the logged in user's roles) so long as we can provide enough context in the UI that the creator can make the right decision for the business purpose of the alert that is being created.

Regarding the two points Ionnais raises, I think we will need to store a piece of metadata on the alert that tells us what user and roles the alert was created with, and then do 2 things: 1. Wire into the Kibana user UI a task to check the API keys for a given user to make sure they match the changes that were made, and to regenerate them if not. 2. Run basically the same process on some interval so that if roles change for a user not through the UI we eventually synchronize the API keys with the current roles. For the second point I think the choice in the UI for which user the alert should run as will suffice.

All of this leaves us with one still unaddressed issue: we need API keys to be a basic feature (alerting itself will be basic) and we also need TLS not to be required for using API keys. If TLS remains a requirement for API keys, then what that means is that users can't use alerting without TLS, and by extension, it also effectively means that they can't use any solution that relies on alerting for full functionality (uptime, SIEM, APM, etc.). We are hoping to deliver some subset of the features of alerting for 7.3. Can the ES security team commit to delivering API keys in basic and the relaxation of the TLS requirement for 7.3?

[kobelb]

. I think it would be an OK user experience to ask the user when the alert is created how they want the alert to run (as the logged in user or as a system user with the logged in user's roles) so long as we can provide enough context in the UI that the creator can make the right decision for the business purpose of the alert that is being created.

I disagree with this. We shouldn't allow end-users to decide the context of their alert because as soon as we allow free-form alerts, we open up the possibility of information disclosure if the internal Kibana server user has more privileges than the user creating the alert itself.

1. Wire into the Kibana user UI a task to check the API keys for a given user to make sure they match the changes that were made, and to regenerate them if not. 2. Run basically the same process on some interval so that if roles change for a user not through the UI we eventually synchronize the API keys with the current roles. For the second point I think the choice in the UI for which user the alert should run as will suffice.

This will only be possible for alerts created by the logged in end-user. We need the user to be actively logged in to generate the API keys.