cmd/grafiti: env variables precedence over config

[estroz]

cmd/grafiti: env variables take precedence over config fields

Environment variables should take precedence over fields in a configuration file. This ordering makes dynamically running grafiti easier.

Tests: tested manually by setting env vars and ensuring they've overridden config fields

[ecordell]

This seems like an odd choice to me if the config file is passed explicitly as a command argument.

I would expect: command line arg config > env var config > default config file config

[brianredbeard]

@ecordell Just to add some color to this part of the issue arises that when processing the cloudtrail files the majority existing environment variables are superfluous (e.g. a cloudtrail file already has all the data it will have for that time block so specifying start/stop timestamps would be an anti-pattern. Similarly if i needed to see the raw data as per GRF_INCLUDE_EVENT i would just look at the raw data, not use grafiti as an intermediary.)

[ecordell]

In the scheme of things I think this is a fairly minor design choice to go one way or the other, so if it makes it significantly easier to run it with this precedence that seems fine to me.

a cloudtrail file already has all the data it will have for that time block so specifying start/stop timestamps would be an anti-pattern

Wouldn't timestamps still be useful for filtering over a fixed time-box, or performing the same operations over the same period across multiple cloudtrail files (since you can set up multiple cloudtrails in different accounts, etc)?

Similarly if i needed to see the raw data as per GRF_INCLUDE_EVENT i would just look at the raw data, not use grafiti as an intermediary.

GRF_INCLUDE_EVENT can still be useful if you want to tag a resource using data only available in the raw event. parse gets you the ARN needed to tag with the rtga APIs, but if you want to add a tag based on something not in the default output for parse, you would include the raw event to extract the data you need. If you're just looking at the raw cloudtrail event data in a cloudtrail file, you have to do ARN extraction yourself.

Anyway the code LGTM since it all looks good, the design choice seems odd to me but I don't think it's a huge deal either way.

[ecordell]

LGTM

[estroz]

@ecordell updated to not log when no config file is provided, i.e. when a user intends to only use env vars to configure grafiti.