feat: Add local server hostname to tags

[jrconlin]

Description

Adds the local server hostname to the metric and logging tags.

Testing

Unit test modified to include check for presence.

Issue(s)

Closes #200

[jbuck]

I am concerned this tag will be too high granularity - every single k8s pod will have a different hostname. What are you trying to measure by adding the hostname?

[jrconlin]

per the source issue:
So we can group some metrics by the k8s pod let's include the hostname in metrics as a tag.

We can probably skip adding this tag to anything other than metric reporting, but we'll probably still hit granularity issues.

[jrconlin]

Ok, refactored a bit.
I added a metric section to Tags that includes tags that only go to metrics (they're mixed in right before sending them out via cadence). That should prevent the tag from going out to Sentry by accident, as well as make it very clear when adding what's happening.

I also added the tag only in get_tiles, which is only when we get the tiles. FWIW, also renamed the old periodic reporter, since it's less about re-fetching everything, and more about doing garbage collection.

[pjenvey]

I am concerned this tag will be too high granularity - every single k8s pod will have a different hostname. What are you trying to measure by adding the hostname?

Nothing too specific but it's helpful for diagnosing general individual pod behavior, allowing more correlation across metrics. E.g. the most recently spun up pod has this certain CPU usage with this number of adM request calls.

Though this is a good point, one I hadn't considered because we emit hostname tags for all autopush and some syncstorage metrics. I suppose it's a potential issue there but nobody's complained (yet).

Would a hostname tag in practice be much different from e.g. k8s metrics' resource.label.pod_name?

[jrconlin]

huh. So, looking at: https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/ I don't believe that pods generally know what their name is, that info would have to be exposed in some way.

I'm going to drop this one in favor of better identifying the requirement in #200

[jrconlin]

In light of the conversation in #200, reopening.

change applied