Update documentation for 2.2.0 #42

jakedipity · 2020-11-06T17:24:12Z

Signed-off-by: Jacob Hull jacob@planethull.com

Signed-off-by: Jacob Hull <jacob@planethull.com>

tanberry · 2020-11-06T18:34:47Z

docs/KUBERNETES.md

+## Installing
+
+The agent can be effortless installed in your cluster using a set of yamls we provide. These yamls contain the minimum necessary Kubernetes Objects and settings to run the agent. Teams should review and modify these yamls for the specific needs of their clusters.
+


it's a nit, but we need to say either "YAML files" or ".yml files"... can't just say "yamls". ;-)

Updated all references to correctly specify YAML files

"Objects" shouldn't be capitalized

Fixed this and other capitalization errors around Kubernetes primitives.

docs/README.md

tanberry · 2020-11-06T20:56:37Z

docs/README.md

-* [Installing](#installing)
-    * [Prerequisites](#prerequisites)
-    * [Installing on Kubernetes](#installing-on-kubernetes)
+* [Deploying](#deploying)
 * [Building](#building)


@jakedipity as we discussed in Slack, I think we need the words "installing" and "upgrading" in here somewhere. I still feel deployment is a separate act from installing and upgrading, and... SEO. ;-)

Perhaps we title this section "Installation, Upgrade, and Deployment" then do exactly as you already have in the section, link off to each of the separate files for k8s and OS.

There's no two separate steps for deploying and installing in our documentation - they're one and the same. Deploying is a better term for it but we've been going with installing for a while.

I added some text in the install step to reference both installing and deploying.

I'd agree with "deploying" as well. It's the whole process. Also "Running as Non-Root"? (parallel)

Fixed the parallel issue. Leaving the Installing for now to see what @tanberry has to say

tanberry · 2020-11-06T20:59:41Z

docs/README.md

-* [Installing](#installing)
-    * [Prerequisites](#prerequisites)
-    * [Installing on Kubernetes](#installing-on-kubernetes)
+* [Deploying](#deploying)
 * [Building](#building)
    * [Building on Linux](#building-on-linux)
    * [Building on Docker](#building-on-docker)
 * [Configuration](#configuration)
    * [Options](#options)


As discussed, let's add a section here for "Run as non-root user" and then link to both files (k8s and OS).

docs/README.md

tanberry · 2020-11-06T21:04:03Z

docs/README.md

+* `never` - Never capture Events
+* __Note:__ The default option is `always`
+
+> :warning: Due to a number of issues with Kubernetes, the agent collects events from the entire cluster including multiple nodes. To prevent duplicate logs when running multiple pods, the agents defer responsibilty of capturing Events to the oldest pod in the cluster. If that pod is killed, the next oldest pod will take over responsibility and continue from where the previous pod left off.


let's try to find a word other than "issues" here. Perhaps say something like "Doe to the native Kubernetes networking configurations, the agent collects..." or "... the default Kubernetes configuration..."

I've updated this to directly call out the issue in question

docs/OPENSHIFT.md

alfeng6

A few changes to current PR — overall looks great.

alfeng6 · 2020-11-06T21:40:26Z

docs/KUBERNETES.md

+
+## Installing
+
+The agent can be effortless installed in your cluster using a set of yamls we provide. These yamls contain the minimum necessary Kubernetes Objects and settings to run the agent. Teams should review and modify these yamls for the specific needs of their clusters.


"can be effortlessly installed"

Fixed by removing the subjective word

alfeng6 · 2020-11-06T21:44:27Z

docs/KUBERNETES.md

+* **Example Configuration Yamls:**
+  * [v1.x.x](https://raw.githubusercontent.com/logdna/logdna-agent/master/logdna-agent-ds.yaml)
+  * [v2.0.x](https://raw.githubusercontent.com/logdna/logdna-agent/master/logdna-agent-v2.yaml)
+* **Differences:** This configuration does not include the new logdna-agent namespace and is lacking a number of new Kubernetes Objects.


Maybe "Differences with latest Agent v2"

I think the documentation will be easier to maintain and more robust if we don't specify the current version the docs refer to.

Agreed with @jakedipity . I'd say better to use terminology that's absolute (agent 1.x.x, 2.0.x) instead of relative "latest Agent v2"

And "Objects" shouldn't be capitalized.

I'm torn on this one. The fact that our docs are currently very clearly split between Agent v1 and Agent v2 docs makes it ok, even necessary, to be explicit about which one we are referring to.

Going forward, once we have the Agent names based on OS/env instead of version, I plan to bust in and quickly update the all docs to reflect that, and purge the old version-based names.

So for now... I feel we could say "Differences with previous Agent v2". BUt not a big deal either way, as readers will kow they are reading about v2.

Fix the "Objects"

alfeng6 · 2020-11-06T21:48:23Z

docs/KUBERNETES.md

+  * [v1.x.x](https://raw.githubusercontent.com/logdna/logdna-agent/master/logdna-agent-ds.yaml)
+  * [v2.0.x](https://raw.githubusercontent.com/logdna/logdna-agent/master/logdna-agent-v2.yaml)
+* **Differences:** This configuration does not include the new logdna-agent namespace and is lacking a number of new Kubernetes Objects.
+* **Upgrade Steps:**


It's a little unclear with sub 1. 2. etc. Can you add in some headers?

I don't understand the ask here

I think what he's saying is that maybe 1, 2 under "1" is unclear. I don't see that. However, a, b, c are generally used. Also for 2, I'd just say "Remove the old DaemonSet in the default namespace by running 'blah blah blah'" Better to eliminate the substep when there's only one.

I'd also go with Install the latest agent.

oh. I think he means ordered lists with sub-steps? Like it should be:

XXX
a. xxx
b. xxx

XXX

ughhhh looks like markdown doesn't support that style I proposed above. That's crazy.
Well, you could use bullets for the sub-steps, though that isn't best practices in tech docs, since bullets are for items where the order doesn;t matter, and the order does matter here.

AFAIK I can't control the notation used for lists. Fixed the unnecessary substep here and somewhere else. Changed the last step to to how you suggested.

Yup, keeping it numbered / roman numerated for that reason

alfeng6 · 2020-11-06T21:50:31Z

docs/KUBERNETES.md

+```
+
+The specific tag you should use depends on your requirements, we offer a list of tags for varying compatibility:
+1. `stable` - Updates with each major, minor, and patch version updates


don't we also have a latest tag?

alfeng6 · 2020-11-06T21:53:40Z

docs/KUBERNETES.md

+### Enabling Journald on the Node
+
+Follow the steps below to ensure journald logs are written to `/var/log/journald`:
+1. Gain root access to your node


by root access you mean at least global read right?

This doesn't pertain to the agent's container and running as a non-root user. This is for administrators to directly enable journald logging on the node itsself. They will require READ+WRITE permissions for some systemd configuration files and root is the easiest option to get that.

log in as root?

Simple enough

alfeng6 · 2020-11-06T21:55:46Z

docs/OPENSHIFT.md

@@ -0,0 +1,186 @@
+# LogDNA Agent on OpenShift
+
+The agent has been tested on OpenShift 4.4, but should be compatible with any OpenShift cluster packaged with Kubernetes version 1.9 or greater.


let's just recommend on OpenShift 4.4+ — because the customer doesn't care if you've tested or not, just a guarantee that it works with X version

Sounds good, also this should be 4.6 and not 4.4

alfeng6 · 2020-11-06T21:56:03Z

docs/OPENSHIFT.md

+
+## Installing
+
+The agent can be effortless installed in your cluster using a set of yamls we provide. These yamls contain the minimum necessary OpenShift Objects and settings to run the agent. Teams should review and modify these yamls for the specific needs of their clusters.


"can be effortlessly"

yeah true... we need to avoid subjective words like "effortless" and "easy" and "fast."

alfeng6 · 2020-11-06T21:57:00Z

docs/OPENSHIFT.md

+```
+
+The specific tag you should use depends on your requirements, we offer a list of tags for varying compatibility:
+1. `stable` - Updates with each major, minor, and patch version updates


alfeng6 · 2020-11-06T21:57:39Z

docs/README.md


-### Preamble
+The agent has been tested for deployment to Kubernetes 1.9+ and OpenShift 4.6+ environments. 


let's say something along the lines of "this agent supports X version"

alfeng6 · 2020-11-06T21:59:36Z

docs/README.md

@@ -97,7 +81,7 @@ The resulting image can be found by listing the images:
 ```console
 foo@bar:~$ docker images
 REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
-<none>              <none>              f541543bcd7f        64 seconds ago      119MB
+logdna-agent-v2     dcd54a0             e471b3d8a409        22 seconds ago      135MB
 ```

 ## Configuration


can you add a third column title? also the spacing can be changed so that the second column has more width

Added more width, but didn't understand what you meant with the first ask

Well, I'd say we just show what appears when you run the command...

tanberry · 2020-11-06T22:00:11Z

docs/README.md


-* A LogDNA Account. Create an account with LogDNA by following our [quick start guide](https://docs.logdna.com/docs/logdna-quick-start-guide).
-* A LogDNA Ingestion Key. You can get your ingestion key at the top of [your account's Add a Log Source page](https://app.logdna.com/pages/add-host).
+### More Information


Thanks for adding in mention of "non-root" here.

For the title, maybe "Additional Installation Options"?

Tho... journald... is that really an install option or a configuration option? I'm not sure why journald gets listed here but not Lookback, for example?

I went with your first suggestion of raising the non-root section directly. It does seem like an important feature for cluster maintainers so I do see the case for raising it directly in the landing page documentation.

The configuration section of the Journald feature is environment agnostic and only talks about the feature as it pertains to the agent. If you understand how journald works, you can use that information to do whatever they want with the feature.

The environment specific journald setup instructions talk about how to actually enable journald in their perspective environments. They're actual instructions.

tanberry · 2020-11-06T22:02:35Z

docs/OPENSHIFT.md

+
+## Installing
+
+The agent can be effortless installed in your cluster using a set of yamls we provide. These yamls contain the minimum necessary OpenShift Objects and settings to run the agent. Teams should review and modify these yamls for the specific needs of their clusters.


yeah true... we need to avoid subjective words like "effortless" and "easy" and "fast."

…iles` 2. Removed the screenshot from `docs/README.md` 3. Add back section for running as non-root to `docs/README.md` 4. Updated the reference to Kubernetes issues to specifically outline the "won't fix" bug 5. Add `latest` to list of tags 6. Keep OpenShift support info simple: agent supports 4.6 7. Remove subjective word: `effortlessly` 8. Fix some typos and other issues Signed-off-by: Jacob Hull <jacob@planethull.com>

johnpadilla · 2020-11-09T14:57:05Z

docs/KUBERNETES.md

+
+### Installation Steps
+
+1. Navigate to the root directory of the cloned `logdna-agent` repository.


"logdna-agent-v2"?

johnpadilla

Wow, it's a lot. Sorry...

johnpadilla · 2020-11-09T14:59:03Z

docs/KUBERNETES.md

+logdna-agent-hcvhn   1/1     Running   0          10s
+```
+
+> :warning: By default the agent will run as root. To run the agent as a non-root user, check out the [run as non-root section](#run-as-non-root).


I'd write "refer to "Run as Non-Root" below" (prob bad markup. @tanberry ?

Thought it sounded too casual, fixed.

johnpadilla · 2020-11-09T15:00:49Z

docs/KUBERNETES.md

+
+## Upgrading
+
+There are two components that can be upgraded independent of each other for each new version of the agent. While not strictly required, we always recommend upgrading both components together.


There are two components--blah and blah--that can be... ? And "independently" I believe.

I list the components now and fixed the spelling mistake

johnpadilla · 2020-11-09T15:01:42Z

docs/KUBERNETES.md

+
+There are two components that can be upgraded independent of each other for each new version of the agent. While not strictly required, we always recommend upgrading both components together.
+
+### Upgrading the Configuration


"the Agent Configuration"? "your Configuration"? Not a fan of "the"

Agent feels heavily implied at this point. Changed "the" to "your"; makes sense as these are direct instructions for the reader.

johnpadilla · 2020-11-09T15:04:04Z

docs/KUBERNETES.md

+
+### Upgrading the Configuration
+
+Not every version update of the agent makes a change to our supplied configuration YAML files. These changes will be outlined in our release page to help you determine if you need to update your configuration.


"requires a change"? "results in a change"? I'd also say "If there are changes, they will be outlined..." (and point to the release page)

johnpadilla · 2020-11-09T15:08:19Z

docs/KUBERNETES.md

+
+Not every version update of the agent makes a change to our supplied configuration YAML files. These changes will be outlined in our release page to help you determine if you need to update your configuration.
+
+Due to how the agent has evolved over time, certain versions of the agent configuration YAML files require different paths to be updated successfully.


"different steps?" And maybe "Depending on which version of the agent you're using, different steps are required to update it" or something similar? And I hate to add a bunch of comments, so below is it "version of the configuration" or "version of the agent?"

I see what we're trying to say here. Perhaps "YAML files for older versions of our agent" or something below?

Simplified the statement per your example. Not sure what reference to version is confusing, they all seem to point to the version of the configuration.

johnpadilla · 2020-11-09T15:52:31Z

docs/README.md

@@ -97,7 +81,7 @@ The resulting image can be found by listing the images:
 ```console
 foo@bar:~$ docker images
 REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
-<none>              <none>              f541543bcd7f        64 seconds ago      119MB
+logdna-agent-v2     dcd54a0             e471b3d8a409        22 seconds ago      135MB
 ```

 ## Configuration


Well, I'd say we just show what appears when you run the command...

johnpadilla · 2020-11-09T15:53:51Z

docs/README.md

@@ -125,12 +119,14 @@ The agent accepts configuration from two sources, environment variables and a co
 |`LOGDNA_INCLUSION_RULES`<br>**Deprecated**: `LOGDNA_INCLUDE`|Comma separated list of glob patterns to includes files for monitoring <sup>1</sup>|`*.log,!(*.*)`|
 |`LOGDNA_INCLUSION_REGEX_RULES`<br>**Deprecated**: `LOGDNA_INCLUDE_REGEX`|Comma separated list of regex patterns to exclude files from monitoring||
 |`LOGDNA_JOURNALD_PATHS`|Comma separated list of paths (directories or files) of journald paths to monitor||
+|`LOGDNA_LOOKBACK`|The lookback strategy on startup|`smallfiles`|
+|`LOGDNA_LOG_K8S_EVENTS`|Whether the agent shoudl capture Kubernetes Events|`always`|


"should" and I'm not sure if "events" is capitalized. Don't think so.

johnpadilla · 2020-11-09T15:54:27Z

docs/README.md


-To configure the kubernetes daemonset, copy the [logdna-agent yaml](../k8s/logdna-agent.yaml) and modify the `env` section. For example, to change the hostname add the following:
+To configure the DaemonSet, make modifications to the envs section of the DameonSet [`spec.template.spec.containers.0.env`]. For example, to change the hostname add the following environment variable to the `env` list:


"modify " instead of "make modifications to"

sorry. I'm so used to trying to meet those minimum word counts :D

johnpadilla · 2020-11-09T15:55:17Z

docs/README.md


-Beginning with version 2.2 of the agent, the Dockerfile supports running the agent as a non-root user. This behavior; however, is not the default as to make the new version backwards compatible with old versions of the agent yamls. Make the following changes to your DaemonSet to run the agent as non-root:
+The lookback strategy determines how the agent handles existing container logs on startup. This strategy is determined by `LOGDNA_LOOKBACK`. There's a limited set of valid values for this option:


I'd eliminate "there's a limited set" - it sounds like a weakness that way.

Removed most traces of weakness 💪

johnpadilla · 2020-11-09T15:58:01Z

docs/README.md

+
+### Configuring Events
+
+Kubernetes and OpenShift Events are by default automatically captured by the agent. This feature can be controlled by the `LOGDNA_LOG_K8S_EVENTS` option with only two valid values:


You can control this with the following values for the 'LOGDNA_LOG_K8S_EVENTS option:

Signed-off-by: Jacob Hull <jacob@planethull.com>

tanberry · 2020-11-09T15:45:38Z

docs/KUBERNETES.md

+## Upgrading
+
+There are two components that can be upgraded independent of each other for each new version of the agent. While not strictly required, we always recommend upgrading both components together.
+


We definitely need to state what those two components are, by their names.
Otherwise, we leave the reader hanging, perhaps anxiously. ;)

This is why I rarely got better than a B+ on my college essays :). fixed

tanberry · 2020-11-09T15:46:07Z

docs/KUBERNETES.md

+#### Upgrading from Configuration v1.x.x or v2.0.x
+
+* **Example Configuration Yamls:**
+  * [v1.x.x](https://raw.githubusercontent.com/logdna/logdna-agent/master/logdna-agent-ds.yaml)


ah sorry, my search function was case sensitive :P. fixed

tanberry · 2020-11-09T15:57:33Z

docs/KUBERNETES.md

+* **Example Configuration Yamls:**
+  * [v1.x.x](https://raw.githubusercontent.com/logdna/logdna-agent/master/logdna-agent-ds.yaml)
+  * [v2.0.x](https://raw.githubusercontent.com/logdna/logdna-agent/master/logdna-agent-v2.yaml)
+* **Differences:** This configuration does not include the new logdna-agent namespace and is lacking a number of new Kubernetes Objects.


I'm torn on this one. The fact that our docs are currently very clearly split between Agent v1 and Agent v2 docs makes it ok, even necessary, to be explicit about which one we are referring to.

Going forward, once we have the Agent names based on OS/env instead of version, I plan to bust in and quickly update the all docs to reflect that, and purge the old version-based names.

So for now... I feel we could say "Differences with previous Agent v2". BUt not a big deal either way, as readers will kow they are reading about v2.

tanberry · 2020-11-09T16:00:17Z

docs/KUBERNETES.md

+  * [v1.x.x](https://raw.githubusercontent.com/logdna/logdna-agent/master/logdna-agent-ds.yaml)
+  * [v2.0.x](https://raw.githubusercontent.com/logdna/logdna-agent/master/logdna-agent-v2.yaml)
+* **Differences:** This configuration does not include the new logdna-agent namespace and is lacking a number of new Kubernetes Objects.
+* **Upgrade Steps:**


oh. I think he means ordered lists with sub-steps? Like it should be:

XXX
a. xxx
b. xxx

XXX

tanberry · 2020-11-09T16:01:28Z

docs/KUBERNETES.md

+  * [v1.x.x](https://raw.githubusercontent.com/logdna/logdna-agent/master/logdna-agent-ds.yaml)
+  * [v2.0.x](https://raw.githubusercontent.com/logdna/logdna-agent/master/logdna-agent-v2.yaml)
+* **Differences:** This configuration does not include the new logdna-agent namespace and is lacking a number of new Kubernetes Objects.
+* **Upgrade Steps:**


ughhhh looks like markdown doesn't support that style I proposed above. That's crazy.
Well, you could use bullets for the sub-steps, though that isn't best practices in tech docs, since bullets are for items where the order doesn;t matter, and the order does matter here.

tanberry · 2020-11-09T16:21:45Z

docs/KUBERNETES.md

+
+## Run as Non-Root
+
+By default the agent is configured to run as root. If this behavior is not desired, the DaemonSet can be modified to run the agent as a non-root user.


Add "user" to the first sentence, please.
"By default the agent is configured to run as root user."

Also, let's sepecify exactly where/which DaemonSet needs to be modified... I got a bit confused in third paragraph between the "local configuration file" and the "DaemonSet inside of `k8s/agent-resources.yaml".

Already talked about this bit in a DM, but just sharing here again:

That change would imply the existence of multiple root users when in fact there's just one and it's generally referred to by as just root.

Also removed the reference to local configuration file to make it less confusing

Agreed: "root"

tanberry · 2020-11-09T16:25:21Z

docs/KUBERNETES.md

+
+By default the agent is configured to run as root. If this behavior is not desired, the DaemonSet can be modified to run the agent as a non-root user.
+
+This is accomplished through Linux Capabilities that makes the agent a "Capability-dumb binary." Specifically, the agent is only allowed global read access with `CAP_DAC_READ_SEARCH`. This capability is already baked into the image and configuration. The only required step is configuring the agent DaemonSet to run as the user and group `5000:5000`.


I did a ton of googling on whether or not Capabilities should be capitalized, and it looks like No.

Appreciate the thoroughness and accuracy! fixed

tanberry · 2020-11-09T16:57:02Z

docs/KUBERNETES.md

+This is accomplished through Linux Capabilities that makes the agent a "Capability-dumb binary." Specifically, the agent is only allowed global read access with `CAP_DAC_READ_SEARCH`. This capability is already baked into the image and configuration. The only required step is configuring the agent DaemonSet to run as the user and group `5000:5000`.
+
+To update the local configuration file, add two new fields, `runAsUser` and `runAsGroup`, to the `securityContext` section found in the `logdna-agent` container in the `logdna-agent` DaemonSet inside of `k8s/agent-resources.yaml` [`spec.template.spec.containers.0.securityContext`]:
+


So is this what they want to end up with?
++++
securityContext:
capabilities:
add:
- DAC_READ_SEARCH
- runAsUser
- runAsGroup
drop:
- all

I'm hesitant to reference what the updated yaml will look like. This doesn't take into account what the customer might have added themselves and is likely to fall behind as new changes are made to the yamls. But I also understand why it's helpful to have it.

beta yaml update @c-nixon

tanberry · 2020-11-09T18:56:27Z

docs/README.md


-Add two new fields, `runAsUser` and `runAsGroup`, to the `securityContext` section found in the `logdna-agent` container in the `logdna-agent` DaemonSet [`spec.template.spec.containers.0.securityContext`]:
+* `start` - Always start at the beginning of the files.


We need to tell them where they add the desired value. Is it in the YAML file?

The section above, Configuring the Environment tells them where and how to make their changes.

tanberry · 2020-11-09T19:00:27Z

docs/README.md

+* `start` - Always start at the beginning of the files.
+* `smallfiles` - If the file is less than 8KiB, start at the beginning. Otherwise, start at the end.
+* `none` - Always start at the end of the files.
+* __Note:__ The default option is `smallfiles`


This sentence doesn't work for my brain. Do you mean "When the agent restarts, begin tailing at the point-in-time ..."??

You kind of have to combine the information here with the statement above: The lookback strategy determines how the agent handles existing files on startup.

So on startup the agent will either tail from the beginning of a file, the end of a file, or by the 8KiB heuristic.

tanberry · 2020-11-10T00:38:03Z

docs/README.md

+* `never` - Never capture events
+* __Note:__ The default option is `always`
+
+> :warning: Due to a ["won't fix" bug in the Kubernetes API](https://github.com/kubernetes/kubernetes/issues/41743), the agent collects events from the entire cluster including multiple nodes. To prevent duplicate logs when running multiple pods, the agents defer responsibilty of capturing events to the oldest pod in the cluster. If that pod is killed, the next oldest pod will take over responsibility and continue from where the previous pod left off.


Consider this rewrite:
"Due to a "won't fix" bug in the Kubernetes API, the LogDNA agent collects events from the entire cluster, including multiple nodes. To prevent duplicate logs when running multiple pods, the LogDNA agents on each pod defer responsibility of capturing events to the agent on the oldest pod in the cluster. If that pod is down, the LogDNA agent on the next oldest pod will take over responsibility and continue from where the previous pod left off.

Made this change mostly, made some slight tweaks:

the LogDNA agents on each pod => the LogDNA agent pods

the LogDNA agent on the next oldest pod => the next oldest LogDNA agent pod

I understand the distinction you're trying to make. I think these changes weed out some wordiness and keep the same distinctions :). lmk what you think

tanberry · 2020-11-10T00:41:43Z

docs/README.md

+### Configuring Events
+
+Kubernetes and OpenShift events are by default automatically captured by the agent. This feature can be controlled by the `LOGDNA_LOG_K8S_EVENTS` option with only two valid values:
+


Consider this instead, which highlights the "by default"part and avoids passive voice:

"By default, the LogDNA agent captures Kubernetes events (and Openshift events, as well, since Openshift is built on Kubernetes clusters).

To control whether the LogDNA agent collects Kubernetes events, configure the LOGDNA_LOG_K8S_EVENTS option using one of these two values:"
+++++

Also, we need to tell readers WHERE to configure the LOGDNA_LOG_K8S_EVENTS option.

@jakedipity where exactly do they configure the LOGDNA_LOG_K8S_EVENTS option? Is in the logdna.yaml file? I couldn't find it there.

Yeh, it's in the k8s yaml env section, I think this should be fairly intuitive based on how the rest of the configuration is done

tanberry · 2020-11-10T00:49:25Z

docs/README.md

+Take a look at enabling journald monitoring for [Kubernetes](KUBERNETES.md#collecting-node-journald-logs) or [OpenShift](OPENSHIFT.md#collecting-node-journald-logs).
+
+### Configuring Events
+


Let's add a bit about what the Kube Events resource is:

"A Kubernetes event is exactly what it sounds like: a resource type that is automatically generated when state changes occur in other resources, or when errors or other messages manifest across the system. Monitoring events is useful for debugging your Kubernetes cluster.

Makes sense to add and that's a very nice to read paragraph 👍

Signed-off-by: Jacob Hull <jacob@planethull.com>

c-nixon · 2020-11-13T12:58:28Z

docs/OPENSHIFT.md

+oc new-project logdna-agent
+oc create serviceaccount logdna-agent
+oc create secret generic logdna-agent-key --from-literal=logdna-agent-key=<YOUR LOGDNA INGESTION KEY>
+oc adm policy add-scc-to-user privileged system:serviceaccount:logdna-agent:logdna-agent


Just testing if we can avoid this and get away with one of the less powerful SCCs. Watch this space

Nope, doesn't look like it's possible, we may need a custom SCC or something. This is how fluentd does it, so at least we're not doing anything worse than they are

@c-nixon is line 39 where the config for marking the OpenShift container as privileged? Since we didn't have time to find a better home for this info, I am going to add a Note below that in order to run as non-root, you need to verify that your OS Container is set to privileged. And I want to cross-reference this line 39.

It always need to run as privileged, non-root or not, it's what openshift requires if you want to access the host

docs/OPENSHIFT.md

stephen-cameron-logdna · 2020-11-16T22:17:08Z

docs/KUBERNETES.md

+```console
+kubectl apply -f k8s/agent-namespace.yaml
+kubectl create secret generic logdna-agent-key -n logdna-agent --from-literal=logdna-agent-key=<YOUR LOGDNA INGESTION KEY>
+kubectl apply -f k8s/agent-resources.yaml


Will we need to update this to the beta yamls? @c-nixon

stephen-cameron-logdna · 2020-11-16T22:17:56Z

docs/KUBERNETES.md

+* **Upgrade Steps:**
+  1. If you have changes you want to persist to the new DaemonSet, backup the old DaemonSet.
+     1. Run `kubectl get daemonset -o yaml logdna-agent > old-logdna-agent-daemon-set.yaml`.
+     2. Copy any desired changes from `old-logdna-agent-daemon-set.yaml` to the DaemonSet object in `k8s/agent-resources.yaml`.


Another one for the beta yaml? @c-nixon

stephen-cameron-logdna · 2020-11-16T22:19:07Z

docs/KUBERNETES.md

+  1. If you have changes you want to persist to the new DaemonSet, backup the old DaemonSet.
+     1. Run `kubectl get daemonset -o yaml -n logdna-agent logdna-agent > old-logdna-agent-daemon-set.yaml`.
+     2. Copy any desired changes from `old-logdna-agent-daemon-set.yaml` to the DaemonSet object in `k8s/agent-resources.yaml`.
+  2. Apply the latest configuration YAML file; run `kubectl apply -f k8s/agent-resources.yaml`.


beta yaml update @c-nixon

stephen-cameron-logdna · 2020-11-16T22:21:44Z

docs/KUBERNETES.md

+Apply the updated configuration to your cluster:
+
+```console
+kubectl apply -f k8s/agent-resources.yaml


beta yaml @c-nixon

stephen-cameron-logdna · 2020-11-16T22:22:15Z

docs/KUBERNETES.md

+```
+* If you are modifying a YAML file:
+  1. Add the new environment variable to the envs section of the DaemonSet object in `k8s/agent-resources.yaml` [`spec.template.spec.containers.0.env`].
+  2. Apply the new configuration file, run `kubectl apply -f k8s/agent-resources.yaml`.


beta yaml for 190 and 191 @c-nixon

stephen-cameron-logdna · 2020-11-16T22:26:21Z

docs/OPENSHIFT.md

+```
+3. Create the remaining resources:
+```console
+oc apply -f k8s/agent-resources-openshift.yaml


I'm assuming the openshift yamls will be tagged with beta too @c-nixon

stephen-cameron-logdna · 2020-11-16T22:26:42Z

docs/OPENSHIFT.md

+* **Upgrade Steps:**
+  1. If you have changes you want to persist to the new DaemonSet, backup the old DaemonSet.
+     1. Run `oc get daemonset -o yaml logdna-agent > old-logdna-agent-daemon-set.yaml`.
+     2. Copy any desired changes from `old-logdna-agent-daemon-set.yaml` to the DaemonSet object in `k8s/agent-resources-openshift.yaml`.


beta yaml @c-nixon

stephen-cameron-logdna · 2020-11-16T22:26:59Z

docs/OPENSHIFT.md

+     1. Run `oc get daemonset -o yaml logdna-agent > old-logdna-agent-daemon-set.yaml`.
+     2. Copy any desired changes from `old-logdna-agent-daemon-set.yaml` to the DaemonSet object in `k8s/agent-resources-openshift.yaml`.
+  2. If you want to continue using journald, follow the steps for [enabling journald monitoring on the agent](#enabling-journald-monitoring-on-the-agent).
+  3. Overwrite the DaemonSet as well as create the new OpenShift objects; run `oc apply -f k8s/agent-resources-openshift.yaml`.


beta yaml @c-nixon

stephen-cameron-logdna · 2020-11-16T22:27:16Z

docs/OPENSHIFT.md

+
+This is accomplished through Linux capabilities and turning the agent binary into a "capability-dumb binary." The binary is given `CAP_DAC_READ_SEARCH` to read all files on the file system. The image already comes with this change and the necessary user and group. The only required step is configuring the agent DaemonSet to run as the user and group `5000:5000`.
+
+Add two new fields, `runAsUser` and `runAsGroup`, to the `securityContext` section found in the `logdna-agent` container in the `logdna-agent` DaemonSet inside of `k8s/agent-resources-openshift.yaml` [`spec.template.spec.containers.0.securityContext`]:


beta yaml @c-nixon

stephen-cameron-logdna · 2020-11-16T22:27:27Z

docs/OPENSHIFT.md

+Apply the updated configuration to your cluster:
+
+```console
+oc apply -f k8s/agent-resources-openshift.yaml


beta yaml @c-nixon

stephen-cameron-logdna · 2020-11-16T22:27:45Z

docs/OPENSHIFT.md

+```
+* If you are modifying a YAML file:
+  1. Add the new environment variable to the envs section of the DaemonSet object in `k8s/agent-resources-openshift.yaml` [`spec.template.spec.containers.0.env`].
+  2. Apply the new configuration file, run `oc apply -f k8s/agent-resources-openshift.yaml`.


177 and 178 beta yaml @c-nixon

These ones refer to github, so they should be ok, I think it's only the cdn ones that we'll have issues with

tanberry · 2020-11-10T17:07:29Z

docs/README.md

+### Configuring Events
+
+Kubernetes and OpenShift events are by default automatically captured by the agent. This feature can be controlled by the `LOGDNA_LOG_K8S_EVENTS` option with only two valid values:
+


@jakedipity where exactly do they configure the LOGDNA_LOG_K8S_EVENTS option? Is in the logdna.yaml file? I couldn't find it there.

tanberry · 2020-11-16T17:20:52Z

docs/KUBERNETES.md

+The upgrade path for the image depends on which image tag you are using in your DaemonSet.
+
+If your DaemonSet is configured with `logdna/logdna-agent:stable`, our default configuration setting, then you just need to delete the pods to trigger them to recreate and pull down the latest stable version of the logdna-agent image.
+


@alfeng6 I need your help clarifying what exactly the phrase "latest stable version" means. Can we point to a specific URL, or at very least use words to explain what we mean.

tanberry · 2020-11-16T18:45:01Z

docs/README.md

-kubectl apply -f https://assets.logdna.com/clients/agent-resources.yaml
-```
+* Version specific upgrade paths
+* Collecting system logs through Journald

 ## Building



Change title to be more descriptive: ##Building the Image for Agent v2.2

tanberry · 2020-11-16T19:46:35Z

docs/README.md

-REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
-<none>              <none>              f541543bcd7f        64 seconds ago      119MB
+REPOSITORY              TAG                 IMAGE ID            CREATED             SIZE
+logdna-agent-v2         dcd54a0             e471b3d8a409        22 seconds ago      135MB
 ```


@c-nixon please remove the section about building on Linux (per Padilla and Stephen reviews), and make sure Docker is uppercase and has trademark symbol after the first mention. The markdown for the trademark symbol is "Docker&reg"

Also, please add two commands BEFORE the make build-image command... the three commands that they need to run are:
git clone https://github.com/logdna/logdna-agent-v2.git
cd logdna-agent-v2
make build image

tanberry · 2020-11-16T22:02:09Z

docs/README.md

-    * [Building on Linux](#building-on-linux)
-    * [Building on Docker](#building-on-docker)
+  * [Building on Linux](#building-on-linux)
+  * [Building on Docker](#building-on-docker)


@c-nixon Please remove the whole line "Building on Linux."

tanberry · 2020-11-16T22:20:22Z

docs/OPENSHIFT.md

+```
+3. Create the remaining resources:
+```console
+oc apply -f k8s/agent-resources-openshift.yaml


@c-nixon does this yaml file name need to change, to point to Beta one?

tanberry · 2020-11-16T22:25:03Z

docs/OPENSHIFT.md

+* **Upgrade Steps:**
+  1. If you have changes you want to persist to the new DaemonSet, backup the old DaemonSet.
+     1. Run `oc get daemonset -o yaml logdna-agent > old-logdna-agent-daemon-set.yaml`.
+     2. Copy any desired changes from `old-logdna-agent-daemon-set.yaml` to the DaemonSet object in `k8s/agent-resources-openshift.yaml`.


@c-nixon here's another yaml name change?

tanberry · 2020-11-16T22:26:27Z

docs/OPENSHIFT.md

+     1. Run `oc get daemonset -o yaml logdna-agent > old-logdna-agent-daemon-set.yaml`.
+     2. Copy any desired changes from `old-logdna-agent-daemon-set.yaml` to the DaemonSet object in `k8s/agent-resources-openshift.yaml`.
+  2. If you want to continue using journald, follow the steps for [enabling journald monitoring on the agent](#enabling-journald-monitoring-on-the-agent).
+  3. Overwrite the DaemonSet as well as create the new OpenShift objects; run `oc apply -f k8s/agent-resources-openshift.yaml`.


@c-nixon another yaml file change...

tanberry · 2020-11-16T22:41:44Z

docs/OPENSHIFT.md

+oc new-project logdna-agent
+oc create serviceaccount logdna-agent
+oc create secret generic logdna-agent-key --from-literal=logdna-agent-key=<YOUR LOGDNA INGESTION KEY>
+oc adm policy add-scc-to-user privileged system:serviceaccount:logdna-agent:logdna-agent


@c-nixon is line 39 where the config for marking the OpenShift container as privileged? Since we didn't have time to find a better home for this info, I am going to add a Note below that in order to run as non-root, you need to verify that your OS Container is set to privileged. And I want to cross-reference this line 39.

tanberry · 2020-11-16T22:47:36Z

docs/OPENSHIFT.md

+## Run as Non-Root
+
+By default the agent is configured to run as root; however, the DaemonSet can be modified to run the agent as a non-root user.
+


Please replace first sentence with the following (and tweak as needed):

"By default the agent is configured to run as root; however, you can modify DaemonSet in the YAML file to run the agent as a non-root user.

And please add this Note on a separate line:

NOTE: To run as non-root, your OpenShift container must be marked as privileged.

You can also modify the DaemonSet directly on the cluster, without touching the YAML file, so I think it's better to leave that bit out

docs/README.md

doc tweaks

jakedipity self-assigned this Nov 6, 2020

jakedipity requested review from johnpadilla, alfeng6 and tanberry November 6, 2020 17:42

Documentation update for 2.2.0

f507973

Signed-off-by: Jacob Hull <jacob@planethull.com>

jakedipity force-pushed the docs branch from d7b347c to f507973 Compare November 6, 2020 21:17

tanberry reviewed Nov 6, 2020

View reviewed changes

alfeng6 suggested changes Nov 6, 2020

View reviewed changes

tanberry previously approved these changes Nov 6, 2020

View reviewed changes

jakedipity dismissed tanberry’s stale review via 29a580f November 9, 2020 11:09

jakedipity force-pushed the docs branch 2 times, most recently from 32d37b8 to 81cc8de Compare November 9, 2020 11:23

jakedipity force-pushed the docs branch from 81cc8de to 8ba396e Compare November 9, 2020 11:24

johnpadilla reviewed Nov 9, 2020

View reviewed changes

johnpadilla suggested changes Nov 9, 2020

View reviewed changes

jakedipity added 5 commits November 9, 2020 09:51

Correct capitalization issues

1c249a4

Signed-off-by: Jacob Hull <jacob@planethull.com>

Fix hyphenation and version reference issues

4a79c8f

Signed-off-by: Jacob Hull <jacob@planethull.com>

Fix parallelness in list

4b33ab1

Signed-off-by: Jacob Hull <jacob@planethull.com>

Fix some issues with various step instructions

978d67b

Signed-off-by: Jacob Hull <jacob@planethull.com>

Fix journald docs

125251b

Signed-off-by: Jacob Hull <jacob@planethull.com>

tanberry reviewed Nov 10, 2020

View reviewed changes

tanberry suggested changes Nov 10, 2020

View reviewed changes

jakedipity added 3 commits November 9, 2020 18:55

Implement a literal ton of changes

9d6b5e8

Signed-off-by: Jacob Hull <jacob@planethull.com>

Ensure Journald documentation uses the correct path: /var/log/journal

bbc2d1e

Signed-off-by: Jacob Hull <jacob@planethull.com>

Fix bad value issue with Journald patch command

19d1c42

Signed-off-by: Jacob Hull <jacob@planethull.com>

c-nixon reviewed Nov 13, 2020

View reviewed changes

c-nixon assigned tanberry and c-nixon Nov 16, 2020

c-nixon reviewed Nov 16, 2020

View reviewed changes

docs/OPENSHIFT.md Outdated Show resolved Hide resolved

stephen-cameron-logdna reviewed Nov 16, 2020

View reviewed changes

tanberry reviewed Nov 16, 2020

View reviewed changes

c-nixon reviewed Nov 16, 2020

View reviewed changes

docs/README.md Outdated Show resolved Hide resolved

c-nixon added 2 commits November 17, 2020 00:12

Apply suggestions from code review

3c6d5ac

doc tweaks

Merge branch 'master' into docs

1c61e60

c-nixon merged commit 049a98e into logdna:master Nov 17, 2020

		## Installing

		The agent can be effortless installed in your cluster using a set of yamls we provide. These yamls contain the minimum necessary Kubernetes Objects and settings to run the agent. Teams should review and modify these yamls for the specific needs of their clusters.

		@@ -0,0 +1,186 @@
		# LogDNA Agent on OpenShift

		The agent has been tested on OpenShift 4.4, but should be compatible with any OpenShift cluster packaged with Kubernetes version 1.9 or greater.


		### Preamble
		The agent has been tested for deployment to Kubernetes 1.9+ and OpenShift 4.6+ environments.


		### Installation Steps

		1. Navigate to the root directory of the cloned `logdna-agent` repository.


		## Upgrading

		There are two components that can be upgraded independent of each other for each new version of the agent. While not strictly required, we always recommend upgrading both components together.


		There are two components that can be upgraded independent of each other for each new version of the agent. While not strictly required, we always recommend upgrading both components together.

		### Upgrading the Configuration


		### Upgrading the Configuration

		Not every version update of the agent makes a change to our supplied configuration YAML files. These changes will be outlined in our release page to help you determine if you need to update your configuration.


		Not every version update of the agent makes a change to our supplied configuration YAML files. These changes will be outlined in our release page to help you determine if you need to update your configuration.

		Due to how the agent has evolved over time, certain versions of the agent configuration YAML files require different paths to be updated successfully.


		To configure the kubernetes daemonset, copy the [logdna-agent yaml](../k8s/logdna-agent.yaml) and modify the `env` section. For example, to change the hostname add the following:
		To configure the DaemonSet, make modifications to the envs section of the DameonSet [`spec.template.spec.containers.0.env`]. For example, to change the hostname add the following environment variable to the `env` list:


		Beginning with version 2.2 of the agent, the Dockerfile supports running the agent as a non-root user. This behavior; however, is not the default as to make the new version backwards compatible with old versions of the agent yamls. Make the following changes to your DaemonSet to run the agent as non-root:
		The lookback strategy determines how the agent handles existing container logs on startup. This strategy is determined by `LOGDNA_LOOKBACK`. There's a limited set of valid values for this option:


		### Configuring Events

		Kubernetes and OpenShift Events are by default automatically captured by the agent. This feature can be controlled by the `LOGDNA_LOG_K8S_EVENTS` option with only two valid values:


		## Run as Non-Root

		By default the agent is configured to run as root. If this behavior is not desired, the DaemonSet can be modified to run the agent as a non-root user.


		By default the agent is configured to run as root. If this behavior is not desired, the DaemonSet can be modified to run the agent as a non-root user.

		This is accomplished through Linux Capabilities that makes the agent a "Capability-dumb binary." Specifically, the agent is only allowed global read access with `CAP_DAC_READ_SEARCH`. This capability is already baked into the image and configuration. The only required step is configuring the agent DaemonSet to run as the user and group `5000:5000`.

		This is accomplished through Linux Capabilities that makes the agent a "Capability-dumb binary." Specifically, the agent is only allowed global read access with `CAP_DAC_READ_SEARCH`. This capability is already baked into the image and configuration. The only required step is configuring the agent DaemonSet to run as the user and group `5000:5000`.

		To update the local configuration file, add two new fields, `runAsUser` and `runAsGroup`, to the `securityContext` section found in the `logdna-agent` container in the `logdna-agent` DaemonSet inside of `k8s/agent-resources.yaml` [`spec.template.spec.containers.0.securityContext`]:


		Add two new fields, `runAsUser` and `runAsGroup`, to the `securityContext` section found in the `logdna-agent` container in the `logdna-agent` DaemonSet [`spec.template.spec.containers.0.securityContext`]:
		* `start` - Always start at the beginning of the files.

		Take a look at enabling journald monitoring for [Kubernetes](KUBERNETES.md#collecting-node-journald-logs) or [OpenShift](OPENSHIFT.md#collecting-node-journald-logs).

		### Configuring Events

		The upgrade path for the image depends on which image tag you are using in your DaemonSet.

		If your DaemonSet is configured with `logdna/logdna-agent:stable`, our default configuration setting, then you just need to delete the pods to trigger them to recreate and pull down the latest stable version of the logdna-agent image.

Update documentation for 2.2.0 #42

Update documentation for 2.2.0 #42

Conversation

jakedipity commented Nov 6, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jakedipity Nov 9, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alfeng6 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

johnpadilla left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jakedipity Nov 9, 2020 •

edited

Loading