Skip to content
This repository has been archived by the owner on Jul 7, 2023. It is now read-only.
This repository has been archived by the owner on Jul 7, 2023. It is now read-only.

Metrics scraper using recycled tokens in kubernetes 1.21 #42

Closed
@shalver

Description

Metric scraper version: v1.0.6

The Bound Service Account Tokens feature is now enabled by default in kubernetes 1.21

Service account tokens are now regularly regenerated and replaced on pods. Kubernetes currently extends the life of the token after they are swapped, but also increments the serviceaccount_stale_tokens_total of the kube-apiserver and also audits the information about the offender. The dashboard-metrics-scraper pod is using these tokens past their life and is being flagged in audit log (note the annotation: authentication.k8s.io/stale-token). Moving up to go 1.15 or greater may possible correct this problem.

{
  "level": "Metadata",
  "auditID": "41b24987-fc6f-468c-ac17-0b990d96d214",
  "stage": "RequestReceived",
  "requestURI": "/apis/metrics.k8s.io/v1beta1/nodes",
  "verb": "list",
  "user": {
    "username": "system:serviceaccount:kube-system:kubernetes-dashboard",
    "uid": "90021a00-b991-497b-9b70-93c657e6c569",
    "groups": [
      "system:serviceaccounts",
      "system:serviceaccounts:kube-system",
      "system:authenticated"
    ],
    "extra": {
      "authentication.kubernetes.io/pod-name": [
        "dashboard-metrics-scraper-79f744b7dd-jpld7"
      ],
      "authentication.kubernetes.io/pod-uid": [
        "d3e55f47-b195-41f2-b52e-a14d567d4782"
      ]
    }
  },
  "sourceIPs": [
    "172.18.137.0"
  ],
  "userAgent": "metrics-sidecar/v0.0.0 (linux/amd64) kubernetes/$Format",
  "objectRef": {
    "resource": "nodes",
    "apiGroup": "metrics.k8s.io",
    "apiVersion": "v1beta1"
  },
  "requestReceivedTimestamp": "2021-04-13T18:58:14.483224Z",
  "stageTimestamp": "2021-04-13T18:58:14.483224Z",
  "annotations": {
    "authentication.k8s.io/stale-token": "subject: system:serviceaccount:kube-system:kubernetes-dashboard, seconds after warning threshold: 82321"
  }
}

Metadata

Assignees

Labels

lifecycle/frozenIndicates that an issue or PR should not be auto-closed due to staleness.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions