Step: gather-loki

The pre-deprovision artifacts step collects logs of all containers captured by loki. Even of containers created but no longer presented in the cluster.

Container image used for this step: ocp/cli-jq:latest

ocp/cli-jq:latest resolves to an image imported from the specified imagestream tag on the build farm (documentation).

Environment

Step exposes no environmental variables except the defaults.

Source Code

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
#!/bin/bash -x

set -o nounset
set -o errexit
set -o pipefail

export LOKI_ADDR=http://localhost:3100
export LOKI_ENDPOINT=${LOKI_ADDR}/loki/api/v1

function queue() {
  local TARGET="${1}"
  shift
  local LIVE
  LIVE="$(jobs | wc -l)"
  while [[ "${LIVE}" -ge 45 ]]; do
    sleep 1
    LIVE="$(jobs | wc -l)"
  done
  echo "${@}"
  if [[ -n "${FILTER:-}" ]]; then
    "${@}" | "${FILTER}" >"${TARGET}" &
  else
    "${@}" >"${TARGET}" &
  fi
}

if test ! -f "${KUBECONFIG}"
then
	echo "No kubeconfig, so no point in gathering loki logs."
	exit 0
fi

mkdir -p /tmp/loki-container-logs
mkdir -p "${ARTIFACT_DIR}/loki-container-logs"

echo "Checking if 'loki' namespace exists"
if [[ $(oc get ns -o json | jq '.items[].metadata.name' | grep '"loki"' | wc -l) -eq 0 ]]; then
  echo "Namespace 'loki' not found, skipping"
  exit 0
fi

oc port-forward -n loki loki-0 3100:3100 &
ocpordforwardpid="$!"

echo "Waiting for oc port-forward -n loki loki-0 3100:3100 connection"
timeout 30s bash -c "while [[ \"\$(curl -s -o /dev/null -w '%{http_code}' ${LOKI_ADDR}/ready)\" != \"200\" ]]; do echo \"Waiting...\"; sleep 1s; done"

if [[ "$(curl -s -o /dev/null -w '%{http_code}' ${LOKI_ADDR}/ready)" != "200" ]]; then
  echo "Timeout waiting for oc port-forward -n loki loki-0 3100:3100 connection"
  oc get pods -n loki
  oc describe pod -n loki
  kill ${ocpordforwardpid}
  exit 1
fi

echo "Flushing loki data on disk"
curl -s -X POST ${LOKI_ENDPOINT}/flush

kill ${ocpordforwardpid}

echo "Backup loki index and chunks ..."
queue ${ARTIFACT_DIR}/loki-container-logs/loki-data.tar.gz oc --insecure-skip-tls-verify exec -n loki loki-0 -- tar cvzf - -C /data .
wait

Properties

Property Value Description
Step timeout[?] 10m0s Limits the execution time of the step.
Resource requests (cpu) 300m Used in .resources.requests of the pod running this step.
Resource requests (memory) 300Mi Used in .resources.requests of the pod running this step.
Best effort[?] true This step's failure will not cause whole job to fail if the step is run in post phase.

GitHub Link:

https://github.com/openshift/release/blob/master/ci-operator/step-registry/gather/loki/gather-loki-ref.yaml

Owners:

Approvers:

Source code for this page located on GitHub