Prometheus job generated by the PodMonitor does not exclude non-running pods. All the "completed" Pods are still going to be listed as targets in Prometheus and marked as down. This issue is related to PodMonitor implementation and is discussed in prometheus-operator/prometheus-operator#4816
Signed-off-by: Arcadie Condrat <arcadie.condrat@gmail.com>
- add loki-stack HelmRelease to install Loki and Promtail in the monitoring namespace
- make the loki-stack HelmRelease depend on kube-prometheus-stack to install Loki's datasource and service monitors in the correct order
- add a Grafana dashboard for displaying and filtering the Flux controllers JSON logs
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
- Automate kube-prometheus-stack helm release upgrades for the v35.x range
- Remove deprecated Grafana settings
- Set Prometheus retention to 24h
- Label Flux dashboards and PodMonitors with `app.kubernetes.io/component: monitoring`
- Change the `podMonitorSelector` to match the label `app.kubernetes.io/component: monitoring`
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
Since kube-prometheus-stack helm chart v32.2.0, the `labelValue` has to
be set to "1" for the default grafana dashboard label selector to select
the flux dashboard configuration.
Also, update kube-prometheus-stack to v34.7.0, latest.
Refer: eba5b198f5
Signed-off-by: Sunny <darkowlzz@protonmail.com>