Filter out non-running pods in Prometheus

Prometheus job generated by the PodMonitor does not exclude non-running pods. All the "completed" Pods are still going to be  listed as targets in Prometheus and marked as down. This issue is related to PodMonitor implementation and is discussed in prometheus-operator/prometheus-operator#4816

Signed-off-by: Arcadie Condrat <arcadie.condrat@gmail.com>

manifests/monitoring/monitoring-config/podmonitor.yaml

@@ -23,3 +23,8 @@ spec:
           - image-reflector-controller
   podMetricsEndpoints:
     - port: http-prom
+      relabelings:
+        # https://github.com/prometheus-operator/prometheus-operator/issues/4816
+        - sourceLabels: [__meta_kubernetes_pod_phase]
+          action: keep
+          regex: Running