Monitoring

Background Grafana has become the de-facto visualization tool for Prometheus. While it is cool to run a central Grafana hooked up to an RDS database, I think it is even better if you can make Grafana completely configurable via git and thus have stateless Grafana instances which you can scale horizontally. Based on this philosophy, I have been running a Grafana setup at Red Hat, here’s some key points: Grafana runs as pods on a Kubernetes (OpenShift) cluster Each dashboard is mounted into the pod via ConfigMap Our GitOps pipeline takes care of adding the dashboard configmaps into the namespace, so all dashboards and their changes ultimately must end up in Git One of the best benefits of this approach is that you never have to worry about Grafana upgrades/downgrades. Because the pods are stateless, you can simply roll out a new version as long as the dashboard schema stays consistent. ...

This is a quick knowledge sharing post before it gets out of my head :) I’m sure many (if not most) of you use Alertmanager as the go-to alerting system with Prometheus I really like the simplicity of Alertmanager’s configuration file and how nicely you can plug it into your configuration generation. The deployment pattern, however, is always a confusion for new adopters. I am going to try to solve some of that confusion in this post. ...

What is Prometheus If you’re an engineer working with Cloud technologies, chances are that you’ve already heard of Prometheus Prometheus is an Open Source monitoring tool. Its development started at SoundCloud and it has now evolved into being a go-to choice for metrics collection. I often relate its rise in popularity to its simple, gitops friendly configuration management, simple setup and modularity. Prometheus does a few things and does it well. While doing this, it does have some nice modularity as you can mix and match it with other tooling such as Grafana and Alertmanager. ...