Grafana
A web interface for searching, graphing and alerting on data. In my stack it's where I hunt through logs and where the Telegram alerts come from.
#What it is
Grafana is an open-source web app for exploring, visualising and alerting on data. It doesn't store anything itself; it connects to data sources (here, Loki) and queries them. Its Explore view is for ad-hoc searching, dashboards are for at-a-glance views, and alerting evaluates queries on a schedule and notifies a contact point when something crosses a threshold.
#Why I use it
It's the front end of my home SIEM: the place I search DNS and honeypot logs with LogQL, and the thing that pings my phone through a Telegram bot the moment the honeypot is touched.
#Benefits
- One place to search every log source.
- Built-in alerting with Telegram, email, Slack and many other contact points, no extra software.
- Provisioning files let you pre-wire data sources in YAML, so a rebuild is repeatable.
- Runs fine on a Pi.
#Trade-offs
- It's a web app with an admin account on your LAN. The default login is
admin/admin. Change it on first login. - Alert rules take some care. Set "Alert state if no data" to Normal, or a quiet honeypot (the normal state) makes Grafana alert on the empty result.
#Gotchas I hit
- "Live tailing was stopped due to following error: undefined". Live mode uses a websocket that fails if Grafana doesn't know its own address. Set
root_urlingrafana.ini, or turn Live off and run a normal query with a time range.