Using time series as alert thresholds – Robust Perception | Prometheus Monitoring Experts
This is neat, I must try this out one day to let our dev teams define their own alerting thresholds per service. #sre #alerting #prometheus
I loved that one too : https://about.gitlab.com/blog/2019/07/23/anomaly-detection-using-prometheus/
@tkanos@twtxt.net Oh that’s cool! I’ll have a play with this 👌
@prologic@twtxt.net yep very good to play, and to understand some concepts, but I did not find it much useful in my area.
@prologic@twtxt.net But after that what you can also do is : a python script that get prometheus metrics, do a bit of machine learning, and then inject the result on prometheus (very fun, but again it’s hard to have a sufficient big training set, except if your app is buggy :D)