Table of Contents 1. Introduction 2. What is Observability? * Metrics, Logs, and Traces * Monitoring vs. Observability 3. Key Components * Metrics * Logs * Traces * Dashboards and Alerts 4. Benefits of Cloud-native Observability 5. Building a Scalable Stack * Metrics with Prometheus * Logs with Loki * Traces with Jaeger 6. Challenges and Best Practices 7.
Measure what is measurable, and make measurable what is not so - Galileo Galilei Startups are notorious for moving fast. Rapid iteration, scaling and user satisfaction is all dependent on keeping your systems reliable. OpenTelemetry (OTel), is an open-source framework for metrics, logs and traces, which can be used to
Introduction: In my post here: link, I outlined with an application which I wrote, how we can read data from Modbus devices, and store them at a high frequency into InfluxDB for visualisation and alerting. In this post, were going to be taking a look at Telegraf, and how we
Introduction In this previous blog post, I demonstrated a small application I made to retrieve Modbus metrics from a Teltonika Wi-Fi router which would collect metrics and send them via REST, this system works fine if you are happy with a lag in your metrics of up to a second
Parts: 1. Getting images from ECR (Regcred) 2. Getting Helm chart from ECR (AWS CLI Cronjob) 3. Renovate checking for updates from ECR (Don't need to cover) 4. Example GitOps repo Notes: For the Argo Repo updater, note that we can do this with OIDC stuff if we
Introduction: This post is part 2 of a series outlining streamlining continuous delivery of microservices. You can read part 1 here: link In part 1, we streamlined the process of building and storing images for our microservices in ECR using GitHub Actions, in this part, we will focus on the
Introduction: This will be the first in a series of posts outlining a continuous delivery pipeline that takes code from a GitHub repo and ultimately deploys running pods into a Kubernetes cluster via ArgoCD, the full workflow of which is demonstrated above. For this blog post we will be focusing
Introduction Quite often when something goes wrong with a system, we only know that something is happening, we rarely immediately know what the issue or its cause is, we therefore need to use certain methods for identifying the issue. In this blog post, I will be going over a few
Introduction: I have recently been reading Alex Hidalgo's SLO book, and one quote that struck me, which I will paraphrase below is: In a perfect world, an SLO would be built from the ground up This implies that for us to build an SLO for our application, system
Introduction In this post, I will outline the process of building a GTFS-Realtime Prometheus exporter for monitoring and visualising the Sydney transit system with Grafana. Additionally, I will provide details about working with the GTFS API alongside Prometheus and Grafana. Check out the live Grafana dashboard here: link Overview Building
Infrastructure
Introduction: As technology evolves, so does the need for efficient, reliable and consistent infrastructure. In this post, I will be introducing the Apache Pinot Terraform provider - a powerful tool for streamlining the process of managing your Apache Pinot cluster configuration. Over the past few months, I've had
ArgoCD
Use case: To be able to deploy and manage PostgreSQL clusters with GitOps and to provide an auditable and easily understandable system for developers to manage their own Postgres clusters from the same git repo. Brief description of the tech involved: GitOps with ArgoCD: GitOps is a software and deployment
Projects
📢 Important Announcement The following article is about first retrieving Modbus data from devices with a Teltonika 4G router. If you are not too concerned about latency of the data from the devices, this article is fine. However, if you are concerned with latency and would like a solution that works
Kubernetes
In this final post of the Going Mobile series, I'll go over some issues I have faced while deploying the system, and provide a summary of the solutions I chose to resolve them. Networking (again): svclb timing out: Upon deploying the nodes in different locations I realised that
Kubernetes
3 Parts: This post will be split into 3 parts: * Networking * Monitoring * Chaos Engineering Networking: The networking stack is comprised of 3 main parts: * MagicDNS with Headscale * HAProxy * Traefik MagicDNS with Headscale The Headscale server handles the DNS resolution with MagicDNS, an example of how this is implemented in the
Kubernetes
A change of plans: The original design in part 2 had a few missing parts: * It didn't include the hosting of the Headscale server required for our Tailscale overlay network. * The assumption of being able to have 2 physical location wasn't correct in the context of