Software engineers can’t imagine their lives without a centralized view across their systems, and they are ingrained in every part of the engineering process. In hindsight, it makes sense why software observability tools have taken off. Traces coupled with an easy to use interface makes it simple for an engineer to conclude that their APIs are slow because of a specific Postgres query that needs an index. It’s not enough to know that an API endpoint is slow an engineer needs to be able to have context to understand why that endpoint is slow so they can fix the underlying issue. They added another integrated layer: application performance monitoring (APM). Once you are collecting this metadata, is it easy to make them actionable? Tools like those we’ve mentioned make it simple to create dashboards containing the most critical metrics, allowing you to add alerting and anomaly detection, and easily send notifications to Slack and Pagerduty to notify your team when something goes wrong. Engineers need to be able to easily add monitoring agents within their systems to collect metrics, traces, and logs. But the companies that separated themselves from the rest were those that built incredible developer experiences.

Software observability tools gave every engineering team the ability to easily collect metrics over time across their systems, giving them instant insight into the health of these systems. That’s why the term “observability” was borrowed from control theory, where observability “is a measure of how well internal states of a system can be inferred from knowledge of its external outputs.” The thought is that, by measuring enough pieces of the information of your system over time, its internal state can be inferred over time. As you have more and more infrastructure, the need to monitor it becomes more and more critical. The premise was simple: when you host components of infrastructure like databases, servers, and API endpoints in the cloud, you should know the state of them, whether it's the memory usage of a database, the CPU usage of a server or the latency of an API endpoint.

As children of the cloud, these software observability tools rose from the foundation laid by Amazon Web Services and the cloud services to come. Ask the closest software engineer if they remember what building software was like before tools like Datadog, AppDynamics, New Relic, Grafana, Splunk, and Sumo Logic.

Core systems software