Observability for Digital Transformation

Exploring observability's role in modern technology challenges and the emerging trends

Hi there! This week on Eximius Echo, we're exploring the SaaS sector in India, talking about Observability and the key trends we’re eyeing. In the context of technology, Observability refers to the ability to understand and gain insights into the inner workings of complex systems, particularly software applications. It provides a more comprehensive and flexible approach to monitoring, involving the collection and analysis of a broad range of data, including logs, metrics, and traces, to provide a holistic view of system behaviour. 

If you’ve not heard of us, Eximius is a sector-differentiated fund focusing on FinTech, SaaS, Media & Gaming, and HealthTech. You can find out more here. This newsletter is an attempt to share ideas, insights, and context within the realms of our chosen sectors. Let’s dive in.

The long-standing paradigm of application performance monitoring is facing challenges due to the emergence of modern development trends, including hybrid infrastructure, microservices architectures, and edge computing. These advancements, while enhancing agility and scalability, have introduced complexities that legacy APM tools are ill-equipped to handle. Thus, observability has emerged as a solution. With advanced analytical capabilities, enterprises are enabled to monitor applications running in a wide range of modern environments.

The observability market has seen explosive growth with a TAM of $60B+. While current vendors like Splunk (4B ARR, 20% YoY), Dynatrace (1.3B ARR, 18% YoY), and Datadog (2B ARR, 20% YoY) have already built enormous businesses, there is room for new startups to address gaps in the market.

Emerging trends and opportunities

• Data explosion: Over the past two years, 90% of the world's data has been generated, with the global data volume doubling every two years. According to IDC projections, the Global Datasphere is expected to surge from 45 Zettabytes (ZB) in 2019 to 175 ZB by 2025.

• Rising costs of ‘log everything, make sense later’: While the collection of telemetry data itself may be inexpensive, the expenses associated with streaming terabytes over networks, storing vast amounts of data, and conducting ongoing analyses have become significant. Clients of Splunk, Datadog, and similar companies have raised concerns about escalating costs, particularly as these services often charge based on data ingestion. This presents an opportunity for companies to target different aspects of the Splunk stack.

• Observability at the edge: To address the challenges related to cost and scalability, a fresh wave of vendors, like Cribl and Edge Delta, is emerging. These vendors employ smarter agents to assist customers in directing various subsets of data to diverse destinations, optimising costs. They enable customers to categorise data across observability tools, economical log search platforms, and archive storage, resulting in significant cost reductions. As a result, we are seeing the increasing popularity of these vendors.

• Emergence of the additional pillars: Datadog successfully integrated the three core observability components: metrics, logs, and traces. Yet, with the increasing complexity of cloud infrastructure, systems engineers demand more detailed performance insights beyond these fundamental pillars. Companies like Pyroscope, PolarSignals, and Opsian are stepping up to address this need by offering continuous profiling solutions.

• Predictive security in observability: Observability-driven security presents several key advantages over traditional security monitoring due to the wealth of data collected. These benefits encompass identifying root causes, predictive threat analysis, and expediting incident response. The integration of AI further enhances these capabilities.

• End-to-end observability: There are tremendous opportunities for startups to connect the dots between business data and observability data. These two live in disparate systems/tools- business data resides in data warehouses like Snowflake and BigQuery, whereas observability sits within Splunk and Sumologic. Unifying these will enable businesses to make more informed, data-driven business decisions at all times. 

As the market continues to evolve, fueled by trends like data explosion and rising costs, new opportunities arise for startups to innovate and address emerging challenges. With the emergence of additional observability pillars and the integration of AI, the potential for enhanced performance monitoring and predictive security underscores the importance of observability in shaping the future of technology.

If you are an entrepreneur in the SaaS or AI space in the pre-seed/seed stage, kindly fill up a short form - https://bit.ly/saas-eximius and we would love to hear more about what you are building!