Processing Time Series data in Azure: the options

Time series data are data points that are collected or recorded at successive points in time. This type of data is becoming increasingly important in today’s digital world as more systems, sensors, and applications generate continuous streams of information.
From IoT devices and financial markets to website analytics and industrial monitoring, time-stamped data enables organizations to detect trends, predict outcomes, and respond to changes in real time. As industries strive for smarter operations and faster decision-making, the ability to efficiently collect, store, and analyze time series data has become a critical capability for modern data solutions.

When working with several companies in different industries, I came across many different options and approaches to store time series data and that’s why I decided to write this blog post series.

The different options in Microsoft Azure

There are several approaches and options to process and store timeseries data in Microsoft Azure, and this article will quickly list and touch upon them.

Some technologies will get a more detailed article later on.

What is Time Series data?

Time series data is a set of events. In messaging solutions, there are several types of messages that get exchanged between systems:

• Messages: these are intents. A message can be a command, a query, a request. It implies something should happen and is expected.
• Events: these are facts. An event just states something has happened and can be a measurement, a report or notification.

Time series falls under that second category. However, they are not discrete events (who are independent and actionable on their own).

Time series are order in time and are typically partitioned by a given context. (for example: MeterId , WebSiteVisitorId…). So one application will have multiple time series, typically. And these can be analysable on their own, or compared to each other.

The 4 components of Time Series data

There are four main components that make up time series data.

1. The trend: the long term direction of time series data
2. Cyclical effect: the larger fluctuations around the long term variation. Typically over longer periods than seasonality and without a fixed frequency.
3. Seasonality: the variations that occur and repeat over fixed time periods (every year, every Sunday, etc). One time series can have multiple frequencies of seasonality.
4. Residual effect (also referred to as noise). Unpredictable variations without a pattern that cannot be linked to the previous components.

Storing and processing Time Series in Azure

Several possibilities to store time series in Azure are listed below. Just know that this list is not exhaustive.

Azure Data Explorer

Azure Data Explorer (ADX, aka Kusto) is a fast and highly scalable data analytics service optimized for analyzing large volumes of structured, semi-structured, and unstructured data in near real-time. It is particularly well-suited for time series data, where high ingestion rates and quick querying over large datasets are critical. ADX supports time series-specific capabilities like automatic aggregation, downsampling, and time-based joins, making it ideal for monitoring trends, detecting anomalies, and visualizing data over time.
Its powerful Kusto Query Language (KQL) enables users to extract insights efficiently, even from billions of records. With native support for high-throughput ingestion pipelines and seamless integration with other Azure services, Azure Data Explorer provides a robust and scalable solution for managing and analyzing time series data.

Azure Fabric Realtime Analytics

Azure Fabric Real-Time Analytics is a service within Microsoft Fabric, which is an all-in-one analytics platform that combines data engineering, data warehousing, real-time analytics, and B.I. It is built on top of Kusto, the engine that powers ADX, so it inherits ADX’s strengths—fast ingestion, time series support, and the Kusto Query Language (KQL). However, it adds deeper integration with the broader Fabric ecosystem, like OneLake (a unified data lake), Power BI, and other collaborative and governance tools in Microsoft 365.
This also comes with a much different pricing model, which mostly means it’s more expensive.

Azure Database for PostgreSQL with TimescaleDB

PostgreSQL is an open-source relational database known for its robustness, extensibility, and support for complex queries, making it a solid foundation for time series data management. When combined with TimescaleDB, a PostgreSQL extension specifically designed for time series workloads, its capabilities are significantly enhanced. TimescaleDB adds features like automatic partitioning (hypertables), time-based compression, and continuous aggregates, enabling PostgreSQL to efficiently store, query, and analyse massive volumes of time-stamped data.

Azure Database for PostgreSQL provides a fully managed database service, in which TimescaleDB can also be provided.

Azure managed instance for Appache Cassandra

Apache Cassandra is a highly scalable, distributed NoSQL database designed for handling large volumes of data across multiple nodes with high availability and no single point of failure. It is particularly well-suited for time series data due to its write-optimized architecture, horizontal scalability, and support for high-throughput operations. Cassandra’s data model allows for efficient storage and retrieval of time-stamped records using techniques like wide rows and composite keys, making it a strong choice for time series applications where speed, scalability, and fault tolerance are critical.

Microsoft Azure allows to set up a managed instance of Appache Cassandra in your Azure subscription, so the actual management and setup of the cluster goes much faster, compared to setting up your own cluster in the environment.

Azure Timeseries Insights

This was a very easy to use service, but the service has been deprecated, in favour for Azure Fabric and Azure Data Explorer. But especially the Time Series Explorer was a very intuitive and interesting tool to explore and visualize time series data. You can see this in the screen shot below.