A multi-node, petabyte-scale, time-series database on Postgres

Today we have a big announcement: we’re officially making multi-node TimescaleDB, a petabyte-scale distributed time-series database on PostgreSQL (currently in beta and slated for general release in a couple months as part of TimescaleDB 2.0) available for free.

This decision complements our latest release, TimescaleDB 1.7, which also made a few other formerly enterprise-only capabilities free: data retention policies, downsampling, and data reorder policies (for more efficiently organizing data on disk).

All of these capabilities are being released under the Timescale License, our source-available license that permits broad usage, except for where organizations are providing TimescaleDB-as-a-service.

(Note: This announcement only affects the multi-node source code. Most of the TimescaleDB code base is still Apache 2 licensed. We have never re-licensed existing code, and we have no plans to do that.)

To try out multi-node TimescaleDB today, follow the instructions here and join our public Slack (#multi-node) to share feedback, ask questions, and get updates about new releases.

And if you’d like to dig deeper: the multi-node code base, the culmination of nearly two years of dedicated engineering effort, can be seen in this 67,000+ line pull request. (And while you're on our Github, give us a star ⭐️!)

Why we are doing this

Our mission is to enable every software developer to store, analyze, and build on top of their time-series data, so that they can measure what matters in their world: IoT devices, IT systems, marketing analytics, user behavior, financial metrics, and more. We aim to help software developers build better applications and become heroes to their customers and colleagues.

We are proud of the progress we have made towards that mission in just 3 years. Today TimescaleDB has a large and active user community, with tens of millions of downloads, hundreds of thousands of active deployments, and a public Slack channel with over 4,000 members. TimescaleDB is available for download for a variety of platforms (including Kubernetes and Docker), or via Timescale Cloud, our fully-managed, multi-cloud service on AWS, Azure, and GCP.

During these 3 years, TimescaleDB has been available as a single-node time-series database (with replicas for failover and read-only queries). This architecture has scaled to millions of metrics per second and 100+ terabytes of storage, enabling a broad set of organizations like Siemens, Schneider Electric, Warner Music, Fujitsu, Comcast, and thousands of others, to trust TimescaleDB.

Here are some detailed examples of how organizations use TimescaleDB today to power a myriad of mission-critical business operations and customer-facing applications:

Zabbix, the open-source IT monitoring platform, for storing metrics from servers, virtual machines, and network devices
Sakura Internet, a leading Internet infrastructure service provider for businesses and individuals in Japan, for monitoring network traffic
Senseforce, as a centralized datastore for all their industrial IoT data and machine metrics data
Sentinel Marine, for maritime fleet management, managing boats and other assets
LAIKA, the acclaimed animation studio (Coraline, ParaNorman, The Boxtrolls, Kubo and the Two Strings, Missing Link), for IT monitoring consolidation
TransferWise, for providing instant global monetary transfers with accurate conversion estimates
European Space Agency, for high-resolution studies of the Sun and inner heliosphere
Blue Sky Analytics, for monitoring environmental data using time-series and geo-spatial data
k6, for powering their load testing SaaS service for developers, DevOps, QA, and SRE teams
Grillo, for monitoring earthquakes in Mexico in real-time with low-cost sensors

But, we're just getting started. Making multi-node free allows us to invest even more into this community, even at the cost of immediate revenue. Long-term thinking should always win over short-term revenue, and we are building for the long-run.

Multi-node, elastically expandable, petabyte scale, for free

Last fall, we announced the multi-node version of TimescaleDB, which offers:

Scalable reads and writes, by parallelizing operations across multiple nodes and disks and increasing aggregate disk IOPS
Faster queries, via push-down aggregation
Elastic scale-out, with the ability to add new data nodes to a live system
Data replication for fault tolerance and load balancing

We also published benchmarks: ingesting over 10 million metrics per second, with faster queries via parallelization.

Our benchmarking results (see announcement blog post for more detail)

That announcement spurred a lot of questions and excitement, in particular on Hacker News. Yet many readers still walked away with one major question: will multi-node TimescaleDB be free?

Today, we are proud to say yes, multi-node TimescaleDB, offering all of the capabilities listed above, is free.

Investing even more in our community

The TimescaleDB community is vibrant. We are here to serve our community, and continuously look for ways to invest even more in our community.

With that goal in mind, we recently (with TimescaleDB 1.7) made a few other advanced capabilities free: data retention policies, downsampling (i.e., continuous aggregates paired with data retention policies), and data reorder policies (for more efficiently organizing data on disk).

It would have been easy for us to charge for these capabilities, especially multi-node. Many database providers today charge to scale beyond one node – it’s an easy and common way to monetize a database product.

But we know that our community has helped us get to where we are today. While we still need to make a buck, we also want to continue to serve you - and developers everywhere. In the long-term, helping you measure everything that matters fuels our success.

Looking ahead

Our mission is to enable every software developer to store, analyze, and build on top of their time-series data, so that they can measure what matters in their world and be heroes to their colleagues and customers. Multi-node is just one step in that long journey.

To everyone who has joined us in this adventure so far, all of the Timescale community members, thank you. We appreciate your help and contributions as we continue to make progress towards this mission.

If you’d like to get more involved with the community, we developed the Timescale Heroes program to recognize community members who are dedicated to teaching the world and building strong developer communities.

We’re always looking for new Heroes: Learn more and submit an application here.

If you’re new to TimescaleDB, we would love to hear your feedback. To join our (active and passionate) community of users, please join our public Slack here.

We (myself, and the entire Timescale engineering team) are active in all channels – and you’ll also find global developers asking questions and sharing advice every day.

Sneak peek: An open-source analytical platform for Prometheus

If you've read this far, then you might like a sneak peek at one of our next major product releases.

Another of our engineering teams has been quietly working on a new open-source analytical platform for Prometheus data: PromQL, SQL, native compression, single-node and distributed, high-cardinality, long-term storage, k8s native deployment via helm charts or Docker compose.

This architecture arose out of the pain we felt while monitoring our own infrastructure. We also realized it could be built on TimescaleDB, so the team decided to go ahead and build it themselves.

Here is a sneak peek at that Design Document, which outlines the architecture, design decisions, current status, and next steps. The team welcomes your comments:

Building an open-source analytical platform for Prometheus.

🐯🚀