Short and concise: ClickHouse vs. Apache Druid

Table of Contents

Introduction

ClickHouse and Druid are both columnar storage databases designed for handling large amounts of data and performing real-time analytics. However, there are some differences between the two that may make one more suitable than the other depending on your specific use case.

ClickHouse is a high-performance, open-source, column-oriented database management system that was developed by Yandex. It’s optimized for OLAP (Online Analytical Processing) workloads, and it’s particularly well-suited for handling complex analytical queries over large volumes of data. ClickHouse is known for its high processing speed and low latency, and it supports a wide range of data ingestion and query processing techniques.

Druid, on the other hand, is an open-source, distributed, column-oriented data store designed for OLAP workloads. It was originally developed by Metamarkets and is now an Apache Software Foundation project. Druid is particularly well-suited for handling real-time streaming data and supports efficient querying and aggregation of large data sets.

In general, ClickHouse may be a better choice if you need to handle complex analytical queries over large volumes of data, while Druid may be more suitable for handling real-time streaming data and supporting efficient querying and aggregation of large data sets. However, the choice between the two will ultimately depend on your specific use case and requirements.

Similarities

  • They both don’t support point updates and deletes natively.
  • Both of them are proven at large scale.
  • Because of their fundamental architectural similarity, they have approximately the same “limit” of efficiency and performance optimization. No “magic pill” allows any of those systems to be significantly faster than the others in all scenarios.

Differences

ClickHouse Druid
Simple and monolith (a single binary) Modular, configurable, and consequently more complicated
Cheaper setup More expensive as it requires more machines and systems
Famous for being resource efficient Notoriously famous for using so much memory
ClickHouse is built on a “shared nothing” architecture where each node in a cluster has both compute and storage resources. Druid features both worlds: shared-nothing query performance combined with the flexibility of separate storage and compute.
Nodes are responsible for both query processing and persistence/durability of the data, stored on them. There are at least three types of nodes: Master, Query, and Data.
Cannot return data immediately after insert Can return data immediately after insert
Partitions data Segments data
When a cluster grows rapidly, data is not rebalanced automatically without humans manually changing “node weights” in a partitioned table. Data management with segments. Segments could be moved between the nodes relatively easily.
Doesn’t need deep storage Needs deep storage (e.g. HDFS)
There is a special ephemeral “distributed” table that could be set up on any node, and queries to this table do everything for what “broker” nodes are responsible in Druid. It has dedicated nodes called “brokers”, which accept all queries to the system.
If there are multiple nodes, it needs Zookeeper Needs ZooKeeper
Query amplification factor becomes as large as the partitioning factor, even for queries, that cover small interval of data. A query for a little interval of data will usually hit just the required nodes.
Data ingestion in ClickHouse is simple but expects users to batch data (> 1000 rows per batch is recommended). Data ingestion is heavy. It consists of several different services.
Resembles traditional RDMBS, e. g. PostgreSQL. Resembles Big Data systems such as HBase.
Single data set Multiple unrelated data sets (multitenancy)
Professional SREs must pay close attention to very big clusters. Druid is more comparable to other Hadoop Big Data systems. Even at very high scales (more than 500 nodes), It retains “self-driving” properties even on very large scale (more than 500 nodes).
Each ClickHouse instance plays a critical role in your application’s stability Constantly removing and adding nodes is easy

References