ClickHouse resources for data engineers

ClickHouse introduced GroeneAI, an autonomous agent resolving CI test failures across the codebase. It reproduces bugs, builds ClickHouse, runs tests, and opens pull requests without humans in the loop, targeting database stability and continuous integration hygiene. Maintainers review each fix.

ClickHouse introduced a cost-based join optimizer using DPsub to reorder queries up to 12 tables. It safely reorders non-inner joins, including outer, semi, and anti joins, using a conflict detector that prevents semantic errors. It will be enabled by default in release 26.10.

ClickHouse Cloud provides high-throughput ingestion and sub-second analytical queries on massive datasets. In CostBench tests streaming 113.2 billion rows, it achieved lower preparation and query costs than Snowflake, Databricks, BigQuery, and Redshift Serverless.

Twilio replaced Kafka, Spark Streaming, and Elasticsearch with a consolidated ClickHouse pipeline managed by dbt. Using the dbt-clickhouse adapter, materialized views handle streaming transformations, while GitOps and Argo Workflows enable automated testing and multi-region deployment.

ADBC standardizes database access by returning Apache Arrow columnar data to bypass row-oriented protocol bottlenecks. While native engines like DuckDB enable zero-copy transfers, drivers face severe type-mapping mismatches across engines like Postgres, BigQuery, and Spark. Test schemas early.

ClickHouse Managed Postgres expanded alerts to target CDC pipeline issues, including inactive logical replication slots retaining unconfirmed WAL. It warns when inactive slots hold at least 10 GiB of WAL over two hours, preventing unbounded disk growth from detached downstream consumers.

Shipsidekick migrated from Neon to ClickHouse Managed Postgres, using ClickPipes to replicate 500 GB in hours. The move dropped CPU utilization from 100% to under 10%, increased the cache hit rate to 99%, and delivered 10x faster query execution across their transaction and analytics workloads.

Langfuse traces Cursor agent model calls and tool executions using ClickHouse as its analytical backend. In Langfuse v4, a wide ClickHouse observations table stores session context with each record, avoiding costly table joins when querying aggregates for latency, cost, and usage metrics.

OpenReplay scales front-end session replay by using ClickHouse for real-time analytics. Its largest deployments ingest billions of events and ~6 TB daily on a single vertically scaled instance with sub-second queries, leveraging built-in TTLs and materialized views to streamline data pipelines.

ClickHouse topped ClickBench on NVIDIA Vera's ARM platform using standard binaries. The dual-socket setup features 176 cores, 352 threads, and up to 1.2 TB/s memory bandwidth, demonstrating linear query scaling without custom optimizations.

ObsessionDB launched a BYOC model to run ClickHouse on your own cloud object storage. It decouples stateless compute from storage using a distributed NVMe cache, isolating ingestion and query workloads over a single bucket without re-sharding.

The benchmark tested LLMs executing agentic SQL loops over a ClickHouse data warehouse. Five new models scored 68.7% to 69.7%, trailing Claude Fable 5.1 at 76.6%. Sonnet 5.5 improved 6.5 points over Sonnet 5, completing runs within the 60-turn limit without exhausting budget.

Query Insights for ClickHouse Managed Postgres now tracks wait event metrics. Engineers can monitor average active sessions grouped by wait types like IO and Lock to pinpoint contention bottlenecks alongside CPU, read, and write usage.

ClickHouse released an experimental .NET native TCP client using native columnar format. Built with zero-allocation and zero-copy designs, it is 1.5x-2x faster than HTTP, streaming up to 1.1M rows/s and cutting GC allocations by over 99%.

Offloading data from OLTP databases like Postgres to ClickHouse yields order-of-magnitude cost savings as data scales. Scaling transactional engines leads to linear infrastructure cost growth, but moving non-transactional workloads to an OLAP engine avoids paying high ACID compliance overhead.

ClickHouse 26.9 introduces AFTER and UNTIL boundary conditions to the LIMIT clause to simplify log analysis pipelines. The release also adds incremental refresh for append-only materialized views, disk spilling for DISTINCT operations, and faster reads for the TimeSeries engine.

Osmosis cut real-time data API costs from $60k to $30k by migrating from Tinybird to ObsessionDB. The shift avoided per-vCPU-second billing on constant workloads. The team dual-wrote block data to ClickHouse, ported history, and diffed responses field-by-field before switching endpoint groups.

Managed ClickStack added built-in z-score anomaly detection to query-builder charts, removing manual baseline SQL. To scale large workloads, ClickHouse updated the scheduler with sharding across workers to distribute evaluation.

Ridgeline unified its investment analytics by migrating downstream performance queries to ClickHouse Cloud on AWS. Facing scale limits with Postgres, the team adopted ClickHouse for fast, accurate OLAP querying over alternatives like Redshift and SingleStore to power real-time reporting.

ClickHouse Cloud outperformed Redshift Serverless with 1,996x better performance per dollar during continuous ingestion of 113.2 billion rows. It leverages asynchronous inserts and pre-aggregations to optimize data preparation and minimize scanning costs for real-time streaming queries.

ClickHouse integrates Postgres for OLTP and ClickHouse for OLAP into a single managed stack. To accelerate analytics pipelines, it launched WalShadow in private preview and introduced the chdb Postgres extension alongside Managed Postgres on Google Cloud to handle heavy write bursts.

ClickHouse tail latency spiked on idle Native TCP connections because unbatched metadata packets exceeded a reset TCP congestion window, triggering an extra round trip. Engineers eliminated the latency by batching those metadata packets at the application layer.

AI agents query ClickHouse with bursts of small key lookups rather than wide scans. Concurrency, caching, and workload isolation govern performance over scan speed. Size clusters for parallel in-flight queries and cache working sets, isolating agent compute from human analyst reporting.

🎙️ The Data Flowcast by Astronomer — Ramp orchestrates warehouse integrations, ML models, and pipelines by pairing Apache Airflow with Snowflake. To resolve DAG dependency bottlenecks, the team leverages Airflow datasets, while adopting ClickHouse for OLAP workloads to achieve 100x query performance gains by minimizing joins.

iSAM Funds migrated its options research data from Postgres to ClickHouse Cloud to solve massive fan-out scaling. The move delivered 35-50x compression, boosted ingestion to 800,000 rows/sec, and eliminated materialized views via on-demand vectorized queries.

Clera migrated its 500 GB production database across 500+ tables overnight to ClickHouse Managed Postgres. Selected for NVMe-backed performance over alternatives in sysbench TPC-C tests, the live cross-region migration slashed database CPU utilization from continuous 100% spikes down to 10-20%.

ClickHouse Cloud now supports JWT authentication via OpenID Connect providers like Okta and Entra. Replacing static database passwords, ClickHouse verifies incoming tokens to generate ephemeral users with role mappings, ensuring short-lived, governed access for pipelines and users.

ClickHouse Cloud delivered over 400x better performance per dollar than BigQuery during concurrent real-time ingestion and querying. Benchmarking 113.2 billion rows showed its columnar storage, ordering, and native pre-aggregations minimized data preparation overhead and reduced query runtime.

ClickHouse integrates with Postgres to unite OLTP and OLAP workloads. Tools include pgclickhouse to query ClickHouse via native Postgres SQL with query pushdowns, and chdbhook to import and export data lake files through Postgres COPY commands across cloud object storage.

ClickHouse Cloud outperformed Databricks Serverless SQL on continuous ingestion benchmarks, achieving 752x higher performance per dollar while streaming 113.2 billion quotes. Its efficient data preparation and real-time pre-aggregations reduce query cost and eliminate pipeline lag.