dataaaaa!

The article emphasizes that AI SRE tools require a solid observability foundation, using ClickHouse as an ideal OLAP database for long retention and high-cardinality data, rather than relying solely on LLM models.

The article advocates for opinionated, code-first systems like Dagster, Modal, and dlt to decouple infrastructure from business logic, leveraging AI to assemble proven patterns within structured environments.

DuckDB outperforms Polars for 1TB data processing with its streaming execution engine, configurable memory limits, and strong integrations, while Polars fails on large datasets despite lazy execution.

The article examines the gap between stated data needs and actual usage, highlighting analytics agents as a solution to reduce access friction, with insights from query logs and data governance challenges in enterprise settings.

Squirreling is a browser-native SQL engine designed for interactive LLM data exploration, featuring async execution, late materialization, and a minimal 9 KB footprint.

Showcases using ClickHouse SQL to solve complex algorithmic challenges like Advent of Code, leveraging advanced analytical functions and CTEs for stream processing without intermediate tables.

2026 forecasts highlight AI-driven metadata automation, unified Data & AI platforms, and automated governance as critical foundations for sustainable AI projects and balanced hybrid architectures.

Fifth edition of the 'Red Book' presenting data management research, with chapters on DBMS architectures, large-scale dataflow engines, query optimization, and data integration.

Predictions for data infrastructure in 2026, covering the blurring of analytical and operational systems, growth of the Apache Arrow ecosystem with maintenance challenges, broader ADBC adoption as a standard connectivity layer, and Arrow's expansion into JavaScript/TypeScript ecosystems.

The article examines AI biases despite clean data, covering data engineering concepts such as data quality, processing pipelines, and bias detection tools relevant for data engineers.

Covers common SQL interview questions using Employee and Department tables, focusing on key concepts like joins, aggregations, and subqueries relevant for data engineers.

Palantir's Pipeline Builder is highlighted as a data engineering tool that simplifies data pipeline creation through an intuitive interface and automated features, catering to year-end requirements.

GizmoSQL is an open-source data engineering tool that leverages columnar databases such as DuckDB and the Apache Arrow ecosystem to enhance SQL query performance.

Application of visual language models to structure product image data at marketplace scale, highlighting data engineering approaches for large-scale processing.

Technical demonstration of internal usage data analysis using SQL and MotherDuck, featuring queries for percentiles, consecutive streaks, and filtering out service accounts.

Benchmark of DuckDB processing 1TB of Parquet files on a single node, assessing read and aggregation performance.

rustocr is a Rust crate for OCR, providing data engineering tools for text extraction from images, with potential integration into data processing pipelines.

The article examines why traditional data catalogs fail at enterprise scale and presents a three-part framework for implementing them as products, featuring decoupled architecture, metadata contracts in CI/CD workflows, and ruthless curation through tiering.

The article discusses emerging skills for data engineers in 2026, focusing on AI integration and open table formats like Iceberg to enhance data pipelines and automation.

Introduction to 8 fundamental principles for data engineering, covering best practices, architectures, and essential tools for building robust and scalable data systems.

Explains building a semantic search engine with PartyKit, using Vectorize as a vector database and an embedding model to index and query documents through vectors.

The article explores DataOps methodology, applying DevOps principles to data engineering to enhance collaboration, CI/CD automation, data quality, and pipeline observability.

Introduces a local-first reversible PII scrubber tool using ONNX and regex for AI workflows, emphasizing privacy and efficiency in data engineering processes.

Risk-based data quality approach to optimize testing, focusing on critical aspects and pipeline automation for data engineers.

The article examines current arguments that AI makes data modeling obsolete, including context windows as schemas, denormalized tables for LLM queries, and agentic modeling, while highlighting technical pitfalls of these approaches.