dataaaaa!

Triggering AI agents requires choosing between webhooks, CDC, message buses, or polling. Webhooks often lack ordering and delivery guarantees, risking stale data. A webhook tells me something changed. It does not tell me the truth about what changed, or in what order. Use CDC for durability.

External tables act as pointers to data stored outside a database, enabling SQL access without ingestion. Modern versions support columnar formats like Parquet and JSON.

dbt's rapid venture-backed growth creates tension between its community roots and commercial pressures. While it revolutionized analytics engineering, the core CLI tool faces stagnation and the community slack has become a vendor-heavy marketing channel. Where's the love it used to get?

BigQuery is optimizing LLM-powered SQL queries using proxy models to reduce latency and costs. This architecture uses smaller models for simpler tasks, routing only complex requests to larger LLMs. The proxy model approach can make LLM-based data processing more than 100x faster and cheaper.

AlterTable Workers introduces always-on agents to model, monitor, and analyze data. This AI-native platform aims to unify teams by surfacing insights and reducing costs. It functions as an AI-native data platform with always-on agents to streamline engineering workflows and data observability.

DuckDB's new Quack protocol introduces a client-server model using HTTP to transmit internal data vector blocks without transcoding. This enables multiple processes to write to a single database simultaneously, overcoming previous file-locking limitations.

DuckDB-Quack introduces a specialized extension designed to optimize analytical workloads within AI-driven environments. The architecture focuses on high-performance in-process execution, leveraging vectorized processing to minimize data movement between storage and compute layers.

OpenFoundry is an open-source operational data platform built as a Go monorepo. It replicates the Palantir Foundry capability model, including datasets, ontology, and governance. The architecture features 41 microservices and 32 shared libraries.

DuckDB's Quack protocol enables client-server architectures with multiple concurrent writers by allowing instances to communicate over HTTP(S). It supports workloads from small transactions to bulk operations. DuckDB will act both as a client and as a server! Use ATTACH to query remote tables.

The Cortex Code Skill routes Snowflake operations to the Cortex Code CLI, allowing coding agents to use specialized Snowflake expertise in headless mode. It integrates with tools like Claude Code and Cursor to automate data workflows. The skill routes Snowflake-related operations to Cortex Code CLI.

Nao is a headless analytics agent that removes the need for BI dashboards by integrating data access directly into Slack, Teams, and MCP-compatible tools like Claude. It shifts from analytics-as-a-tool to analytics-as-infrastructure, using a governed context layer for data quality.

The dlt library provides a Python-based framework for building data pipelines from REST APIs. The platform focuses on agentic data engineering, offering tools to deploy, monitor, and scale automated data loading workflows.

The Paris meetup focuses on columnar data formats for in-memory and disk storage. Technical sessions cover zero-materialization merging to solve the small file problem via run-end encoded Arrow manipulation and implementing Arrow specifications in C++20.

Cortex Code has evolved into a plugin host using portable, versioned building blocks called skills. These are simple Markdown files with YAML front matter that define agent workflows. The product is the inventory of opinionated defaults that ship with it, plus the social fabric.

XGBoost models can be converted into SQL queries to eliminate Python containers in batch scoring. By translating binary trees into nested CASE WHEN statements, engineers can run predictions directly in the warehouse. ML models are just math and arithmetic translates to SQL.

Data engineering is shifting from human-centric design to architectures that accommodate AI agents. Modern systems must now support robots reading docstrings and tools generating code.

Rust is becoming the standard for modern data infrastructure. Projects like Turso, Neon, and DataFusion leverage its performance. Most, if not all, the most-impactful projects in the database world are now Rust-based.

OpenAI's data agent stack manages 600 PB of structured data, but unstructured files in S3 lack the necessary schema and lineage. Bridging this gap requires four primitives: schema, datasets, file references, and lineage. For files, schema is something you have to invent the language to express.

The dbc 0.3.0 release transforms the ADBC driver manager into a library for Go applications and CI pipelines. It adds a GitHub Action and a list command for better automation. The agent guessed it and it worked is a design goal, focusing on machine-readable JSON and sharper error handling.

Snowflake now supports Apache Iceberg version 3, enabling features like deletion vectors for faster updates and row lineage for change data capture. Support for version 3 (v3) of the Apache Iceberg table specification is now generally available. The GET_DDL function now tracks table versions.

The Looker France User Group launch in Paris focuses on building a technical community for data professionals to exchange feedback and explore business intelligence trends.

Modern data stacks are shifting from human-centric semantic layers to agent-native context layers. While vendors promote YAML-based standards, an alternative approach uses Python code to define schema contracts and logic.

DuckDB has stabilized its Delta and Unity Catalog extensions, adding write support and time travel. Users can now perform atomic inserts and query specific table versions. The big addition since we last chatted is INSERT support! It works as simply as you expect. This enables lakehouse governance.

SAP is acquiring Dremio to transform its Business Data Cloud into an Apache Iceberg-native lakehouse. This move enables federated analytics across SAP and non-SAP sources without data movement. Dremio helps eliminate that data fragmentation and integration friction to power agentic AI.

DuckDB is an in-process analytical database that eliminates network overhead by running as a library. It uses replacement scans for zero-copy data access from formats like pandas.