dataaaaa!

Claude Opus 4.7 improves software engineering by handling complex, long-running tasks with better instruction following and self-verification. It resolves 13% more tasks on coding benchmarks than its predecessor.

Snowflake now offers documentation optimized for AI agents and LLMs via a hierarchical llms.txt file and Markdown versions of every page. This structure allows tools to fetch only the sections they need, reducing token usage. Markdown files strip HTML noise to provide cleaner context for agents.

Snowflake Storage for Apache Iceberg tables is now available on AWS and Azure, offering a managed storage layer that eliminates the operational burden of self-managed buckets. It provides built-in fail-safe recovery and automated file compaction.

Water Town proposes a framework for agentic data stacks where swarms of AI agents manage analytics and pipelines. For engineers, this evolves from simple code completion to AI-coded transformations. Agents are like violence; the only solution to the problems they cause is to use more of them.

Agentic BI requires a foundation of data consistency and modeling. SafetyCulture transitioned from a slow Data Vault to a Kimball architecture using dbt and SCD Type 2. Ensuring we had a single source of truth for each element in the model was a must-do to enable reliable AI-driven analytics.

MotherDuck is positioning itself as the premier cloud data warehouse for AI workflows. By optimizing for modern data agent requirements, it challenges traditional platforms. Jordan Tigani notes it has quickly become the best data warehouse for AI workflows by prioritizing speed and integration.

Modern BI is shifting from static dashboards to the underlying primitives like dimensional modeling and semantic layers. AI agents require these governed structures to function. BI was never about dashboards but rather the metrics and query performance that drive reliable, context-aware insights.

ADE-bench is a new open-source benchmark designed to test AI agents in realistic data environments featuring complex dbt projects and layered databases. It evaluates agents on their ability to fix bugs and refactor models rather than just writing SQL. Writing queries is not our job; not really.

DuckLineage is a DuckDB extension that automatically captures and emits OpenLineage events for every query. It tracks column-level lineage, schema facets, and dataset detection across operations like JOINs and CTAS.

Omni Agent Skills turn repeatable workflows into governed agents by combining instructions with a semantic layer. This ensures data consistency and access control. The semantic layer tells the agent what the data means. The skill controls the workflow.

Snowflake now supports Apache Iceberg tables using internal Snowflake storage. This preview feature removes the need for external cloud storage management while maintaining Fail-safe data protection for permanent tables.

BigQuery Graph integrates graph database capabilities directly into the data warehouse, allowing engineers to analyze complex relationships using the Cypher query language. It eliminates the need for data movement by letting you unlock hidden relationships in your data using existing tables.

AI-native data platforms now utilize always-on agents to automate core data engineering tasks. These systems focus on modeling, monitoring, and analyzing data to reduce operational costs. By unifying teams through automated insights, they streamline the management of modern lakehouse table formats.

Software history shows automation absorbing manual work into new layers. Agentic coding is the next shift, but it currently lacks the discipline of prior layers like CI or Terraform. The agent writes code but the process around it is unmanaged and that gap has the same shape as every previous one.

Data leaders report a gap between AI-driven coding speed and the governance needed for reliability. While 72 percent prioritize AI-assisted development, only 24 percent focus on AI-driven testing. Trust in data and data teams as an organizational priority surged from 66% to 83%.

OpenDuck is an open-source implementation of MotherDuck's architecture, featuring differential storage and hybrid execution for DuckDB. It allows users to join local and remote tables in a single query by splitting execution plans between local machines and remote workers.

The Common AI Provider integrates LLM and agent capabilities directly into Apache Airflow, supporting 20+ model providers. It features TaskFlow decorators like @task.llm and @task.agent for structured outputs and multi-step reasoning. HookToolset turns any of them into AI agent tools.

ClickHouse enables a single developer to build complex full-stack applications by unifying real-time analytics and transactional workflows. Using ClickPipes CDC to sync PostgreSQL with ClickHouse ensures sub-second queries across billions of rows.

DuckLake v1.0 is a production-ready lakehouse format that stores metadata in a SQL database rather than scattered files. It supports data inlining to prevent small file problems by staging updates in the catalog. It makes much more sense to store all metadata of a lakehouse in a database.

DuckDB v1.5.2 introduces the stable DuckLake v1.0 lakehouse format, featuring data inlining and bucket partitioning. The update enhances Iceberg support with ALTER TABLE and partitioned deletes. The test suite has uncovered a bug that was triggered by INSERT INTO statements, now resolved.

Docglow is a modern documentation generator for dbt Core that replaces the standard static site with an interactive SPA. It features a visual lineage explorer and column-level tracing. Docglow replaces it with a modern, interactive single-page application and it works with any dbt Core project.

Cortex Code is a Snowflake-native agent built on Claude Code, adding tools like snowflakesqlexecute and DataDiff. While Claude excels at repo-level context, Cortex Code provides live account introspection. The agent picks the tool that gets the job done, not the one marked native.

Analytics engineers must transition from ticket queues to building agentic systems. Success requires securing access to production data and best-in-class tooling now. The experience of working with a real data agent on real data is so different from reading about it that it is essential.

AI-generated dashboards often prioritize aesthetics over utility. Effective data engineering for visualization requires matching chart types to specific questions, such as using line charts for evolution and bar charts for ranking.

clickhousectl is the official CLI for managing local ClickHouse installations and Cloud infrastructure. It functions as a version manager and project scaffolder while providing a predictable interface for agentic development.