dataaaaa!

Orchestra introduced .oml and rolled out anomaly detection for long-running dbt models alongside an enterprise audit trail. The platform added state machine task pausing, environment branch controls, augmented managed agents, and Google Kubernetes integration.

Engineers ported Doom directly into SQL, running the game loop and renderer inside a database. Game state is modeled relationally from WAD data, executing logic via in-database procedures at 35 Hz. A separate query pipeline renders the frame buffer directly from state tables.

MotherDuck introduces prompt_jev() to query unstructured text columns like agent traces, comments, and job ads directly in SQL. Running roughly 20 times faster than standard LLM calls at lower cost, it lets data engineers extract insights and group business metrics within a single SQL pipeline.

Dexaflow is a container-native workflow orchestrator compatible with the Apache Airflow UI and REST API. It replaces XCom plumbing with native map-reduce syntax. Built with a Go control plane, it supports isolated per-DAG virtual environments and pod-per-task execution.

Leoflow is a container-native workflow orchestrator compatible with the Apache Airflow UI and REST API. Powered by a Go control plane and Postgres, it offers native map-reduce fan-out without XCom plumbing or brokers, plus isolated per-DAG virtual environments.

Embrasure Flow is a Rust service that streams PostgreSQL changes into Apache Iceberg. It captures initial COPY and logical replication, writing Parquet data with v2 position deletes or v3 deletion vectors to S3 storage via an Iceberg REST catalog, while handling local compaction.

dbt Charts 0.9.0 adds Athena and ClickHouse warehouse support alongside a new y_start YAML field. This property enables floating bars starting away from zero, unlocking waterfall, Gantt, and candlestick charts directly within code-reviewed data pipelines.

A benchmark evaluates Cube Core, MetricFlow, Malloy, and SLayer across 54 features. SLayer and Malloy lead in dynamic query-time expressiveness, while Cube and MetricFlow rely on configuration-time modeling.

Astronomer launched the astronomer-data plugin for ChatGPT, connecting to managed or self-hosted Apache Airflow. Data engineers can debug DAG failures from task logs, scaffold pipelines, query data warehouses, and trace lineage using natural language.

Speed up string aggregations by moving repeated strings into dimension tables with narrow integer keys. Hash aggregation on fixed-width integers reduces CPU overhead and memory footprint compared to variable-length text. The strings are joined back only after grouping.

Data engineering teams are updating workflows and architectures to adapt to AI. Tools like MotherDuck and dltHub are evolving to meet these demands, shifting requirements for engineers.

Aurora PostgreSQL now embeds DuckDB to directly query Apache Iceberg and Parquet in S3 without ETL pipelines. Using the aurora_analytics extension, it supports foreign tables with predicate pushdown and column pruning to join operational and lake data.

Updates to pgclickhouse and chdb improve Postgres-ClickHouse integration with a new checkencoding option. It handles invalid byte sequences across foreign tables and chdb queries using fail, remove, replace, or truncate handlers.

Databricks integrates Postgres for data apps and AI agents, pairing ETL and orchestration for batch and streaming data with full-text and vector search to support unified analytics.

Apache Ossie now supports Microsoft ecosystem integration via a Power BI converter translating to TMDL. This enables bidirectional conversions with Snowflake Semantic Views while adding DAX query support, preserving metric logic without duplicating underlying data across platforms.

Snowflake Expedition features technical workshops for data engineers and developers to build data pipelines, AI models, and agentic workflows. Focus areas include using Snowflake CoCo to accelerate data engineering with coding agents, utilizing Snowflake CoWork, and governing enterprise data at

Building a Postgres CDC connector relies on reading the Write-Ahead Log via logical decoding and replication slots. If a consumer halts and fails to acknowledge its slot, Postgres retains unbounded WAL files, filling disk space and halting writes across the production database.

To infer missing CRM salutations from first names, a Python script queried Jev and local Laya models. Jev achieved 18/20 accuracy with usable confidence scores, while local Unsloth runs processed 1,000 records in 5-10 seconds using parallel calls.

The dbt Charts team is hosting open office hours to provide technical guidance on building visualizations with dbt Charts. Data engineers can explore implementation tips, get hands-on troubleshooting support, discuss pipeline visualization ideas, and share direct product feedback with the

Data teams struggle with rising cloud and data warehouse costs even after resizing warehouses and configuring auto-suspend. Engineers are addressing this by gaining cross-cloud spend visibility and dynamically routing eligible workloads to DuckDB compute without requiring migrations or query

Tusk is a native database client built in Rust using GPUI, featuring a virtualized, GPU-rendered grid to scroll tables with millions of rows smoothly. Built for data engineering workflows, it connects to 20 engines including ClickHouse, Snowflake, BigQuery, DuckDB, Redshift, and PostgreSQL.

Astro launched its browser IDE to general availability for testing and deploying Airflow DAGs without local setup. A new VS Code extension integrates local runtimes, logs, and deployment health checks directly into developer environments.

Jev acts as a fast SQL decision function to classify unstructured text into typed choices without token-by-token LLM generation. MotherDuck's prompt_jev() processed 100,000 text rows in 82.2 seconds for $0.50, outputting a struct with the chosen label and confidence scores directly in the query.

TypeSafe AI's Jev outputs typed decisions as JSON. Representing these outputs via Apache Arrow schemas enables zero-copy ingestion into engines like DuckDB and Polars, avoiding serialization overhead. However, analytical batch evaluation remains bottlenecked by the API's single-state request design.

Snowflake Gen2 warehouses cost 25% to 35% more credits, requiring faster runtimes to break even. They accelerate table scans, merges, and updates, but workloads must finish in under 74% of Gen1 time on AWS. Bursty queries under 60 seconds are poor fits due to minimum billing floors.