dataaaaa!

Amazon Redshift Data Sharing enables cross-cluster querying without copying data between clusters. Producers grant datashare access, allowing consumers to read live data directly. Teams can then run dbt Cloud on consumer clusters to build and materialize transformed models via SQL pipelines.

Timestral provides standalone multivariate probabilistic time series forecasting with support for up to 32,768 steps and covariates. It runs inference on CPU or GPU with minimal dependencies like PyTorch, NumPy, and safetensors, generating 21 quantiles per time step across multiple series.

Snowflake announced the general availability of data mirroring for Snowflake Postgres. This release expands database replication tooling, alongside the v5.0.0 Python Connector. Recent platform additions also include Elastic Channels for Snowpipe Streaming and partitioned Apache Iceberg tables.

Fivetran introduced LakeCompute, letting dbt users run transformations on Iceberg tables via DuckDB compute instead of Snowflake. By managing the landing layer and compute, Fivetran disintermediates the data warehouse to cut query costs, shifting workloads away from Snowflake.

Google Cloud is integrating its borderless lakehouse architecture with Gemini to enable agentic workflows across data analytics systems. The initiative focuses on bridging data lakehouse environments with AI tooling for enhanced data engineering.

Offloading data from OLTP databases like Postgres to ClickHouse yields order-of-magnitude cost savings as data scales. Scaling transactional engines leads to linear infrastructure cost growth, but moving non-transactional workloads to an OLAP engine avoids paying high ACID compliance overhead.

Snowflake plans to partner with Bleu to deliver its AI Data Cloud on French sovereign infrastructure. Data engineers can store data in customer-controlled, S3-compatible object storage using Apache Iceberg, avoiding vendor lock-in and migration hurdles while meeting strict SecNumCloud standards.

Databow 0.2.0 adds Parquet, JSON Lines, and Arrow IPC stream exports to the Rust-based ADBC database query CLI. Because data arrives as Arrow columnar batches, databow converts directly to Parquet without row-by-row serialization. It also introduces URI-based driver inference and shell dot commands.

Modern semantic layers force engineers to write custom APIs and glue code for downstream apps and AI agents. SLayer offers an open-source, agentic-first alternative that bundles context primitives and a query engine supporting multi-stage metrics out of the box to prevent metric drift.

BigQuery introduces cost management features for generative AI functions, enabling data engineers to maintain predictable AI spend within their data analytics workflows.

The dbt Wizard agent harness automates data transformations by thinking in DAGs. It runs a native validation loop that compiles SQL, resolves references, and tests code before review. To curb warehouse compute costs, it restricts execution scope and automatically defers unchanged upstream models.

DuckDB 2.0 introduces major performance upgrades without query changes. Asynchronous I/O accelerates object storage reads, dropping S3 Parquet read latency significantly. The release optimizes recursive CTE execution, introduces a VARIANT type over text JSON, and adds disk-spilling file caches.

Snowflake released the Cortex Agent SDK in public preview alongside data engineering updates. Key additions include Snowpipe Streaming SDK v1.9.0, lineage tracking for external tables and Cortex Search, and incremental refresh for dynamic tables after failover.

Dual-axis charts distort data pipelines and reports through false crossings and competing gridlines. Tools like dbt Charts warn that intersections are scale artifacts. Instead, normalize series to a common index in your SQL models, use scatter plots, or split data into stacked single-axis charts.

Google Cloud details cross-cloud conversational analytics using open standards and a borderless lakehouse pattern to unify data across platforms.

ClickHouse tail latency spiked on idle Native TCP connections because unbatched metadata packets exceeded a reset TCP congestion window, triggering an extra round trip. Engineers eliminated the latency by batching those metadata packets at the application layer.

AWS Lambda Web Functions runs port-listening HTTP apps without event wrappers. It processes up to 128 concurrent requests per environment using active CPU billing, significantly lowering costs during database queries and I/O wait times. Built-in latency routing simplifies multi-region APIs.

dbt Charts introduces dashboards as code via an open-source, declarative BI framework. Stored as YAML alongside dbt data models, dashboards undergo version control, pull request reviews, and CI testing. This provides machine-readable context for AI and engineers to reliably query and visualize data.

The dbt Charts team is introducing updates to help data teams build and share dashboards directly within their workflows. Key technical features include using AI to accelerate dashboard development, alongside live troubleshooting and product feedback sessions to help engineers optimize data

An Iceberg table is a tree of JSON metadata, Avro manifest files, and Parquet data files. DuckDB can query each layer using SQL to trace table state. Because table files are immutable, writes create new files and update the catalog pointer to commit snapshots.

AI agents query ClickHouse with bursts of small key lookups rather than wide scans. Concurrency, caching, and workload isolation govern performance over scan speed. Size clusters for parallel in-flight queries and cache working sets, isolating agent compute from human analyst reporting.

Snowflake announced Horizon Context in preview along with named workload identity management. Recent data engineering updates also include Snowpipe Streaming SDK v1.9.0 and data lineage support for storage integrations, external tables, and Cortex Search services.

Responsive is joining Databricks to advance Lakebase and online databases built on object storage. Their architecture disaggregates compute and storage via SlateDB, an embedded LSM engine, enabling transactional and analytical engines to share open data lake storage.

Metabase automated their CI/CD release pipeline by replacing manual scripts with GitHub Actions and TypeScript. They reused pre-tested binaries to cut 30 minutes from build runs and scheduled daily cron releases to deploy patches for the latest versions from commits passing all test gates.

Snowflake has released Semantic Studio to general availability as a dedicated IDE for semantic views. Integrated within Snowflake Workspaces, it streamlines data modeling pipelines by supporting authoring, debugging, and deployment alongside git versioning and CI/CD.