Snowflake resources for data engineers

dltHub updated its REST API, breaking pipelines by renaming scripts to jobs, runs to job-runs, and returning 404s on legacy trigger endpoints. Inspecting the dlthub-client wheel reveals payload and SDK changes like jobs.getbyref, requiring updates to Snowflake stored procedures.

Snowflake published its October 8, 2026 feature update for Snowflake Data Clean Rooms alongside broader releases, including Snowpipe Streaming for partitioned Apache Iceberg tables and incremental refresh support for dynamic tables using the VALUES clause.

Snowflake previewed Snowflake Decision with AI_COMPLETE for automated decisions. Supporting updates for data pipelines include dynamic table incremental refreshes after failover and Snowpipe Streaming support for partitioned Apache Iceberg tables.

Snowflake previewed the Kimi K3 model alongside pipeline enhancements, including data lineage for storage integrations, external tables, and Cortex Search. Recent updates also expand dynamic table incremental refresh support and Snowpipe Streaming for partitioned Apache Iceberg tables.

Snowflake released the OpenAI GPT-6.1 Sol model in preview. Other key data engineering updates include GA for dynamic tables incremental refresh after failover, partitioned Apache Iceberg tables in Snowpipe Streaming, and data lineage for storage integrations and external tables.

Snowflake released the Anthropic Claude Sonnet 5.5 model in preview alongside response caching for Cortex AI Functions. Data engineers also gained general availability for inline stored procedures and check constraints on hybrid tables, plus Snowpipe Streaming support for partitioned Iceberg tables.

Snowflake cost optimization requires granular tracking across compute pools and Cortex AI functions. Simple fixes like right-sizing warehouses and reducing job frequencies yield the highest ROI, while observability views help detect token spikes before limits are breached.

Snowflake released zero-copy integration for Workday Data Cloud into general availability. This feature allows direct data access without manual ETL or data duplication pipelines. Snowflake also added lineage preservation across temporary tables and views.

Snowflake completed an independent NZISM Restricted assessment on AWS in Auckland, enabling New Zealand public sector data pipelines to meet local security compliance while centralizing workloads and deploying AI models securely.

Snowflake SQL patterns resolve common data issues in revenue pipelines. Engineers can handle gaps using date spines, deduplicate upserts via QUALIFY with ROW_NUMBER, and reconcile mismatched ledgers with FULL OUTER JOIN. Converting time zones before truncating prevents shifted period metrics.

Snowflake expanded data lineage support across key architectural components, adding tracking for storage integrations, external tables, model monitors, and Cortex Search services.

Snowflake previewed the ability to enforce CoCo policy with MDM. Related updates include incremental refresh support for dynamic tables, partition evolution for Apache Iceberg tables, and Snowpipe Streaming monitoring via event tables.

Databricks and Snowflake drive growth by tying compute engines directly to data state, warehousing, and AI tools. Databricks expands beyond core lakehouse analytics into serverless operational Postgres with Lakebase while scaling AI gateways to sustain massive revenue and enterprise valuations.

Snowflake added support to change the type of an organization user, refining account administration. Recent pipeline and governance updates include WITH GRANT OPTION for role grants and dynamic tables continuing incremental refresh after failover.

Snowflake announced the general availability of the Cortex Agent code execution tool. This feature enhances agentic workflows by enabling automated code execution directly within the Snowflake data platform.

Snowflake announced the public preview of Agentic Marketplace Discovery in its feature updates. Recent data engineering updates also include Iceberg partition evolution, Elastic Channels for Snowpipe Streaming, and dynamic table incremental refresh support for non-deterministic functions.

Snowflake introduced AI Gateway for Advertising, connecting Cortex Agents to Meta and TikTok via Model Context Protocol. It bridges warehouse data like transactions and KPIs with live platform signals to automate cross-channel campaign analysis and optimization.

Orchestra introduced .oml and rolled out anomaly detection for long-running dbt models alongside an enterprise audit trail. The platform added state machine task pausing, environment branch controls, augmented managed agents, and Google Kubernetes integration.

Spec-driven data engineering replaces ad-hoc AI prompts with formal specifications, business concepts, and engineering standards. Using agentic AI in Snowflake, engineers transition from subjective trial and error to repeatable pipelines, transforming one-off assistance into measurable

Snowflake dynamic tables now support incremental refresh after failover, maintaining continuous data pipeline processing without requiring full refreshes. The release also adds general availability for Optimized Refresh and RPO Assurance for failover groups to improve disaster recovery.

Snowflake has released observability support for Cortex Agents within Snowflake Native Apps. This update expands telemetry and tracking for agentic workflows deployed natively, complementing data pipeline lineage tracking.

Snowflake added WITH GRANT OPTION for role grants, streamlining access governance. Recent platform updates also delivered Apache Iceberg partition evolution, dynamic table incremental refreshes, and Snowpipe Streaming elastic channels to optimize data pipelines.

Snowflake released Organization Command Center 3rd party access configuration to general availability, streamlining organizational governance and third-party integrations across accounts. Additional updates include Iceberg table partition evolution and Snowpipe Streaming support for Iceberg.

Remote app operations for Snowflake Native Apps reached general availability, expanding operational management for distributed data applications. Recent updates also brought Snowpipe Streaming support for partitioned Apache Iceberg tables and incremental refresh for dynamic tables.

Snowflake announced general availability for Organization Features in Organization Command Center alongside third-party access configuration. Recent updates also deliver optimized refresh for failover groups and Snowpipe Streaming support for partitioned Apache Iceberg tables.

Snowflake announced improved data freshness for selected ORGANIZATION_USAGE history views. Recent updates also expand data pipeline capabilities, adding support for the VALUES clause and non-deterministic aggregate functions during dynamic table incremental refresh.

Snowflake introduced User-managed Restricted Session Scope in CoCo as generally available. Alongside updates like zero-copy support for all table formats and write support for nested namespaces, this feature enhances session security and access control across data engineering workflows.

Snowflake has reached general availability for the migration of gen 1 Openflow deployments and runtimes to gen 2. This milestone follows the retirement of gen 1 deployment creation and expands data pipeline infrastructure alongside recent additions like Iceberg tables and Snowpipe Streaming.

Snowflake released updates for Data Clean Rooms alongside key pipeline features. Data engineers gain Snowpipe Streaming for partitioned Apache Iceberg tables and incremental refresh support for dynamic tables using non-deterministic aggregates and the VALUES clause.

Snowflake released CoCo Desktop v1.21.6 alongside major data engineering updates. Key additions include GA for inline stored procedures and check constraints on hybrid tables, plus Snowpipe Streaming support for partitioned Apache Iceberg tables and incremental refreshes on dynamic tables.