Semantic model resources for data engineers

Google Cloud is expanding Knowledge Catalog to ground data agents in a unified context graph. The architecture connects trusted semantic models across analytics pipelines, enabling engineers to link disparate metadata frameworks and deliver consistent context to automated systems.

Nao launched a cloud UI for its open-source agentic BI builder. It syncs warehouse metadata across systems like Snowflake and ClickHouse to build a governed context layer of schemas, metrics, and rules. It features offline testing and MCP integration to serve governed data directly to AI agents.

Data engineers are shifting from manual modeling to building automated systems that continuously maintain semantic layers. Using production query feedback and Semantic SQL, these architectures let consumers combine governed measures while agents propose model updates and pre-aggregations.

🎙️ The Joe Reis Show — AI cannot solve poor data modeling. Feeding an entire enterprise schema into an LLM produces data slop. Instead, agents are shifting pipelines from bottom-up ETL to deterministic top-down domain modeling, highlighting semantic layers and Malloy to separate modeling logic from queries.

Silent pipeline bugs often stem from boundary mismatches and unwritten definitions rather than crashes. A healthy job run does not guarantee correctness. Robust platforms centralize business rules across modular models and use external reconciliation checks to catch silent join drops.

BigQuery measures has reached general availability to unify business intelligence metrics and knowledge graphs. This capability enhances data analytics architectures by standardizing metric definitions and integrating graph structures to boost accuracy for AI workloads.

Survey data shows 59% of teams use AI analytics, with 50% wanting AI agents querying data via MCP. Infrastructure control remains critical, as 44% require running on their own servers. Yet 31% do not know what a semantic layer is, despite it being essential for trustworthy AI.

Optimizing an analytics agent across 59 data warehouse marts improved accuracy from 49.6% to 96.5%. While basic schema profiling lowered query costs, reaching top performance required strict schema documentation, table grain definitions, join rules, and constrained semantic context.

Sigma manages its semantic layer as versioned dbt code orchestrated by Dagster across 2,883 models. Custom packages compile definitions into Sigma data models, Snowflake semantic views, and Cortex agents. Dagster automates state-aware dbt builds and syncs them through Sigma's API.

DoorDash consolidated its analytics architecture by decoupling the semantic layer from consumption. Data stored in Apache Iceberg and Snowflake is modeled via dbt metrics, serving ThoughtSpot and AI agents centrally. DoorDash treats semantic definitions as version-controlled code.

AWS architects proposed an architecture pairing the dbt Semantic Layer with the Amazon Redshift MCP Server to stop AI agents from writing raw SQL. MetricFlow compiles certified metrics while Redshift enforces read-only execution, preventing hallucinations and silent join errors.

KOHO built KoDa, an LLM self-serve tool connecting Claude to Redshift via MCP. It queries only governed dbt marts without PII or raw data. The team maintains an eval council of 375 real analyst questions to benchmark quality, proving foundational data modeling and evals matter more than the LLM.

LangChain built a layered context architecture for AI analytics agents. They sync detailed dbt docs into BigQuery, expose DAG SQL dependencies, and use a semantic layer for metrics. Context guides are versioned in Git and imported via CI/CD, boosting self-serve questions by 40x.

Checkr solved LLM metric drift across 6,000 dbt models by pairing Snowflake with a semantic layer and structured context. Using eval loops, they feed synonyms and business rules into Omni to prevent incorrect joins. Context should be added surgically to fix specific query failures.

Grounding AI analysts with a semantic layer achieved 96-100% accuracy, versus 80-91% with raw SQL against database schemas. Reliable text-to-SQL systems require data modeling, tests, and metadata context over raw schema access.

AI agents are reviving the dbt Semantic Layer by requiring governed data paths. To prevent cascading pipeline errors in multi-step LLM chains, dbt is developing "semantic SQL" to expose metrics as a standard database interface while pushing pre-aggregations upstream to reduce warehouse spend.

KOHO built KoDa, an internal tool pairing Claude and Redshift MCP with 1,400 governed dbt models. Restricting access to certified marts prevents PII exposure while an evaluation council logs and audits query accuracy. Reliable self-serve AI relies on robust semantic foundations over LLMs.

To stop LLMs from hallucinating metrics, agents use a compiled semantic bundle rather than direct SQL. The pipeline validates terms, fetches certified logic, and queries warehouse tables with forced filters via an immutable in-memory bundle.

Data Culture uses evaluation-driven development to validate AI analytics agents against modeled dbt tables. By testing queries on golden questions via a four-tier comparison cascade, accuracy grew from 0% on raw data to 94.7% with business rules and verified queries.

Bolt scaled its analytics engineering by standardizing dbt architectures into strict intermediate shapes (spine, feature, metrics) and exposed marts. Project dependencies flow strictly one-way from horizontal to vertical domains, and models enforce single ownership with no shared logic.

Okta solved AI context fragmentation by moving agent governance into dbt. Analytics engineers manage metadata, markdown context, and semantic views via pull requests. Context syncs to Snowflake Agents Schema, serving Cortex and Claude from one unified source of truth.

Semantic layers must provide a grammar instead of predefined metrics for AI agents. Rather than hardcoding every query, define base measures, context, and composable rules like time shifts or aggregations. Tools like SLayer compile these into correct, deterministic SQL.

LookML metadata indexing in Google Cloud Knowledge Catalog is now GA. This integration unifies enterprise context and semantics across data architectures, enabling automated cataloging and governance of business intelligence assets directly within the metadata platform.

Green Dot created Meta:Context, embedding a five-layer schema in dbt semantic layer YAML meta blocks. This delivers typed business context alongside query metrics via API calls, governed by CI and pull requests to guide AI agents.

Greenpeace used a 3-layer dbt package architecture to unify transformations across 25 autonomous entities on BigQuery and Fivetran. Shared dbt packages hold 75% of the logic, standardizing metrics and semantic definitions while allowing automated CI rollouts to 170 projects.

Integrating dbt MCP and semantic layers gives AI agents technical lineage, but lacks organizational context. Pedro Heyerdahl solved this with a pipeline extracting atomic facts with temporal validity and supersession, functioning like a dbt snapshot to track business decisions over time.

Snowflake automated sales forecasting by establishing a centralized context layer over certified data sources. Using Snowflake CoCo profiles, data engineers encode metric definitions and source routing, reducing query resolution from 3.0 to 1.3 steps and cutting compute costs.

Open formats like Apache Iceberg decouple storage from compute, letting teams match engines like DuckDB or Snowflake to specific workloads. True interoperability requires unified governance through standards like the Iceberg REST protocol so policies travel with data across engines.

AI data agents struggle with complex analytics across disparate databases and unstructured text. In the Data Agent Bench, top agents achieved only a 61% pass@1 rate. Adding a semantic layer significantly improves query accuracy, reinforcing the need for curated data modeling over raw text-to-SQL.

🎙️ The Analytics Engineering Podcast (dbt) — AI data agents struggle with raw data, often pulling everything into pandas instead of pushing down compute. Tested on messy benchmarks, top agents solve only 61% of queries on the first try. Data engineers must provide gold-layer tables and robust semantic layers to enable reliable queries.