dataaaaa!

Jeppesen ForeFlight uses Airflow to manage aviation data pipelines on strict 28-day cycles. They are integrating AI agents to automate debugging and reduce manual monitoring. The Data Port agent is a two-task DAG that routes Slack pipeline alerts to either a predefined command list or an AI token.

MotherDuck is positioning itself as the premier cloud data warehouse for AI workflows. By optimizing for modern data agent requirements, it challenges traditional platforms. Jordan Tigani notes it has quickly become the best data warehouse for AI workflows by prioritizing speed and integration.

Bob Seiner advocates for Non-Invasive Data Governance, a framework that formalizes existing organizational behaviors rather than imposing new burdens. He addresses security risks posed by LLMs and autonomous agents. Non-Invasive Data Governance formalizes what organizations are already doing.

AI agents are transforming data stacks by automating query drafting and system monitoring. While unified platforms like Microsoft Fabric simplify workflows, engineers must still manage trade-offs between consistency and scale.

Jean-Baptiste Berthoux transitioned from Web3 to AI training, scaling his B2B daily rates to 4,000 euros. He focuses on helping companies integrate LLMs into their workflows. Aujourd'hui, il forme beaucoup d'entreprises et facture ses journées entre 2 000 et 4 000 euros.

Microsoft Fabric IQ integrates traditional signal-based machine learning with LLM reasoning to enhance data analytics. This hybrid architecture allows for more sophisticated pattern recognition and automated insights.

Data engineering fundamentals remain vital despite the AI boom. Building platforms without a theoretical framework leads to failure, as AI makes rigorous data engineering more critical. The rise of AI actually guarantees that data engineering is going to become more critical, not less.

The MCP Dev Summit explored transitioning the Model Context Protocol from a minimum viable product to an enterprise-ready standard. Key discussions focused on adapting the model context protocol for SLMs and the edge to support agentic AI workflows and scalable data integration.

The semantic layer is now a critical foundation for AI, providing the context necessary to prevent LLM hallucinations. Starburst uses its Trino engine to integrate semantics and AI via the AIDA agent. The semantic context is the vital element that allows LLMs to understand complex data structures.

Europe is shifting from data privacy compliance to a focus on availability and resilience to avoid service disruptions. While models like Mistral show progress, hardware dependencies on foreign GPUs remain a hurdle. The challenge is not isolation, but mastering risk exposure through diversification.

Genspark Claw evolves open-source agents by hosting them on cloud virtual machines to ensure security and data safety. It integrates specialized sub-agents for spreadsheets and slides while managing API keys to prevent credential leaks.

Wes McKinney discusses transitioning to AI coding agents, using Go as a primary language for multi-agent workflows. He focuses on local data sovereignty to combat platform decay. Wes shares how he overcame his initial existential dread about the future of software engineering.

Agentic AI is shifting data engineering from manual coding to autonomous workflows where agents execute queries and debug models. This transition promises 10-50x productivity gains. The role of data engineers is shifting from code authors to operators of autonomous agents.

Decision intelligence shifts focus from dashboards to contextual decisions using entity resolution and graph analytics. By linking siloed records across systems, engineers can create trusted data products.

Gorgias transitioned to a GenAI-first architecture to power conversational agents for 15,000 e-commerce clients. The engineering team evolved by introducing the Machine Learning Analyst role to manage feedback loops and quality.

AI is shifting data engineering from output-driven tasks to outcome-focused workflows. By embedding natural language prompts directly into SQL, practitioners improve model accuracy. Good commenting practices already make your codebase easier for humans to navigate.

AI acceleration is causing identity crises in tech as rapid change disorients workers. Engineers face a reverse centaur effect where humans become parts of a machine. Are white-collar engineers about to face the Amazon warehouse treatment through token consumption leaderboards?

Data engineering has shifted from Hadoop to cloud platforms, yet batch processing remains the dominant workflow. Airflow provides essential dependency management for complex, multi-team pipelines. Batch pipelines remain the most common approach even as real-time use cases grow.

Multi-agent systems are evolving into a formal engineering discipline, moving away from "YOLO agent engineering" toward systematic evaluation. These workflows act as generalizations of classical ML ensembles, using multiple models to improve robustness and reduce prediction error.

Joe Reis discusses the evolving landscape of data engineering, focusing on the resurgence of data modeling and AI's integration into the field. He emphasizes that data modeling is the foundation of everything we do in data. The discussion explores how AI tools are reshaping coding workflows.

Claude Code is a command-line interface tool leaked from Anthropic that allows for direct terminal-based agentic coding. It functions as a specialized agent capable of executing shell commands and managing complex file edits. The leak reveals a shift toward autonomous developer workflows.

Traditional data engineering often forces messy reality into rigid tables, losing 90% of information. Amit Prakash proposes using dynamic ontologies to extract high-fidelity insights from unstructured sources. We've been forcing the messy reality of business into rigid database tables.

A security vulnerability in a popular LLM library highlights risks in data engineering pipelines. Attackers can exploit dependencies to exfiltrate sensitive credentials and environment variables.

Axel de Goursac details a three-phase GenAI strategy: top-down task identification, bottom-up ROI reconciliation, and full-scale operationalization. Success requires identifying the roles and the tasks and estimating the ROI and the feasibility of projects to ensure scalable value.

Data engineering in 2026 requires balancing AI tools like GitHub Copilot and Claude with core fundamentals. Success depends on portfolio strategies and technical depth. Mastering the fundamentals of data engineering remains crucial despite the shift caused by AI coding tools.