dataaaaa!

Gust is a task orchestration system designed as a lightweight alternative to Airflow. It aims to eliminate complex database setups and outdated interfaces by being 10x more efficient, faster, and far easier to set up. It uses an Elixir-based DSL to define DAGs with built-in secret management.

DuckLineage is a DuckDB extension that automatically captures and emits OpenLineage events for every query. It tracks column-level lineage, schema facets, and dataset detection across operations like JOINs and CTAS.

OpenDuck is an open-source implementation of MotherDuck's architecture, featuring differential storage and hybrid execution for DuckDB. It allows users to join local and remote tables in a single query by splitting execution plans between local machines and remote workers.

OpenDataLoader PDF is an open-source parser designed to convert PDFs into AI-ready Markdown, JSON, and HTML. It features a hybrid mode combining deterministic local processing with AI for complex layouts. Extract Markdown, JSON (with bounding boxes), and HTML from any PDF. #1 in benchmarks.

Docglow is a modern documentation generator for dbt Core that replaces the standard static site with an interactive SPA. It features a visual lineage explorer and column-level tracing. Docglow replaces it with a modern, interactive single-page application and it works with any dbt Core project.

Marimo-pair enables AI agents to interact with reactive Python notebooks as execution environments. It supports the Agent Skills standard and Claude Code, allowing developers to drop agents inside running marimo notebook sessions to automate data tasks and code execution in real-time.

Laravel Ingest is a configuration-driven ETL framework for Laravel that replaces procedural import scripts with declarative classes. It uses generators and queues to handle massive datasets with flat memory usage. Define what to import, not how to loop over it.

FlowScope is a privacy-first SQL lineage engine built with Rust and WebAssembly that runs entirely in the browser. It analyzes queries to produce table and column-level graphs. All processing happens in your browser - your SQL never leaves your machine. It supports dbt templates and linting.

AlphaPurify is a Python library for factor construction and backtesting. It processes millions of rows in seconds using memory-optimized modules for IC testing and return attribution. AlphaPurify goes far beyond IC analysis and simple graphs by offering 40+ preprocessing methods.

SLayer is an open-source semantic layer enabling AI agents to query data without writing SQL. It features auto-ingestion with foreign key awareness and dynamic model manipulation at runtime.

Pubfig is a Matplotlib-native library for generating publication-ready scientific figures. It features journal-aware export and a panel-first Figma workflow to streamline multi-panel assembly.

One-Eval is an automated agentic framework for LLM evaluation using a graph-based execution process. It enables NL2Eval, converting natural language goals into orchestrated workflows. Everything that can be automated is handled by the Agent, from benchmark selection to report generation.

Data Workers is an open-source swarm of 11 AI agents designed to automate data engineering via the Model Context Protocol. It features 160+ tools and 15 connectors to handle pipelines and incidents locally. The agents handle execution of tasks like schema management and anomaly detection.

Datahike is a durable Datalog database featuring Datomic-compatible APIs and git-like semantics. It uses persistent data structures to treat database snapshots as immutable values. Database snapshots are immutable values that can be held, shared, and queried anywhere—without locks or copying.

Visitran is an open-source tool for building Pythonic data transformation pipelines using Ibis for SQL generation. It features a visual IDE and an AI assistant for natural language modeling.

DAB is a benchmark for evaluating AI agents on complex data tasks across heterogeneous systems like PostgreSQL and MongoDB. It addresses challenges where real-world data is often fragmented across multiple heterogeneous database systems.

SlowQL is an offline SQL static analyzer that detects performance regressions and security risks without a database connection. It supports 14 dialects and 272 rules. SlowQL ships with 272 built-in rules covering 14 SQL dialects, and is built for modern engineering teams supporting CI/CD pipelines.

Way Back Home is an AI workshop platform for building intelligent agents on Google Cloud. It uses a narrative-driven architecture where participants deploy Cloud Run backends to manage registration and evidence.

DecisionBox is an AI-powered data discovery platform that connects directly to warehouses like BigQuery and Redshift. It uses autonomous agents to explore data without manual pipelines. AI writes and executes SQL queries, iterates based on results to provide severity-ranked insights.

dbterd converts dbt artifacts into Entity Relationship Diagrams using formats like DBML, Mermaid, and PlantUML. It supports multiple detection algorithms, including dbt relationships data tests and model contract's foreign_key constraints. This tool automates ERD-as-code for better data modeling.

This Claude Code plugin enables DuckDB-powered data exploration and session memory. It allows engineers to attach databases, explore schemas, and query local or remote files like Parquet and S3. Run SQL queries against attached databases or ad-hoc against files.

Text2SQirreL is a fine-tuned small language model that converts natural language into SQL for local CSV analysis. The 4B model matches the accuracy of a 685B teacher model while being 170x smaller. The 0.6B model achieves 74% accuracy with a 2x improvement over its base model.

Altimate Code is an open-source harness providing a deterministic intelligence layer for data engineering. It offers over 100 tools for SQL analysis, column-level lineage, and dbt integration. Altimate gives any LLM a deterministic data engineering intelligence layer to prevent hallucinations.

Altimate Code is an open-source agentic harness for data engineering that uses compiled engines to validate SQL against schemas and lineage. It prevents silent errors like cartesian joins and schema mismatches. The core idea: the LLM reasons; compiled engines validate; neither replaces the other.

Autoschematic is an infrastructure and policy as code tool using a push-pull state model to resolve drift. Unlike traditional tools, it enables automatic importing of existing infrastructure. Autoschematic is built around a push-pull state model. This means that it can resolve state drift.