dataaaaa!

While automated tools manage data transformations and queries, daily drills train analytical judgment. Practitioners navigate real-world data issues, sudden metric shifts, messy schemas, and rule precision, reinforcing critical decision-making beyond raw code syntax.

dbmask scans SQL databases for sensitive columns, masks them with deterministic fakes, and verifies changes via row-by-row validation. It offers an auditable workflow for sanitizing data when replicating production databases into dev, test, analytics, or AI pipelines.

This self-hosted quantitative workbench features pluggable data sources and local data pipelines for stock screening, monitoring, and backtesting. It leverages Polars for millisecond-level stock scanning across custom metrics and handles post-market synchronization for daily bars, minute data, and

data-eng-bench evaluates AI coding agents on real dbt tasks across a retail data warehouse. Agents build or modify dbt models tested on DuckDB and Snowflake, with pytest verifying materialized tables row-by-row against reference solutions.

QwenPaw-Data is an agentic system for enterprise data analytics that unifies data warehouses, logs, and business assets into governed context. It manages metadata graphs, lineage, and reusable methodology to automate long-horizon workflows across retrieval, execution, and artifact provenance.

Simple Checker is a data validation suite built for dataset verification. Its multi-threaded architecture executes thousands of concurrent checks per minute with automatic proxy rotation, exporting results to structured JSON and text files.

Netra is a single-binary network traffic visualization platform that processes mirrored SPAN packets in the kernel via eBPF and XDP. For storage, it persists high-frequency traffic as hourly Parquet files, querying them with an embedded DuckDB columnar engine to eliminate external database

opensdmx is a Python CLI and library for querying SDMX 2.1 REST APIs across providers like Eurostat, OECD, and the World Bank. Built as an execution layer for AI agents, it translates structured requests into deterministic data retrieval.

Diagram Design provides an agent skill to generate technical diagrams directly in static HTML without build steps. It supports 38 visual grammars—including database schemas, dependency graphs, and architectural flywheels—allowing engineers to model system patterns and redraw Mermaid sources quickly.

Provenance is an open-source geospatial data pipeline integrating 41 live layers across disparate public formats. Its architecture uses serverless edge instances with module-level caching, gracefully degrading dormant upstreams to empty 200 responses rather than failing with 5xx errors.

Dosi is an OSI-native semantic layer that compiles metrics into native SQL across 13+ dialects like Snowflake and DuckDB. Built with a Rust engine and zero-copy Arrow streaming, it eliminates metric drift and prevents fan-out double-counting across data warehouses.

ERD Builder Pro is a database design and documentation tool for developers. It supports DBML editing with bidirectional ERD sync, schema inspection for PostgreSQL, MySQL, and SQLite, and SQL DDL generation. It enables engineers to import remote structures directly into visual schema models.

Apache Fluss provides lakehouse-native streaming storage, uniting message queues, key-value stores, and lakehouses into one substrate. It externalizes stream state and offers Arrow-based streaming with Iceberg, Paimon, and Lance integrations.

Maestro is Netflix's general-purpose workflow orchestrator providing workflow-as-a-service across data and ML platforms. It schedules hundreds of thousands of workflows daily with high scalability and supports incremental processing using Apache Iceberg.

The DACH Data Job Market Tracker analyzes 3,000+ live job listings daily using an automated data pipeline. A Databricks pipeline queries the Adzuna API for 16 job titles, processes listings through a bronze/silver/gold Delta Lake, and scans postings against a 71-term skills dictionary.

This Power BI data architecture combines native visuals with DAX-driven HTML templates to create custom visual cards. Metrics are normalized across units to build consolidated scoring, while field parameters manage dynamic layers to avoid bookmark sprawl.

This sports trajectory Skill processes FIT and KML files to analyze cycling, running, or hiking data, generating analysis reports and opening local 3D route pages. It supports rendering 3D route stories into custom H.264 MP4 videos using Three.js visualization pipelines.

This end-to-end recommendation platform processes e-commerce events via PostgreSQL, MinIO, Debezium, and Kafka. Spark builds Iceberg tables while Flink processes real-time streams. Airflow orchestrates pipelines, feeding Feast with PostgreSQL offline and Redis online feature stores.

Marketingdashboard is a one-screen real-time market dashboard for financial and industry research. It tracks global stocks, commodities, US Treasury yields, sector heat, and money flow. It features an AI cockpit using OpenRouter APIs to monitor token consumption trends across 50+ LLM providers.

Rekal is a CLI tool that captures AI coding session reasoning at git commits, storing memory directly in a repo's .rekal directory. It indexes locally and syncs team context via git push. Rekal captures session decisions "including the approaches that were tried and rejected" without a SaaS.

The World of Peaches repo provides the winning report file for the Power BI World Championship. Design tokens live in CSS and JSON to keep visual consistency, while "field parameters driving the dynamic layer instead of bookmark sprawl" optimize the architecture alongside clean DAX patterns.

Quack on Demand is an open-source serving layer for DuckLake that turns it into a multi-tenant SQL warehouse using Arrow Flight SQL. It provides least-loaded routing, table-level RBAC with column-level security, and dynamic data masking, acting as "self-hosted MotherDuck, scoped to serving, on your

duckdb-zarr is a Rust-based extension that allows users to query Zarr stores directly with SQL. The tool provides table functions like readzarr, readzarrmetadata, and readzarr_groups to read remote or local datasets without external query engines.

Proof-of-Deliberation attaches a credibility signal to human labels using interaction traces and deliberation time to gate streaming updates. The Python library provides a closed-loop runner and SGD wrapper to handle human-in-the-loop active learning.

QuackWrangler is a VS Code extension for data engineering workflows. Powered by an embedded DuckDB engine, it allows querying, joining, and transforming local and remote CSV, Parquet, and JSON files without Python setup. Open CSV, Parquet, JSON, Excel, and ODS files directly—no Python environment.