Faire · Posted 3w ago
Staff Data Infrastructure Engineer
Job relayed by Stapply Data, its source: you apply there.
- Location
- 🇺🇸 San Francisco, United States
- Contract
- Full-time
- Role
- Platform & databases
- Source
- Stapply Data
Stack
The posting
Join our team as a Staff Data Infrastructure Engineer, where you will set the technical direction for data movement from production systems to analytical stores. You will build the CDC and streaming ingestion layer, implement data contracts and quality checks, and own the reliability of the platform. You will also mentor engineers and lead the migration of existing pipelines onto the new one.
- Set the technical direction for data movement from production systems to analytical stores, and own the roadmap for implementation. - Build the CDC and streaming ingestion layer, ensuring proper handling of ordering, deduplication, late data, schema changes, and backfills. - Implement data contracts and quality checks throughout the platform, and take ownership of the datasets the business relies on.
- You take ownership of things that are broken or unowned, and you're willing to be on call for the systems you build - Experience with data quality and observability in practice, including data contracts, SLAs, and tools like Anomalo or Monte Carlo - Experience operating Airflow at scale and working with managed ingestion like Fivetran - Hands-on experience with lakehouse architectures on an open table format. Iceberg on S3 is what we use, so that's especially valuable. You should be comfortable talking about partitioning, compaction, catalogs, and copy-on-write versus merge-on-read - You've built and run data infrastructure that other teams depended on, at meaningful scale, and you've been the person setting direction for it, not just working on it - Experience in a marketplace, e-commerce, or other transaction-heavy business is a plus - Solid Python plus at least one of Kotlin, Java, Scala, or Go. Experience shipping infrastructure on AWS with Terraform - A track record of leading cross-team data initiatives and migrations, and of mentoring senior engineers - Strong Spark skills, and experience running Databricks and Snowflake against shared storage - A working understanding of data governance: access control, PII, retention and deletion, lineage, and audit - You can explain a technical tradeoff to a leadership team and to a new grad, and you can get people who disagree with each other to a decision - Strong SQL, and good instincts for how to model data so analysts and data scientists can actually use it - Deep experience with change data capture and streaming ingestion from operational databases through Kafka. You know what goes wrong with ordering, duplicates, snapshots, and schema evolution because you've dealt with it