Senior Data Platform Engineer

Join Komodo Health as a Senior Data Platform Engineer, where you will design, build, operate, and improve large-scale data pipelines and foundational data platforms. This hands-on engineering role focuses on processing complex healthcare data at scale, improving reliability and performance, and contributing to the technical direction of core data systems. You will work with Python, SQL, Spark, and Airflow on AWS, and make critical decisions regarding data storage and layout. Additionally, you will strengthen data reliability through quality checks, validation, lineage, observability, and alerting.

- Design, build, operate, and improve large-scale data pipelines and foundational data platform. - Make storage and data layout decisions to keep large datasets fast to query and cost-efficient. - Strengthen reliability through data quality checks, validation, lineage, observability, and alerting.

- Strong hands-on experience building, owning, and debugging production data pipelines at multi-terabyte scale - Proven experience designing and operating data systems in AWS - Comfort using AI-assisted engineering tools for coding, debugging, documentation, and technical exploration - Advanced Python, including performant, memory-efficient code for large-scale data processing, plus strong SQL - Deep production experience with Airflow, including operating and scaling it, not just writing DAGs - A solid understanding of how data is physically stored: columnar vs. row formats, and open table formats like Iceberg, Delta Lake, or Hudi (updates, compaction, small files, partitioning) - You will be expected to leverage AI-augmented engineering tools, such as ChatGPT, Gemini, or Claude, to improve productivity and technical decision-making. This may include using AI to generate and refine code, accelerate documentation, automate test case creation, debug complex issues, explore unfamiliar technical concepts, and assess architectural trade-offs and risks - The ability to communicate technical trade-offs clearly with engineering, product, and data partners - Strong instincts for data correctness, root-cause analysis, and production troubleshooting - Experience with Spark or a comparable distributed processing framework - AI-Augmented Engineering Expectations: - Containers and orchestration (Docker, Kubernetes) - Strong core software engineering fundamentals and experience in more than one language - Data governance and lineage experience - Experience applying AI or agentic workflows to engineering, data quality, delivery, or operations - Experience delivering external-facing data products through APIs, serving layers, or customer access patterns - Success in high-growth or ambiguous environments - No healthcare background required. You’ll build that domain expertise here