Senior Data Engineer

Join People Data Labs (PDL), a remote-first company focused on providing the best people and company data. As a Senior Data Engineer, you will build infrastructure for data ingestion, transformation, and loading, develop CI/CD pipelines, and solve complex data engineering and data science problems. Enjoy unlimited PTO, monthly employee stipends, and comprehensive medical, dental, and vision insurance.

- Construire l'infrastructure pour l'ingestion, la transformation et le chargement d'un volume de données en constante augmentation à partir de diverses sources. - Développer des pipelines CI/CD et des systèmes de détection d'anomalies capables d'améliorer continuellement la qualité des données. - Imaginer des solutions à des problèmes d'ingénierie et de science des données largement indéfinis.

- Experience building scalable data processing systems (e.g., cleaning, transformation) from the ground up - Experience with Python - Experience working in Databricks (including delta live tables, data lakehouse patterns, etc.) - Experience with data warehousing (e.g., Databricks, Snowflake, Redshift, BigQuery, or similar) - Expertise with Apache Spark (Java, Scala, and/or Python-based) - Understanding of modern data storage formats and tools (e.g., parquet, ORC, Avro, Delta Lake) - 5-7+ years of industry experience with clear examples of strategic technical problem-solving and implementation - Experience using developer-oriented data pipeline and workflow orchestration (e.g., Airflow (preferred), dbt, dagster or similar) - Knowledge of modern data design and storage patterns (e.g., incremental updating, partitioning and segmentation, rebuilds and backfills) - Experience with cloud computing services (AWS (preferred), GCP, Azure or similar) - Strong software development fundamentals - Experience with SQL - Balance high ownership and autonomy with a strong ability to collaborate - Scope and breakdown projects, communicate and collaborate progress and blockers effectively with your manager, team, and stakeholders - Demonstrate strong written communication skills on Slack/Chat and in documents - Work effectively remotely (able to be proactive about managing blockers, proactive on reaching out and asking questions, and participating in team activities) - Exhibt experience in writing data design docs (pipeline design, dataflow, schema design) - People Thrive Here Who Can: - Experience working with data acquisition / data integration - Experience working with entity data (entity resolution / record linkage) - Degree in a quantitative discipline such as computer science, mathematics, statistics, or engineering - Expertise with Python and the Python data stack (e.g., numpy, pandas) - Experience with streaming platforms (e.g., Kafka) - Experience evaluating data quality and maintaining consistently high data standards across new feature releases (e.g., consistency, accuracy, validity, completeness) - If you are looking to be part of a team discovering the next frontier of data-as-a-service (DaaS) with a high level of autonomy and opportunity for direct contributions, this might be the role for you. We like our engineers to be thoughtful, quirky, and willing to fearlessly try new things. Failure is embraced at PDL as long as we continue to learn and grow from it