Software Developer (Content Integration & Data Pipelines)

Accountabilities •

Build scalable web scrapers and crawlers: Develop and maintain production-ready scraping and crawling solutions that collect online content responsibly, following standards such as robots.txt, rate limiting, and applicable terms of use.

•

Develop content integration services: Implement parsing, extraction, and normalization logic using TypeScript and JavaScript to transform content from diverse sources into consistent, usable data.

•

Design event-driven pipelines: Build and maintain Kafka-based services that process, enrich, and route content in near real time, supporting reliable and scalable data flows.

•

Integrate external content sources: Connect third-party content providers, RSS and Atom feeds, and external APIs to expand and maintain content coverage.

•

Develop backend services: Design and implement Java-based RESTful services that support content processing, integration, and delivery across the platform.

•

Improve data quality and reliability: Establish and apply best practices for error handling, source management, validation, data normalization, and fault-tolerant pipeline operations.

•

Monitor and optimize pipeline performance: Implement observability and monitoring practices, investigate processing issues, and improve system performance, reliability, and operating costs.

•

Contribute to technical architecture: Participate in system design discussions, evaluate scalability trade-offs, and help define the technical direction of content acquisition and data processing systems.

•

Support search and indexing capabilities: Contribute to content indexing and retrieval workflows, collaborating with search technologies such as Solr to ensure content is structured and accessible.

•

Collaborate across the development team: Work with technical colleagues and other stakeholders to deliver maintainable solutions aligned with product and business needs.

•

Mentor and share knowledge: Support junior developers, contribute to code reviews and engineering best practices, and promote continuous learning within the team.

Requirements:

•

Education and experience: A degree in Computer Science or a related field, or equivalent practical experience, with at least three years of professional software development experience.

•

TypeScript and JavaScript proficiency: Strong programming skills in TypeScript or JavaScript, particularly for content integration, parsing, and data processing.

•

Backend development experience: Solid experience building backend services with Java or a comparable programming language.

•

Production web scraping expertise: Hands-on experience developing and operating web scrapers and crawlers in production environments, with an understanding of robots.txt, rate limiting, responsible crawling practices, and applicable terms of use.

•

Event-driven architecture: Practical experience with event-driven systems and message-streaming technologies, particularly Kafka.

•

Content processing skills: Knowledge of HTML parsing, content extraction, data transformation, and normalization techniques.

•

Data pipeline engineering: Experience designing or maintaining high-volume, fault-tolerant data pipelines that process and deliver information reliably.

•

Database and data modeling knowledge: Familiarity with SQL and data modeling principles.

•

Development tools and environments: Experience working in Linux environments and using Git and modern software development workflows. Familiarity with Maven is also expected.

•

Search technology knowledge: Experience with Solr, including indexing, schema design, and query tuning, is an asset.

•

Relevant industry experience: Experience building content ingestion pipelines for news, media, or similar data-intensive applications is strongly preferred.

•

System design capabilities: Understanding of distributed systems, scalability trade-offs, reliability, and performance optimization.

•

Ownership and initiative: Ability to work independently, take responsibility for projects, and proactively identify opportunities for technical improvement.

•

Analytical problem-solving: Strong troubleshooting skills and the ability to investigate complex technical issues and develop practical solutions.

•

Communication and teamwork: Ability to communicate effectively with technical and non-technical stakeholders, collaborate across teams, and share knowledge.

•

Interest in innovation: Enthusiasm for data engineering, automation, emerging technologies, and AI-powered capabilities.

Benefits:

•

Competitive compensation: Expected annual salary range of CAD $75,000–$110,000, depending on experience, skills, and internal equity.

•

Remote work: Fully remote working environment.

•

Health coverage: Health, dental, and vision benefits.

•

Retirement savings: RRSP matching.

•

Financial protection: Life insurance.

•

Employee support: Access to an Employee Assistance Program (EAP).

•

Paid time off: Paid vacation, personal days, and sick days.

•

Holiday break: A week off between Christmas and New Year's.

•

Meeting-free Fridays: No internal meetings scheduled on Fridays.

•

Career development: Opportunities for professional growth and career progression.

•

Collaborative culture: A supportive working environment with tools and practices that encourage teamwork, knowledge sharing, and continuous improvement.

•

Inclusive workplace: A commitment to an inclusive environment where people with diverse backgrounds and perspectives are encouraged to apply. Reasonable accommodations can be arranged for candidates who require support during the recruitment process. How Jobgether works: We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team. We appreciate your interest and wish you the best!  Why Apply Through Jobgether?    Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.     #LI-CL1