Senior Staff Machine Learning Engineer (DevOps)

Join ServiceNow's AI Services Platform Group as a Senior Staff Machine Learning Engineer (DevOps). In this role, you will lead the architecture, optimization, and operational excellence of our AI inference platform. You will own the end-to-end inference lifecycle, deploy and operate services in production Kubernetes environments, and ensure observability for what you build. This position requires deep technical expertise in GPU compute, LLM APIs, inference frameworks, containerization, orchestration, and API gateway architecture.

- Lead the architecture, optimization, and operational excellence of the AI inference platform, owning the end-to-end inference lifecycle. - Architect the intelligent gateway layer using Envoy and ext_proc to enforce access control, provide observability, and enable advanced request routing. - Deploy, scale, and operate services in production Kubernetes environments, including Helm-based deployments and CI/CD pipelines.

- Expertise in coding efficient, object-oriented, modularized and quality software - Experience building new products that use challenging algorithms - Knowledge of core AI/ML techniques and algorithms - Expertise in Go or Python, OOP, Design Patterns, time and space-efficient algorithms - Knowledge of unit testing, profiling, and code tuning - AI-assisted debugging and trace analysis, and a clear understanding of the data privacy rules governing what goes into a prompt - Confident use of AI coding assistants such as GitHub Copilot or Cursor, paired with the critical eye to review AI-generated code as rigorously as any other — catching logic errors, security anti-patterns, and missed edge cases rather than treating model output as production-ready - Artificial intelligence strategy is a critical skill at Experienced proficiency for IC5 — it applies to how you build and to what you build - 3+ years of Kubernetes in production, including Helm and CI/CD - 8+ years of Backend software engineering experience - 4+ years of designing and operating distributed systems - A master’s degree may offset up to one year of the experience minimum. Equivalent practical experience is considered where technical depth is clearly demonstrable - Bachelor’s degree in computer science, software engineering, or a closely related technical field required