Overview
Inferact's mission is to grow vLLM as the world's AI inference engine and accelerate AI progress by making inference cheaper and faster. Founded by the creators and core maintainers of vLLM, we sit at the intersection of models and hardware, a position that took years to build.
About the Role
We're looking for a Site Reliability Engineer to help make vLLM-powered inference systems reliable, observable, and operationally simple at production scale. This role is for someone who thinks about failure before launch, designs systems that are easier to operate, and knows how to turn incidents into durable improvements rather than one-off fixes.
You'll work across engineering and infrastructure to define SLOs, improve monitoring and alerting, strengthen incident response, drive post-mortems, and reduce operational risk before it reaches users. Your work will directly impact the reliability, availability, and production readiness of the systems powering AI inference at scale.
Skills and Qualifications
Minimum qualifications:
- Bachelor's degree or equivalent experience in computer science, engineering, systems, infrastructure, or similar.
- Strong experience operating production systems with meaningful traffic, user impact, or infrastructure criticality.
- Deep understanding of SLOs, SLIs, error budgets, alerting, incident response, and post-mortem processes.
- Experience live-fighting major production incidents, including mitigation, root cause analysis, escalation, and follow-through on prevention work.
- Strong Linux, networking, systems debugging, observability, and distributed systems fundamentals.
- Ability to design operationally simple systems and identify likely failure modes before launch.
- Strong programming or scripting ability in Python, Go, Bash, or similar for automation, tooling, and reliability improvements.
Preferred qualifications:
- Experience supporting ML infrastructure, inference systems, GPU workloads, Kubernetes-based platforms, or high-scale backend services.
- Experience building or improving observability systems using metrics, logs, traces, dashboards, alerts, and runbooks.
- Experience with Kubernetes, Docker, Terraform, cloud infrastructure, service meshes, CI/CD systems, or production deployment platforms.
- Experience driving incident review culture, post-mortem processes, reliability reviews, and prevention-oriented engineering work.
- Ability to partner with engineering teams to improve service design, release safety, capacity planning, and operational readiness.
Bonus points if you have:
- Owned reliability for high-throughput, latency-sensitive, or mission-critical production systems.
- Supported AI inference, model serving, GPU clusters, ML platforms, or distributed serving infrastructure.
- Built automation that reduced toil, improved recovery time, or prevented repeat incidents.
- Led incident response for severe outages with clear communication across engineering and leadership.
- Created practical SLOs, dashboards, alerts, runbooks, or release gates that improved production reliability.
Logistics
- Location: This role is based in San Francisco, California. Will consider remote in the US for exceptional candidates.
- Compensation: Depending on background, skills, and experience, the expected annual salary range for this position is $200,000 - $400,000 USD + equity.
- Visa sponsorship: We sponsor visas on a case-by-case basis.
- Benefits: We offers generous health, dental, and vision benefits as well as 401(k) company match.