Periodic Labs

RL Systems Engineer

Periodic Labs · Menlo Park, CA
Menlo Park, CA was $250K–$350K Closed
Applications are closed for this role. It was originally posted 2026-04-29. It’s no longer accepting applicants — see roles Periodic Labs is still hiring for →, or browse the live openings below.
Salary
$250K–$350K
Type
Full-time
Experience
7+ yr

ABOUT PERIODIC LABS

We're an AI and physical sciences company building state-of-the-art models to accelerate breakthroughs across materials, energy, and beyond. Backed by world-class investors and growing rapidly, we operate at the pace the frontier requires. Our team brings deep expertise, genuine ownership, and an insatiable drive to push the boundaries of what's scientifically possible.

ABOUT THE ROLE

You will own the systems layer that makes our frontier model training and inference fast, efficient, and tightly coupled to the RL feedback loop that drives scientific discovery.

This is not a pure infrastructure role and it is not a pure research role — it sits exactly at their intersection. You will go deep into the stack: scheduling, kernels, RDMA, weight synchronization, and communication primitives, while working shoulder-to-shoulder with researchers to co-design the algorithms and infrastructure together.

The RL loop is central to how Periodic Labs works. Models propose experiments, experiments generate data, data feeds back into training. The speed and reliability of that loop is a direct multiplier on the pace of scientific discovery. You will own the infrastructure that makes it fast.

WHAT YOU'LL DO

  • Build rack and topology-aware scheduling for GB series GPUs across Ray, Slurm, and Kubernetes, minimizing latency and maximizing utilization across heterogeneous cluster configurations
  • Build online and offline profilers that surface bottlenecks across the training and inference stack and translate findings into actionable optimizations
  • Implement direct S3 checkpoint streaming to eliminate I/O bottlenecks in large-scale training runs
  • Run methodical benchmarking to identify optimal RL training configurations across model sizes, batch strategies, and hardware topologies
  • Write and optimize communication and GPU kernels to extract maximum throughput from the hardware
  • Design and implement zero-copy RDMA weight synchronization between training and inference to keep the RL loop tight and low-latency
  • Build fast sandbox execution environments that allow rapid rollout of model-generated actions and return of rewards without blocking the training pipeline
  • Engage directly with the SGLang, Megatron, and Ray communities — contributing upstream, influencing roadmaps, and pulling in improvements that benefit Periodic Labs’ workloads
  • Work in close collaboration with RL and pretraining researchers to co-design algorithms and infrastructure together — you will shape what is possible at the research level by knowing what is achievable at the systems level, and vice versa

The net result: high-throughput, fault-tolerant training and inference systems tightly coupled with a low-latency RL feedback loop that accelerates scientific discovery at every turn.

YOU MIGHT THRIVE IN THIS ROLE IF YOU HAVE EXPERIENCE WITH

  • Large-scale inference infrastructure: load balancing, traffic shifting, scheduling, and serving architecture at production scale
  • Low-level systems programming: RDMA, NVLink, kernel-level work, and network stack optimization
  • GPU cluster scheduling and orchestration across Ray, Slurm, or Kubernetes, with awareness of rack topology and hardware locality
  • Writing and optimizing CUDA kernels, communication primitives, or distributed training collective operations
  • Profiling and benchmarking distributed ML systems to identify and eliminate bottlenecks across compute, memory, and network
  • Checkpoint management and streaming at scale, including direct cloud storage integration
  • Building or contributing to open source ML infrastructure projects (e.g., SGLang, Megatron-LM, vLLM, Ray)
  • Working directly with ML researchers on algorithm-infrastructure co-design — you understand the research well enough to make systems decisions that serve it

MECHANICS

Minimum education: Bachelor’s degree or similar experience

Location: Menlo Park, CA (Soon: San Francisco, too)

Compensation: $250,000-$350,000 base + equity

Visa sponsorship: Yes, we sponsor visas and will do everything we can to assist in this process with our legal support.

We’re building a team of the world’s best — the scientists, engineers, and problem-solvers who don’t just follow the frontier, they define it. If you’re driven to bring AI to life in the physical world and make discoveries that have never been made before, you belong here.

Kubernetes
E
Research Engineer - Midtraining
Menlo Park, CA
Engineering
$250K–$350K
D
Research Scientist, Data
Menlo Park, CA
Data & ML
$250K–$350K
D
Research Scientist, Thin Films
Menlo Park, CA
Data & ML
$225K–$325K
See all live roles at Periodic Labs →
A
Senior Software Engineer (Full Stack), Intelligence Systems
Anduril Industries Reston, VA
Engineering
$191K–$253K
A
Senior AI Systems Engineer
Archer Aviation San Jose, CA
Engineering
$160K–$198K
C
Systems Engineer 2 - Confluence/Jira/Kibana/Grafana
Captivation Software Annapolis Junction, MD
Engineering
$130K–$270K
C
Senior Distributed Systems Engineer
CENSYS Remote (US) Remote
Engineering
$174K–$206K
See all Engineering roles →
Applications closed