About this role
We are seeking a highly skilled LLM Pre-training & Distributed Systems Engineer. This role is essential for orchestrating large-scale machine learning training runs and optimizing distributed infrastructure. The ideal candidate will have a deep understanding of GPU clusters and extensive experience in system engineering to ensure efficient and reliable training processes.
Responsibilities:
• Orchestrate distributed training runs across 1,000+ GPUs using PyTorch, DeepSpeed, or Megatron-LM.
• Optimize networking (InfiniBand/RDMA) and memory management to prevent out-of-memory errors.
• Automate checkpointing and failure recovery during month-long training runs.
Required Skills:
• Deep expertise in 3D parallelism (Data, Tensor, Pipeline).
• Experience managing SLURM or Kubernetes-based GPU clusters.
• Strong systems engineering background (C++, CUDA, Python).
Responsibilities:
• Orchestrate distributed training runs across 1,000+ GPUs using PyTorch, DeepSpeed, or Megatron-LM.
• Optimize networking (InfiniBand/RDMA) and memory management to prevent out-of-memory errors.
• Automate checkpointing and failure recovery during month-long training runs.
Required Skills:
• Deep expertise in 3D parallelism (Data, Tensor, Pipeline).
• Experience managing SLURM or Kubernetes-based GPU clusters.
• Strong systems engineering background (C++, CUDA, Python).
Tech stack
LLMPyTorchKubernetesPythonC++
About Hyphen Connect Limited
Hyphen Connect Limited is hiring for the llm pre-training & distributed engineer role. NewJob aggregates active openings directly from Hyphen Connect Limited's applicant tracking system, so this listing is current.
More jobs at Hyphen Connect Limited →