Manager, Supercomputer Site Reliability Engineering - Memphis

SpaceX was founded under the belief that a future where humanity is out exploring the stars is fundamentally more exciting than one where we are not. Today, SpaceX is actively developing the technologies to make this possible, with the ultimate goal of making life multiplanetary and expanding consciousness to the stars.

SpaceXSI is building Super Intelligence to help humanity understand the universe. Our teams are building the full Super Intelligence stack: from our Colossus supercomputers, to Grok models, to our suite of applications, including Grok Bot, Grok Build, Grok Imagine, Grok Voice, and X. We're creating a powerful flywheel where more compute trains better models, better models power more useful products, and what we learn from people using those products shapes what we build next.

MANAGER, SUPERCOMPUTER SITE RELIABILITY ENGINEERING

SpaceXSI operates multi-campus, hyperscale GPU infrastructure where cluster availability directly determines training throughput, customer handover commitments, and capital efficiency of the fleet. The Manager of Supercomputer Site Reliability Engineering will be responsible for both compute and infrastructure performance, and will be responsible for significantly increasing the SRE team size. On the compute side, focus will be put on continuous site monitoring, cross-discipline incident command, deep hardware root-cause analysis (RCA), and emphasis on software systems that orchestrate repair velocity. Additional focus will be on physical infrastructure reliability as well. Understanding system margins and resiliency will be key.

Site Reliability engineering manager owns >99% compute uptime outcome, drives unplanned facilities compute losses to 0, ensures efficient allocation of > 3GW site power, upgrade robust fault-free megapack failover systems capable of supporting all halls for >4 hours (160 MWhr per hall), drives robust hardware inventory systems and repair velocity to eliminate hardware chokepoints.

RESPONSIBILITIES:

Serve as the senior operational authority for the SRE organization and represent production reliability concerns in engineering and leadership discussions

Lead integrated Supercompute systems engineering reliability team responsible for reactive and increasingly proactive cluster and facilities health and reliance initiatives

Build out the site reliability engineering sub-teams: compute hardware failure analysis, software tools, cluster monitoring, and integrated site reliability engineering

Drive operational superiority

Define SRE organizational structure and develop culture of 24/7 operational excellence

Physics limit anomaly response times and robust proactive health processes

Define on-call responsibilities, incident management protocols, leadership structure, and communication discipline

Develop training schemes for tough, competent, and disciplined monitoring and response teams

Establish clear operational interfaces with Datacenter Operations, engineering teams, vendors, and other service owners

Own vendor oversight and accountability mechanisms

Drive mean time to repair toward physics limits

Define and continuously improve operational procedures and analysis documentation mechanisms with strong emphasis on intelligent automation

Lead robustification of documentation workflow

Ensure efficient signal to noise in execution

Define operational targets, health indicators, alerting standards, and incident severity definitions that facilitate measurable team performance insights and accountability mechanisms

Drive high technical standard for hardware root cause analysis and remediation teams ensuring industry-leading hardware failure diagnostic expertise

Integrate with platform engineering teams to develop sophisticated hardware and system understanding to drive high operational reliability

Drive repeat hardware issues to a minimum and lead automation efforts to eliminate manual overhead for unavoidable remediation needs

BASIC QUALIFICATIONS:

Bachelor’s degree and 3+ years of experience working in a large scale operations with 2+ years leading people leaders of technical teams; OR 10+ years of experience working in a large scale operations with 5+ years leading people leaders of technical teams

PREFERRED SKILLS AND EXPERIENCE:

Proven ability to lead large, multi-site, 24/7 operations organizations in fast-paced, high-responsibility settings

Deep expertise in server hardware, cluster reliability, and data center technologies, from deployment through lifecycle management

Experience supporting compute-heavy environments like AI, machine learning, or high-performance computing at scale

A track record of owning uptime, Service Level Agreements, or reliability metrics for large compute clusters

Experience leading site reliability engineering or equivalent reliability-focused teams, including root cause analysis and procedure ownership

Strong analytical skills and the ability to explain technical concepts clearly to diverse audiences, from technicians to executive and tenant partners

A history of partnering with vendors at scale, driving mean time to repair down, and scaling operations across multiple sites

Familiarity with tooling and automation (e.g., Jira, Python, Bash) used to monitor cluster health and improve team efficiency

Enthusiasm for SpaceXAI's mission to accelerate human discovery and unravel the universe

Ability to thrive in a dynamic, mission-focused environment with on-call ownership of cluster-impacting events

ADDITIONAL REQUIREMENTS:

Willingness to travel frequently to data center locations to support operations across sites

Physical capability to handle data center tasks, including lifting up to 50 lbs unassisted, standing for long periods, and occasional ladder use

Must be willing to work extended hours and/or weekends as needed

SpaceXSI is an equal opportunity employer. For details on data processing, view our Recruitment Privacy Notice .