Opportunity Software Engineering work from LinkedIn, X, Reddit and Upwork, in one list. Find me work
techroles

Member of Technical Staff - Training Platform

Prime Intellect primeintellect.ai Open · verified Oct 24, 2026
Remote San Francisco Full-time Senior Software Engineering

At a glance

Senior Software Engineering role at Prime Intellect. Remote, United States · full-time · $150,000–$300,000 base.

SalaryStated by Prime Intellect
$150,000 – $300,000
Base salary for this role, in USD per year, as published in the posting.
This role, from Prime IntellectOur estimate for the market: $195,575–$250,000Median $220,500
The range includes the market median of $220,500 for senior software engineering roles in San Francisco, compared against the San Francisco market.
Worked out from 214 postings that stated pay, median $220,500. Last updated Oct 24, 2026.
RoleSoftware Engineering
SenioritySenior
LocationSan Francisco
WorkplaceRemote
EmploymentFull-time
Base pay$150,000–$300,000
PostedJul 8, 2026 · 3mo ago
Role brief

techroles summary, based on the employer's posting.

What you'll do

  • Run Kubernetes training and inference orchestration across GPU clusters and cloud providers
  • Package trainers, inference servers and supporting components into repeatable training stacks with Helm
  • Build Python agents that track cluster jobs and report their status to the platform
  • Create hosted-training features for submitting jobs, following runs, reviewing logs and managing models
  • Connect platform services to cluster workloads and turn new training methods into usable product features

What you bring

  • You bring hands-on Kubernetes operations, including Helm, cluster debugging and GPU workload scheduling
  • You have strong Python backend skills and can build services and agents that use Kubernetes APIs
  • You understand fine-tuning, inference systems, GPU hardware and distributed training fundamentals
  • You can also ship developer-facing interfaces with TypeScript and React or Next.js

Who this fits

This senior role suits an engineer with depth across AI infrastructure, Kubernetes and platform development, with experience in either platform or infrastructure work and ideally both. You can work remotely or from the San Francisco office. Visa sponsorship and relocation support are available.

From the employer

Own Your Intelligence

Prime Intellect is building the open superintelligence stack: the infrastructure frontier AI labs build internally, made available to every ambitious AI team.

Our platform, Lab, unifies compute, environments, evaluations, secure sandboxes, high-performance training, and deployment into one full-stack system for post-training at frontier scale - from SFT and RL to tool use, agent workflows, and continuously improving production models. We are building open frontier AI: open-source models trained end to end for long-horizon tasks like autonomous research, and the full-stack platform our own research team uses to build them. The next generation of AI companies, enterprises, and research teams do not just need more GPUs. They need the ability to turn their own workflows, tools, data, and feedback loops into superintelligence they own.

Prime Intellect has raised $150M in total funding from Founders Fund, Radical Ventures, NVIDIA, and exceptional AI, infrastructure, and enterprise operators — including Andrej Karpathy, Dwarkesh Patel, and leaders and founders from Ramp, Perplexity, Harvey, Mercor, Zapier, Datadog, Cognition, OpenAI, Thinking Machines, Together AI, SemiAnalysis, LangChain, Browserbase, Cloudflare, Sierra, Databricks, Airbnb, OpenRouter, Standard Intelligence, Fleet, Core Auto, and more. We are looking for people who want to build at the intersection of frontier research, real infrastructure, and go-to-market for a category that does not fully exist yet.

Role Impact

You'll help build our hosted training platform - the product that lets users launch LoRA and full fine-tuning runs on managed GPU clusters with a single API call or a few clicks. The role spans the developer-facing platform and the underlying Kubernetes-based training infrastructure that runs the jobs.

Core Technical Responsibilities

Hosted Training Infrastructure

  • Design and operate Kubernetes-based training and inference orchestration across multi-cluster, multi-cloud GPU fleets

  • Build and maintain Helm charts that compose trainers, inference servers, environment servers, and supporting services into reproducible "Training stacks"

  • Develop the Python control-plane agents that watch pods, report run state to the platform, and keep clusters in sync

  • Implement scheduling and autoscaling for heterogeneous hardware (H100/H200/B200) using KEDA, LeaderWorkerSet, taints/tolerations, and gang scheduling

  • Run a tight GitOps workflow - every change ships through PRs, Helm values, and CI

  • Build node-local model caches, checkpoint pipelines, and shared storage for fast cold starts

  • Operate the observability stack (Prometheus, Grafana, Loki, DCGM) and make GPU cluster debugging fast

Platform Development

  • Build the developer-facing surfaces for hosted training: job submission, live run monitoring, logs, metrics, model/adapter management, comparisons

  • Develop FastAPI backend services and REST APIs that bridge the platform to running clusters

  • Build real-time monitoring and debugging tools (streaming logs, step-level metrics, failure analysis)

  • Ship product UI in Next.js / React / TypeScript with shadcn, Tailwind, tRPC, and TanStack Query

Research Bridge

  • Interface with the RL trainer, inference servers, and environment servers running inside our clusters

  • Productize new training capabilities (new model architectures, RL algorithms, modes)

Technical Requirements

We're looking for engineers who are fluent across three areas - you don't need to be the world's best at any one, but you should have real depth in all three and a clear point of view on how they connect.

AI & GPU Landscape

  • Strong working knowledge of the modern AI stack - open model families, finetuning techniques (LoRA, QLoRA, full FT, RLHF/RLAIF), inference engines (vLLM, SGLang, TensorRT-LLM)

  • Familiarity with GPU hardware tradeoffs (H100 / H200 / B200, NVLink, interconnects, memory hierarchy) and what they mean for training and inference workloads

  • Understanding of distributed training fundamentals (data/tensor/pipeline/expert parallelism, NCCL, multi-node scheduling)

  • Awareness of what's happening at the frontier - new models, training methods, infra patterns - and the ability to translate that into product decisions

Kubernetes & Infrastructure

  • Strong Kubernetes operations experience - Helm, CRDs, operators, KEDA, gang scheduling, GPU operator

  • Comfortable debugging real production clusters (kubectl, pod lifecycle, node issues, networking)

  • Cloud platform experience (GCP preferred - GCS, GKE, Cloud Run, Cloud Tasks)

  • Infrastructure automation (Helm, Terraform, Ansible) and a GitOps mindset

  • Observability: Prometheus, Grafana, Loki, OpenTelemetry, DCGM

  • Linux fundamentals: networking, namespaces, performance tuning

Programming & Platform

  • Strong Python backend development (FastAPI, async, SQLAlchemy)

  • Comfortable building Python control-plane agents that talk to Kubernetes APIs

  • Modern frontend development (TypeScript, React/Next.js, Tailwind, shadcn) - enough to ship product surfaces end-to-end

  • REST and tRPC API design

  • Experience building developer tools, dashboards, and live-monitoring UIs

What We Offer

  • Cash compensation $150K–$300K with significant equity

  • Flexible work arrangement (remote or San Francisco office)

  • Full visa sponsorship and relocation support

  • Professional development budget for courses and conferences

  • Regular team off-sites and conference attendance

  • Opportunity to shape the future of decentralized AI development

Growth Opportunity

You'll join a team of experienced engineers and researchers working on cutting-edge problems in AI infrastructure. We believe in open development and encourage team members to contribute to the broader AI community through research and open-source work.

We value potential over perfection - if you're passionate about democratizing AI development and have experience in either platform or infrastructure development (ideally both), we want to talk to you.

Ready to help shape the future of AI? Apply now and join us in our mission to make powerful AI models accessible to everyone.

Opportunity

Software Engineering work from LinkedIn, X, Reddit and Upwork, in one list.

Each morning you get everything that fits, strongest first.

Software EngineeringRemote$150k+

Find me work

By the team behind techroles.

Find me work
Opportunity

Software Engineering work from LinkedIn, X, Reddit and Upwork, in one list.

Each morning you get everything that fits, strongest first.

Find me work

By the team behind techroles.