HHiring Reality
← Scale AI

Staff Software Engineer, RL Environments

Scale AI
Location
San Francisco, CA; New York, NY
Posted
18 days ago
Department
Gen AI Engineering
What they actually want (must-haves)
  • 8+ years of software engineering experience with strong fundamentals in distributed systems, system design, data structures, and algorithms
  • Strong Python skills and a track record of shipping production software; comfort in at least one other part of the stack (TypeScript/React, Go, Rust, or similar)
  • Deep experience with containerization and sandboxed execution, including Docker, VMs, gVisor/Firecracker, Kubernetes, or equivalent
  • Experience building or operating high-throughput backend systems: orchestration, job scheduling, queuing, and large-scale data pipelines
  • Hands-on experience building with LLMs including agent loops, tool calling, MCP, or eval harnesses, and enough intuition about model behavior to reason about what a training signal actually teaches
  • Demonstrated ability to own ambiguous, undefined problems end to end and drive them to a shipped system
Nice to have
  • Direct experience building RL environments, agentic benchmarks, or eval harnesses (SWE-bench-style task suites, terminal or browser environments, tool-use benchmarks, or in-house equivalents)
  • Familiarity with post-training methods: RLHF, RLAIF, RLVR, GRPO/PPO-family algorithms, rejection sampling, reward modeling, and the practical failure modes of each
  • Experience designing verifiable reward signals, and firsthand experience with reward hacking and how to defend against it
  • Experience with RL training or serving stacks (verl, TRL, Ray, vLLM, SGLang, or similar)
  • Experience with high-scale sandbox or code-execution infrastructure, remote development environments, or CI systems
What the job really is

As a Staff Software Engineer focused on RL Environments at Scale AI, you will be responsible for creating and managing the technical foundation for reinforcement learning environments. This involves designing platforms for execution, packaging, and orchestration while also developing the environments themselves, ensuring they are robust and capable of handling complex tasks. The role requires both hands-on engineering and leadership across teams, emphasizing the delivery of high-quality, scalable systems.

Benefits
  • Comprehensive health, dental and vision coverage
  • Retirement benefits
  • Learning and development stipend
  • Generous PTO
  • Potential for commuter stipend
  • Equity-based compensation subject to Board of Director approval
Things to weigh
  • No specific information on team size or structure provided
  • Role involves both technical and leadership responsibilities, which may require balancing multiple priorities
  • High level of experience required may limit candidate pool
  • Salary range provided, but actual compensation may vary based on multiple factors
Job score3.4/5
Benefits3/5
Freshness4/5
Career value5/5
Role clarity5/5
Pay transparency0/5

Applying to Scale AI?

See how your résumé matches this role — and tailor it from what actually gets interviews.