HHiring Reality
← Scale AI

Machine Learning Research Scientist, Evaluations

Scale AI
Location
San Francisco, CA; Seattle, WA; New York, NY
Posted
25 days ago
Department
Research
What they actually want (must-haves)
  • Expertise in LLM post-training (SFT, RLHF, reward modeling) and evaluation
  • Ph.D. or Master's degree in Computer Science, Machine Learning, AI, or a related field
  • Deep understanding of deep learning, reinforcement learning, and large-scale model fine-tuning
  • Experience with LLM evaluation or benchmark development
  • Excellent written and verbal communication skills
Nice to have
  • Published research in areas of machine learning at major conferences (NeurIPS, ICML, ICLR, ACL, EMNLP, CVPR, etc.) and/or journals
  • Previous experience in a customer facing role
What the job really is

In this role as a Machine Learning Research Scientist focused on evaluations, you will analyze model behavior to identify and diagnose failure modes in large language models (LLMs) and agents. You will design benchmarks and evaluation methods for both text and multimodal modalities, apply post-training techniques, and publish your research findings in top-tier AI conferences. Collaboration with researchers and engineers will be key to defining best practices in evaluation-driven AI development.

Benefits
  • Comprehensive health, dental and vision coverage
  • Retirement benefits
  • Learning and development stipend
  • Generous PTO
  • Commuter stipend (may be eligible)
Things to weigh
  • Salary range provided is broad and may vary based on multiple factors
  • 90-day waiting period for reconsideration of candidates for the same role
  • Role involves significant collaboration and may require strong interpersonal skills
Job score3.2/5
Benefits3/5
Freshness4/5
Career value5/5
Role clarity4/5
Pay transparency0/5

Applying to Scale AI?

See how your résumé matches this role — and tailor it from what actually gets interviews.