AI eval & model-quality engineer jobs

Model-quality roles — eval harnesses, LLM-as-judge, observability (LangSmith, Arize).

413 open roles

Open roles: 413
Posted this week: 57
Median salary: $238k
Remote: 16%
Top skill: Eval harnesses

Hiring most:LangChain17Anthropic12Fair Isaac (FICO)11ServiceNow11Mercor9Etched8

By level:Intern6Junior3Senior86Staff78Principal24Lead20Manager31

Common stack:Eval harnesses331Python258Orchestration168Go151Tool use142MCP116RAG110AWS102TypeScript95CI/CD82

All jobs RSS feed

Search

Member of Technical Staff, Data & Evals
Harvey · San Francisco
$230k–340k/yr
Posted todayHybridStaffEval harnesses Python
Member of Technical Staff, Data & Evals
Harvey · New York
$230k–340k/yr
Posted todayHybridStaffEval harnesses Python
Member of Technical Staff, Infrastructure
Harvey · San Francisco
$230k–340k/yr
Posted todayHybridStaffEval harnesses GPU PyTorch Python
Member of Technical Staff, Infrastructure
Harvey · New York
$230k–340k/yr
Posted todayHybridStaffEval harnesses GPU PyTorch Python
Member of Technical Staff, Modeling
Harvey · San Francisco
$230k–340k/yr
Posted todayHybridStaffEval harnesses Python
Member of Technical Staff, Modeling
Harvey · New York
$230k–340k/yr
Posted todayHybridStaffEval harnesses Python
Engineering Manager, Agent Builder
DataSnipper · New York
$275k–300k/yr
Posted todayOn-siteManagerEmbeddings Tool use Eval harnesses Azure
Engineering Manager, Evidence Request Management
DataSnipper · New York
$275k–300k/yr
Posted todayHybridManagerTool use Eval harnesses Azure Python
Sr Staff Software Engineer
ServiceNow · Tel Aviv, Israel
Posted todayOn-siteStaffTool use Orchestration Eval harnesses Python
Senior Engineering Manager - AI Foundations Team
Toast · Dublin, Ireland
Posted todayOn-siteManagerMCP Eval harnesses
Applied AI Engineer
Fair Isaac (FICO) · Work from Home, United States
$105k–165k/yr
Posted 1d agoRemoteRAG Tool use LLM-as-judge CI/CD
Senior Staff ML Engineer, AI Safeguards - Moveworks
ServiceNow · Mountain View, CALIFORNIA, United States
Posted 1d agoOn-siteStaffEval harnesses Python Go
Senior Staff Security Engineer, AI Safeguards - Moveworks
ServiceNow · Mountain View, CALIFORNIA, United States
Posted 1d agoOn-siteStaffEval harnesses Python Go
Senior Software Engineer (AI/Data)
Codat · Remote
£90k–110k/yr
Posted 1d agoRemoteSeniorMCP Eval harnesses Docker CI/CD
Senior Software Engineer (AI/Data)
Codat · London
£95k–120k/yr
Posted 1d agoHybridSeniorMCP Eval harnesses Docker CI/CD
Senior Backend Engineer - Developer Experience & Productivity
Taktile · Berlin Office
Posted 1d agoHybridSeniorOrchestration MCP Eval harnesses Docker
Sr AI Engineer, Agentic Systems
Rubrik · Bengaluru, India
Posted 1d agoOn-siteSeniorMulti-agent Orchestration MCP Eval harnesses
Director - Machine Learning Engineering
Capital One · Bangalore, In
Posted 2d agoOn-siteManagerEval harnesses PyTorch TensorFlow Kubernetes
Principal Machine Learning Engineer, Business Process Optimization
Workday · USA, CA, Pleasanton
Posted 2d agoOn-sitePrincipalLangChain LlamaIndex RAG Embeddings
Principal, Machine Learning Software Engineer, Admin Central
Workday · USA, CA, Pleasanton
Posted 2d agoOn-sitePrincipalLangChain LlamaIndex RAG Embeddings
Director - AI Engineering
Capital One · Bangalore, In
Posted 2d agoOn-siteManagerHugging Face Eval harnesses PyTorch AWS
AI/ML Engineer
GSK · USA - California - San Francisco
$136k–227k/yr
Posted 2d agoOn-siteEval harnesses PyTorch TensorFlow Deep learning
Quant / Agentic AI Developer - AVP
Barclays · Canary Wharf, 1 Churchill Place
Posted 2d agoOn-siteRAG Embeddings Tool use Multi-agent
AI Engineer
Workday · IRL.Dublin.College Square
Posted 2d agoOn-siteLangChain LangSmith AWS Azure
Senior Director - Machine Learning Engineering
Capital One · Bangalore, In
Posted 2d agoOn-siteManagerEval harnesses PyTorch TensorFlow Kubernetes
Sr ML Software Development Engineer, Admin Central
Workday · USA, CA, Pleasanton
Posted 2d agoOn-siteSeniorLangChain LlamaIndex DSPy RAG
Director of AI Research, AI for Oncology Clinical Development
AstraZeneca · US - Cambridge Kendall SQ - MA
Posted 2d agoOn-siteManagerMulti-agent Eval harnesses
Senior AI Impact & Applied Innovation Specialist
AstraZeneca · UK - Cambridge
Posted 2d agoOn-siteSeniorEmbeddings LLM-as-judge
Sr ML Engineer, Business Process Optimization
Workday · USA, CA, Pleasanton
Posted 2d agoOn-siteSeniorLangChain LlamaIndex DSPy RAG
Research Engineer, Intern – Benchmarking
Mercor · San Francisco
$70/hour
Posted 2d agoOn-siteInternTool use Eval harnesses
Staff AI Engineer, Agentic AI Harness & Quality - Moveworks
ServiceNow · Mountain View, CALIFORNIA, United States
Posted 2d agoOn-siteStaffOrchestration Eval harnesses Python Go
Mid-Level Python Developer (GenAI)
Sopra Steria · Sofia, Bulgaria
Posted 2d agoOn-siteLangChain LlamaIndex RAG Pinecone
Staff AI Engineer, Developer Productivity
Oura · Hybrid - London, England
Posted 2d agoHybridStaffTool use Orchestration MCP Eval harnesses
Staff AI Engineer, Developer Productivity
Oura · Hybrid - Helsinki, Uusimaa
Posted 2d agoHybridStaffTool use Orchestration MCP Eval harnesses
Senior / Staff Software Engineer — Product (Backend-Focused)
Snorkel AI · San Francisco, CA (Hybrid)
$208k–315k/yr
Posted 3d agoHybridStaffEval harnesses Python Go Rust
Senior Engineer- Agentic Development
Fair Isaac (FICO) · Work from Home, Bulgaria
Posted 3d agoRemoteSeniorOrchestration LLM-as-judge Kubernetes CI/CD
Principal Harness Engineer
Fair Isaac (FICO) · Work from Home, United Kingdom
Posted 3d agoRemotePrincipalOrchestration LLM-as-judge CI/CD
Principal Engineer- Agentic Development
Fair Isaac (FICO) · Work from Home, Bulgaria
Posted 3d agoRemotePrincipalOrchestration LLM-as-judge Kubernetes CI/CD
Senior, Forward Deployed / Full Stack Engineer (Document Authoring & Knowledge Systems)
GSK · UK – London – New Oxford Street
Posted 3d agoOn-siteSeniorRAG Embeddings Multi-agent Eval harnesses
Principal Engineer - Agentic Development
Fair Isaac (FICO) · Work from Home, Bulgaria
Posted 3d agoRemotePrincipalOrchestration LLM-as-judge Kubernetes CI/CD
Senior Harness Engineer
Fair Isaac (FICO) · Work from Home, Bulgaria
Posted 3d agoRemoteSeniorOrchestration LLM-as-judge CI/CD
Sr. Staff Applied AI Engineer, GTM
Zscaler · Remote - USA; Santa Clara, California, USA
$180k–225k/yr
Posted 3d agoRemoteStaffRAG Embeddings Orchestration MCP
Senior Software Engineer, Backend (AI Platform)
Affirm · Remote US
Posted 3d agoRemoteSeniorTool use MCP Eval harnesses Kubernetes
Staff Machine Learning Engineer
Twilio · Remote - Ontario, Canada (DNU)
CAD 160k–200k/yr
Posted 3d agoRemoteStaffOrchestration Eval harnesses Distillation AWS
Staff Software Engineer (AI Agents)
Traba · New York City, NY
$240k–300k/yr
Posted 3d agoOn-siteStaffTool use Orchestration MCP Eval harnesses
Senior Software Engineer (Applied AI)
Traba · New York City, NY
$200k–240k/yr
Posted 3d agoOn-siteSeniorOrchestration Eval harnesses Python TypeScript
Senior Software Engineer (AI Agents)
Traba · New York City, NY
$200k–240k/yr
Posted 3d agoOn-siteSeniorTool use Orchestration MCP Eval harnesses
Model Operations Lead, AI Gateway
Cloudflare · In-Office
$230k–317k/yr
Posted 3d agoOn-siteLeadTool use Eval harnesses Quantization Python
Staff/Principal Engineer, Search & Retrieval
Lila Sciences · San Francisco, CA USA
$224k–336k/yr
Posted 3d agoOn-sitePrincipalRAG Embeddings Reranking Eval harnesses
Fullstack Software Engineer (x/f/m) - Agentic Platform
Alan · Anywhere in France, Belgium, Spain
Posted 3d agoOn-siteMulti-agent MCP Eval harnesses AWS