AI eval & model-quality engineer jobs

Model-quality roles — eval harnesses, LLM-as-judge, observability (LangSmith, Arize).

432 open roles

Open roles: 432
Median salary: $238k
Remote: 13%
Top skill: Eval harnesses

Hiring most:LangChain19Anthropic14OpenAI14ServiceNow12Mercor9Cisco8

Common stack:Eval harnesses340Python261Orchestration175Go157Tool use135AWS114RAG107MCP100TypeScript100Kubernetes92

← All jobs RSS feed

Search

Senior Fullstack Engineer, AI Observability & Evals Platform
LangChain · Boston, MA
$175k–240k/yr
Posted 1d agoOn-siteSeniorLangChain LangSmith Python TypeScript
Senior Backend Software Engineer, AI Observability & Evals Platform (LangSmith)
LangChain · Boston, MA
$180k–240k/yr
Posted 1d agoOn-siteSeniorLangChain Eval harnesses LangSmith AWS
Senior Software Engineer I
CoreWeave · Bellevue, WA / Sunnyvale, CA / San Francisco, CA / Livingston, NJ / Manhattan, NY / Philadelphia, PA / Dallas, TX
$153k–204k/yr
Posted 2d agoOn-siteSeniorEval harnesses Kubernetes Docker CI/CD
Staff Engineer, Applied AI (Customer Agent)
Klaviyo · Boston, MA
$176k–264k/yr
Posted 2d agoOn-siteStaffOrchestration Eval harnesses Go
Senior / Staff Software Engineer, AI Tech Strike Team
Waabi · Toronto, ON
Posted 2d agoHybridStaffTool use Eval harnesses CI/CD Python
Forward Deployed Solution Engineer – Applied AI FDE
ServiceNow · Montreal, QUEBEC, Canada
Posted 2d agoHybridLangChain LlamaIndex vLLM Hugging Face
Senior Product Engineer (Agent Training and Evals)
Prolific · Remote, UK
Posted 2d agoRemoteSeniorOrchestration Eval harnesses Kubernetes Python
Senior AI Engineer
Grab · Bangalore, India
Posted 2d agoOn-siteSeniorLangChain Orchestration LangSmith LoRA
Software Engineering Manager, AI Observability & Evals Platform
LangChain · Boston, MA
$200k–240k/yr
Posted 2d agoOn-siteManagerLangChain LangSmith Python TypeScript
Machine Learning Engineering Technical Leader
Cisco · Seattle, Washington, US
$234k–297k/yr
Posted 3d agoOn-siteTool use Eval harnesses Quantization LoRA
Applied AI Engineer
Arena · Bay Area
$150k–250k/yr
Posted 3d agoHybridEval harnesses CI/CD TypeScript Go
Senior ML Researcher
Toloka · European Union
Posted 3d agoOn-siteSeniorDSPy vLLM Tool use Eval harnesses
Senior/Staff Software Engineer, Agent
OpenArt · San Carlos, California, USA
$500k–600k/yr
Posted 3d agoHybridStaffTool use Orchestration MCP Eval harnesses
Senior Applied AI Engineer
Mintlify · San Francisco
$190k–265k/yr
Posted 3d agoOn-siteSeniorTool use Orchestration Eval harnesses
Forward Deployed Engineer, EMEA
Brainbase Labs · Remote (EMEA)
Posted 3d agoRemoteEmbeddings Eval harnesses
Principal Forward-Deployed Engineer
Thoughtworks · Riyadh, KSA
Posted 3d agoOn-sitePrincipalLangChain LlamaIndex RAG Pinecone
ML & Agentic Systems Engineer [IC4]
Sourcegraph · Remote
Posted 3d agoRemoteEmbeddings MCP Eval harnesses Distillation
Principal Machine Learning Engineer
Workday · Ireland, Dublin
Posted 4d agoOn-sitePrincipalHugging Face Orchestration LangSmith GPU
Senior Software Development Engineer
Workday · Ireland, Dublin
Posted 4d agoOn-siteSeniorHugging Face Orchestration LangSmith GPU
Staff/Sr. Staff Software Engineer, Product Engineering
Harvey · New York
$231k–340k/yr
Posted 4d agoHybridStaffTool use Eval harnesses Python TypeScript
Staff/Sr. Staff Software Engineer, Product Engineering
Harvey · San Francisco
$231k–340k/yr
Posted 4d agoHybridStaffTool use Eval harnesses Python TypeScript
OB - Lead AI Engineer
Empirical Health · LATAM
Posted 4d agoRemoteLeadEval harnesses CI/CD
Engineering Manager, AI
PermitFlow · New York City, NY
$180k–250k/yr
Posted 4d agoHybridManagerOrchestration LLM-as-judge
Applied AI Architect, Beneficial Deployments (Life Sciences)
Anthropic · London, UK
£165k–190k/yr
Posted 4d agoOn-siteMCP Eval harnesses Go
Senior Software Engineer (Applied AI)
Traba · New York City, NY
$200k–240k/yr
Posted 5d agoOn-siteSeniorOrchestration Eval harnesses Python TypeScript
Senior Software Engineer (AI Agents)
Traba · New York City, NY
$200k–240k/yr
Posted 5d agoOn-siteSeniorTool use Orchestration MCP Eval harnesses
Machine Learning Engineer - Summer Intern 2027
S&P Global · Cambridge, MA
Posted 5d agoOn-siteInternvLLM Hugging Face RAG pgvector
VP, Site Reliability Engineering (SRE & Observability Platform)
FIS · US FL JAX 347
Posted 5d agoOn-siteManagerOrchestration Eval harnesses Kubernetes
Software Engineer, Cloud Infrastructure (Multiple Seniority Levels)
Beacon AI · San Carlos - Hybrid
$135k–260k/yr
Posted 5d agoHybridLangChain RAG Pinecone pgvector
Research Engineer, Takeoff Intel
Anthropic · Remote-Friendly (Travel Required) | San Francisco, CA
$350k–850k/yr
Posted 5d agoRemoteEval harnesses Go
Lead GenAI Engineer
Fractal Analytics · California
Posted 6d agoOn-siteLeadLangChain LlamaIndex RAG Tool use
Software Engineering MTS
Salesforce · Washington - Bellevue
$117k–177k/yr
Posted 6d agoOn-siteRAG Embeddings MCP Eval harnesses
Lead AI & Data Engineer – APD Platform
AstraZeneca · Spain - Barcelona
Posted 6d agoOn-siteLeadLangChain RAG LangSmith Kubernetes
Member Technical Staff - Applied AI Engineer (US Timing)
Composio · Bangalore
Posted 6d agoOn-siteStaffTool use Eval harnesses Python TypeScript
Software Engineer
ServiceNow · Hyderabad, India
Posted 6d agoOn-siteLangChain LlamaIndex RAG Pinecone
Senior Backend Engineer (m/f/d) - Agent SDK
voize · Berlin
Posted 6d agoRemoteSeniorOrchestration Eval harnesses
Senior Applied AI Engineer
Culture Amp · Sydney
Posted 6d agoOn-siteSeniorRAG pgvector Orchestration LLM-as-judge
Agent Reliability Engineer, GTM
LangChain · San Francisco, CA
$150k–190k/yr
Posted 7d agoOn-siteLangChain Eval harnesses LangSmith Python
Software Engineer: Internal Automation
Rogo · New York City
$200k–275k/yr
Posted 8d agoOn-siteTool use Eval harnesses GCP Python
Software Engineer, AI Automation
Coalition · Any location, Canada
Posted 9d agoOn-siteTool use MCP Eval harnesses Python
Software Engineer, AI Automation
Coalition · Any location, United States
Posted 9d agoOn-siteTool use MCP Eval harnesses Python
Lead Applied AI Engineer
LangChain · New York, NY
$160k–200k/yr
Posted 9d agoOn-siteLeadLangChain Embeddings Orchestration Eval harnesses
Senior AI/ML Engineer
Metriport · San Francisco
$200k–260k/yr
Posted 9d agoHybridSeniorEval harnesses Deep learning SageMaker AWS
Applied AI, Research Engineer
Anthropic · San Francisco, CA | New York City, NY
$300k–400k/yr
Posted 9d agoOn-siteEval harnesses Go
Research Engineer, Forge
Mistral AI · Paris
Posted 9d agoHybridOrchestration Eval harnesses Distillation PyTorch
Senior Software Engineer, Expense
Navan · London, UK
Posted 9d agoOn-siteSeniorRAG Tool use Multi-agent Orchestration
Senior Software Engineer, Expense
Navan · Berlin, Germany
Posted 9d agoOn-siteSeniorRAG Tool use Multi-agent Orchestration
Forward Deployed Engineer (Audit experience required)
DataSnipper · Amsterdam
Posted 9d agoHybridEval harnesses
Staff+ Software Engineer, Claude Science
Anthropic · San Francisco, CA
$405k–485k/yr
Posted 9d agoOn-siteStaffMulti-agent Eval harnesses
AI Engineer
9fin · London
Posted 10d agoOn-siteEmbeddings Eval harnesses Computer vision AWS