RLHF & Model Evaluation Specialist
Rank, grade and red-team model outputs against clear rubrics to improve model quality.
Location -
Remote
Posted -
As an RLHF & Model Evaluation Specialist you will judge model responses in your area of expertise, such as software engineering, math or science. You will apply rubrics consistently, explain your reasoning and help refine guidelines as projects evolve.
Consistent, well-reasoned judgment is the signal models learn from.
Responsibilities
Rank and grade model responses against project rubrics.
Write short, clear explanations for your ratings.
Identify subtle errors, unsafe outputs and unclear prompts.
Take part in calibration rounds so scoring stays consistent across reviewers.
Requirements
Strong background in software engineering, mathematics or another technical field.
Excellent written communication and attention to detail.
Willingness to follow detailed guidelines and give feedback on them.
Experience with model evaluation or annotation is a plus, not a requirement.
