About The RoleRex.zone is hiring Germany-based English/German AI Generalist Trainers to support RLHF and large language model evaluation by assessing, ranking, and validating model outputs to improve training data quality and drive model performance improvement.
Responsibilities
- * Evaluate and rank model-generated outputs in English and German using defined rubrics and annotation guidelines
- Perform RLHF-style preference ranking and prompt evaluation to identify the best responses
- Conduct QA evaluation to ensure training data quality, consistency, and annotation guidelines compliance
- Write concise, evidence-based rationales explaining evaluation decisions
- Validate outputs for completeness, policy adherence, and content safety labeling requirements
- Flag ambiguous prompts, edge cases, and guideline gaps; propose clarifications
- Track errors, patterns, and failure modes that affect evaluation outcomes and model performance
Basic Qualifications
- * Based in Germany and able to work remotely on a full-time schedule
- Fluent in German and English (reading, writing, and nuance in both languages)
- Strong analytical reasoning and attention to detail; consistent guideline compliance
- Comfortable working with structured workflows, task queues, and quality targets
- Able to handle sensitive or safety-related content in line with labeling policies
Preferred Qualifications
- * Experience with data labeling, LLM evaluation, prompt evaluation, or QA evaluation
- Familiarity with RLHF concepts, preference ranking, and common LLM failure modes
- Self-driven and reliable in a remote environment
Compensation$35–$40 USD per hour (hourly).
Feed salary range: $30–$50 USD per hour.
How To ApplyApply with a short summary of your bilingual English/German experience, your location in Germany, and any relevant evaluation/QA/data labeling background.
