Opens LinkedIn
- Employment type
- Part-time
- Experience level
- Senior · 5+ years
- Posting language
- English
- Working hours
- 40 hours per week
- Seniority
- Mid-Senior level
Job summary
Design and refine rubrics and evaluation criteria for preference data, review and label data to a high quality standard, and build infrastructure and pipelines for data workflows. Summarize findings and collaborate with researchers and engineers to turn qualitative judgments into scalable processes.
Job details
We're hiring for a leading AGI infrastructure company that partners with frontier AI labs to accelerate model development. This organization is at the forefront of AI innovation, providing essential training data, evaluations, and engineering talent to push the boundaries of artificial intelligence. The Role • Design, refine, and iterate on rubrics and evaluation criteria for preference data. • Review and label data with a high bar for quality, identifying subtle issues. • Build and maintain pipelines and infrastructure for data generation, collection, and evaluation workflows. • Synthesize findings from data work into clear write-ups, updates, or recommendations. • Collaborate with researchers and engineers to translate qualitative judgment into scalable processes. What You'll Need • 5+ years of proficiency in Python (especially building/maintaining pipelines and infra) or Typescript. • Hands-on experience with preference data collection and/or rubric creation. • Strong written and verbal communication skills, able to distill technical work into clear, actionable summaries. • Demonstrated high-taste judgment, with the ability to articulate the quality of work. • Comfort in ambiguous, fast-moving environments, with a bias for shipping. What's On Offer • Competitive hourly compensation of $100 per hour. • Opportunity to work on cutting-edge AI projects with leading LLM companies. • Fully remote work environment. Apply via Haystack today!
What you’ll do
Design and refine rubrics and evaluation criteria for preference data, review and label data to a high quality standard, and build infrastructure and pipelines for data workflows. Summarize findings and collaborate with researchers and engineers to turn qualitative judgments into scalable processes.
Requirements
Requires at least five years of proficiency in Python or TypeScript, particularly for building and maintaining pipelines and infrastructure, plus hands-on experience with preference data collection or rubric creation. Candidates should communicate technical work clearly, demonstrate strong judgment about quality, and be comfortable working in ambiguous, fast-moving environments.
Benefits
- Fully Remote Work Environment
- Opportunity to Work on Cutting-Edge AI Projects
Listed skills
- Pipeline Development · Preferred
- Communication · Preferred
- TypeScript · Preferred
- Python · Preferred
Other relevant skills
Identified from the job description. Confirm important requirements above.
- Python
- TypeScript
- Pipeline Development
- Infrastructure Maintenance
- Preference Data Collection
- Rubric Creation
- Data Labeling
- Data Quality Evaluation
- Evaluation Criteria Design
- Data Generation
- Data Collection
- Workflow Evaluation
- Technical Writing
- Communication
- Qualitative Judgment
- Researcher and Engineer Collaboration
Job areas
- Software
- Technology
- Data & Analytics
- Engineering
- Science & Research