Board Game Reasoning Expert (AI Training & Evaluation)

About this opportunity

Turing is seeking Board Game Reasoning Experts to help train and evaluate next-generation AI models. The role involves using expertise in board games, game mechanics, strategic reasoning, probability, and complex rule systems to create, review, and evaluate high-quality datasets. Experts will work on tasks involving game logic, decision-making, strategic planning, rule interpretation, and scenario analysis to help improve the reasoning capabilities of frontier AI systems.

What the listing describes

• ✓Create and review game-based reasoning tasks designed to evaluate AI systems
• ✓Analyze board game scenarios, strategic decision trees, and rule-based systems
• ✓Evaluate AI-generated responses for correctness, consistency, and reasoning quality
• ✓Develop prompts for board game and strategy-focused reasoning tasks
• ✓Create rubrics and evaluation guidelines for game-based AI assessments
• ✓Identify logical errors, rule violations, and flawed reasoning in AI-generated outputs
• ✓Analyze strategic decision-making scenarios involving complex game mechanics
• ✓Apply probability and logical reasoning to evaluate proposed game strategies and outcomes
• ✓Contribute to benchmark creation for AI reasoning and evaluation projects
• ✓Support quality assurance processes for AI training and evaluation datasets
• ✓Collaborate with project teams to improve dataset quality and evaluation methodologies

Requirements shown on the source

• ✓Bachelor’s degree in Computer Science, Mathematics, Cognitive Science, Game Design, Philosophy, Economics, or a related analytical field
• ✓At least 2 years of professional or semi-professional experience in board game design, playtesting, tabletop game communities, or related strategy-focused environments
• ✓Strong understanding of logic and probabilistic reasoning
• ✓Strong understanding of game mechanics and complex rule systems
• ✓Background in game theory, behavioral economics, decision science, formal logic, or a related analytical discipline
• ✓Familiarity with modern board games, trading card games (TCGs), tabletop RPGs, strategy games, or competitive game systems
• ✓Strong analytical and problem-solving skills
• ✓Excellent written communication skills
• ✓Ability to analyze complex game scenarios and strategic decision trees
• ✓Ability to consistently interpret and apply detailed game rules
• ✓Ability to evaluate AI-generated reasoning for correctness and logical consistency
• ✓Ability to work independently in a remote environment
• ✓Availability for at least 4 hours per day and a minimum of 20 hours per week
• ✓Ability to provide 4 hours of overlap with PST working hours

Relevant backgrounds mentioned

• ✓Prior experience in data annotation
• ✓Experience with AI training or evaluation
• ✓Experience with prompt engineering
• ✓Experience in quality assurance
• ✓Experience with game design or puzzle design
• ✓Experience analyzing rules-based systems
• ✓Experience creating evaluation rubrics
• ✓Experience creating benchmark datasets
• ✓Experience developing quality assurance frameworks
• ✓Experience with Python, SQL, or data analysis tools
• ✓Experience working on RLHF projects
• ✓Experience with AI model evaluation
• ✓Experience with synthetic data generation
• ✓Experience with LLM benchmarking projects
• ✓Familiarity with recently released strategy or tabletop games
• ✓Ability to quickly learn and analyze new game rule systems