Senior Software Engineer – LLM Evaluation

About this opportunity

Turing is seeking a Senior Software Engineer to work as a Software Engineering evaluator creating cutting-edge datasets for training, benchmarking, and advancing large language models. The role involves curating code examples, developing precise solutions and corrections across multiple programming languages, evaluating AI-generated code for efficiency, scalability, and reliability, collaborating with cross-functional teams, and designing mechanisms that can automatically verify software engineering solutions.

What the listing describes

• ✓Curate high-quality code examples for AI model training and evaluation
• ✓Build software solutions and correct code in Python, JavaScript, ReactJS, C/C++, Java, Rust, and Go
• ✓Evaluate and refine AI-generated code for efficiency, scalability, reliability, and correctness
• ✓Collaborate with cross-functional teams to improve AI-driven coding solutions against industry performance benchmarks
• ✓Build agents capable of verifying software code quality and identifying recurring error patterns
• ✓Hypothesize and evaluate different stages of the software engineering lifecycle, including prototyping, architecture design, API design, production implementation, launch, experimentation, monitoring, and operational maintenance
• ✓Evaluate the capabilities of AI models across different software engineering workflows
• ✓Design verification mechanisms capable of automatically validating solutions to software engineering tasks
• ✓Create precise solutions and corrections that can be used to improve AI model performance
• ✓Provide structured technical evaluations and communicate findings clearly to researchers and engineering teams

Requirements shown on the source

• ✓Several years of professional software engineering experience
• ✓At least 2 years of continuous full-time experience at a top-tier product company or comparable high-level engineering environment
• ✓Strong expertise in building full-stack applications
• ✓Experience deploying scalable, production-grade software using modern languages and tools
• ✓Deep understanding of software architecture and system design
• ✓Strong software development and debugging skills
• ✓Strong understanding of code quality and code review practices
• ✓Ability to evaluate software implementations for correctness, efficiency, scalability, and reliability
• ✓Experience working with multiple modern programming languages and software development ecosystems
• ✓Ability to reason about the full software engineering lifecycle from prototyping through production and maintenance
• ✓Excellent oral communication skills
• ✓Excellent written communication skills
• ✓Ability to produce clear and structured technical evaluation rationales
• ✓Ability to work independently in a remote contractor environment

Relevant backgrounds mentioned

• ✓Experience at a top-tier technology or product company such as Google, Stripe, Amazon, Apple, Meta, Netflix, Microsoft, Datadog, Dropbox, Shopify, PayPal, or IBM Research
• ✓Strong full-stack engineering background
• ✓Experience with Python, JavaScript, ReactJS, C/C++, Java, Rust, or Go
• ✓Experience building scalable production systems
• ✓Experience with AI-assisted software development or LLM evaluation
• ✓Experience designing automated testing and verification systems
• ✓Experience developing software agents or automated code-quality tools
• ✓Experience collaborating with AI researchers or machine learning teams