Senior QA Engineer - GenAI & Systems Quality

Our Company

Gateway is a stealth technology company building an agentic platform that brings enterprise-grade operational and compliance rigor to startups and SMEs through automation and intelligent systems

We focus on system design, automation, and agent-driven execution in environments where correctness, auditability, and failure handling matter. Teams are small, specifications are explicit, and ownership is end-to-end.

Role Overview

For QA and quality engineering, this means working on systems where correctness matters: automated workflows, agent behavior, approvals, auditability, and failure recovery. Quality is not a final checkpoint at Gatewayit is designed into specifications, test cases, and system boundaries from the start.

QA engineers at Gateway work closely with product and engineering to define acceptance criteria, design adversarial and edge-case tests, validate agent behavior, and ensure systems behave predictably under real-world conditions. Ownership is end-to-end, and testability is treated as a core system requirement.

We are looking for a Senior QA Engineer to lead the quality programme for our AI-native startup OS platform & products. This role is a hybrid of QA for AI (validating the accuracy and safety of our models) and AI for QA (leveraging agentic technology to accelerate our testing lifecycle). You will move beyond simple pass/fail scripts to build systems that evaluate the nuance of conversational intelligence.

Key Responsibilities

1. QA for AI: Validating Conversational Intelligence

• Conversational Evaluation: Design test frameworks to verify conversation correctness, coherence, and effectiveness across multi-turn dialogues.
• LLM-as-a-Judge: Implement and fine-tune "LLM Judge" architectures to automate the grading of non-deterministic outputs against ground-truth datasets.
• Behavioral Simulation: Develop user behavior simulation agents to stress-test conversational flows, identifying "hallucinations" or dead-ends in the user journey.
• Safety & Alignment: Build test suites to detect bias, toxicity, and prompt injection vulnerabilities in AI responses.

2. AI for QA: Engineering Velocity

• Strategy Augmentation: Use GenAI to analyze product requirements and automatically generate comprehensive test strategies and edge-case scenarios.
• Agentic Automation: Deploy AI agents (including browser-based ) to perform autonomous exploratory testing and self-healing UI automation.
• Gap Analysis: Leverage LLMs to analyze existing test suites against codebase changes to identify coverage gaps and "hidden" regression risks.
• Synthetic Data Generation: Build pipelines that use AI to generate realistic, high-variance test data for complex user scenarios.

Technical Requirements

• Experience: 5+ years in Software Quality Engineering with a track record of owning complex, ambiguous sub-systems.
• Deep understanding of test methodologies for AI/ML applications.
• AI for Testing: Hands-on programming experience in using GenAI frameworks. Proven experience of using GenAI tools to assist in writing code, generating test cases, or automating documentation
• Testing for AI: Hands-on experience with LLM evaluation metrics and building "LLM-as-a-Judge" workflows.
• Simulation & Tooling: Experience developing agents or scripts that simulate human-like interactions within a digital interface.
• Modern Stack: Deep proficiency in Python (preferred) or JavaScript, and experience with modern frameworks like Playwright, Selenium, AI browser automation frameworks like Stagehand
• Ability to process ongoing customer feedback to identify failure points that can be improved systemically with more comprehensive edge-case testing.
• Knowledge of prompt evaluation, prompt injections, and adversarial testing for AI systems
• Understanding of fairness, bias, and explainability testing in AI
• Familiar with test management tools and CI/CD pipelines
• Excellent communication and documentation skills

Why Join Us?

• Be at the forefront of AI quality assurance innovation
• Influence how modern AI systems are tested, evaluated, and trusted
• Work with a passionate, mission-driven team building impactful products
• Competitive compensation and flexible work options

How to Apply?

Email us your latest resume to jobs@gogateway.ai