Software Engineer in Test

What youll do

Test Infrastructure & Automation

Design, build, and maintain scalable test automation frameworks for API, UI, and end-to-

end testing across Xometrys suite of products

Integrate automated test suites into CI/CD pipelines to enable fast, reliable feedback

loops for development teams

Build and maintain test observability tooling including dashboards, flakiness tracking,

and failure triage automation

Partner with engineers to define testability requirements and embed quality practices

early in the software development lifecycle

Agentic Test Authoring

Design and maintain agent-readable context files (CLAUDE.md, AGENTS.md, etc.) that

Give AI coding agents the knowledge they need to write correct, idiomatic E2E tests

Write reusable agent skills and rules that encode your teams testing conventions, page

object patterns, assertion strategies, and fixture management - so agents produce tests

that pass code review

Build and maintain agent workflows (e.g., Claude Code agents, Cursor rules) that can

Take a feature description or ticket and produce a working E2E test suite with minimal

human intervention

Define skills, test scaffolding, and guardrails that constrain agent output to your

framework conventions and prevent common failure modes (flaky selectors, missing

teardown, improper auth state)

Evaluate and iterate on agent-generated test quality - tracking pass rates, flakiness, and

maintainability of agent-authored tests vs. human-authored tests

Collaborate with engineers to identify high-value test coverage gaps that can be closed

through agentic generation, and build the pipelines to close them continuously

Stay current on the agentic coding tooling landscape (Claude Code, Cursor, Copilot

Workspace, etc.) and bring a point of view on what belongs in our stack

What youll bring

Required

4+ years of software engineering experience with a test infrastructure or automation

focus

Proficiency in one or more of: Python, Java, TypeScript, or Go

Strong experience building and maintaining UI automation frameworks (Playwright,

Cypress, or Selenium)

Solid API testing experience (REST, GraphQL)

Experience integrating tests into CI/CD systems (GitHub Actions, Jenkins, CircleCI, etc.)

Familiarity with containerization and test environment orchestration (Docker,

Kubernetes)

Strong software engineering fundamentals - you write test code the same way youd

write production code

Hands-on experience with AI coding agents (Claude Code, Cursor, Copilot, or similar) -

you use these tools daily and understand their capabilities and failure modes

Experience writing structured context files, rules, or instructions that shape agent

behavior - you know the difference between an agent that writes a test and an agent that writes a good test

Nice to have

Experience with performance or load testing (k6, Locust, Gatling)

Exposure to observability tooling (Datadog, Grafana, PagerDuty, Coralogix)

Familiarity with chaos engineering or fault injection testing

Experience in marketplace, e-commerce, or B2B SaaS environments

Background building self-healing or AI-assisted test tooling

Experience scaling test infrastructure across multiple teams or product areas

Contributions to open-source testing tools or agent frameworks