Senior Software Engineer

Skill: ETL Automation Testing

Experience: 3-10 Years

Location: Hyderabad/Pune/Coimbatore & Bangalore

About the Role

As an ETL Automation Tester, you will design, implement, and execute automated testing solutions for large-scale data pipelines. You will validate extraction, transformation, and loading processes, ensure data accuracy, and build automation frameworks to streamline testing across data warehouses and big data environments.

Responsibilities

• Develop and maintain automation frameworks for ETL testing using Python and PySpark.
• Design and execute test cases and scripts to validate data extraction, transformation, and loading.
• Perform data quality checks, including completeness, accuracy, and consistency across source and target systems.
• Automate regression testing to ensure new changes do not impact existing workflows.
• Collaborate with data engineers, analysts, and QA teams to identify and resolve defects.
• Conduct performance and scalability testing of ETL jobs in distributed environments.
• Document test results, defects, and provide detailed reports to stakeholders.
• Integrate ETL testing into CI/CD pipelines for continuous validation.
• Ensure compliance with data governance and security standards.

Qualifications

• Bachelors degree in Computer Science, Information Technology, or related field.

Required Skills

• Strong knowledge of ETL concepts, data warehousing, and big data frameworks.
• Hands-on experience with PySpark for distributed data processing and validation.
• Proficiency in Python for automation scripting and test framework development.
• Strong SQL skills for data validation and query optimization.
• Experience with automation tools (QuerySurge, Datagaps, or custom Python frameworks).
• Familiarity with CI/CD tools (Jenkins, Git, Azure DevOps) for test integration.
• Excellent analytical, problem-solving, and communication skills.