Senior QA engineer (Data Platforms)

We are seeking a Senior QA Engineer to ensure the stability, accuracy, and reliability of our Databricks-based data platforms. This role will primarily focus on validating and monitoring data pipelines, ETL processes, and backend data workflows within our Data Lake and Big Data environments. You will be responsible for implementing automated data quality checks, validating Databricks jobs regularly, and ensuring data integrity across the Data Market Hub platform.

The position involves hands-on testing of data transformations using Python, PySpark, and SQL, executing pipeline validation checks, and supporting release cycles through structured QA processes. You will work closely with data engineers and platform teams to proactively identify issues, validate data accuracy, and maintain consistent quality standards across all data assets. Additionally, you will contribute to test reporting, dashboarding, and continuous improvement of QA practices within the data platform ecosystem.

Open for locations: Bangalore, Pune, Chennai, Hyderabad

RESPONSIBILITIES

Automate data quality checks for Databricks (DBRX), Data Lake, and Big Data environments.

Create and maintain test plans for the Data Market Hub platform.

Develop and maintain Data Quality and Data Expectation checks within the Market Data Hub Platform using Python/PySpark

Participate in requirements refinement and specification.

Use SQL to validate data integrity and support efficient data processing workflows.

Collaborate in the incident management process to ensure swift issue resolution.

Test Data Management: Use SQL to manage test data, retrieve relevant test information, and store test execution results in a centralized database for historical analysis and reporting.

Dashboard Design: Create PowerBI dashboards to provide insights into test results, code coverage, and performance trends across projects and test levels.

REQUIREMENTS

Minimum 5 years in QA Automation.

minimum 3 years of experience testing big data on the Databricks platform.

Strong knowledge of SQL and Python for creating test scripts.

Solid understanding of PySpark (3y+).

Hands-on experience with Azure DevOps.

Hands-on experience with backend and API testing, including REST services, authentication, and data validation

Experience with test automation tools.

Familiarity with data warehouses and ETL tools for quality testing.

Understanding of CI/CD processes and integration.

Fast learner with the ability to adapt to new technologies and tools.