Senior Data Engineer

Job Title: Senior /Lead Data Engineer

Location: Velachery ,Chennai

Employment Type: Full-Time (onsite, 5days )

Experience Level: 7-12 years

Department: Data Engineering

Job Summary:

We are looking for a Lead Data Engineer with strong expertise in Azure, advanced skills in Spark, SQL, PySpark, and data modeling/design. The role involves leading the design and development of scalable data pipelines, driving best practices, mentoring the team, and managing complex real-time/streaming data solutions.

Key Responsibilities:

• Lead the design, development, and optimization of scalable and efficient ETL/ELT data pipelines.
• Oversee data cleaning, transformation, and validation from diverse structured and unstructured sources.
• Collaborate closely with Data Scientists, Analysts, and stakeholders to define and deliver robust data solutions aligned with business goals.
• Ensure data quality, consistency, and integrity across all data platforms and pipelines.
• Develop and maintain comprehensive documentation for data architecture, processes, and best practices.
• Drive code reviews, enforce engineering standards, and promote continuous improvement across the team.
• Lead the implementation of streaming data technologies and manage real-time data processing solutions.
• Mentor and coach junior and mid-level data engineers, fostering technical growth and ensuring successful project delivery.
• Provide technical leadership in architectural decisions and contribute to the overall data strategy.

Required Skills & Qualifications:

• 7+ years of experience in data engineering or related roles, with proven leadership in complex projects.
• Strong expertise in Azure Data Services (ADF, Databricks, ADLS Gen2, Key Vault, Event Hub).
• Advanced proficiency in SQL and programming languages such as Python or Scala.
• Deep understanding of Apache Spark and Spark Core concepts.
• Extensive experience with Azure Databricks for large-scale data engineering workloads.
• Strong knowledge of data warehousing, data modeling, and relational database concepts.
• Exposure to cloud platforms (Azure preferred; AWS or GCP is a plus).
• Hands-on experience with ETL tools and enterprise data integration frameworks.
• Excellent analytical, debugging, and problem-solving skills.
• Proven ability to lead teams, mentor engineers, and collaborate across functions.
• Knowledge of streaming technologies such as Apache Flink and/or programming in Java is an added advantage.
• Familiarity with version control and CI/CD tools like Git and Azure DevOps.
• Strong communication skills with the ability to drive technical discussions and influence stakeholders.

Note : Immediate Joiners/ Candidates who can join within 15days are preferred.