Senior Data Engineer_96447

Company:A Large Global Organization

Key Skills: Data Engineer, Kafka

Roles and Responsibilities:

• Design, develop, and maintain data engineering solutions using Azure services including Azure Data Factory, Azure SQL Server, and Azure Databricks.
• Implement and manage data pipelines, ensuring data quality and compliance with best practices.
• Develop APIs using FastAPI or similar frameworks in Python.
• Collaborate with stakeholders to interpret business requirements and design effective data solutions.
• Troubleshoot and resolve data-related issues across multiple platforms using SQL and Azure SQL Server.
• Maintain documentation for data pipeline architecture and development processes using tools like Confluence.
• Participate in the DevOps lifecycle, supporting CI/CD processes and utilizing tools such as Git and Jenkins.
• Analyze and optimize existing PySpark/Python code for efficient data transformations.
• Deliver results under demanding timelines while ensuring adherence to Service Level Agreements (SLA) and Turn Around Times (TAT).
• Provide training and support to end-users, developing training materials and release notes as necessary.

Skills Required:

• Strong experience in data engineering and data pipeline development.
• Hands-on expertise with Azure cloud services such as Azure Data Factory, Azure SQL Server, and Azure Databricks.
• Proficiency in Python programming and PySpark for large-scale data processing.
• Experience developing APIs using FastAPI or similar Python frameworks.
• Strong knowledge of SQL and database management.
• Experience working with streaming platforms such as Kafka (preferred).
• Familiarity with DevOps practices and CI/CD pipelines using tools like Git and Jenkins.
• Experience with data architecture design, data transformation, and optimization techniques.
• Strong problem-solving skills to troubleshoot data pipeline and platform issues.
• Experience documenting data architecture, pipelines, and processes using collaboration tools like Confluence.
• Ability to work in fast-paced environments with strict timelines and SLA requirements.

Education: B.Tech / M.Tech (Dual Degree) in Electrical & Electronics, Mechanical Engineering, or Computer Science.