Exp: 4 to 13 years
Skill: Data Bricks+Pyspark
Location : Bangalore/Hyderabad/Kolkota/Pune/Chennai
Technical Skills :
Python,PySpark,Azure Data Lake Store,Databricks Workflows,Databricks SQL
Responsibilities :
- Develop and optimize data solutions using Azure Data Lake Store to support business needs.
- Utilize Python to create efficient and scalable data processing scripts.
- Implement and manage Databricks SQL for querying and analyzing large datasets.
- Design and maintain Databricks Workflows to automate data processing tasks.
- Leverage PySpark to perform large-scale data transformations and analytics.
- Collaborate with cross-functional teams to understand data requirements and deliver solutions.
- Ensure data integrity and quality through rigorous testing and validation processes.
- Provide technical guidance and support to junior developers and team members.
- Monitor and troubleshoot data pipelines to ensure smooth operation.
- Stay updated with the latest industry trends and technologies to continuously improve data solutions.
- Document all development processes and workflows for future reference.
- Contribute to the overall data strategy and architecture of the organization.
- Drive innovation by exploring new tools and techniques to enhance data capabilities.
-Qualifications
- Possess strong experience in Azure Data Lake Store, demonstrating the ability to manage and optimize large data repositories.
- Have extensive knowledge of Python, with a proven track record of developing efficient data processing scripts.
- Show proficiency in Databricks SQL, capable of querying and analyzing complex datasets.
- Demonstrate expertise in designing and maintaining Databricks Workflows for automated data processing.
- Exhibit strong skills in PySpark, with experience in large-scale data transformations and analytics.
- Nice to have experience in Investment Banking Operations, providing valuable domain insights.